CN111506374A - Cloud application UI interaction method and device based on different objects - Google Patents
Cloud application UI interaction method and device based on different objects Download PDFInfo
- Publication number
- CN111506374A CN111506374A CN202010282394.XA CN202010282394A CN111506374A CN 111506374 A CN111506374 A CN 111506374A CN 202010282394 A CN202010282394 A CN 202010282394A CN 111506374 A CN111506374 A CN 111506374A
- Authority
- CN
- China
- Prior art keywords
- image
- compression
- graphic data
- compression algorithm
- cloud application
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000003993 interaction Effects 0.000 title claims abstract description 31
- 230000006835 compression Effects 0.000 claims abstract description 101
- 238000007906 compression Methods 0.000 claims abstract description 101
- 238000011084 recovery Methods 0.000 claims abstract description 5
- 230000003068 static effect Effects 0.000 claims description 39
- 230000005540 biological transmission Effects 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 12
- 238000005516 engineering process Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 3
- 230000015556 catabolic process Effects 0.000 description 2
- 238000006731 degradation reaction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000002427 irreversible effect Effects 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
- G06F9/452—Remote windowing, e.g. X-Window System, desktop virtualisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
- H04N21/64723—Monitoring of network processes or resources, e.g. monitoring of network load
- H04N21/64738—Monitoring network characteristics, e.g. bandwidth, congestion level
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Security & Cryptography (AREA)
- Information Transfer Between Computers (AREA)
- Compression Of Band Width Or Redundancy In Fax (AREA)
Abstract
A cloud application UI interaction method and device based on different objects are disclosed, the method comprises the steps of obtaining graphic data to be transmitted from a server side; judging whether the type of the graphic data to be transmitted belongs to a first image type or a second image type; performing image compression coding on the first image type by adopting a first compression algorithm, and performing image compression coding on the second image type by adopting a second compression algorithm; transmitting the compressed graphic data to a remote client; pattern recovery step S150: and decoding and displaying the compressed graphic data. The invention performs compression through two different compression algorithms, reduces the bandwidth on the premise of ensuring the image quality, seeks the balance between the image quality and the bandwidth, and realizes better user experience.
Description
Technical Field
The invention relates to the field of image transmission, in particular to a cloud application UI interaction method and device based on different objects in cloud application and adopting different compression algorithms.
Background
The cloud application is a subset of cloud computing, and the working principle of the cloud application is a novel application that the mode of local installation and local operation of traditional software is changed into a service of 'instant taking and using', and a remote server cluster is connected and controlled through the Internet or a local area network to complete service logic or an operation task. Because the software has the characteristics of cross-platform property, usability and lightness, the development of the traditional software to the cloud application transformation is great tendency.
The development of cloud computing brings the advantage of data concentration to people, so that resources can be reasonably distributed and managed, data of a data center can be protected, the security is higher, and the method and the system can be helpful for preventing the loss of sensitive information compared with a plurality of scattered computers which are not easy to manage. Therefore, many people try to propose an efficient cloud desktop system by combining the idea of data concentration to realize virtualization of a server desktop.
According to the traditional cloud desktop interaction mode, after a terminal logs in, a server provides a compression algorithm to compress and output desktop images of a virtual machine, and the terminal displays the images through decoding. The image processing is a very complex technology, and common manufacturers adopt a compression rate to compress after full screen capture, so that the protocol is simplest and the potential problems are the most. Obviously, the larger the image compression rate, the smaller the bandwidth, and the faster the speed, the greater the image distortion. And the smaller the image compression rate, the lower the image distortion rate, and the corresponding bandwidth is increased by a large amount. In order to ensure that the image is not distorted, a lossless compression algorithm is generally adopted, and the method occupies more bandwidth, affects the display efficiency of the terminal and causes poor user experience.
Therefore, in cloud application transmission, how to reduce the image transmission bandwidth, ensure the display quality of the terminal, and maintain the user experience becomes a technical problem to be solved urgently in the prior art.
Disclosure of Invention
The invention aims to provide a cloud application UI interaction method and device based on different objects, which are used for processing a static image part and a non-static image part separately and compressing the static image part and the non-static image part through different compression algorithms to seek the balance of image quality and bandwidth and realize better user experience.
In order to achieve the purpose, the invention adopts the following technical scheme:
a cloud application UI interaction method based on different objects comprises the following steps:
a graphic data acquisition step S110: acquiring graphic data to be transmitted from a server side;
graphic data type determination step S120: judging whether the type of the graphic data to be transmitted belongs to a first image type or a second image type;
image compression encoding step S130: performing image compression coding on the first image type by adopting a first compression algorithm, and performing image compression coding on the second image type by adopting a second compression algorithm;
pattern transmission step S140: transmitting the compressed graphic data to a remote client; for example, to the Web site over a network.
Pattern recovery step S150: and decoding and displaying the compressed graphic data.
Optionally, the step S110 of obtaining the graphic data is specifically to obtain the graphic data through Hook in the process of calling the API interface of the bottom graphic library at the server side to draw the graphic.
Optionally, the first image type is a static image, and the second image type is a non-static image;
wherein the compression loss of the first compression algorithm is less than the compression loss of the second compression algorithm.
Optionally, the first compression algorithm is a lossless compression algorithm, and the second compression algorithm is a lossy compression algorithm.
The invention also discloses a cloud application UI interaction device based on different objects, which is characterized by comprising the following units:
the graphic data acquisition unit 210: the system comprises a server side, a data transmission unit and a data transmission unit, wherein the server side is used for acquiring graphic data to be transmitted from the server side;
the graphic data type determination unit 220: for judging whether the type of the graphic data to be transmitted belongs to a first image type or a second image type;
the first encoding unit 230: the image compression coding device is used for carrying out image compression coding on the first image type by adopting a first compression algorithm;
the second encoding unit 240: the image compression coding is carried out on the second image type by adopting a second compression algorithm;
the graphic transmission unit 250: the system is used for transmitting the compressed graphic data to a remote client; for example, to the Web site over a network.
The graphic restoration unit 260: for decoding and displaying the compressed graphic data.
Optionally, the graphic data obtaining unit 210 is specifically configured to obtain the graphic data through Hook in a process of calling an API interface of a bottom graphic library at the server side to perform graphic drawing.
Optionally, the first image type is a static image, and the second image type is a non-static image;
wherein the compression loss of the first compression algorithm is less than the compression loss of the second compression algorithm.
Optionally, the first compression algorithm is a lossless compression algorithm, and the second compression algorithm is a lossy compression algorithm.
The invention also discloses a storage medium for storing computer executable instructions, which is characterized in that:
the computer executable instructions, when executed by the processor, perform the different object based cloud application UI interaction method described above.
According to the cloud application UI interaction method and device based on different objects, in the process that the server side calls the API interface of the bottom graphics library to draw the graphics, the graphics data are obtained through Hook, the obtained data are processed separately according to the static image part and the non-static image part, and the two different compression algorithms are used for compression, so that the bandwidth is reduced on the premise of ensuring the image quality, the balance between the image quality and the bandwidth is sought, and better user experience is achieved.
Drawings
FIG. 1 is a flow chart of a method of different object based cloud application UI interaction according to the present invention;
FIG. 2 is a flow chart of a method of different object based cloud application UI interaction according to the present invention;
fig. 3 is a schematic diagram of a cloud application UI interaction method according to a specific embodiment of the present invention and a UI interaction method according to the related art.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Technical terms in the present invention have the following definitions:
1.OpenGL
OpenG L (Open Graphics L library) is a specialized graphical program interface that defines a cross-programming language, cross-platform programming interface specification for three-dimensional images (two-dimensional as well) and is a powerful, easy-to-call underlying Graphics library.
2.Hook
In computer programs, Hook is a technique for changing or augmenting the behavior of an application by intercepting function calls, or intercepting messages or events passed between software components. The code that handles intercepted function calls, events or messages is called "Hook".
Because of the persistence of vision, animation (i.e., video) formed by changing pictures is relatively insensitive and can be seen as a general picture. For a static image portion (such as an interface control for user interaction), the static image portion is more represented in a static form, and if the distortion degree is higher, the user can feel more obvious. Therefore, the static image part and the non-static image part in the cloud server are processed separately and compressed through two different compression algorithms, so that the balance between the image quality and the bandwidth is sought, and better user experience is realized.
Specifically, referring to fig. 1, a flowchart of a method for interacting a cloud application UI based on different objects according to the present invention is shown, which includes the following steps:
a graphic data acquisition step S110: acquiring graphic data to be transmitted from a server side;
specifically, in the process of calling the API interface of the underlying graphics library at the server side to draw the graphics, the graphics data is obtained by using a Hook technology, for example.
Graphic data type determination step S120: it is determined whether the type of graphics data to be transmitted belongs to a first image type or a second image type.
Image compression encoding step S130: and carrying out image compression coding by adopting a first compression algorithm on the first image type, and carrying out image compression coding by adopting a second compression algorithm on the second image type.
Further, the first image type is a static image, and the second image type is a non-static image;
the static image mainly refers to an image displayed in a static form, and comprises an interface control for user interaction, a desktop display icon and the like.
The non-static image mainly refers to animation, video and the like formed by changing pictures.
Wherein the compression loss of the first compression algorithm is less than the compression loss of the second compression algorithm.
Pattern transmission step S140: transmitting the compressed graphic data to a remote client; for example, to the Web site over a network.
Pattern recovery step S150: and decoding and displaying the compressed graphic data.
The first compression algorithm may be a lossless compression algorithm, and the second compression algorithm may be a lossy compression algorithm. Compared with a lossless compression algorithm, the lossy compression algorithm discards some data from an original file, the process is irreversible, the original state cannot be returned once loss occurs, and the image degradation is more serious as the compression times are more.
Therefore, the type of the bottom layer graphic data is judged, the static image part and the non-static image part are processed separately, the compression is carried out through two different compression algorithms, the graphic data acquired through the Hook technology belong to changed data, the data volume is smaller compared with the full screen capture and compression transmission every time, and the pressure of network transmission is smaller. Seeking the balance between image quality and bandwidth to realize better user experience
Further, referring to fig. 2, a block diagram of a different object based cloud application UI interaction apparatus according to the present invention is shown, including the following elements:
the graphic data acquisition unit 210: the system comprises a server side, a data transmission unit and a data transmission unit, wherein the server side is used for acquiring graphic data to be transmitted from the server side;
specifically, the graphic data obtaining unit obtains the graphic data through a Hook technology, for example, in a process that the server side calls an API interface of a bottom graphic library to perform graphic drawing.
The graphic data type determination unit 220: for determining whether the type of graphics data to be transmitted belongs to a first image type or a second image type.
The first encoding unit 230 is configured to perform image compression encoding on the first image type by using a first compression algorithm.
And a second encoding unit 240 for performing image compression encoding using a second compression algorithm for the second image type.
Further, the first image type is a static image, and the second image type is a non-static image;
the static image mainly refers to an image displayed in a static form, and comprises an interface control for user interaction, a desktop display icon and the like.
The non-static image mainly refers to animation, video and the like formed by changing pictures.
Wherein the compression loss of the first compression algorithm is less than the compression loss of the second compression algorithm.
The graphic transmission unit 250: the system is used for transmitting the compressed graphic data to a remote client; for example, to the Web site over a network.
The graphic restoration unit 260: for decoding and displaying the compressed graphic data.
Example (b):
referring to fig. 3, a schematic diagram of a cloud application UI interaction method according to a specific embodiment of the present invention and a UI interaction method according to the related art is shown.
As can be seen in fig. 3, the conventional display mode is to perform screenshot in the process of drawing a graph, compress the screenshot in a lossy coding algorithm mode or a lossless coding algorithm mode, and transmit the compressed screenshot to the Web end through a network for display.
The method comprises the steps of acquiring graphic data output by an API (application programming interface) interface of a bottom graphic library by a Hook technology in the process of calling the interface to display a graphic, respectively storing the acquired data and dividing the data into a static image part and a non-static image part by calling the API for acquiring the static image part and the API for acquiring the non-static image part when acquiring the graphic data output by the bottom graphic library, and respectively coding and compressing the data.
The non-static image part is coded according to a lossy compression algorithm (for the lossy compression algorithm, some data are discarded from an original file, the process is irreversible, once loss occurs, the original state cannot be returned, the compression times are more, the image degradation is more serious), and the static image part is coded according to the lossless compression algorithm. The encoded data is transmitted to the Web end through a network, and then is decoded and displayed through a decoding technology. The graphic data acquired by the Hook technology belong to changed data, the data volume is smaller relative to each full-screen interception and compression transmission, and the pressure on network transmission is smaller.
The invention further discloses a storage medium for storing computer executable instructions, which is characterized in that: the computer executable instructions, when executed by the processor, perform the different object based cloud application UI interaction method described above.
Therefore, according to the cloud application UI interaction method and device based on different objects, in the process that the server side calls the API (application program interface) of the bottom graphics library to draw graphics, the graphics data are obtained through Hook, the obtained data are processed separately according to the static image part and the non-static image part, and the data are compressed through two different compression algorithms, so that the bandwidth is reduced on the premise of ensuring the image quality, the balance between the image quality and the bandwidth is sought, and better user experience is realized.
While the invention has been described in further detail with reference to specific preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (9)
1. A cloud application UI interaction method based on different objects comprises the following steps:
a graphic data acquisition step S110: acquiring graphic data to be transmitted from a server side;
graphic data type determination step S120: judging whether the type of the graphic data to be transmitted belongs to a first image type or a second image type;
image compression encoding step S130: performing image compression coding on the first image type by adopting a first compression algorithm, and performing image compression coding on the second image type by adopting a second compression algorithm;
pattern transmission step S140: transmitting the compressed graphic data to a remote client;
pattern recovery step S150: and decoding and displaying the compressed graphic data.
2. The cloud application UI interaction method of claim 1, wherein:
the step S110 of obtaining the graphic data is to obtain the graphic data through Hook in the process of calling the API interface of the bottom graphic library at the server side to draw the graphic.
3. The cloud application UI interaction method of claim 1, wherein:
the first image type is a static image, and the second image type is a non-static image;
wherein the compression loss of the first compression algorithm is less than the compression loss of the second compression algorithm.
4. The cloud application UI interaction method of claim 3, wherein:
the first compression algorithm is a lossless compression algorithm, and the second compression algorithm is a lossy compression algorithm.
5. The device for cloud application UI interaction based on different objects is characterized by comprising the following units:
graphics data acquisition unit (210): the system comprises a server side, a data transmission unit and a data transmission unit, wherein the server side is used for acquiring graphic data to be transmitted from the server side;
graphics data type determination unit (220): for judging whether the type of the graphic data to be transmitted belongs to a first image type or a second image type;
first encoding unit (230): the image compression coding device is used for carrying out image compression coding on the first image type by adopting a first compression algorithm;
second encoding unit (240): the image compression coding is carried out on the second image type by adopting a second compression algorithm;
graphics transfer unit (250): the system is used for transmitting the compressed graphic data to a remote client;
graphics recovery unit (260): for decoding and displaying the compressed graphic data.
6. The cloud application UI interaction device of claim 5, wherein:
the graphic data acquisition unit (210) is specifically configured to acquire the graphic data through Hook in the process of calling the API interface of the bottom graphic library at the server side to draw the graphic.
7. The cloud application UI interaction device of claim 5, wherein:
the first image type is a static image, and the second image type is a non-static image;
wherein the compression loss of the first compression algorithm is less than the compression loss of the second compression algorithm.
8. The cloud application UI interaction device of claim 7, wherein:
the first compression algorithm is a lossless compression algorithm, and the second compression algorithm is a lossy compression algorithm.
9. A storage medium for storing computer-executable instructions, characterized in that:
the computer-executable instructions, when executed by a processor, perform the method for heterogeneous object based cloud application UI interaction of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010282394.XA CN111506374A (en) | 2020-04-12 | 2020-04-12 | Cloud application UI interaction method and device based on different objects |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010282394.XA CN111506374A (en) | 2020-04-12 | 2020-04-12 | Cloud application UI interaction method and device based on different objects |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111506374A true CN111506374A (en) | 2020-08-07 |
Family
ID=71863960
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010282394.XA Pending CN111506374A (en) | 2020-04-12 | 2020-04-12 | Cloud application UI interaction method and device based on different objects |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111506374A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104025561A (en) * | 2012-11-23 | 2014-09-03 | 华为技术有限公司 | Image compression method and image processing apparatus |
| CN108184118A (en) * | 2016-12-08 | 2018-06-19 | 中兴通讯股份有限公司 | Cloud desktop contents encode and coding/decoding method and device, system |
| CN108848082A (en) * | 2018-05-31 | 2018-11-20 | 深圳市腾讯网络信息技术有限公司 | Data processing method, device, storage medium and computer equipment |
-
2020
- 2020-04-12 CN CN202010282394.XA patent/CN111506374A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104025561A (en) * | 2012-11-23 | 2014-09-03 | 华为技术有限公司 | Image compression method and image processing apparatus |
| CN108184118A (en) * | 2016-12-08 | 2018-06-19 | 中兴通讯股份有限公司 | Cloud desktop contents encode and coding/decoding method and device, system |
| CN108848082A (en) * | 2018-05-31 | 2018-11-20 | 深圳市腾讯网络信息技术有限公司 | Data processing method, device, storage medium and computer equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11824913B2 (en) | Video stream management for remote graphical user interfaces | |
| CN111399964B (en) | Cloud desktop platform based on video streaming technology | |
| CN111681167B (en) | Image quality adjusting method and device, storage medium and electronic equipment | |
| US8254704B2 (en) | Remote computing platforms providing high-fidelity display and interactivity for clients | |
| CN1856819B (en) | System and method for network transmission of graphical data through a distributed application | |
| Lu et al. | Virtualized screen: A third element for cloud–mobile convergence | |
| US20160255400A1 (en) | Remote Display Performance Measurement Triggered by Application Display Upgrade | |
| US8910228B2 (en) | Measurement of remote display performance with image-embedded markers | |
| CN104660687A (en) | Realization method and system for virtual desktop display | |
| CN109309842B (en) | Live broadcast data processing method and device, computer equipment and storage medium | |
| EP3410302B1 (en) | Graphic instruction data processing method, apparatus | |
| US8995763B2 (en) | Systems and methods for determining compression methods to use for an image | |
| US20090006977A1 (en) | Method and System of Computer Remote Control that Optimized for Low Bandwidth Network and Low Level Personal Communication Terminal Device | |
| WO2022257699A1 (en) | Image picture display method and apparatus, device, storage medium and program product | |
| US8760366B2 (en) | Method and system for remote computing | |
| CN112843676B (en) | Data processing method, device, terminal, server and storage medium | |
| CN104253804B (en) | A kind of method and device for transmitting view data | |
| CN110740352B (en) | SPICE protocol-based difference image display method in video card transparent transmission environment | |
| KR20120058227A (en) | Image data encoding method for presentation virtualization and server therefor | |
| CN114938408A (en) | Data transmission method, system, equipment and medium of cloud mobile phone | |
| CN108769715B (en) | Method and device for processing graphic instruction data | |
| CN111506374A (en) | Cloud application UI interaction method and device based on different objects | |
| CN111327863A (en) | Video call processing method and device | |
| CN110868614B (en) | SPICE protocol-based difference image display system in video card transparent transmission environment | |
| KR101251879B1 (en) | Apparatus and method for displaying advertisement images in accordance with screen changing in multimedia cloud system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200807 |