CN117519887B - Method and system for improving cloud computer remote operation experience - Google Patents
Method and system for improving cloud computer remote operation experience Download PDFInfo
- Publication number
- CN117519887B CN117519887B CN202311711069.0A CN202311711069A CN117519887B CN 117519887 B CN117519887 B CN 117519887B CN 202311711069 A CN202311711069 A CN 202311711069A CN 117519887 B CN117519887 B CN 117519887B
- Authority
- CN
- China
- Prior art keywords
- image
- data
- cloud computer
- picture
- cloud
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 230000008859 change Effects 0.000 claims abstract description 28
- 230000006870 function Effects 0.000 claims abstract description 18
- 230000008569 process Effects 0.000 claims abstract description 17
- 230000006835 compression Effects 0.000 claims description 23
- 238000007906 compression Methods 0.000 claims description 23
- 238000004590 computer program Methods 0.000 claims description 13
- 238000009877 rendering Methods 0.000 claims description 13
- 230000005540 biological transmission Effects 0.000 claims description 12
- 230000002708 enhancing effect Effects 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000003860 storage Methods 0.000 claims description 7
- 238000004422 calculation algorithm Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 230000001960 triggered effect Effects 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 9
- 230000007547 defect Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 230000001133 acceleration Effects 0.000 description 2
- 239000008186 active pharmaceutical agent Substances 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- VYZAMTAEIAYCRO-UHFFFAOYSA-N Chromium Chemical compound [Cr] VYZAMTAEIAYCRO-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/08—Protocols specially adapted for terminal emulation, e.g. Telnet
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/4401—Bootstrapping
- G06F9/4411—Configuring for operating with peripheral devices; Loading of device drivers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
- G06F9/452—Remote windowing, e.g. X-Window System, desktop virtualisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/423—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Signal Processing (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Security & Cryptography (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a method and a system for improving remote operation experience of a cloud computer, which belong to the technical field of cloud computer image coding, and specifically comprise the following steps: the method comprises the steps that a user is connected with a cloud computer, the cloud computer is started, a system is initialized, application in a Guest calls a Guest OS drawing engine to draw pictures, in the process that Application in the Guest calls the Guest OS drawing engine, the Guest OS drawing engine is triggered to call a physical display card driving function, the physical display card driving calls the physical display card capacity to render pictures, drawn picture data are obtained, a screen grabbing module obtains picture change data, an image coding strategy is utilized, the picture change data are directly coded in a video memory, finally the coded data are transmitted to a protocol server, the protocol server transmits the coded data to a client, decoding and displaying are carried out, and through the method, remote operation experience of the user is improved, and meanwhile scheme cost is reduced.
Description
Technical Field
The invention belongs to the technical field of cloud computer image coding, and particularly relates to a method and a system for improving cloud computer remote operation experience.
Background
The cloud desktop is an application of cloud computing, a user accesses a virtual machine ("cloud computer") located at a cloud end through a cloud terminal, client software running on the cloud terminal transmits information such as a keyboard, a mouse, a sound recording and the like to the virtual machine through a network, and the virtual machine transmits the change of a picture and playback information back to the cloud terminal for display and play.
In the process, the processing of the picture data most affects the use experience, so that key optimization is needed, and the data processing process of the current picture is divided into two cases that the virtual machine has no physical display card and the virtual machine has a physical display card. In the case of a physical graphic card, the physical graphic card already allocated for use by the virtual machine cannot be used by the host at the same time due to technical limitations. Therefore, the current processing logic cannot fully exert the encoding and decoding capability of the display card, and the encoding and decoding work still needs to be completed in the host.
If the host CPU code is used, the method not only increases the processing time and affects the user experience, but also increases the load of the host, reduces the number of virtual machines which can be borne by the host, and increases the scheme cost. But if the host is additionally configured with coding hardware, the scheme cost is increased as well.
The invention mainly aims to solve the problems, and fully plays the role of the video card under the scene that the video card exists in the virtual machine, so that the user experience is improved, and coding hardware is not required to be additionally added to a host, thereby reducing the scheme cost.
For example, chinese patent with grant number CN114338550B discloses a method and system for real-time encoding of cloud computer with reduced bandwidth, and a desktop screen capturing module performs desktop screen capturing according to a set maximum frame rate; the network congestion detection module calculates a current network state value according to the received data packet state reply and transmits the current network state value to the frame rate control module; the system rendering surface detection module calculates a system rendering frame rate and a mouse cursor refresh rate; the frame rate control module calculates a target frame rate according to the system rendering frame rate and the mouse cursor refresh rate, and finally calculates an actual setting frame rate to be set in the image control module according to the maximum available frame rate of the network and the smoothing process; the image control module resamples the image data of the screen capturing according to the actual set frame rate, sends the image to the encoder for encoding, and sends the encoded image to the cloud box end. The method uses the dynamic frame rate to effectively reduce the overall bandwidth flow of the cloud computer in use on the premise of ensuring the video image quality and the video fluency.
For example, chinese patent with publication number CN113703904B discloses a method, system and readable storage medium for improving text definition by cloud computer, wherein the method comprises: marking acquisition time, obtaining the output frame rate of the current display card, and extracting the identity identification code of the front-end equipment running program; marking a target key frame in the running program, and extracting transmission data of the target key frame to form a key data group; judging the attribution of the type of the running program so as to adjust the output frame rate of the display card and the output code rate of the encoder according to different types; and retransmitting the key data set to the front-end equipment running program for presentation based on the adjusted output frame rate and the output code rate. According to the invention, the network environment or hardware facilities of the existing cloud computer are not required to be changed, the visual experience of a user is not reduced, the improvement of the definition of characters is realized in a dynamic switching mode, redundant space resources are not occupied in the operation process, and the frame rate and the code rate can be rapidly adapted according to different requirements.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a method and a system for improving the remote operation experience of a cloud computer, a user is connected with the cloud computer, the cloud computer is started, the system is initialized, the Application in the Guest calls a Guest OS drawing engine to draw a picture, the Application in the Guest calls the Guest OS drawing engine, the Guest OS drawing engine is triggered to call a physical display card driving function, the physical display card driving calls the physical display card to render the picture, the drawn picture data is obtained, a screen grabbing module obtains picture change data, the picture change data is directly coded in a display memory by utilizing an image coding strategy, finally, the coded data is transmitted to a protocol server, the coded data is transmitted to a client, and the client is decoded and displayed.
In order to achieve the above purpose, the present invention provides the following technical solutions:
a method for improving cloud computer remote operation experience comprises the following specific steps:
step S1: the user connects the cloud computer, the cloud computer is started, the system is initialized, and Application in the Guest calls Guest OS drawing engine to draw the picture;
step S2: triggering the Guest OS drawing engine to call a physical display card driving function in the process of calling the Guest OS drawing engine by Application in the Guest;
step S3: the physical display card drive calls the physical display card capability to render the picture, and drawn picture data is obtained;
step S4: the screen capturing module acquires picture change data, directly codes in a video memory by utilizing an image coding strategy, and finally transmits the coded data to a protocol server;
step S5: and the protocol server transmits the encoded data to the protocol client and decodes and displays the encoded data.
The decoding calculation formula in step S5 is:
wherein, the method comprises the steps of, wherein,representing the encoded data decoded at coordinates (x, y), the +_>Representing the transformed 3 bytes of data at coordinates (i, j), is +.>And->Representing the transformed orthogonal coefficients.
Specifically, the cloud computer in step S1 includes: cloud resources, transmission protocols and cloud terminals.
Specifically, the screen capturing manner of the screen capturing module in step S4 includes: GDI, DDRAW and DXGI.
Specifically, the specific steps of the image encoding strategy in the step S4 include:
step S401: setting an image of the screen capturing module acquired picture change data as T, and calculating the gray value, the red concentration offset and the blue concentration offset of the image T of the picture change data, wherein the calculation formula is as follows:
，
，
，
wherein Gray represents the Gray value of the image T of the picture variation data, LS represents the blue density shift amount of the image T of the picture variation data, HS represents the red density shift amount of the image T of the picture variation data, G represents the value of the green channel of the image T of the picture variation data, R represents the value of the red channel of the image T of the picture variation data, and B represents the value of the blue channel of the image T of the picture variation data;
step S402: dividing the image into blocks of 8×8 pixels, and if the edge area is less than 8×8 pixels, complementing the edge area into blocks of 8×8 pixels;
step S403: 3 bytes of data stored in each pixel in the 8×8 pixel block are transformed, and the transformation formula is as follows:
，
wherein i represents a horizontal direction coordinate,j represents the vertical direction coordinate,/>，/>representing 3 bytes of data at coordinates (x, y), a +.>Representing the transformed 3 bytes of data at coordinates (i, j), is +.>And->Representing the transformed orthogonal coefficients;
step S404: compressing the data in the transformed 8×8 pixel block, and the calculation formula is:
，
where YS denotes data in a block of 8 x 8 pixels after compression-conversion, t denotes a compression coefficient,representing a rounding function in rounding;
step S405: repeating the steps S403-S404 to complete the compression of the image T of the picture change data;
step S406: the compressed image T is encoded using an encoding algorithm.
Specifically, the 3 bytes of data in step S403 are: gray value, red density shift, and blue density shift of the image T.
Specifically, in the step S403The expression mode is as follows: />，Gray value at coordinates (x, y) in a block representing 8 x 8 pixels,/->Red density offset at coordinates (x, y) in a block representing 8 x 8 pixels,/->The blue density shift amount at coordinates (x, y) in the block representing 8×8 pixels.
Specifically, the encoding algorithm in step S406 includes: VLI encoding and huffman encoding.
A system for enhancing cloud computer remote operation experience, comprising: the system comprises a user protocol client, a cloud computer, a rendering module, a screen capturing module and an image coding module;
the user protocol client is used for connecting a user with the cloud computer;
the cloud computer is used for providing desktop, application and hardware resources to a user in a service mode of on-demand service and elastic allocation by using an open cloud terminal through a transmission protocol;
the rendering module is used for rendering and displaying pictures of the cloud terminal;
the screen grabbing module is used for grabbing images of the picture change data;
the image coding module is used for carrying out compression coding on the image by utilizing an image coding strategy.
Specifically, the user protocol client includes: the protocol client and the decoding display unit,
the protocol client is used for connecting a user with the cloud computer;
the decoding display unit is used for decoding the compression encoded data sent by the cloud computer and displaying the compression encoded data in the form of images in the protocol client.
Specifically, the image coding module comprises an image compression unit and an image coding unit,
the image compression unit is used for compressing the image of the picture change data;
the image encoding unit is used for encoding the image data of the compressed picture change data.
An electronic device comprising a memory and a processor, the memory storing a computer program, the processor implementing steps of a method of enhancing a cloud computer remote operation experience when executing the computer program.
A computer readable storage medium having stored thereon computer instructions which, when executed, perform the steps of a method of enhancing the remote operation experience of a cloud computer.
Compared with the prior art, the invention has the beneficial effects that:
1. the invention provides a system for improving the remote operation experience of a cloud computer, which is optimized and improved in terms of architecture, operation steps and flow, and has the advantages of simple flow, low investment and operation cost and low production and working costs.
2. The invention provides a method for improving remote operation experience of a cloud computer, which is characterized in that a user is connected with the cloud computer, the cloud computer is started, a system is initialized, application in a Guest calls a Guest OS drawing engine to draw a picture, the Guest OS drawing engine is triggered to call a physical display card driving function in the process of calling the Guest OS drawing engine, the physical display card driving function calls a physical display card capacity to render the picture, drawn picture data are obtained, a screen grabbing module obtains picture change data, an image coding strategy is utilized, the picture is directly coded in a display memory, finally, the coded data are transmitted to a protocol server, the protocol server transmits the coded data to a client, decoding and displaying are carried out.
Drawings
FIG. 1 is a flow chart of a method for improving the remote operation experience of a cloud computer;
FIG. 2 is a schematic diagram of components of a method for enhancing the remote operation experience of a cloud computer according to the present invention;
FIG. 3 is a system architecture diagram for improving the remote operation experience of a cloud computer according to the present invention;
fig. 4 is an electronic device diagram of a method for improving remote operation experience of a cloud computer according to the present invention.
Detailed Description
In order that the technical means, the creation characteristics, the achievement of the objects and the effects of the present invention may be easily understood, it should be noted that in the description of the present invention, the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the apparatus or elements to be referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as limiting the present invention. Furthermore, the terms "a", "an", "the" and "the" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. The invention is further described below in conjunction with the detailed description.
Example 1
Referring to fig. 1-2, an embodiment of the present invention is provided: a method for improving cloud computer remote operation experience comprises the following specific steps:
step S1: the user connects the cloud computer, the cloud computer is started, the system is initialized, and Application in the Guest calls Guest OS drawing engine to draw the picture;
step S2: triggering the Guest OS drawing engine to call a physical display card driving function in the process of calling the Guest OS drawing engine by Application in the Guest;
step S3: the physical display card drive calls the physical display card capability to render the picture, and drawn picture data is obtained;
step S4: the screen capturing module acquires picture change data, directly codes in a video memory by utilizing an image coding strategy, and finally transmits the coded data to a protocol server;
step S5: and the protocol server transmits the encoded data to the protocol client and decodes and displays the encoded data.
The prior art has the following processing flow and defects: the data processing process of the current picture is divided into two cases that the virtual machine has no physical display card and the virtual machine has a physical display card. 1) If the virtual machine does not have a physical display card, the processing flow is as follows: 1. application in Guest calls Guest OS drawing engine to draw picture, 2, above-mentioned process triggers Guest OS drawing engine to call the drive function of the virtual display card, 3, the virtual display card drives and can therefore obtain the picture data to the virtual display card apparatus on the host computer, 4, the virtual display card apparatus gives the data to the agreement server through the interface, 5, the agreement server encodes the image data, 6, the agreement server decodes and displays the data after encoding to the customer end, because the virtual machine has no physical display card, so put the encoding work in the host computer is reasonable; 2) If the virtual machine has a physical display card, the processing flow is as follows: 1. the Application in the Guest calls Guest OS drawing engine to draw the picture, 2, the above-mentioned process triggers Guest OS drawing engine to call the physical display card to drive the function, 3, the physical display card drives and calls the ability of the physical display card to render the picture, thus get the picture data, 4, grasp the screen module and obtain the picture change data through some way, and transmit the data to the agreement server, 5, the agreement server encodes the image data, 6, the agreement server transmits the data after encoding to the customer end and decodes and reveals, because the virtual machine has physical display card, so it is not too appropriate to put the encoding work in the host computer.
Defects and deficiencies:
as can be seen from the above flow, in the case of a physical video card, the physical video card already allocated to the virtual machine for use cannot be used by the host at the same time due to technical limitations. Therefore, the current processing logic cannot fully exert the encoding and decoding capability of the display card, and the encoding and decoding work still needs to be completed in the host.
If the host CPU code is used, the method not only increases the processing time and affects the user experience, but also increases the load of the host, reduces the number of virtual machines which can be borne by the host, and increases the scheme cost. But if the host is additionally configured with coding hardware, the scheme cost is increased as well.
The cloud computer in step S1 includes: cloud resources, transmission protocols and cloud terminals.
The screen capturing mode of the screen capturing module in step S4 includes: GDI, DDRAW and DXGI.
GDI screen capture: the method is more general, all windows versions support the screen capturing in the method, and the screen capturing can be completed by only using the API in the user32 without depending on other APIs; however, the screen capturing mode is relatively slow, and the time for capturing a desktop of one frame 1080p is about 5-8 ms. This time is acceptable for simple screen capture, but often requires much image processing after screen capture, so this time is less friendly. And on the system after vista, if Aero special effects are started, the time for capturing one frame is increased to 300-500 ms, and basically the system belongs to an unavailable level.
DDRAW screen capture: the screen capturing speed is faster than that of GDI, and about 4-5 ms is needed for capturing one frame of image, but the method has the defects that the method is same as GDI, and if special effects are started, the speed is very slow and is basically more than 400 ms.
DXGI screen capture: the screen capturing mode is very fast, usually one frame of image can be completed within 2-3 ms, and even if an Aero special effect is started, the screen capturing efficiency is the same. Especially on the system behind windows10, when the desktop is not changed, you cannot grasp any image, and only when the desktop is changed, you can grasp the image, which brings higher screen capturing efficiency and less system overhead.
The specific steps of the image coding strategy in the step S4 comprise:
step S401: setting an image of the screen capturing module acquired picture change data as T, and calculating the gray value, the red concentration offset and the blue concentration offset of the image T of the picture change data, wherein the calculation formula is as follows:
，
，
，
wherein Gray represents the Gray value of the image T of the picture variation data, LS represents the blue density shift amount of the image T of the picture variation data, HS represents the red density shift amount of the image T of the picture variation data, G represents the value of the green channel of the image T of the picture variation data, R represents the value of the red channel of the image T of the picture variation data, and B represents the value of the blue channel of the image T of the picture variation data;
step S402: dividing the image into blocks of 8×8 pixels, and if the edge area is less than 8×8 pixels, complementing the edge area into blocks of 8×8 pixels;
step S403: 3 bytes of data stored in each pixel in the 8×8 pixel block are transformed, and the transformation formula is as follows:
，
wherein i represents a horizontal direction coordinate,j represents the vertical coordinate, ++>，/>Representing 3 bytes of data at coordinates (x, y), a +.>Representing the transformed 3 bytes of data at coordinates (i, j), is +.>And->Representing the transformed orthogonal coefficients;
，/>。
step S404: compressing the data in the transformed 8×8 pixel block, and the calculation formula is:
，
where YS denotes data in a block of 8 x 8 pixels after compression-conversion, t denotes a compression coefficient,representing a rounding function in rounding;
step S405: repeating the steps S403-S404 to complete the compression of the image T of the picture change data;
step S406: the compressed image T is encoded using an encoding algorithm.
The 3 bytes of data in step S403 are: gray value, red density shift, and blue density shift of the image T.
In step S403The expression mode is as follows: />，/>Gray value at coordinates (x, y) in a block representing 8 x 8 pixels,/->Red density offset at coordinates (x, y) in a block representing 8 x 8 pixels,/->Coordinates (x, y) in a block representing 8 x 8 pixelsBlue color concentration shift amount of (c).
The encoding algorithm in step S406 includes: VLI encoding and huffman encoding.
VLI encoding: is a coding mode for representing digits by different digits, and the coding is characterized in that: the smaller the absolute value, the fewer the number of digits occupied, because the bit depth of the coding mode is indefinite, the number of digits to be read must be specified before the VLI code is read, when the number of digits to be read is specified to be 0, the number of digits are not occupied, the binary representation of the positive number is the positive number, and the binary representation of the negative number is the binary representation of the positive number which is inverted by the digits;
huffman coding: huffman coding is a type of variable length coding, i.e., codewords with high probability of occurrence use short codes and codewords with low probability of occurrence use long codes. The idea is to map a frequently occurring fixed length sequence into a shorter binary sequence, and map a fixed length sequence with a lower frequency of occurrence into a longer binary sequence, comprising the following specific steps: 1) The information source information symbols are sequentially arranged according to the occurrence probability; 2) Taking two letters with the smallest probability, respectively matching with 0 and 1 code elements, adding the two probabilities as the probability of a new letter, and re-queuing with the letters of unassigned binary symbols; 3) Repeating the process of the step (2) on the rearranged two least probability symbols; 4) The above process continues until the last two symbols are assigned 0 and 1; 5) Starting from the last stage, the symbol sequence corresponding to each source symbol, namely the corresponding code word, is obtained in a forward return mode. Huffman coding is not unique. The allocation of "0" and "1" symbols to two least probable symbols of the reduced source at a time is arbitrary, so different codewords are available. The separable codewords can be obtained as long as the consistency of symbol allocation is maintained in each reduced source. Different code elements are allocated, the obtained specific code words are different, but the code length Ki is unchanged, and the average code length is also unchanged, so that no essential difference exists; when the information source is reduced, if the probability of the new combined symbol is equal to that of other symbols, the sequences of the symbols can be arranged randomly in terms of the coding method, and the coded codes are correct, but the obtained code words are different; the codeword lengths Ki obtained by different coding methods are also different.
In the Huffman coding process, when the reduced source symbols are rearranged according to the order from big to small probability, the combined new symbols should be arranged at the front position as much as possible, thus the repeated coding times of the combined new symbols can be reduced, and the short codes can be fully utilized.
The decoding calculation formula in step S5 is:
wherein, the method comprises the steps of, wherein,representing the encoded data decoded at coordinates (x, y), the +_>Representing the transformed 3 bytes of data at coordinates (i, j), is +.>And->Representing the transformed orthogonal coefficients.
The deep learning image compression scheme is significantly improved in image fidelity compared to the conventional hybrid image coding scheme due to its efficient nonlinear transformation, end-to-end optimization framework, etc.
For deep learning image compression schemes, several models with different code rates are usually trained to adapt to different bandwidth requirements, which results in different amounts of distortion of the compressed image. In addition, during the training process, even if the same code rate is in different epochs, the distortion amount is continuously changed. Therefore, it is not reasonable to evaluate the perceived loss of different amounts of distortion if the same JNDMap is still used. A higher JND may result in a negligible actual perceptual penalty, thereby reducing the perceived quality of the compressed image. A lower JND will account for distortion that is not perceived by the HVS as a loss of perception, which will result in a waste of bits. To better evaluate the perceptual penalty, JNDMap is adjusted according to the distortion level of different epochs at different QPs.
The method for improving the cloud computer remote operation experience further comprises the following steps: 1) Selecting the nearest server position, selecting the nearest server position can reduce data transmission delay, thereby improving the response speed of the cloud computer, if the response speed is far away, the private network hung by the server can be different, and a certain time is needed for transmitting data between different private networks; 2) The high-speed network connection ensures high-speed and stable network connection between the computer and the cloud server, better performance can be provided by using wired connection, and the wireless connection can have delay and unstable conditions; 3) Selecting a high-quality cloud service provider, selecting a reliable cloud service provider, and providing a stable and high-performance cloud server, which can ensure that the speed and response time of your remote access are better; 4) Selecting a suitable cloud computing tool, selecting a powerful and easy to use cloud computing tool, such as TeamViewer, anyDesk, chrome remote desktop, which provides remote access and remote computer operation functions, is a popular tool; 5) Optimizing remote desktop settings, adjusting remote desktop settings according to your actual needs, e.g., you can choose to adjust resolution and display quality to balance image quality and response speed; 6) Using hardware acceleration, if your cloud computer supports hardware acceleration, the function can be started to optimize operations such as image rendering and video playing; 7) The large file transmission or the network downloading is avoided, and when the remote operation is performed, bandwidth consumption operations such as the large file transmission or the network downloading are avoided as much as possible, so that the situations of network delay and blocking can be reduced; 8) Periodically checking and updating software to ensure that your cloud computing tool and operating system are both the latest versions, which can provide better performance and security; 9) Considering the use of VPN, if your cloud server is located abroad, the use of VPN can help you optimize network connection, reducing delay; 10 Optimizing the local and remote devices, keeping the operating systems, drivers, etc. of the local and remote devices up-to-date, which may improve compatibility and stability; 11 Multiple users are prevented from remotely accessing at the same time, and multiple users are prevented from remotely accessing at the same time as much as possible, which increases the load of the server and network delay.
Example 2
Referring to fig. 3, another embodiment of the present invention is provided: a system for enhancing cloud computer remote operation experience, comprising:
the system comprises a user protocol client, a cloud computer, a rendering module, a screen capturing module and an image coding module;
the user protocol client is used for connecting a user with the cloud computer;
the cloud computer is used for providing desktop, application and hardware resources to a user in a service mode of on-demand service and elastic allocation by using an open cloud terminal through a transmission protocol;
the cloud computer is an overall service scheme and comprises cloud resources, a transmission protocol and a cloud terminal. And providing resources such as desktop, application, hardware and the like for users in a service mode of on-demand service and flexible distribution through a transmission protocol by using the open cloud terminal. The user can realize single machine multi-user without considering the complex IT construction. Compared with the traditional computer, the cloud computer box is internally provided with hardware such as a CPU, a memory, a hard disk and the like, and the hardware is arranged at the cloud, so that the resource sharing of the hardware is realized, and the hardware is distributed as required.
The rendering module is used for rendering and displaying pictures of the cloud terminal;
the screen grabbing module is used for grabbing images of the picture change data;
the image coding module is used for carrying out compression coding on the image by utilizing an image coding strategy.
The user protocol client comprises: the protocol client and the decoding display unit,
the protocol client is used for connecting a user with the cloud computer;
the decoding display unit is used for decoding the compression encoded data sent by the cloud computer and displaying the compression encoded data in the form of images in the protocol client.
The image encoding module includes an image compression unit and an image encoding unit,
the image compression unit is used for compressing the image of the picture change data;
the image encoding unit is used for encoding the image data of the compressed picture change data.
Example 3
Referring to fig. 4, an electronic device includes a memory and a processor, where the memory stores a computer program, and the processor implements steps of a method for improving remote operation experience of a cloud computer when executing the computer program.
A computer readable storage medium having stored thereon computer instructions which, when executed, perform the steps of a method of enhancing the remote operation experience of a cloud computer.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The embodiments of the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present invention and the scope of the claims, which are all within the protection of the present invention.
Claims (9)
1. The method for improving the remote operation experience of the cloud computer is characterized by comprising the following specific steps of:
step S1: the user connects the cloud computer, the cloud computer is started, the system is initialized, and Application in the Guest calls Guest OS drawing engine to draw the picture;
step S2: triggering the Guest OS drawing engine to call a physical display card driving function in the process of calling the Guest OS drawing engine by Application in the Guest;
step S3: the physical display card drive calls the physical display card capability to render the picture, and drawn picture data is obtained;
step S4: the screen capturing module acquires picture change data, directly codes in a video memory by utilizing an image coding strategy, and finally transmits the coded data to a protocol server;
step S5: the protocol server transmits the encoded data to the protocol client and decodes and displays the encoded data;
the specific steps of the image coding strategy in the step S4 include:
step S401: setting an image of the screen capturing module acquired picture change data as T, and calculating the gray value, the red concentration offset and the blue concentration offset of the image T of the picture change data, wherein the calculation formula is as follows:
，
，
，
wherein Gray represents the Gray value of the image T of the picture variation data, LS represents the blue density shift amount of the image T of the picture variation data, HS represents the red density shift amount of the image T of the picture variation data, G represents the value of the green channel of the image T of the picture variation data, R represents the value of the red channel of the image T of the picture variation data, and B represents the value of the blue channel of the image T of the picture variation data;
step S402: dividing the image into blocks of 8×8 pixels, and if the edge area is less than 8×8 pixels, complementing the edge area into blocks of 8×8 pixels;
step S403: 3 bytes of data stored in each pixel in the 8×8 pixel block are transformed, and the transformation formula is as follows:
，
wherein i represents a horizontal direction coordinate,j represents the vertical coordinate, ++>，/>Representing 3 bytes of data at coordinates (x, y), a +.>Representing the transformed 3 bytes of data at coordinates (i, j), is +.>Andrepresenting the transformed orthogonal coefficients;
step S404: compressing the data in the transformed 8×8 pixel block, and the calculation formula is:
，
where YS denotes data in a block of 8 x 8 pixels after compression-conversion, t denotes a compression coefficient,representing a rounding function in rounding;
step S405: repeating the steps S403-S404 to complete the compression of the image T of the picture change data;
step S406: encoding the compressed image T by using an encoding algorithm;
the 3 bytes of data in the step S403 are: gray value, red density offset, and blue density offset of image T;
in said step S403The expression mode is as follows: />，/>Gray value at coordinates (x, y) in a block representing 8 x 8 pixels,/->Red density offset at coordinates (x, y) in a block representing 8 x 8 pixels,/->The blue density shift amount at coordinates (x, y) in the block representing 8×8 pixels.
2. The method of claim 1, wherein the cloud computer in step S1 comprises: cloud resources, transmission protocols and cloud terminals.
3. The method for improving the remote operation experience of the cloud computer according to claim 2, wherein the screen capturing mode of the screen capturing module in step S4 includes: GDI, DDRAW and DXGI.
4. The method of claim 3, wherein the encoding algorithm in step S406 includes: VLI encoding and huffman encoding.
5. A system for enhancing cloud computer remote operation experience, which is implemented based on the method for enhancing cloud computer remote operation experience according to any one of claims 1-4, and is characterized by comprising: the system comprises a user protocol client, a cloud computer, a rendering module, a screen capturing module and an image coding module;
the user protocol client is used for connecting a user with the cloud computer;
the cloud computer is used for providing desktop, application and hardware resources to a user in a service mode of on-demand service and elastic allocation by using an open cloud terminal through a transmission protocol;
the rendering module is used for rendering and displaying pictures of the cloud terminal;
the screen grabbing module is used for grabbing images of the picture change data;
the image coding module is used for carrying out compression coding on the image by utilizing an image coding strategy.
6. The system for enhancing a cloud computing remote operation experience as recited in claim 5, wherein said user protocol client comprises: the protocol client and the decoding display unit,
the protocol client is used for connecting a user with the cloud computer;
the decoding display unit is used for decoding the compression encoded data sent by the cloud computer and displaying the compression encoded data in the form of images in the protocol client.
7. The system for improving the remote operation experience of a cloud computer as recited in claim 6, wherein the image encoding module comprises an image compression unit and an image encoding unit,
the image compression unit is used for compressing the image of the picture change data;
the image encoding unit is used for encoding the image data of the compressed picture change data.
8. An electronic device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of a method of enhancing the remote operation experience of a cloud computer as claimed in any of claims 1-4.
9. A computer readable storage medium having stored thereon computer instructions which when executed perform the steps of a method of enhancing the remote operation experience of a cloud computer as claimed in any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311711069.0A CN117519887B (en) | 2023-12-13 | 2023-12-13 | Method and system for improving cloud computer remote operation experience |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311711069.0A CN117519887B (en) | 2023-12-13 | 2023-12-13 | Method and system for improving cloud computer remote operation experience |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117519887A CN117519887A (en) | 2024-02-06 |
| CN117519887B true CN117519887B (en) | 2024-03-12 |
Family
ID=89745747
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311711069.0A Active CN117519887B (en) | 2023-12-13 | 2023-12-13 | Method and system for improving cloud computer remote operation experience |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117519887B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106534866A (en) * | 2016-11-25 | 2017-03-22 | 中北大学 | Image compression method and apparatus |
| CN107333136A (en) * | 2017-06-26 | 2017-11-07 | 西安万像电子科技有限公司 | Method for encoding images and device |
| CN110198445A (en) * | 2018-02-26 | 2019-09-03 | 浙江宇视科技有限公司 | Image encoding method, device, NVR and readable storage medium storing program for executing |
| CN110647375A (en) * | 2019-09-23 | 2020-01-03 | 天津卓朗科技发展有限公司 | Image rendering method and device of cloud desktop and cloud server |
| CN111552554A (en) * | 2020-05-09 | 2020-08-18 | 中瓴智行(成都)科技有限公司 | Graphic library API agent-based GPU virtualization method, system and medium |
| CN112486609A (en) * | 2020-11-25 | 2021-03-12 | 湖南麒麟信安科技股份有限公司 | Virtual display card implementation method and device based on cloud desktop |
| CN112929624A (en) * | 2021-01-21 | 2021-06-08 | 杭州雾联科技有限公司 | Encoding method, encoding device, electronic equipment and computer readable storage medium |
| CN113067887A (en) * | 2021-04-02 | 2021-07-02 | 东莞市九鼎实业有限公司 | Multi-target application program window information capturing and network pushing method and system |
| CN113574881A (en) * | 2019-03-11 | 2021-10-29 | 佳能株式会社 | Image encoding device, image encoding method, and program |
| CN113891080A (en) * | 2021-12-06 | 2022-01-04 | 麒麟软件有限公司 | Optimization method for image processing in SPICE cloud desktop |
| CN115248714A (en) * | 2022-06-27 | 2022-10-28 | 北京和信创天科技股份有限公司 | Method for improving VDI remote desktop performance by using VGPU |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11496461B2 (en) * | 2021-02-23 | 2022-11-08 | Sophos Limited | Gateway management for a zero trust environment |
-
2023
- 2023-12-13 CN CN202311711069.0A patent/CN117519887B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106534866A (en) * | 2016-11-25 | 2017-03-22 | 中北大学 | Image compression method and apparatus |
| CN107333136A (en) * | 2017-06-26 | 2017-11-07 | 西安万像电子科技有限公司 | Method for encoding images and device |
| CN110198445A (en) * | 2018-02-26 | 2019-09-03 | 浙江宇视科技有限公司 | Image encoding method, device, NVR and readable storage medium storing program for executing |
| CN113574881A (en) * | 2019-03-11 | 2021-10-29 | 佳能株式会社 | Image encoding device, image encoding method, and program |
| CN110647375A (en) * | 2019-09-23 | 2020-01-03 | 天津卓朗科技发展有限公司 | Image rendering method and device of cloud desktop and cloud server |
| CN111552554A (en) * | 2020-05-09 | 2020-08-18 | 中瓴智行(成都)科技有限公司 | Graphic library API agent-based GPU virtualization method, system and medium |
| CN112486609A (en) * | 2020-11-25 | 2021-03-12 | 湖南麒麟信安科技股份有限公司 | Virtual display card implementation method and device based on cloud desktop |
| CN112929624A (en) * | 2021-01-21 | 2021-06-08 | 杭州雾联科技有限公司 | Encoding method, encoding device, electronic equipment and computer readable storage medium |
| CN113067887A (en) * | 2021-04-02 | 2021-07-02 | 东莞市九鼎实业有限公司 | Multi-target application program window information capturing and network pushing method and system |
| CN113891080A (en) * | 2021-12-06 | 2022-01-04 | 麒麟软件有限公司 | Optimization method for image processing in SPICE cloud desktop |
| CN115248714A (en) * | 2022-06-27 | 2022-10-28 | 北京和信创天科技股份有限公司 | Method for improving VDI remote desktop performance by using VGPU |
Non-Patent Citations (4)
| Title |
|---|
| Wang Suhong 等." An encoder–decoder neural model for high-precision UAV positioning via point cloud super resolution and image semantic segmentation".《 Computer Communications》.2022,第197卷第52-60页. * |
| 余启家 ; 熊庭刚 ; 张宇 ; .基于VPU加速的虚拟桌面客户端设计.舰船电子工程.2018,(第08期),第17-21页. * |
| 杨飞."基于SPICE协议的虚拟桌面设计与实现".《中国优秀硕士学位论文全文数据库 (信息科技辑)》.2017,第I138-2156页. * |
| 许琦."基于KVM架构的云桌面GPU虚拟化技术研究".《自动化与仪器仪表》.2023,第3卷第138-142页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117519887A (en) | 2024-02-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113015021B (en) | Cloud game implementation method, device, medium and electronic equipment | |
| CN103688240B (en) | For sending method and the transmitters and receivers scene process equipment of numeral scene description data | |
| CN1856819B (en) | System and method for network transmission of graphical data through a distributed application | |
| JP4698739B2 (en) | Image compression for computer graphics | |
| JP4748219B2 (en) | Remote display processing method based on server / client structure | |
| JP4116006B2 (en) | Screen transfer device, screen transfer system, screen transfer method, and program | |
| CN104660687A (en) | Realization method and system for virtual desktop display | |
| CN101977312A (en) | Video compression encoder | |
| EP3410302B1 (en) | Graphic instruction data processing method, apparatus | |
| CN107665128B (en) | Image processing method, system, server and readable storage medium | |
| CN102156611A (en) | Method and apparatus for creating animation message | |
| CN101802872A (en) | Depth buffer compression | |
| JP5995864B2 (en) | Display data encoding method and system | |
| TW201806386A (en) | Image compression method and apparatus | |
| US20190200030A1 (en) | Split framebuffer encoding | |
| CN110751711B (en) | Remote extension rendering system applied to mobile terminal virtual/augmented reality | |
| CN117519887B (en) | Method and system for improving cloud computer remote operation experience | |
| CN104469400B (en) | Image data compression method based on RFB protocol | |
| US7675525B2 (en) | Deep pixel display and data format | |
| CN106791855B (en) | Method and system for reducing cost of agent management system | |
| CN115225615B (en) | Illusion engine pixel streaming method and device | |
| CN108668170B (en) | Image information processing method and device, and storage medium | |
| CN109413445B (en) | Video transmission method and device | |
| CN116668741A (en) | Cloud desktop display method, image display method, device and storage medium | |
| WO2023050921A1 (en) | Video and audio data sending method, display method, sending end and receiving end |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |