CN111078104B - Redirection method of multi-point touch equipment on cloud desktop and computer equipment - Google Patents
Redirection method of multi-point touch equipment on cloud desktop and computer equipment Download PDFInfo
- Publication number
- CN111078104B CN111078104B CN201911000040.5A CN201911000040A CN111078104B CN 111078104 B CN111078104 B CN 111078104B CN 201911000040 A CN201911000040 A CN 201911000040A CN 111078104 B CN111078104 B CN 111078104B
- Authority
- CN
- China
- Prior art keywords
- virtual machine
- touch
- touch screen
- virtual
- point touch
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
- G06F9/452—Remote windowing, e.g. X-Window System, desktop virtualisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention provides a redirection method of multi-point touch equipment on a cloud desktop, wherein a virtual machine receives multi-point touch data, an operating system of a local terminal and touch screen information; translating the multi-point touch data into an execution instruction according to the operating system, touch screen information, virtual machine hardware information and the type of the operating system of the local terminal; the virtual machine executes corresponding operation according to the execution instruction; the invention also provides the computer equipment, so that a user can use multi-point touch control on the cloud desktop.
Description
Technical Field
The invention relates to a redirection method of multi-point touch equipment on a cloud desktop and computer equipment.
Background
The cloud desktop is an application of cloud computing technology, simply, a virtualization software is installed in a server in a data center machine room through a virtualization technology, and then a plurality of virtual machines are created through configuration of the software, so that an actual server can be divided into a plurality of virtual machines to be used independently by a plurality of users, different users can use different virtual machines, the CPU and the memory resources cannot be preempted mutually, and the virtual machines cannot conflict mutually due to operation in the same machine. In the application scene of the cloud desktop, in order to meet the use habit of most users, a Windows operating system is generally installed in a virtual machine, and the users use desktop services of the virtual machine, similar to the normal use of a computer, to perform operations of browsing webpages, editing documents and the like.
The cloud desktop protocol is a core technology in the cloud desktop technology, and can transmit display contents in a virtual machine to a local terminal (including a computer, a mobile phone and a smart television) for display, and transmit local operations (keyboard operations and mouse operations) to the virtual machine for execution, so that an effect of remotely controlling the virtual machine is achieved. The user can normally browse the web page and edit the document on the virtual machine.
As shown in fig. 2, since the cloud desktop technology is implemented by using a virtualization technology, most virtual machines created by using the virtualization technology currently only support basic hardware, including cpu, memory, disk, and the like, and only have a mouse and a keyboard in terms of user input, and no multi-touch device. Therefore, when a user connects a cloud desktop service using a local terminal supporting multi-touch, such as a mobile phone and a smart television, the multi-touch function cannot be realized. The multi-touch can only be converted into single-point touch, and converted into mouse again for operation. Therefore, when the software which can be operated only by multi-point touch is used on the cloud desktop, the software cannot be used.
The name of the comparison file is: an RDP protocol-based mobile cloud desktop accurate touch method, application number 201310026044.7 and application date 2013.01.24; a virtual cursor moving module and a touch event response module are established between the RDP protocol functional data layer and the interface display layer, so that a touch screen local to the terminal equipment is simulated to be used as a touch pad; intercepting all touch operations of a current touch screen on a cloud desktop window, and then indicating a current clicking position by using a virtual cursor, wherein the movement and the position of the virtual cursor are determined by the current touch operations; when a finger moves on the touch screen, the virtual cursor is moved relatively, so that the virtual cursor always keeps the same offset as the touch track to move, and a touch event message of the current virtual cursor position is sent to the remote cloud desktop; therefore, the function of simulating the touch pad by the touch screen is realized to position the virtual cursor, and then accurate touch control is realized through the virtual cursor. According to the invention, the touch screen is adopted as a touch pad application method, so that accurate touch control is realized after the cloud desktop is connected under the RDP protocol; the comparison file does not create virtual multi-point touch screen equipment in the virtual machine, the multi-point touch screen equipment cannot be supported, only a virtual cursor can be created in the comparison file for operation of the single-point touch screen equipment, the multi-point touch screen equipment (such as a touch screen mobile phone) does not display the cursor, and only a single-point operation mode can display the cursor (such as a mouse), so that the comparison file only performs some optimization operations between an RDP protocol functional data layer and an interface layer on the terminal equipment, the operation precision of the touch screen is improved, and the whole scheme is operated on the terminal equipment.
Disclosure of Invention
The invention aims to solve the technical problem of providing a redirection method of multi-point touch equipment on a cloud desktop and computer equipment, so that a user can use multi-point touch on the cloud desktop.
One of the present invention is realized as follows: a redirection method of multi-touch equipment on a cloud desktop comprises the following steps:
step 1, a virtual machine receives multi-point touch data, an operating system of a local terminal and touch screen information;
step 2, translating the multi-point touch data into an execution instruction according to the operating system, touch screen information, virtual machine hardware information and the type of the operating system of the local terminal;
and step 3, the virtual machine executes corresponding operation according to the execution instruction.
Further, the step 1 is further specifically: and generating a virtual multi-touch device, which is used for transmitting the execution instruction to a device bus of the virtual machine, and receiving multi-touch data, an operating system of the local terminal and touch screen information by the virtual machine through the virtual multi-touch device.
Further, the step 2 is further specifically:
step 21, establishing a terminal equipment touch screen data control block according to the operating system types and touch screen information of different terminal equipment;
step 22, establishing a virtual machine touch screen data control block according to the HID descriptor of the created virtual device in the virtual machine;
step 23, establishing a one-to-one correspondence between the touch screen data control blocks of the terminal equipment and the virtual machine touch screen data control blocks;
step 24, translate the multi-touch data into the execution instruction according to the established one-to-one correspondence.
The second invention is realized in the following way: a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of:
step 1, a virtual machine receives multi-point touch data, an operating system of a local terminal and touch screen information;
step 2, translating the multi-point touch data into an execution instruction according to the operating system, touch screen information, virtual machine hardware information and the type of the operating system of the local terminal;
and step 3, the virtual machine executes corresponding operation according to the execution instruction.
Further, the step 1 is further specifically: and generating a virtual multi-touch device, which is used for transmitting the execution instruction to a device bus of the virtual machine, and receiving multi-touch data, an operating system of the local terminal and touch screen information by the virtual machine through the virtual multi-touch device.
Further, the step 2 is further specifically:
step 21, establishing a terminal equipment touch screen data control block according to the operating system types and touch screen information of different terminal equipment;
step 22, establishing a virtual machine touch screen data control block according to the HID descriptor of the created virtual device in the virtual machine;
step 23, establishing a one-to-one correspondence between the touch screen data control blocks of the terminal equipment and the virtual machine touch screen data control blocks;
step 24, translate the multi-touch data into the execution instruction according to the established one-to-one correspondence.
The invention has the following advantages: virtual multi-touch equipment is created in a virtual machine of a cloud desktop, meanwhile, multi-touch information is captured on a local terminal and transmitted to the virtual machine, the multi-touch information is translated into the multi-touch information which can be identified by the virtual machine, and then the virtual machine is operated through the virtual multi-touch equipment, so that a user can use the multi-touch equipment on the cloud desktop.
Drawings
The invention will be further described with reference to examples of embodiments with reference to the accompanying drawings.
FIG. 1 is a flow chart of the method of the present invention.
Fig. 2 is a schematic diagram of a cloud desktop technology in the prior art.
Fig. 3 is a block diagram of an embodiment of the present invention.
Fig. 4 is a flow chart of an embodiment of the present invention.
FIG. 5 is a flow chart of translation of an embodiment of the present invention.
Detailed Description
As shown in fig. 1, the redirection method of the multi-touch device on the cloud desktop of the present invention includes:
step 1, generating a virtual multi-point touch device, which is used for transmitting an execution instruction to a device bus of a virtual machine, wherein the virtual machine receives multi-point touch data, an operating system of a local terminal and touch screen information through the virtual multi-point touch device;
step 2, establishing a terminal equipment touch screen data control block according to the operating system types and touch screen information of different terminal equipment; establishing a virtual machine touch screen data control block according to the HID descriptor of the created virtual device in the virtual machine; establishing a one-to-one correspondence between a terminal equipment touch screen data control block and a virtual machine touch screen data control block; translating the multi-point touch data into execution instructions according to the established one-to-one correspondence;
and step 3, the virtual machine executes corresponding operation according to the execution instruction.
The computer device of the invention comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the following steps when executing the program:
step 1, generating a virtual multi-point touch device, which is used for transmitting an execution instruction to a device bus of a virtual machine, wherein the virtual machine receives multi-point touch data, an operating system of a local terminal and touch screen information through the virtual multi-point touch device;
step 2, establishing a terminal equipment touch screen data control block according to the operating system types and touch screen information of different terminal equipment; establishing a virtual machine touch screen data control block according to the HID descriptor of the created virtual device in the virtual machine; establishing a one-to-one correspondence between a terminal equipment touch screen data control block and a virtual machine touch screen data control block; translating the multi-point touch data into execution instructions according to the established one-to-one correspondence;
and step 3, the virtual machine executes corresponding operation according to the execution instruction.
The invention is a specific embodiment:
as shown in fig. 3, a virtual multi-touch device is first created on a virtual machine, and meanwhile, a multi-touch message is captured on a local terminal, and the locally captured multi-touch message is transmitted to the virtual machine through a network and translated into a multi-touch message that can be identified by the virtual machine, and then the virtual machine is operated through the virtual multi-touch device, so that the virtual machine can support multi-touch operation. The virtual machine is respectively provided with virtual multi-point touch equipment, a virtual multi-point touch translation module and a network communication module. The local terminal is respectively provided with a multi-point touch control capturing module and a network communication module.
The description of each module is as follows:
1) As shown in fig. 4, in the multi-touch capturing module on the local terminal, a local terminal operating system (e.g.: windows, android, IOS) the multi-touch interface provided obtains the multi-touch message. These messages will vary depending on the operating system of the local terminal and the hardware of the touch screen. The multi-point touch capturing module needs to send the information of the system and the information of the touch screen through the network communication module firstly during initialization, then starts to circularly capture the touch points, and immediately sends the captured multi-point touch information to the network communication module; the information of the touch screen comprises the information of the touch screen, wherein the information comprises the number of fingers supported by touch, the length and the width of the touch screen and the sensitivity degree of pressure sensing of the touch screen.
2) The network communication module of the local terminal is only a basic network communication module, and is used for establishing connection with the network communication module of the virtual machine, receiving the message of the multi-touch capturing module and sending the message to the virtual machine.
3) The network communication module of the virtual machine is also just a basic network communication module, and is used for establishing connection with the network communication module of the local terminal, receiving the message of the multi-touch capturing module sent by the local terminal, and sending the message to the virtual multi-touch translating module.
4) As shown in fig. 5, the translation module of the virtual multi-touch system analyzes the multi-touch message according to the type of the local terminal operating system after receiving the message, translates the analyzed multi-touch message according to the type of the virtual machine operating system, generates an instruction that can be identified by the virtual machine, and sends the instruction to the virtual multi-touch device for execution.
The specific translation method comprises the following steps:
1) Firstly, the touch screen data control blocks of the terminal equipment are established according to the operating system types (such as android system, IOS system and Windows system) of different terminal equipment. These data blocks contain a lot of information. Including but not limited to:
finger unique identifier
Whether or not the finger is in contact
X, Y coordinates of finger
Finger size
2) The touch screen data control block of the virtual machine is established according to the HID descriptor of the created virtual device in the virtual machine (according to the definition of the HID device standard of USB). These data blocks contain a lot of information. Including but not limited to:
finger unique identifier
Whether or not the finger is in contact
X, Y coordinates of finger
Finger size
3) The information of the data blocks is in one-to-one correspondence, such as a length unit like a coordinate or a size, and a ratio of the length to the screen can be installed to establish a conversion relation.
4) The translation is performed according to the suggested correspondence.
For example, when the Windows operating system is installed in the virtual machine, the translated command may include information about one or more fingers, and each finger includes information that the virtual machine can only recognize the multi-touch command.
Unique identifier of finger
Whether the finger is in contact
X-axis coordinates of finger
Y-axis coordinates of finger
Width of finger
Finger height
After having this translation module, the multi-touch device can be compatible after instruction translation no matter how different the hardware and the operating system of the local terminal and the virtual machine are.
5) The virtual multi-touch device is generated by software simulation and is only responsible for delivering instructions to the device bus of the virtual machine. Thus, the interaction between the multi-touch control and the virtual machine is realized, the device is needed to be added, and an operating system in some virtual machines sometimes needs to recognize that the device of one type is needed to start the multi-touch control function, such as handwriting input and the like.
While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that the specific embodiments described are illustrative only and not intended to limit the scope of the invention, and that equivalent modifications and variations of the invention in light of the spirit of the invention will be covered by the claims of the present invention.
Claims (2)
1. A redirection method of multi-point touch equipment on a cloud desktop is characterized by comprising the following steps of: comprising the following steps:
step 1, generating a virtual multi-point touch device, which is used for transmitting an execution instruction to a device bus of a virtual machine, wherein the virtual machine receives multi-point touch data, an operating system of a local terminal and touch screen information through the virtual multi-point touch device;
step 2, establishing a terminal equipment touch screen data control block according to the operating system types and touch screen information of different terminal equipment; establishing a virtual machine touch screen data control block according to the HID descriptor of the created virtual device in the virtual machine; establishing a one-to-one correspondence between a terminal equipment touch screen data control block and a virtual machine touch screen data control block; translating the multi-point touch data into execution instructions according to the established one-to-one correspondence;
and step 3, the virtual machine executes corresponding operation according to the execution instruction.
2. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized by: the processor, when executing the program, performs the steps of:
step 1, generating a virtual multi-point touch device, which is used for transmitting an execution instruction to a device bus of a virtual machine, wherein the virtual machine receives multi-point touch data, an operating system of a local terminal and touch screen information through the virtual multi-point touch device;
step 2, establishing a terminal equipment touch screen data control block according to the operating system types and touch screen information of different terminal equipment; establishing a virtual machine touch screen data control block according to the HID descriptor of the created virtual device in the virtual machine; establishing a one-to-one correspondence between a terminal equipment touch screen data control block and a virtual machine touch screen data control block; translating the multi-point touch data into execution instructions according to the established one-to-one correspondence;
and step 3, the virtual machine executes corresponding operation according to the execution instruction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911000040.5A CN111078104B (en) | 2019-10-21 | 2019-10-21 | Redirection method of multi-point touch equipment on cloud desktop and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911000040.5A CN111078104B (en) | 2019-10-21 | 2019-10-21 | Redirection method of multi-point touch equipment on cloud desktop and computer equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111078104A CN111078104A (en) | 2020-04-28 |
| CN111078104B true CN111078104B (en) | 2023-08-08 |
Family
ID=70310474
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911000040.5A Active CN111078104B (en) | 2019-10-21 | 2019-10-21 | Redirection method of multi-point touch equipment on cloud desktop and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111078104B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112156450A (en) * | 2020-08-20 | 2021-01-01 | 福建升腾资讯有限公司 | Game handle redirection method and system applied to cloud desktop |
| CN112394872B (en) * | 2020-11-30 | 2022-01-11 | 北京麟卓信息科技有限公司 | Method and device for optimizing android system multi-touch supported by Linux platform |
| CN112394871B (en) * | 2020-11-30 | 2021-11-02 | 北京麟卓信息科技有限公司 | Method and device for supporting android system multi-point touch by Linux platform |
| CN112202825A (en) * | 2020-12-09 | 2021-01-08 | 江苏北弓智能科技有限公司 | Remote mobile cloud desktop transmission system and method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104932820A (en) * | 2015-06-01 | 2015-09-23 | 深圳市深信服电子科技有限公司 | Touch screen use method and system based on USB mapping |
| CN106354546A (en) * | 2016-08-29 | 2017-01-25 | 北京奇虎科技有限公司 | Virtual machine remote control method and system |
| CN107392002A (en) * | 2016-05-17 | 2017-11-24 | 中兴通讯股份有限公司 | A kind of method and device for the dummy machine system for logging in cloud desktop |
| CN107844243A (en) * | 2017-11-09 | 2018-03-27 | 新华三云计算技术有限公司 | Cloud tabletop touch implementation method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104216726B (en) * | 2013-06-04 | 2016-08-10 | 腾讯科技（深圳）有限公司 | Android virtual machine loading method and device |
-
2019
- 2019-10-21 CN CN201911000040.5A patent/CN111078104B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104932820A (en) * | 2015-06-01 | 2015-09-23 | 深圳市深信服电子科技有限公司 | Touch screen use method and system based on USB mapping |
| CN107392002A (en) * | 2016-05-17 | 2017-11-24 | 中兴通讯股份有限公司 | A kind of method and device for the dummy machine system for logging in cloud desktop |
| CN106354546A (en) * | 2016-08-29 | 2017-01-25 | 北京奇虎科技有限公司 | Virtual machine remote control method and system |
| CN107844243A (en) * | 2017-11-09 | 2018-03-27 | 新华三云计算技术有限公司 | Cloud tabletop touch implementation method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111078104A (en) | 2020-04-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111078104B (en) | Redirection method of multi-point touch equipment on cloud desktop and computer equipment | |
| CN102567074B (en) | USB (universal serial bus) device redirecting method facing virtual machines | |
| CN104823186B (en) | The method of long-range presentation between at least two terminals of network connection | |
| CN103049190B (en) | A kind of method of mobile communication terminal and control appliance operation | |
| JP2013175242A (en) | Method system and software for providing image sensor based human machine interfacing | |
| JP5458708B2 (en) | Image processing apparatus, display control method, and display control program | |
| CN103631556A (en) | Method, mobile terminal and system for remote control over display device | |
| KR20120046807A (en) | Method and apparatus for providing virtual platform | |
| JP2016164769A (en) | Method for implementing multi-screen by distributing plural screens and sharing input interface between user apparatuses, and user apparatuses for setting up as multi-screen | |
| CN112231017B (en) | Virtual keyboard and mouse method and device compatible with Android running environment running on Linux | |
| US20210326151A1 (en) | Methods, devices and computer-readable storage media for processing a hosted application | |
| CN109635216B (en) | Method for annotating pictures of mobile terminal and browser on same screen | |
| CN113032273B (en) | Application program debugging method and device, computer equipment and storage medium | |
| CN112221127B (en) | Multi-point control method and device for cloud game | |
| US20150244799A1 (en) | Display system, display apparatus, display method, and computer program | |
| CN113778583A (en) | Method, device, equipment and medium for publishing local application of cloud desktop | |
| WO2023029526A1 (en) | Display control method and apparatus for pointer in window, device, and storage medium | |
| CN103970279A (en) | Control method and system for cloud game | |
| CN105262880A (en) | Mobile phone control demonstration system applied to WiFi network | |
| CN113391782A (en) | Method, system, electronic device and storage medium for controlling mobile terminal by computer | |
| CN103412704A (en) | Optimization schemes for controlling user interfaces through gesture or touch | |
| US9823826B2 (en) | User interface module sharing | |
| CN108667902B (en) | Remote control system, method, device and equipment of iOS equipment | |
| CN112156450A (en) | Game handle redirection method and system applied to cloud desktop | |
| KR20040108028A (en) | Remote login method for mobile communication terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |