CN114510191A - Screen projection method and device - Google Patents
Screen projection method and device Download PDFInfo
- Publication number
- CN114510191A CN114510191A CN202210143178.6A CN202210143178A CN114510191A CN 114510191 A CN114510191 A CN 114510191A CN 202210143178 A CN202210143178 A CN 202210143178A CN 114510191 A CN114510191 A CN 114510191A
- Authority
- CN
- China
- Prior art keywords
- mouse
- screen
- data
- screen projection
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 230000004044 response Effects 0.000 claims description 34
- 238000009877 rendering Methods 0.000 claims description 31
- 230000001960 triggered effect Effects 0.000 claims description 6
- 238000003672 processing method Methods 0.000 claims 1
- 230000009471 action Effects 0.000 abstract description 7
- 230000005540 biological transmission Effects 0.000 abstract description 6
- 238000005266 casting Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 238000010172 mouse model Methods 0.000 description 3
- 238000004590 computer program Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000011664 signaling Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Controls And Circuits For Display Device (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The application discloses a screen projection method which can be applied to shared equipment. In one example, a sharing device may collect screen stream data at a first frame rate and mouse data at a second frame rate, and then send the screen stream data and the mouse data to a screen projection device, respectively, so that the screen projection device renders a screen projection interface based on the screen stream data and the mouse data. Since the sharing device respectively sends the screen stream data and the mouse data to the screen projection device, the transmission of the mouse data is less affected by the bandwidth, and therefore, the second frame rate can be higher, and correspondingly, when the screen projection device renders a screen projection interface based on the mouse data, the fluency of the mouse action is also higher.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a screen projection method and apparatus.
Background
In the screen projection scene, the user can project the content displayed on the screen of the sharing device to the screen of the screen projection device for display. When the content displayed on the screen of the sharing device is projected to the screen of the screen projecting device for display, the content displayed on the screen of the sharing device can be transmitted to the screen projecting device as a video stream, and the screen projecting device renders the received video stream.
At present, in a screen projection scene, a mouse displayed on a screen of a screen projection device has a problem of unsmooth movement, for example, a phenomenon of jamming occurs.
Therefore, a scheme capable of improving the fluency of the movement of the mouse displayed on the screen of the screen projection device is urgently needed.
Disclosure of Invention
The technical problem that this application will solve is: how to improve the action fluency of a mouse displayed on a screen of the screen projection equipment, and a screen projection method and a device are provided.
In a first aspect, an embodiment of the present application provides a screen projection method, which is applied to a sharing device, and the method includes:
acquiring screen flow data at a first frame rate;
collecting mouse data at a second frame rate;
and respectively sending the screen streaming data and the mouse data to screen projection equipment so that the screen projection equipment renders a screen projection interface based on the screen streaming data and the mouse data.
Optionally, the method further includes:
responding to screen projection operation triggered by a user, and sending a screen projection request to the screen projection equipment;
and receiving indication information sent by the screen projection equipment according to the screen projection request, wherein the indication information is used for indicating that the screen projection equipment supports independent mouse data receiving.
Optionally, the content displayed on the screen of the sharing device is projected to a plurality of screen projection devices for display, and the plurality of screen projection devices all support to independently receive mouse data.
Optionally, the acquiring mouse data includes:
in response to detecting that a characteristic of a mouse on a screen of the sharing device changes, obtaining the mouse data, the characteristic of the mouse including: mouse position and/or mouse style.
Optionally, when the position of the mouse is changed and the model of the mouse is not changed, the characteristics of the mouse include the mouse position and do not include the mouse model.
Optionally, when the position of the mouse is changed and the style of the mouse is changed, the characteristics of the mouse include the mouse position and the mouse style.
Optionally, when the position of the mouse is not changed and the style of the mouse is changed, the characteristics of the mouse include the mouse style and do not include the mouse position.
Optionally, the method further includes:
receiving a first response sent by the screen projection equipment aiming at the received mouse data;
and determining the second frame rate based on sending information and response information, wherein the sending information and the receiving information are used for indicating the time delay of the sharing equipment for sending the mouse data to the screen projection equipment, or the sending information and the receiving information are used for indicating the packet loss condition of the sharing equipment for sending the mouse data to the screen projection equipment.
Alternatively to this, the first and second parts may,
the sending information includes: the sharing device sends a first timestamp of the mouse data to the screen projection device;
the response information includes: the sharing device receives a second timestamp of the first reply.
Alternatively to this, the first and second parts may,
the sending information includes: the number of the mouse data sent by the sharing device to the screen projection device;
the response information includes: and the sharing equipment receives the number of the responses sent by the screen projection equipment.
Optionally, the sharing device supports sending mouse data separately.
In a second aspect, an embodiment of the present application provides a method for processing mouse data based on screen projection, which is applied to a screen projection device, and the method includes:
receiving mouse data sent by a sharing device, wherein the mouse data is mouse data of a mouse displayed on a screen of the sharing device, and content displayed on the screen of the sharing device is projected to the screen of the screen projecting device for displaying;
caching the mouse data;
rendering a mouse on a screen of the screen projection device based on the mouse data read from the cache.
Optionally, the mouse data includes: mouse position and/or mouse style.
Optionally, in a case that the mouse data does not include the mouse style, the method further includes:
obtaining a cached mouse style;
the rendering a mouse on a screen of the screen projection device based on the mouse data read from the cache includes:
rendering a mouse on a screen of the screen projection device based on the mouse data read from the cache and the cached mouse style.
Optionally, in a case that the mouse data does not include the mouse position, the method further includes:
obtaining the cached mouse position;
the rendering a mouse on a screen of the screen projection device based on the mouse data read from the cache includes:
rendering a mouse on a screen of the screen projection device based on the mouse data read from the cache and the cached mouse position.
Optionally, the method further includes:
receiving a screen projection request sent by the sharing equipment;
and sending indication information to the sharing equipment based on the screen projection request, wherein the indication information is used for indicating that the screen projection equipment supports independent mouse data receiving.
In a third aspect, an embodiment of the present application provides a screen projection apparatus, which is applied to a sharing device, and the apparatus includes:
the first acquisition unit is used for acquiring screen flow data at a first frame rate;
the second acquisition unit is used for acquiring the mouse data at a second frame rate;
the first sending unit is used for respectively sending the screen streaming data and the mouse data to a screen projection device so that the screen projection device renders a screen projection interface based on the screen streaming data and the mouse data.
Optionally, the apparatus further comprises:
the second sending unit is used for responding to screen projection operation triggered by a user and sending a screen projection request to the screen projection equipment;
the first receiving unit is used for receiving indication information sent by the screen projection equipment according to the screen projection request, and the indication information is used for indicating that the screen projection equipment supports independent mouse data receiving.
Optionally, the content displayed on the screen of the sharing device is projected to a plurality of projection devices for display, and the plurality of projection devices support independent reception of mouse data.
Optionally, the acquiring mouse data includes:
in response to detecting that a characteristic of a mouse on a screen of the sharing device changes, obtaining the mouse data, the characteristic of the mouse including: mouse position and/or mouse style.
Optionally, when the position of the mouse is changed and the model of the mouse is not changed, the characteristics of the mouse include the mouse position and do not include the mouse model.
Optionally, when the position of the mouse is changed and the style of the mouse is changed, the characteristics of the mouse include the mouse position and the mouse style.
Optionally, when the position of the mouse is not changed and the style of the mouse is changed, the characteristics of the mouse include the mouse style and do not include the mouse position.
Optionally, the apparatus further comprises:
the second receiving unit is used for receiving a first response sent by the screen projection equipment aiming at the received mouse data;
and a determining unit, configured to determine the second frame rate based on sending information and response information, where the sending information and the receiving information are used to indicate a time delay for the sharing device to send the mouse data to the screen projection device, or the sending information and the receiving information are used to indicate a packet loss situation for the sharing device to send the mouse data to the screen projection device.
Alternatively to this, the first and second parts may,
the sending information includes: the sharing device sends a first timestamp of the mouse data to the screen projection device;
the response information includes: the sharing device receives a second timestamp of the first reply.
Alternatively to this, the first and second parts may,
the sending information includes: the number of the mouse data sent by the sharing device to the screen projection device;
the response information includes: and the sharing equipment receives the number of the responses sent by the screen projection equipment.
Optionally, the sharing device supports sending mouse data separately.
In a fourth aspect, an embodiment of the present application provides a mouse data processing apparatus based on screen projection, which is applied to a screen projection device, and the apparatus includes:
the device comprises a first receiving unit, a second receiving unit and a display unit, wherein the first receiving unit is used for receiving mouse data sent by sharing equipment, the mouse data is mouse data of a mouse displayed on a screen of the sharing equipment, and content displayed on the screen of the sharing equipment is projected to the screen of the screen projecting equipment for displaying;
the cache unit is used for caching the mouse data;
and the rendering unit is used for rendering a mouse on the screen of the screen projection equipment based on the mouse data read from the cache.
Optionally, the mouse data includes: mouse position and/or mouse style.
Optionally, in a case that the mouse data does not include the mouse style, the apparatus further includes:
the first acquisition unit is used for acquiring the cached mouse style;
the rendering unit is configured to:
rendering a mouse on a screen of the screen projection device based on the mouse data read from the cache and the cached mouse style.
Optionally, in a case that the mouse data does not include the mouse position, the apparatus further includes:
the second acquisition unit is used for acquiring the cached mouse position;
the rendering unit is configured to:
rendering a mouse on a screen of the screen projection device based on the mouse data read from the cache and the cached mouse position.
Optionally, the apparatus further comprises:
the second receiving unit is used for receiving a screen projection request sent by the sharing equipment;
and the sending unit is used for sending indication information to the sharing equipment based on the screen projection request, wherein the indication information is used for indicating that the screen projection equipment supports independent mouse data receiving.
In a fifth aspect, an embodiment of the present application provides an apparatus, which includes a processor and a memory;
the processor is configured to execute instructions stored in the memory to cause the apparatus to perform the method of any of the above first aspects or to cause the apparatus to perform the method of any of the above second aspects.
In a sixth aspect, embodiments of the present application provide a computer-readable storage medium, including instructions that instruct a device to perform the method of any one of the above first aspects, or instruct a device to perform the method of any one of the above second aspects.
In a seventh aspect, the present application provides a computer program product, which when run on a computer, causes the computer to execute the method of any one of the above first aspects, or causes the computer to execute the method of any one of the above second aspects.
Compared with the prior art, the embodiment of the application has the following advantages:
the embodiment of the application provides a screen projection method which can be applied to shared equipment. In one example, a sharing device may collect screen stream data at a first frame rate and mouse data at a second frame rate, and then send the screen stream data and the mouse data to a screen projection device, respectively, so that the screen projection device renders a screen projection interface based on the screen stream data and the mouse data. Since the sharing device respectively sends the screen stream data and the mouse data to the screen projection device, the transmission of the mouse data is less affected by the bandwidth, and therefore, the second frame rate can be higher, and correspondingly, when the screen projection device renders a screen projection interface based on the mouse data, the fluency of the mouse action is also higher.
The embodiment of the application provides a screen projection method which can be applied to screen projection equipment. In one example, the screen projection device may receive mouse data sent by the sharing device and cache the received mouse data. And then, reading the mouse data from the cache and rendering the mouse based on the mouse data. By adopting the mode, when the screen projection equipment receives a plurality of mouse data sent by the sharing equipment at the same time, the plurality of mouse data can be firstly cached, instead of rendering the mouse based on one of the plurality of mouse data and discarding other mouse data in the plurality of mouse data, by adopting the scheme, the problem that the action of the mouse displayed on the screen of the screen projection equipment is not smooth due to the discarding of the mouse data can be avoided.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a signaling interaction diagram of a screen projection method according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a method for rendering a mouse according to an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of a screen projection apparatus according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of a screen projection device according to an embodiment of the present application.
Detailed Description
In order to make those skilled in the art better understand the technical solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The inventor of the present application has found through research that, in the conventional technology, when content displayed on a screen of a sharing device is projected onto a screen of a screen projecting device for display, the content displayed on the screen of the sharing device can be transmitted to the screen projecting device as a video stream at a specific frame rate, and the screen projecting device renders the received video stream. Wherein the video stream comprises a screen stream and mouse data. When the sharing device sends the video stream to the screen projection device, the specific frame rate is not too high in general due to the network bandwidth, for example, the specific frame rate is 15 frames per second. The reason is that the specific frame rate is not too high, and therefore, when the mouse change frequency on the side of the sharing device is high, the mouse action on the side of the screen projection device is not smooth.
The inventors of the embodiments of the present application have also found that, in some scenarios, the change in screen stream data of the sharing device is small. For example, in a scene taught by a slide show (PPT), the screen stream data of the shared device may stay on a certain page of the PPT for a period of time. For this situation, if the screen stream data and the mouse data are acquired at the same frame rate and the video stream including the screen stream data and the mouse data is synchronously transmitted to the screen projection device, the frame rate for acquiring the screen stream is not very high because the screen stream transmission is greatly affected by the bandwidth, and the frame rate for acquiring the mouse data is not very high because the frame rate for acquiring the mouse data is the same as the frame rate for acquiring the screen stream data.
In addition, since the screen stream data is the same for a period of time in this scenario, sending a video stream including the screen stream and the mouse data to the screen projection device may cause the same screen stream data to be repeatedly sent to the screen projection device, resulting in unnecessary waste of bandwidth.
In view of this, the present application provides a screen projection method and apparatus.
Various non-limiting embodiments of the present application are described in detail below with reference to the accompanying drawings.
Exemplary method
Referring to fig. 1, the signaling interaction diagram of a screen projection method according to an embodiment of the present application is shown. In this embodiment, the method may include, for example, the following steps S101-S105.
With regard to the sharing device and the screen projection device shown in fig. 1, it should be noted that the content displayed on the screen of the sharing device is projected to the screen of the screen projection device for display. The sharing device and the screen projection device are not particularly limited in the embodiments of the present application. In one example, the sharing device and the screen projection device may both be terminal devices. In one example, content displayed on a screen of the sharing device may be projected onto one projection device for display. In yet another example, for example, in a meeting scenario, content that may be displayed on a screen of a sharing device (a device corresponding to a speaker) may be projected onto multiple projection devices (devices corresponding to multiple other meeting participants) for display. The embodiments of the present application are not particularly limited.
S101: the sharing device collects screen flow data at a first frame rate.
S102: the sharing device collects mouse data at a second frame rate.
In this embodiment, the sharing device may acquire the screen stream data at a first frame rate, which is not specifically limited in this embodiment. The screen stream data mentioned here may be, for example, screen stream data of the sharing device. In one example, the sharing device may determine whether the screen stream data is changed at a certain sampling interval, and collect the screen stream data if it is determined that the screen stream data is changed. By adopting the mode, the screen stream data can be prevented from being collected and sent to the screen projection equipment when the screen stream data is not changed, so that repeated screen stream data is sent to the screen projection equipment, and the bandwidth is saved.
In this embodiment of the present application, the sharing device collects mouse data at a second frame rate, which is not specifically limited in this embodiment of the present application. The mouse data mentioned here refers to the mouse data of the shared device.
The first frame rate and the second frame rate are not specifically limited in the embodiments of the present application, and the first frame rate and the second frame rate may be the same or different. It is understood that, in different application scenarios, the first frame rate and/or the second frame rate may be artificially set according to the needs of an actual scenario or adaptively adjusted according to a set frame rate adjustment manner. In one example, for example, in the scenario of the PPT teaching described above, the second frame rate is greater than the first frame rate.
In one example, in the specific implementation of "acquiring mouse data" in S102, the mouse data may be acquired when a feature of a mouse on a screen of the sharing device is changed. By adopting the mode, the repeated mouse data can be prevented from being collected and sent to the screen projection equipment under the condition that the characteristics of the mouse are not changed, so that the bandwidth is saved.
In the embodiment of the present application, the characteristics of the mouse may include a mouse position and/or a mouse style. In one example, where the position of the mouse is changed and the style of the mouse is not changed, the characteristics of the mouse may include the mouse position and not the mouse style. In yet another example, where the position of the mouse changes and the style of the mouse changes, the characteristics of the mouse may include the mouse position and the mouse style. In another example, where the position of the mouse has not changed and the style of the mouse has changed, the characteristics of the mouse include the mouse style and not the mouse position.
Regarding the mouse data, it should be noted that the mouse data refers to mouse-related data displayed on a screen of the sharing device. The mouse data includes a mouse style and/or a mouse position. In one example, the client may obtain the mouse style and mouse position using an Application Programming Interface (API) of its operating system.
In one example, the mouse data relates to characteristics of a mouse. Specifically, the method comprises the following steps:
in one example, the mouse data may include the mouse position and not include the mouse pattern if the characteristic of the mouse includes the mouse position and not include the mouse pattern.
In yet another example, if the characteristics of the mouse include the mouse position and the mouse style. The mouse data may include the mouse position and the mouse style. The mouse style may be, for example, a picture format.
In another example, the mouse data may include the mouse pattern and not include the mouse position if the characteristic of the mouse includes the mouse pattern and not include the mouse position.
S103: and the sharing equipment respectively sends the screen stream data and the mouse data to the screen projection equipment.
In this embodiment of the present application, the sharing device has a capability of sending mouse data separately, that is: the sharing device supports the separate transmission of mouse data. In other words, the sharing device does not necessarily need to acquire screen stream data and mouse data at the same frame rate and transmit a video stream including the screen stream data and the mouse data to the screen projection device, but may acquire the screen stream data and the mouse data at different frame rates, respectively, and transmit the acquired screen stream data and the mouse data to the screen projection device at different frame rates. Just because the sharing device has the capability of sending mouse data separately, the sharing device can send the screen stream data and the mouse data to the screen projection device respectively.
In one example, another condition for performing S103 is: the screen projection equipment has the capability of independently receiving mouse data, namely: the screen projection equipment supports independent reception of mouse data.
In one example, before performing S103, the sharing device may determine whether the screen projection device supports independent mouse data reception, and if it is determined that the screen projection device supports independent mouse data reception, the sharing device may perform S103. In one example, after a user triggers a screen-casting operation, the sharing device may send a screen-casting request to the screen-casting device in response to the user-triggered screen-casting operation. As an example, if the screen projection device supports independent reception of mouse data, after receiving the screen projection request, the screen projection device sends indication information to the sharing device, where the indication information is used to indicate that the screen projection device supports independent reception of mouse data. After the sharing device receives the indication information, the screen projection device can be determined to support independent mouse data reception. The screen projection device mentioned herein supports separate reception of mouse data, meaning that the screen projection device supports separate rendering of a mouse. As one example, a function module of a separate drawing mouse may be included in the screen projection apparatus. The function module of the single drawing mouse can be executed after receiving mouse data, and is not influenced by screen flow data.
Regarding the screen projection operation triggered by the user, the embodiment of the present Application is not particularly limited, and in an example, the user may trigger the screen projection operation by using an Application (APP) installed on the shared device for providing the screen projection service.
As previously described, content displayed on the screen of the sharing device may be projected onto one or more projection devices for display. When the content displayed on the screen of the sharing device is projected to a plurality of screen projecting devices for displaying, the plurality of screen projecting devices support to independently receive mouse data. For this case, the sharing device may send the screen projection request to each screen projection device, and receive the indication information sent by each screen projection device.
S104: and the screen projection equipment receives screen stream data and mouse data sent by the sharing equipment.
S105: and rendering a screen projection interface by the screen projection equipment based on the screen flow data and the mouse data.
S105, in a specific implementation, may render page content in a screen projection interface based on the screen stream data, and render a mouse on a screen of the screen projection device based on the mouse data.
In one example, after the screen projection device receives the mouse data, a mouse may be rendered on a screen of the screen projection device based directly on the mouse data.
As can be seen from the above description, with the solution provided in the embodiments of the present application, the sharing device can send screen stream data and mouse data to the screen projection device, respectively. In this way, in the scenario explained by the PPT, the screen stream data and the mouse data are respectively collected according to different frame rates, and since the screen stream data has a small change, the frame rate for collecting the screen stream data can be reduced, and the frame rate for collecting the mouse data can be properly increased. The frame rate for acquiring the screen stream data is reduced, so that the frequency of sending the same screen stream data to the screen projection equipment can be reduced, and the network bandwidth is saved. Moreover, the frame rate for acquiring the mouse data is increased, so that the frame rate for receiving the mouse data by the screen projection equipment side is correspondingly increased, and the action fluency of the mouse on the screen projection equipment side can be improved.
In one example, as shown in fig. 2, the screen projection apparatus may render mouse data based on steps S201 to S203 as follows
S201: receiving mouse data sent by a sharing device, wherein the mouse data is the mouse data of a mouse displayed on a screen of the sharing device, and the content displayed on the screen of the sharing device is projected to the screen of the screen projecting device for displaying.
In one example, the sharing device may collect mouse data at a second frame rate. After the sharing device collects the mouse data, the mouse data can be sent to the screen projection device, and correspondingly, the screen projection device can receive the mouse data sent by the sharing device.
With regard to a specific implementation that the sharing device collects the mouse data at the second frame rate, reference may be made to the above related description, and a description thereof is not repeated here.
S202: and caching the mouse data.
S203: rendering a mouse on a screen of the screen projection device based on the mouse data read from the cache.
In some scenarios, if the network between the sharing device and the screen projection device is jittered, the screen projection device may receive multiple mouse data simultaneously. Namely: the mouse data sent by the sharing device for multiple times are received by the screen projection device at the same time. For this situation, if the screen projection device renders a mouse based on one of the plurality of mouse data and discards other mouse data of the plurality of mouse data, the mouse action on the screen of the screen projection device may be not smooth. In this case, the mouse on the screen of the projection device is naturally and smoothly moved. In one example, the screen projection device may cache a plurality of mouse data received. For example, the screen projection device may cache the plurality of mouse data in a first cache region. For another example, the screen projection device may cache the plurality of mouse data in a certain cache queue. When rendering the mouse, the cached mouse data may be sequentially read from the cache, and the mouse may be rendered based on the read mouse data. For example, the screen projection device may read mouse data from the cache at a certain frame rate, and render a mouse based on the read mouse data.
In one example, when a mouse is rendered on a screen of a screen projection device based on mouse data read from a cache, if the mouse data includes a mouse position but does not include a mouse pattern, the screen projection device may obtain the cached mouse pattern, and further, render the mouse on the screen of the screen projection device based on the mouse data read from the cache (i.e., the mouse position) and the cached mouse pattern. The cached mouse style referred to herein refers to the mouse style that was last used by the screen-casting device to render the mouse. In other words, the last time the screen-casting device rendered the mouse, the mouse style used to render the mouse may be cached in the cache (e.g., in the second cache region).
In one example, when a mouse is rendered on a screen of a screen projection device based on cached mouse data, if the mouse data includes a mouse pattern but does not include a mouse position, the screen projection device may obtain the cached mouse position, and further, render the mouse on the screen of the screen projection device based on the mouse data (i.e., the mouse pattern) read from the cache and the cached mouse position. Reference herein to a cached mouse location refers to the mouse location that was last used by the screen-casting device to render the mouse. In other words, the last time the screen-casting device rendered the mouse, the mouse location used to render the mouse may be cached in the cache (e.g., in the third cache region).
In one example, if the mouse position currently used by the screen projection device to render the mouse is obtained from a cache (e.g., the first cache region), it indicates that the position of the mouse has changed compared to the mouse rendered last time, and for this case, the screen projection device may replace the mouse position stored in the third cache region with the mouse position currently used to render the mouse, so that when the mouse is rendered next time, if the mouse position has not changed but the mouse style has changed, the mouse may be rendered based on the mouse position in the third cache region.
In addition, in an example, if the mouse style currently used by the screen projection device to render the mouse is obtained from a cache (e.g., the first cache region), it indicates that the mouse style has changed compared to the mouse rendered last time, and for this case, the screen projection device may replace the mouse style stored in the second cache region with the mouse style currently used to render the mouse, so that when the mouse is rendered next time, if the mouse style has not changed but the mouse position has changed, the mouse may be rendered based on the mouse style in the second cache region.
In one example, after the sharing device sends the mouse data to the screen projection device, the screen projection device receives the mouse data, and may send a response to the mouse data to the sharing device. Further, the sharing device may determine the aforementioned second frame rate based on the transmission information and the response information. Wherein:
the sending information and the receiving information are used for indicating the time delay of the sharing equipment for sending the mouse data to the screen projection equipment, or the sending information and the receiving information are used for indicating the packet loss condition of the sharing equipment for sending the mouse data to the screen projection equipment.
In one example, when the sending information and the receiving information are used to indicate a time delay for the sharing device to send mouse data to the screen projection device, the sending information includes: and the sharing equipment sends the first time stamp of the mouse data to the screen projection equipment. The response information includes: the sharing device receives a second timestamp of the first reply.
As an example, the sharing device may record the first timestamp when sending mouse data to the screen-casting device. Accordingly, the sharing device may record a second timestamp when receiving the first response sent by the screen-casting device. Based on the second time stamp and the first time stamp, the time delay of the sharing device for sending the mouse data to the screen projection device can be calculated. For example, the time delay is 0.5 × (second time stamp — first time stamp). After calculating the time delay, the sharing device may determine the aforementioned second frame rate based on the time delay. For example, the second frame rate may be the inverse of the latency, or the second frame rate may be slightly less than the inverse of the latency.
In one example, when the sending information and the receiving information are used for indicating a packet loss situation that a sharing device sends mouse data to the screen projection device, the sending information includes: the number of the mouse data sent by the sharing device to the screen projection device; the response information includes: and the sharing equipment receives the number of the responses sent by the screen projection equipment.
In one example, the sharing device may adjust the second frame rate during interaction with the screen projection device, wherein the initial value of the second frame rate may be a default value, for example. Specifically, the method comprises the following steps:
the sharing device may send mouse data to the screen projection device at a second frame rate, and then adjust the second frame rate based on the number of the mouse data sent by the sharing device to the screen projection device and the number of responses received by the sharing device from the screen projection device. For example, if the number of the mouse data sent by the sharing device to the screen projection device is greater than the number of the responses received by the sharing device from the screen projection device, it indicates that the mouse data sent by the sharing device to the screen projection device is lost, and the sharing device may decrease the second frame rate. For another example, if the number of the mouse data sent by the sharing device to the screen projection device is equal to the number of the responses received by the sharing device from the screen projection device, it indicates that no packet is lost in the mouse data sent by the sharing device to the screen projection device, and the sharing device may increase the second frame rate or keep the second frame rate unchanged.
Exemplary device
Based on the method provided by the above embodiment, the embodiment of the present application further provides an apparatus, which is described below with reference to the accompanying drawings.
Referring to fig. 3, the figure is a schematic structural diagram of a screen projection device provided in an embodiment of the present application. The apparatus 300 may be applied to a sharing device, and the apparatus 300 may specifically include: a first acquisition unit 301, a second acquisition unit 302 and a first transmission unit 303.
A first collecting unit 301, configured to collect screen stream data at a first frame rate;
a second collecting unit 302, configured to collect mouse data at a second frame rate;
a first sending unit 303, configured to send the screen stream data and the mouse data to a screen projection device, respectively, so that the screen projection device renders a screen projection interface based on the screen stream data and the mouse data.
Optionally, the apparatus further comprises:
the second sending unit is used for responding to screen projection operation triggered by a user and sending a screen projection request to the screen projection equipment;
the first receiving unit is used for receiving indication information sent by the screen projection equipment according to the screen projection request, and the indication information is used for indicating that the screen projection equipment supports independent mouse data receiving.
Optionally, the content displayed on the screen of the sharing device is projected to a plurality of screen projection devices for display, and the plurality of screen projection devices all support to independently receive mouse data.
Optionally, the acquiring mouse data includes:
in response to detecting that a characteristic of a mouse on a screen of the sharing device changes, obtaining the mouse data, the characteristic of the mouse including: mouse position and/or mouse style.
Optionally, when the position of the mouse is changed and the model of the mouse is not changed, the characteristics of the mouse include the mouse position and do not include the mouse model.
Optionally, when the position of the mouse is changed and the style of the mouse is changed, the characteristics of the mouse include the mouse position and the mouse style.
Optionally, when the position of the mouse is not changed and the style of the mouse is changed, the characteristics of the mouse include the mouse style and do not include the mouse position.
Optionally, the apparatus further comprises:
the second receiving unit is used for receiving a first response sent by the screen projection equipment aiming at the received mouse data;
and a determining unit, configured to determine the second frame rate based on sending information and response information, where the sending information and the receiving information are used to indicate a time delay for the sharing device to send the mouse data to the screen projection device, or the sending information and the receiving information are used to indicate a packet loss situation for the sharing device to send the mouse data to the screen projection device.
Alternatively to this, the first and second parts may,
the sending information includes: the sharing device sends a first timestamp of the mouse data to the screen projection device;
the response information includes: the sharing device receives a second timestamp of the first reply.
Alternatively to this, the first and second parts may,
the sending information includes: the number of the mouse data sent by the sharing device to the screen projection device;
the response information includes: and the sharing equipment receives the number of the responses sent by the screen projection equipment.
Optionally, the sharing device supports sending mouse data separately.
Since the apparatus 300 is a device corresponding to the method provided in the above method embodiment, and the specific implementation of each unit of the apparatus 300 is the same as that of the above method embodiment, for the specific implementation of each unit of the apparatus 300, reference may be made to the description part of the steps executed by the shared device in the above method embodiment, and details are not described here.
Referring to fig. 4, the figure is a schematic structural diagram of a screen projection device provided in an embodiment of the present application. The apparatus 400 may be applied to a screen projection device, and the apparatus 400 may specifically include: a first receiving unit 401, a buffering unit 402, and a rendering unit 403.
A first receiving unit 401, configured to receive mouse data sent by a sharing device, where the mouse data is mouse data of a mouse displayed on a screen of the sharing device, and content displayed on the screen of the sharing device is projected onto the screen of the screen projecting device for display;
a cache unit 402, configured to cache the mouse data;
a rendering unit 403, configured to render a mouse on a screen of the screen projection device based on the mouse data read from the cache.
Optionally, the mouse data includes: mouse position and/or mouse style.
Optionally, in a case that the mouse data does not include the mouse style, the apparatus further includes:
the first acquisition unit is used for acquiring the cached mouse style;
the rendering unit 403 is configured to:
rendering a mouse on a screen of the screen projection device based on the mouse data read from the cache and the cached mouse style.
Optionally, in a case that the mouse data does not include the mouse position, the apparatus further includes:
the second acquisition unit is used for acquiring the cached mouse position;
the rendering unit 403 is configured to:
rendering a mouse on a screen of the screen projection device based on the mouse data read from the cache and the cached mouse position.
Optionally, the apparatus further comprises:
the second receiving unit is used for receiving a screen projection request sent by the sharing equipment;
and the sending unit is used for sending indication information to the sharing equipment based on the screen projection request, wherein the indication information is used for indicating that the screen projection equipment supports independent mouse data receiving.
Since the apparatus 400 is a device corresponding to the method provided in the above method embodiment, and the specific implementation of each unit of the apparatus 400 is the same as that of the above method embodiment, for the specific implementation of each unit of the apparatus 400, reference may be made to the description part of the steps executed by the screen projection device in the above method embodiment, and details are not repeated here.
An embodiment of the present application provides an apparatus, which includes a processor and a memory;
the processor is configured to execute instructions stored in the memory to cause the apparatus to perform a method performed by a sharing device or a screen projection device as described in any of the above method embodiments.
The embodiment of the application provides a computer-readable storage medium, which comprises instructions for instructing a device to execute the method of any one of the above method embodiments, wherein the method is executed by a sharing device or a screen projection device.
The embodiment of the application provides a computer program product, which when running on a computer, causes the computer to execute the method performed by any one of the above method embodiments by a sharing device or a screen projection device.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice in the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.
The above description is only exemplary of the present application and should not be taken as limiting the present application, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (20)
1. A screen projection method is applied to a sharing device, and comprises the following steps:
collecting screen flow data at a first frame rate;
collecting mouse data at a second frame rate;
and respectively sending the screen streaming data and the mouse data to screen projection equipment so that the screen projection equipment renders a screen projection interface based on the screen streaming data and the mouse data.
2. The method of claim 1, further comprising:
responding to screen projection operation triggered by a user, and sending a screen projection request to the screen projection equipment;
and receiving indication information sent by the screen projection equipment according to the screen projection request, wherein the indication information is used for indicating that the screen projection equipment supports independent mouse data receiving.
3. The method of claim 2, wherein content displayed on the screen of the sharing device is projected onto a plurality of projection devices, each of the plurality of projection devices supporting separate reception of mouse data.
4. The method of claim 1, wherein the collecting mouse data comprises:
in response to detecting that a characteristic of a mouse on a screen of the sharing device changes, obtaining the mouse data, the characteristic of the mouse including: mouse position and/or mouse style.
5. The method of claim 4, wherein in the case that the position of the mouse is changed and the style of the mouse is not changed, the characteristic of the mouse comprises the mouse position and does not comprise the mouse style.
6. The method of claim 4, wherein the characteristics of the mouse include the mouse position and the mouse style if the mouse position changes and the mouse style changes.
7. The method of claim 4, wherein in the case that the position of the mouse is not changed and the style of the mouse is changed, the characteristics of the mouse include the mouse style and do not include the mouse position.
8. The method of claim 1, further comprising:
receiving a first response sent by the screen projection equipment aiming at the received mouse data;
and determining the second frame rate based on sending information and response information, wherein the sending information and the receiving information are used for indicating the time delay of the sharing equipment for sending the mouse data to the screen projection equipment, or the sending information and the receiving information are used for indicating the packet loss condition of the sharing equipment for sending the mouse data to the screen projection equipment.
9. The method of claim 8,
the sending information includes: the sharing equipment sends a first time stamp of the mouse data to the screen projection equipment;
the response information includes: the sharing device receives a second timestamp of the first reply.
10. The method of claim 8,
the sending information includes: the number of the mouse data sent by the sharing device to the screen projection device;
the response information includes: and the sharing equipment receives the number of the responses sent by the screen projection equipment.
11. The method according to any of claims 1-10, wherein the sharing device supports sending mouse data separately.
12. A mouse data processing method based on screen projection is characterized by being applied to screen projection equipment and comprising the following steps:
receiving mouse data sent by a sharing device, wherein the mouse data is mouse data of a mouse displayed on a screen of the sharing device, and content displayed on the screen of the sharing device is projected to the screen of the screen projecting device for displaying;
caching the mouse data;
rendering a mouse on a screen of the screen projection device based on the mouse data read from the cache.
13. The method of claim 12, wherein the mouse data comprises: mouse position and/or mouse style.
14. The method of claim 13, wherein in the case that the mouse data does not include the mouse pattern, the method further comprises:
obtaining a cached mouse style;
the rendering a mouse on a screen of the screen projection device based on the mouse data read from the cache includes:
rendering a mouse on a screen of the screen projection device based on the mouse data read from the cache and the cached mouse style.
15. The method of claim 13, wherein in the case that the mouse data does not include the mouse location, the method further comprises:
obtaining the cached mouse position;
the rendering a mouse on a screen of the screen projection device based on the mouse data read from the cache includes:
rendering a mouse on a screen of the screen projection device based on the mouse data read from the cache and the cached mouse position.
16. The method of claim 12, further comprising:
receiving a screen projection request sent by the sharing equipment;
and sending indication information to the sharing equipment based on the screen projection request, wherein the indication information is used for indicating that the screen projection equipment supports independent mouse data receiving.
17. A screen projection device is applied to shared equipment, and the screen projection device comprises:
the first acquisition unit is used for acquiring screen flow data at a first frame rate;
the second acquisition unit is used for acquiring the mouse data at a second frame rate;
the first sending unit is used for respectively sending the screen streaming data and the mouse data to a screen projection device so that the screen projection device renders a screen projection interface based on the screen streaming data and the mouse data.
18. A mouse data processing device based on screen projection is characterized in that the device is applied to screen projection equipment and comprises:
the device comprises a first receiving unit, a second receiving unit and a display unit, wherein the first receiving unit is used for receiving mouse data sent by sharing equipment, the mouse data is mouse data of a mouse displayed on a screen of the sharing equipment, and content displayed on the screen of the sharing equipment is projected to the screen of the screen projecting equipment for displaying;
the cache unit is used for caching the mouse data;
and the rendering unit is used for rendering a mouse on the screen of the screen projection equipment based on the mouse data read from the cache.
19. An apparatus, comprising a processor and a memory;
the processor is to execute instructions stored in the memory to cause the device to perform the method of any of claims 1 to 16.
20. A computer-readable storage medium comprising instructions that direct a device to perform the method of any of claims 1-16.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210143178.6A CN114510191A (en) | 2022-02-16 | 2022-02-16 | Screen projection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210143178.6A CN114510191A (en) | 2022-02-16 | 2022-02-16 | Screen projection method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114510191A true CN114510191A (en) | 2022-05-17 |
Family
ID=81552376
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210143178.6A Pending CN114510191A (en) | 2022-02-16 | 2022-02-16 | Screen projection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114510191A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115037978A (en) * | 2022-07-13 | 2022-09-09 | 北京字跳网络技术有限公司 | Screen projection method and related equipment |
| CN115052186A (en) * | 2022-07-12 | 2022-09-13 | 北京字跳网络技术有限公司 | Screen projection method and related equipment |
| CN115209117A (en) * | 2022-07-20 | 2022-10-18 | 北京字跳网络技术有限公司 | Screen projection method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106155606A (en) * | 2015-04-07 | 2016-11-23 | ***通信集团公司 | A kind of multi-screen interaction method and device |
| CN106375841A (en) * | 2015-07-23 | 2017-02-01 | 阿里巴巴集团控股有限公司 | Wireless screen projection data processing method and device, video data display method and device, and electronic device |
| CN107317655A (en) * | 2017-06-06 | 2017-11-03 | 努比亚技术有限公司 | Transfer control method, system and the readable storage medium storing program for executing of screen prjection |
| CN110519632A (en) * | 2019-07-30 | 2019-11-29 | 华为技术有限公司 | Throw screen method and apparatus |
| CN111541919A (en) * | 2020-05-13 | 2020-08-14 | 北京百度网讯科技有限公司 | Video frame transmission method and device, electronic equipment and readable storage medium |
-
2022
- 2022-02-16 CN CN202210143178.6A patent/CN114510191A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106155606A (en) * | 2015-04-07 | 2016-11-23 | ***通信集团公司 | A kind of multi-screen interaction method and device |
| CN106375841A (en) * | 2015-07-23 | 2017-02-01 | 阿里巴巴集团控股有限公司 | Wireless screen projection data processing method and device, video data display method and device, and electronic device |
| CN107317655A (en) * | 2017-06-06 | 2017-11-03 | 努比亚技术有限公司 | Transfer control method, system and the readable storage medium storing program for executing of screen prjection |
| CN110519632A (en) * | 2019-07-30 | 2019-11-29 | 华为技术有限公司 | Throw screen method and apparatus |
| CN111541919A (en) * | 2020-05-13 | 2020-08-14 | 北京百度网讯科技有限公司 | Video frame transmission method and device, electronic equipment and readable storage medium |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115052186A (en) * | 2022-07-12 | 2022-09-13 | 北京字跳网络技术有限公司 | Screen projection method and related equipment |
| CN115052186B (en) * | 2022-07-12 | 2023-09-15 | 北京字跳网络技术有限公司 | Screen projection method and related equipment |
| CN115037978A (en) * | 2022-07-13 | 2022-09-09 | 北京字跳网络技术有限公司 | Screen projection method and related equipment |
| CN115037978B (en) * | 2022-07-13 | 2023-08-25 | 北京字跳网络技术有限公司 | Screen projection method and related equipment |
| CN115209117A (en) * | 2022-07-20 | 2022-10-18 | 北京字跳网络技术有限公司 | Screen projection method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114510191A (en) | Screen projection method and device | |
| JP7110234B2 (en) | Video playback methods, devices and systems | |
| US10231014B2 (en) | Virtual reality (VR) video distribution using edge resources | |
| CN109618565B (en) | Method and system for automatically measuring experience quality of streaming video user | |
| US20150271242A1 (en) | Method and Apparatus for Acquiring Streaming Media Buffer Parameter | |
| WO2008108379A1 (en) | Medium distribution system, distribution server device, medium distribution method used for them, and program thereof | |
| US9813742B2 (en) | Method, device and system for evaluating user experience value of video quality | |
| JP2001094625A (en) | Data communication unit, data communication method and storage medium | |
| US20140344392A1 (en) | Content delivery system, cache server, and content delivery method | |
| CN113068001B (en) | Data processing method, device, equipment and medium based on cascade camera | |
| CN113747194A (en) | Remote video transmission method, transmission device, storage medium and electronic equipment | |
| CN109089131A (en) | A kind of record screen live broadcasting method, device, equipment and storage medium based on IOS system | |
| JP2014182793A (en) | Encoder, video processing server, video processing system, encoding method, and program therefor | |
| US10924782B2 (en) | Method of providing streaming service based on image segmentation and electronic device supporting the same | |
| KR101668540B1 (en) | Distribution network and server, and distribution method | |
| EP3843411A1 (en) | Video frame-based media stream bandwidth reduction | |
| EP3843415A1 (en) | Video image-based media stream bandwidth reduction | |
| CN113038136A (en) | Video self-adaptive adjustment method and device | |
| CN115174846A (en) | Method, system, terminal and medium for improving video conference quality in weak network environment | |
| CN110113641B (en) | Video data transmission method, device, edge service node and medium | |
| CN113766315A (en) | Display device and video information processing method | |
| JP2017204700A (en) | Image reproduction apparatus, image reproduction method, and image reproduction program | |
| KR101795958B1 (en) | Adaptive control method, apparatus and user device for providing video in real time network cameras | |
| CN115278308B (en) | Media stream processing method, device, equipment and storage medium | |
| CN113489745B (en) | Video data transmission method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |