WO2021227688A1 - 屏幕扩展方法、装置、终端设备及计算机可读存储介质 - Google Patents

屏幕扩展方法、装置、终端设备及计算机可读存储介质 Download PDF

Info

Publication number
WO2021227688A1
WO2021227688A1 PCT/CN2021/084205 CN2021084205W WO2021227688A1 WO 2021227688 A1 WO2021227688 A1 WO 2021227688A1 CN 2021084205 W CN2021084205 W CN 2021084205W WO 2021227688 A1 WO2021227688 A1 WO 2021227688A1
Authority
WO
WIPO (PCT)
Prior art keywords
screen
terminal device
display
data
display data
Prior art date
Application number
PCT/CN2021/084205
Other languages
English (en)
French (fr)
Inventor
何书杰
罗朴良
李斌飞
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2021227688A1 publication Critical patent/WO2021227688A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1423—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display
    • G06F3/1446—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display display composed of modules, e.g. video walls
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/80—Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication

Definitions

  • This application belongs to the field of image processing, and in particular relates to a screen expansion method, device, terminal device, and computer-readable storage medium.
  • the screen expansion technology allows users to use multiple displays at the same time for collaborative office, or for entertainment, games, etc., which can greatly enhance the user experience.
  • the main screen device sends the display information that the expansion screen needs to display to the terminal device where the expansion screen is located.
  • the terminal device where the extended screen is located receives the display information, it needs to perform rendering and synthesis operations on the displayed information, and it is difficult to quickly display the displayed information on the screen, causing the extended screen to easily refresh and freeze, and the screen picture displayed on the extended screen Compared with the screen picture displayed on the main screen, there is a larger time delay.
  • the embodiments of this application provide a screen expansion method, device, terminal device, and computer-readable storage medium, which can solve the current expansion screen solution, when the expansion screen is displaying the screen, it is easy to refresh and freeze, and the expansion screen Compared with the screen displayed on the main screen, the displayed screen has a larger time delay.
  • the first aspect of the embodiments of the present application provides a screen expansion method, including:
  • the first terminal device obtains graphic data of each application program, renders the graphic data of each application program, and synthesizes the rendered graphic data corresponding to each application program into screen display data;
  • the first terminal device displays the first screen display data on the display screen of the first terminal device, and sends the second screen display data to the second terminal device to instruct the second terminal
  • the device displays the second screen display data on the display screen of the second terminal device.
  • the first terminal device after obtaining the graphic data of each application, the first terminal device renders the graphic data of each application to obtain the window data corresponding to each application, and synthesize the window data corresponding to each application into a screen Display Data.
  • the first terminal device splits the screen display data into the first screen display data and the second screen display data.
  • the first terminal device can directly display the first screen display data on the first display screen (that is, the display screen of the first terminal device) And send the second screen display data to the second terminal device.
  • the second terminal device can directly display the second screen display data on the second display screen (ie, the display screen of the second terminal device) without Performing the operations of rendering and compositing improves the screen upload speed of the second screen display data, reduces the occurrence of screen refresh jams, and reduces the time delay of the screen picture displayed on the second display screen.
  • the method before the first terminal device obtains the graphic data of each application program, the method further includes:
  • the first terminal device When the first terminal device detects a screen event that meets a preset trigger condition, the first terminal device determines the terminal device corresponding to the screen event as the second terminal device.
  • the first terminal device detects a screen event that meets the preset trigger condition, the first terminal device enters the extended screen mode, the terminal device corresponding to the screen event is determined to be the second terminal device, and the first terminal device The display screen of the second terminal device is determined to be the second display screen.
  • the first terminal device when the first terminal device detects a screen event that meets a preset trigger condition, the first terminal device determines the terminal device corresponding to the screen event as the first Two terminal equipment, including:
  • the first terminal device determines the terminal device corresponding to the screen movement event as a candidate terminal device;
  • the first terminal device determines the candidate terminal device as the second terminal device.
  • the above-mentioned screen event may be a screen movement event.
  • the terminal device corresponding to the screen movement event is determined as a candidate terminal device.
  • the foregoing preset trigger condition may be that the distance between the display screen of the candidate terminal device and the first display screen is less than the preset distance threshold.
  • the first terminal device determines the candidate terminal device meeting the preset trigger condition as the second terminal device, and determines the display screen of the second terminal device as the second display screen.
  • the first terminal device when the first terminal device detects a screen event that meets a preset trigger condition, the first terminal device determines the terminal device corresponding to the screen event as
  • the second terminal device includes:
  • the first terminal device When the first terminal device detects a screen touch event, the first terminal device determines the terminal device corresponding to the screen touch event as the second terminal device.
  • the above-mentioned screen event may also be a screen touch event.
  • the first terminal device may determine the terminal device corresponding to the screen touch event as the second terminal device, and determine the display screen of the second terminal device as the second terminal device. Display screen.
  • the method before the first terminal device obtains the graphic data of each application program, the method further includes:
  • the first terminal device When the first terminal device detects a near field communication NFC signal, and the signal strength of the NFC signal is greater than or equal to the preset strength threshold, the first terminal device determines the terminal device corresponding to the NFC signal as the second terminal equipment.
  • the trigger condition for the first terminal device to enter the extended screen mode may also be that the first terminal device detects a Near Field Communication (NFC) signal, and the signal strength of the NFC signal is greater than or equal to the preset strength Threshold.
  • NFC Near Field Communication
  • the first terminal device may determine the terminal device that generates the NFC signal as the second terminal device, and determine the display screen of the second terminal device It is the second display screen.
  • the method before the first terminal device obtains the graphic data of each application program, the method further includes:
  • the first terminal device determines the terminal device indicated in the screen expansion instruction as the second terminal device.
  • the trigger condition for the first terminal device to enter the extended screen mode may also be that the first terminal device receives a screen extension instruction.
  • the screen expansion instruction includes a screen expansion object, and the screen expansion object may be an identifier of the second terminal device.
  • the first terminal device may determine the terminal device indicated in the screen expansion instruction as the second terminal device, and determine the display screen of the second terminal device as the second display screen.
  • the rendering by the first terminal device of the graphics data of each application includes:
  • the first terminal device renders the graphics data of each application program according to the target display resolution to obtain window data corresponding to each application program.
  • the first terminal device can virtualize the first display resolution of the first display screen and the second display resolution of the second display screen to the target display resolution, and the graphics of each application program can be determined according to the target display resolution. Data is rendered to reduce the difficulty of rendering graphics data.
  • acquiring the target display resolution by the first terminal device includes:
  • the first terminal device obtains the first display resolution corresponding to the first terminal device and the second display resolution corresponding to the second terminal device, and distinguishes the first display resolution from the second display resolution
  • the maximum value in the rate is determined as the target display resolution.
  • the first terminal device determines the target display resolution, it can obtain the first display resolution corresponding to the first terminal device and the second display resolution corresponding to the second terminal device, and take the first display resolution and the second display resolution. The larger value of the resolution is used as the target display resolution.
  • the first terminal device uses the target display resolution to render the graphics data of each application to avoid the problem of picture distortion or picture tearing.
  • the splitting of the screen display data into the first screen display data and the second screen display data by the first terminal device includes:
  • the first terminal device obtains the sum of the number of display screens of the first terminal device and the number of display screens of the second terminal device, and divides the screen display data into equal parts according to the sum of the numbers to obtain The first screen displays data and the second screen displays data.
  • the first terminal device when the first terminal device splits the screen to display data, it may first obtain the total number of the first display screen and the second display screen, and split according to the total number. For example, if there are 1 first display screen and 2 second display screens, the total number is 3, and the screen display data is divided into 3 equal parts to obtain 1 copy of the first screen display data and 2 copies of the second screen display data.
  • the first terminal device displays the first screen display data on the display screen of the first terminal device, and sends the second screen display data To the second terminal device to instruct the second terminal device to display the second screen display data on the display screen of the second terminal device, including:
  • the first terminal device displays the first screen display data on the display screen of the first terminal device, and sends the second screen display data to the second terminal device to instruct the second terminal
  • the device does not render the second screen display data, and displays the second screen display data on the display screen of the second terminal device.
  • the second terminal device since the second screen display data is data that has undergone rendering and synthesis operations, the second terminal device does not need to render the second screen display data, and can directly perform the on-screen operation on the second screen display data.
  • the second screen display data is displayed on the second display screen.
  • the second aspect of the embodiments of the present application provides a screen extension device, including:
  • the rendering and synthesis module is used to obtain the graphic data of each application program, render the graphic data of each application program, and synthesize the rendered graphic data corresponding to each application program into screen display data;
  • a data splitting module for splitting the screen display data into first screen display data and second screen display data
  • the screen expansion module is configured to display the first screen display data on the display screen of the first terminal device, and send the second screen display data to the second terminal device to instruct the second terminal
  • the device displays the second screen display data on the display screen of the second terminal device.
  • the device further includes:
  • the event monitoring module is configured to determine the terminal device corresponding to the screen event as the second terminal device when a screen event meeting a preset trigger condition is monitored.
  • the event monitoring module includes:
  • the candidate movement sub-module is used to determine the terminal device corresponding to the screen movement event as a candidate terminal device when a screen movement event is monitored;
  • the distance determination sub-module is configured to determine the candidate terminal device as the second terminal device if the distance between the display screen of the candidate terminal device and the display screen of the first terminal device is less than a preset distance threshold.
  • the event monitoring module includes:
  • the collision determination sub-module is configured to determine the terminal device corresponding to the screen touch event as the second terminal device when the screen touch event is monitored.
  • the device further includes:
  • the near field communication module is configured to determine the terminal device corresponding to the NFC signal as the second terminal device when the first terminal device detects the NFC signal and the signal strength of the NFC signal is greater than or equal to the preset strength threshold.
  • the device further includes:
  • the instruction determination module is configured to determine the terminal device indicated in the screen expansion instruction as the second terminal device when the screen expansion instruction is acquired.
  • the rendering synthesis module includes:
  • the resolution sub-module is used to obtain the target display resolution
  • the graphics rendering sub-module is used to render the graphics data of each application program according to the target display resolution.
  • the resolution submodule is specifically configured to obtain a first display resolution corresponding to the first terminal device and a second display resolution corresponding to the second terminal device, The maximum value of the first display resolution and the second display resolution is determined as the target display resolution.
  • the data splitting module is specifically configured to obtain the sum of the number of display screens of the first terminal device and the number of display screens of the second terminal device, according to The sum of the numbers divides the screen display data into equal parts to obtain the first screen display data and the second screen display data.
  • the screen expansion module is specifically configured to display the first screen display data on the display screen of the first terminal device, and display the second screen
  • the display data is sent to the second terminal device to instruct the second terminal device not to render the second screen display data, and to display the second screen display data on the display screen of the second terminal device.
  • the third aspect of the embodiments of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and running on the processor.
  • a terminal device including a memory, a processor, and a computer program stored in the memory and running on the processor.
  • the terminal device implements the steps of the above method.
  • the fourth aspect of the embodiments of the present application provides a computer-readable storage medium that stores a computer program, and when the computer program is executed by a processor, the terminal device realizes the steps of the above-mentioned method.
  • the fifth aspect of the embodiments of the present application provides a computer program product, which when the computer program product runs on a terminal device, enables the terminal device to implement the steps of the above-mentioned method.
  • the first terminal device renders and synthesizes the image data of each application program to obtain screen display data that can be directly displayed on the screen. Then, the first terminal device splits the screen display data into first screen display data and second screen display data, displays the first screen display data on the display screen of the first terminal device, and displays the second screen display data Send to the second terminal device. Since the second screen display data is data that has been rendered and synthesized, when the second terminal device receives the second screen display data, it can directly perform the on-screen operation on the second screen display data without the need to display the data on the second screen Re-rendering and compositing operations reduce the occurrence of screen refresh jams and reduce the time delay of the screen displayed on the second display screen.
  • FIG. 1 is a schematic flowchart of a screen expansion method provided by an embodiment of the present application
  • FIG. 2 is a schematic diagram of a screen movement event provided by an embodiment of the present application.
  • FIG. 3 is a schematic diagram of a screen touch event provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of cross-screen display of an application program provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a screen display data splitting method provided by an embodiment of the present application.
  • Fig. 6 is a schematic diagram of a local area network provided by an embodiment of the present application.
  • FIG. 7 is a system architecture diagram of an Android operating system provided by an embodiment of the present application.
  • Fig. 8 is a schematic diagram of a producer-consumer model provided by an embodiment of the present application.
  • FIG. 9 is a system architecture diagram of another Android operating system provided by an embodiment of the present application.
  • Fig. 10 is a schematic diagram of another producer-consumer model provided by an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of a screen extension device provided by an embodiment of the present application.
  • Fig. 12 is a schematic diagram of a terminal device provided by an embodiment of the present application.
  • the term “if” can be interpreted as “when” or “once” or “in response to determination” or “in response to detection” depending on the context .
  • the phrase “if determined” or “if detected [described condition or event]” can be interpreted as meaning “once determined” or “in response to determination” or “once detected [described condition or event]” depending on the context ]” or “in response to detection of [condition or event described]”.
  • the screen extension method provided in the embodiments of the present application can be applied to a terminal device.
  • the terminal device can be any device with a display screen and image processing functions, including but not limited to smart phones, tablets, wearable devices, vehicle-mounted devices, augmented reality (AR)/virtual reality (VR) ) Devices, laptops, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDAs), artificial intelligence (AI) terminals and other terminal devices.
  • the embodiments of the present application do not impose any restrictions on the specific types of terminal devices.
  • the terminal device may be a station (STAION, ST) in a WLAN, a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a wireless local loop (Wireless Local Loop, WLL) station, Personal Digital Assistant (PDA) devices, handheld devices with wireless communication functions, computing devices or other processing devices connected to wireless modems, in-vehicle devices, car networking terminals, computers, laptop computers, handheld communication devices , Handheld computing devices, satellite wireless devices, customer premise equipment (CPE) and/or other devices used to communicate on wireless systems, and next-generation communication systems, such as mobile terminals in 5G networks or the future Mobile terminals in the evolved Public Land Mobile Network (PLMN) network.
  • STAION, ST station
  • WLAN Wireless Local Loop
  • PDA Personal Digital Assistant
  • handheld devices with wireless communication functions computing devices or other processing devices connected to wireless modems
  • computing devices or other processing devices connected to wireless modems in-vehicle devices
  • car networking terminals computers, laptop computers
  • the screen expansion technology allows users to use multiple displays at the same time for collaborative office, or for entertainment, games, etc., which can greatly enhance the user experience.
  • multi-window technology can be implemented in one screen, such as split-screen mode, picture-in-picture mode, and FreeFrom mode.
  • the core idea of multi-window technology is to divide the stack and set the stack boundary.
  • the realization of the multi-window function mainly depends on the activity management service (ActivityManagerService, AMS) and the window management service (WindowManagerService, WMS).
  • AMS mainly manages parameters such as the life cycle of windows
  • WMS mainly manages parameters such as the location and size of windows.
  • the multi-window technology is based on the data structure of the stack.
  • One stack contains multiple tasks, and one task contains multiple window activities.
  • the activity stack (ActivityStack) in AMS corresponds to the task stack (TaskStack) in WMS
  • the activity record (ActivityRecord) in AMS corresponds to the Task in WMS.
  • the above-mentioned multi-window mode can be realized by setting the stack ID and the stack boundary in different multi-window modes.
  • this stack is the home stack (Home Stack), which is the stack where the Android system's desktop launcher (Launcher) is located, except for the desktop launcher , And some system interfaces also run on this stack, such as recent tasks, etc.;
  • Stack ID 2, this stack is where the Activity in free mode is located;
  • Different Activities may belong to different stacks, but they may have the same stack ID.
  • the size of each task in the stack can be adjusted by adjusting the boundary of the above stack, and the size of the final task determines the size of the window.
  • the boundary of the stack can only be the current screen range, the above-mentioned multi-window technology solves the problem of cutting the screen of a single terminal device into multiple windows, such as cutting the screen into two screens, three screens, and four screens. Split screen, etc., but cannot achieve multi-screen expansion.
  • the above-mentioned multi-window technology has poor functional scalability and can only be used in applications that support split screens.
  • some current screen projection technologies can realize multi-screen interaction, such as push mode and mirror mode.
  • the mobile phone and the TV are connected to the same local area network, and the mobile phone and the TV transmit data through the projection protocol (the channel of the projection protocol is similar to the dedicated transmission channel of Bluetooth).
  • the mobile phone After the user clicks the screen projection button, the mobile phone searches the local area network for devices that can apply the screen projection broadcast service.
  • the mobile phone finds the TV, the mobile phone will send a video address to the TV. After the TV receives the address, it will access the video resources in the address to play.
  • the mirroring mode you can project the current screen display on the terminal device to the screen of other terminal devices, that is, display on the same screen.
  • files, desktops, game screens, software applications, and other content displayed on the current terminal device can be projected onto the screen of other terminal devices.
  • the functions of screen mirroring, multi-screen interaction, and mirroring switch in various mobile phones currently belong to the mirroring mode.
  • the principle of mirroring mode is to record the screen of the current terminal device and send it to other terminal devices.
  • the recording speed can usually reach more than 60 frames per second.
  • the mirroring mode is realized. For example, if the mobile phone and the TV are in the same local area network, after the mobile phone searches for the TV, the mirror mode is turned on, and the mobile phone will continuously take screenshots (ie, record the screen). While the mobile phone takes a screenshot at a specific frequency, it sends the screenshot image to the TV, and the TV displays it after receiving it, so that the TV can be used to view the screen of the mobile phone.
  • the push mode and the mirror mode can use the projection protocol for data transmission.
  • the projection protocol includes the Digital Living Network Alliance (DLNA) protocol, the airplay push protocol, and the lelink protocol.
  • DLNA Digital Living Network Alliance
  • the push mode and mirroring mode can expand the display of the screen to a certain extent.
  • the push mode and the mirroring mode are not screen expansions in the true sense, and it is difficult for users to operate the extended screen for collaborative office or entertainment.
  • the terminal device where the expansion screen is located needs to perform rendering and synthesis on the display information of the expansion screen It is difficult to quickly go to the screen due to the operation of, which causes the screen to be refreshed and freezes, which makes the screen displayed on the extended screen have a larger time delay compared with the screen displayed on the main screen.
  • the embodiment of the present application provides a screen expansion method.
  • the first terminal device uses the second display screen of the second terminal device as the expansion screen, the graphics data of each application program is rendered and synthesized to obtain The data can be displayed directly on the screen.
  • the first terminal device splits the screen display data into the first screen display data and the second screen display data, performs an on-screen operation on the first screen display data, and displays the first screen display data on the first display screen, And, sending the second screen display data to the second terminal device.
  • the second screen display data is data that has been rendered and synthesized
  • the second terminal device when the second terminal device receives the second screen display data, it can directly perform the on-screen operation on the second screen display data without the need to display the data on the second screen Re-run rendering and compositing operations to reduce screen refresh jams, thereby reducing the time delay of the screen displayed on the second display, and solving the current extended screen solution, when the extended screen is displaying the screen, the screen will refresh In the case of a freeze, the screen picture displayed on the extended screen has a larger time delay compared with the screen picture displayed on the main screen.
  • FIG. 1 Please refer to the flowchart of the screen expansion method shown in FIG. 1, which includes:
  • the first terminal device obtains graphic data of each application program, renders the graphic data of each application program, obtains window data corresponding to each application program, and synthesizes the window data corresponding to each application program into a screen display data;
  • the first terminal device is the terminal device where the main display screen (that is, the first display screen) is located
  • the second terminal device is the terminal device where the expansion screen (that is, the second display screen) is located.
  • the first terminal device and the second terminal device may perform data transmission through wired communication and/or wireless communication.
  • the first terminal device and the second terminal device may be connected through a wired communication method such as an HDMI cable connection, an RJ45 interface connection, a wifi connection, and/or a wireless communication method.
  • the number of second terminal devices is one or more than one, and the specific number is determined according to actual conditions.
  • the user can set a trigger condition for turning on the extended screen mode in the first terminal device.
  • the first terminal device can detect whether there is a communication connection between other terminal devices and the first terminal device. If no other terminal device is in communication connection with the first terminal device, the first terminal device cannot start the extended screen mode.
  • the first terminal device may determine whether the application program corresponding to the screen picture currently displayed on the first display screen supports the extended screen mode.
  • the first terminal device turns on the extended screen mode, selects the second terminal device from the terminal devices communicatively connected with the first terminal device, and sets the second terminal device in communication with the first terminal device.
  • the second display screen of the terminal device is used as an extended screen.
  • the first terminal device prompts the user that there is an application that does not support the extended screen mode in the screen displayed on the current display screen.
  • the first terminal device monitors the screen events of each terminal device in the local area network, and when a screen event that meets the preset trigger condition is monitored, the terminal device corresponding to the screen event that meets the preset trigger condition is used as the second terminal device, Use the display screen of the second terminal device as the second display screen.
  • Screen events may include screen movement events and screen touch events.
  • a screen movement event refers to an event in which the display screen of a terminal device is moved.
  • a screen touch event refers to an event in which the display screen of a terminal device collides.
  • the trigger condition for turning on the extended screen mode may be that a screen movement event occurs on a certain terminal device, and the distance between the display screen of the terminal device and the first display screen is less than a preset distance threshold. Therefore, when the first terminal device detects a screen movement event, the terminal device corresponding to the screen movement event can be used as a candidate terminal device. If the distance between the candidate terminal device's display screen and the first terminal device's first display screen is less than the expected When the distance threshold is set, the extended screen mode can be started, the candidate terminal device is used as the second terminal device, and the display screen of the second terminal device is used as the second display screen.
  • the user owns the terminal device 1 and the terminal device 2, and the user sets the display screen 201 of the terminal device 1 as the main display screen.
  • the user wants to extend the display of the display screen 1 to the display screen 202 of the terminal device 2
  • the user can move the display screen 202 from position A to the vicinity of the display screen 1 (ie, position B).
  • the terminal device 1 detects that the display screen 202 of the terminal device 2 has a screen movement event, and the distance between the display screen 201 and the display screen 202 is less than the preset distance threshold, the terminal device 1 starts the extended screen mode and uses the terminal device 2 as
  • the second terminal device uses the display screen 202 as a second display screen (that is, an extended screen).
  • the trigger condition for turning on the extended screen mode may be a screen touch event on the display screen of a certain terminal device. Therefore, when the first terminal device detects a screen touch event in the local area network, the terminal device corresponding to the screen touch event can be used as the second terminal device, and the display screen of the second terminal device can be used as the second display screen.
  • the user owns the terminal device 3 and the terminal device 4, and the user sets the display screen 301 of the terminal device 3 as the main display screen.
  • the user wants to expand the content of the screen 3 to the display screen 302 of the terminal device 4, the user can move the display screen 302 from the position C to the position D, so that the display screen 302 and the display screen 301 touch.
  • the terminal device 3 monitors the sensor data of the motion sensor of the terminal device 4 through the local area network.
  • the terminal device 3 When the terminal device 3 detects a touch event on the display screen 302 of the terminal device 4 through the sensor data of the motion sensor of the terminal device 4, the terminal device 3 will The terminal device 4 is used as a second terminal device, and the display screen 302 is used as a second display screen (that is, an extended screen).
  • the user can also tap the display screen 302 with his hand or other objects to cause a screen touch event to occur on the display screen 302.
  • the terminal device 3 monitors the touch event on the display screen 302 of the terminal device 4 through the local area network, and then the terminal device 4 As the second terminal device, the display screen 302 is used as a second display screen (that is, an extended screen).
  • the aforementioned screen movement event, screen touch event, and the distance between the first terminal device and other terminal devices can be obtained by analyzing the sensor data collected by the sensor.
  • the aforementioned sensor may be provided on the first terminal device and/or other terminal devices.
  • the type of sensor is selected according to the actual situation. For example, an acceleration sensor can be set on the display screen, and the screen movement event of the display screen can be detected by the acceleration sensor. This embodiment does not limit the type of sensor and the location where the sensor is installed.
  • the first terminal device may be provided with an NFC tag and the NFC function is enabled.
  • the first terminal device detects the NFC signal of other NFC tags, and the signal strength of the NFC signal is greater than or equal to
  • the intensity threshold is preset
  • the first terminal device performs a pairing operation with the terminal device corresponding to the NFC signal. After the pairing is successful, the first terminal device uses the terminal device corresponding to the NFC signal as the second terminal device, and uses the display screen of the second terminal device as the second display screen.
  • the user turns on the NFC function of the terminal device 5 and the terminal device 6. Then, the terminal device moves the terminal device 6 so that the NFC tag of the terminal device 6 contacts the NFC tag of the terminal device 5. At this time, the terminal device 5 detects the NFC signal of the terminal device 6, and the signal strength of the NFC signal is greater than the preset strength threshold. Therefore, the terminal device 5 sends a pairing request to the terminal device 6, and the user agrees to the pairing on the terminal device 6. If requested, the terminal device 5 and the terminal device 6 are paired successfully. After the pairing is successful, the terminal device 5 uses the terminal device 6 as the second terminal device, and uses the display screen of the terminal device 6 as the second display screen (that is, an extended screen).
  • the trigger condition for the first terminal device to turn on the extended screen mode may also be a screen extension instruction triggered by the user operating the first terminal device.
  • the first terminal device detects the screen expansion instruction
  • the terminal device corresponding to the screen expansion instruction is used as the second terminal device
  • the display screen of the second terminal device is used as the second display screen.
  • the user sets the display screen of the terminal device 7 as the main display screen, and connects the terminal device 7 and the terminal device 8 with an HDMI cable.
  • the user performs corresponding operations on the terminal device 7.
  • the user can drag the application program of the terminal device 7 outside the screen display range of the terminal device 7, trigger the screen expansion instruction, and specify the operation object of the screen expansion instruction For the terminal device 8.
  • the terminal device 7 detects the screen expansion instruction
  • the terminal device 8 corresponding to the screen expansion instruction is used as the second terminal device, and the display screen of the terminal device 8 is used as the second display screen.
  • the first terminal device determines the second terminal device and the second display screen, it can render and synthesize the graphics data of each application.
  • the screen picture displayed on the display screen is usually composed of windows corresponding to multiple applications.
  • the first terminal device usually renders the graphics data corresponding to each application to obtain the window data corresponding to each application.
  • the data is the rendered graphics data.
  • the first terminal device synthesizes the window data corresponding to each application program into screen display data.
  • the screen display data is the data that can be directly displayed on the screen, and the screen display data does not need to be rendered again during the process of being displayed on the screen.
  • the first terminal device Before rendering the graphics data of each application program, the first terminal device needs to virtualize the display resolution of each display screen as the target display resolution, and perform the rendering operation according to the target display resolution.
  • the display resolution of the display screen 401 is 2560 ⁇ 1600, and the display resolution of the display screen 402 is 1280 ⁇ 720.
  • Part of the window interface of the application program A is on the display screen 401, and part of the window interface is on the display screen 402.
  • the first terminal device needs to distinguish between the display resolution of the application A first.
  • the graphics data which part of the graphics data is located on the display screen 401, needs to be rendered according to the display resolution of the display screen 401, and which part of the graphics data is located on the display screen 402, needs to be rendered according to the display resolution of the display screen 402, so that the application
  • the graphic data of program A is divided into graphic data 1 and graphic data 2.
  • the first terminal device allows the graphics data 1 to be rendered according to the display resolution of 2560 ⁇ 1600 of the display screen 401, and the graphics data 2 is rendered according to the display resolution of 1280 ⁇ 720 of the display screen 402.
  • the rendering process is complicated, and the graphics data of the application needs to be split first and then rendered according to the corresponding display resolution.
  • the display resolution of the display screen 401 and the display resolution of the display screen 402 are unified and virtual as the target display resolution of 2560 ⁇ 1600, in the rendering process, whether it is the corresponding graphics data displayed on the display screen 401 or the display screen 402 corresponding
  • the graphics data of the image is rendered according to the target display resolution of 2560 ⁇ 1600. There is no need to split the graphics data of the application before rendering, which simplifies the rendering process.
  • the target display resolution can be set according to the actual situation.
  • the target display resolution may be a value preset by the user.
  • the user can set the target display resolution to 1280 ⁇ 720 in advance, and the first terminal device will be the first terminal device regardless of the first display resolution of the first display screen and the second display resolution of the second display Both the first display resolution of the first display screen and the second display resolution of the second display screen are virtualized to 1280 ⁇ 720, and then the rendering operation is performed.
  • the target display resolution is inconsistent with the display resolution of the display, it is necessary to perform a resolution adaptation operation on the screen display data to adapt the screen display data from the target display resolution To the first display resolution or the second display resolution.
  • the target display resolution is a value preset by the user, it may happen that the target display resolution is higher than the first display resolution and the second display resolution. At this time, the screen display data after the resolution adaptation can be normally displayed on the first display screen and the second display screen.
  • the target display resolution is a value preset by the user
  • the target display resolution is lower than the first display resolution or the target display resolution is lower than the second display resolution.
  • the screen display data needs to be adapted from the target display resolution with a lower resolution to the first display resolution or the second display resolution with a higher resolution, and the screen display data after the adaptation of the resolution is displayed on the display.
  • the screen is on, the screen picture displayed on the display screen is prone to picture distortion or picture tearing.
  • the first display resolution of the first terminal device and the second display resolution of the second terminal device can be obtained, and the largest of the first display resolution and the second display resolution can be obtained.
  • the value is used as the target display resolution.
  • the target display resolution is greater than or equal to the display resolution of each display. After the resolution is adjusted, the screen displayed on the display will not appear to be distorted or torn.
  • the first display resolution of the first display screen is 2560 ⁇ 1600
  • the second display resolution of the second display screen is 1280 ⁇ 720.
  • the first display resolution is greater than the second display resolution, and the first display resolution is selected as the target display resolution.
  • the screen display data is consistent with the target display resolution, and the screen display data does not need resolution adaptation and can be directly displayed on the screen.
  • the screen display data is displayed on the second display screen
  • the second display resolution is lower than the target display resolution
  • the screen display data is adapted from the target display resolution of 2560 ⁇ 1600 to the second display resolution of 1280 ⁇ 720. Adapting the screen display data from a higher display resolution to a lower display resolution can avoid the problem of picture distortion or picture tearing in the screen picture displayed on the second display screen.
  • the first display resolution of the first display screen is 720 ⁇ 480
  • the second display resolution of the second display screen is 1280 ⁇ 720.
  • the second display resolution is greater than the first display resolution, and the second display resolution is selected as the target display resolution.
  • the second display resolution is consistent with the target display resolution, and the screen display data does not require resolution adaptation and can be directly displayed on the screen.
  • the screen display data is displayed on the first display screen
  • the first display resolution is lower than the target display resolution
  • the screen display data is adapted from the target display resolution of 1280 ⁇ 720 to the first display resolution of 720 ⁇ 480. Adapting the screen display data from a higher display resolution to a lower display resolution can avoid the problem of picture distortion or picture tearing of the screen displayed on the first display screen.
  • the resolution adaptation method can be selected according to the actual situation.
  • the resolution of the screen display data can be adapted from high resolution to low resolution by means of downsampling.
  • the foregoing graphics data rendering process may be executed by a component having a graphics rendering function, such as a central processing unit (CPU, central processing unit) and/or a graphics processing unit (GPU).
  • a component having a graphics rendering function such as a central processing unit (CPU, central processing unit) and/or a graphics processing unit (GPU).
  • CPU central processing unit
  • GPU graphics processing unit
  • the screen picture displayed on the display screen is usually composed of windows corresponding to multiple applications. After the first terminal device finishes rendering the graphics data of each application program, the window data corresponding to each application program can be obtained, and the first terminal device will The window data corresponding to each application program is synthesized into screen display data.
  • the first terminal device splits the screen display data into first screen display data and second screen display data.
  • the first terminal device After the first terminal device obtains the screen display data, it can split the screen display data into the first screen display data and the second screen display data.
  • the first screen display data is the data displayed on the first display screen
  • the second screen display data is the data displayed on the second display screen.
  • the number of copies of the data displayed on the second screen is greater than or equal to 1, the number of copies of the data displayed on the second screen is consistent with the number of the second display screens, and one second display screen corresponds to one copy of the second screen display data.
  • the size of the screen display data corresponding to the screen images of each display screen is the same after the rendering is completed.
  • the total number of the first display screen and the second display screen can be obtained, and then the screen display data can be divided equally according to the aforementioned total number, so as to obtain the first screen display corresponding to the first display screen.
  • Data and the second screen display data corresponding to the second display screen are the same after the rendering is completed.
  • the first terminal device splits the screen display data, it can divide the screen display data into three equal parts, and divide the arrays 1 to 3 into the corresponding ones of the display screen 501.
  • the arrays 4 to 6 are divided into the corresponding second screen display data of the display screen 502, and the arrays 7 to 9 are divided into the second screen display data corresponding to the display screen 503.
  • 1 copy of the first screen display data and 2 copies of the second screen display data are obtained.
  • the first terminal device When splitting the first screen display data and the second screen display data, it is also possible to mark application information related to the first screen display data and mark application information related to the second screen display data.
  • the first terminal device needs to know which application windows are included in the first screen display data and which application windows are included in the second screen display data, it can directly call the application information corresponding to the first screen display data and the second screen display The application information corresponding to the data.
  • the first terminal device displays the first screen display data on the first display screen of the first terminal device, and sends the second screen display data to the second terminal device to instruct the The second terminal device displays the second screen display data on the second display screen of the second terminal device.
  • the first terminal device splits the screen display data into the first screen display data and the second screen display data, it can perform an upper screen operation on the first screen display data, and display the first screen display data on the first display screen.
  • the first terminal device may also send the second screen display data to the corresponding second terminal device.
  • the second terminal device After receiving the second screen display data, the second terminal device does not need to re-execute the processing flow of rendering and synthesis, and can directly perform an on-screen operation on the second screen display data, and display the second screen display data on the second display screen.
  • terminal device 601 there are multiple terminal devices in the local area network, such as terminal device 601, terminal device 602, and terminal device 603, and each terminal device in the local area network is connected to each other through wired communication and/or wireless communication.
  • the operating system of each terminal device is the Android operating system.
  • Figure 7 is a system architecture diagram of the Android operating system of the terminal device 601.
  • the system architecture of the Android operating system may include a software (Application, APP) layer, an architecture (Framework) layer, a native (Native) layer, and a core ( Kernel) layer, hardware (Hardware) layer.
  • At least one application (Application) 701 is provided in the APP layer, and the application 701 may be a system application or a third-party application.
  • WMS Window Manager Service
  • AMS Activity Manager Service
  • DMS Window Adaptation Service
  • Multi-Display Policy Service Multi-Display Policy Service
  • Input Manager Input Manager
  • the WMS module 702 is responsible for allocating a window (surface) for the application and managing the properties of the surface (such as the size, display position, display area, and hidden area of the surface).
  • the AMS module 703 is responsible for managing the life cycle of the application and processing items.
  • the DMS module 704 is responsible for managing the display screens currently available in the system. In the non-extended screen mode, the DMS module 704 registers the display screen of the current terminal device as an available display screen; in the extended screen mode, the DMS module 704 registers the display screen and the extended screen of the current terminal device as an available display screen.
  • the Multi-Display Policy Service module 705 is responsible for deciding whether to enable the extended screen mode according to preset trigger conditions for the extended screen.
  • the Input Manager module 706 is responsible for transferring data.
  • a graphics synthesizer (SurfaceFlinger) module 707 a hardware graphics maker (Hardware composer, HWC) module 709, an available image management (OnFrameAvailable) module 708, and a display channel (Display) 710 are provided.
  • the SurfaceFlinger module 707 is responsible for synthesizing the rendered window data of each application program into screen display data.
  • the HWC module 709 is responsible for displaying the screen display data on the screen.
  • the OnFrameAvailable module 708 is responsible for splitting the screen display data.
  • Display710 is responsible for transmitting the split screen display data.
  • a digital rights management (DRM) module 711 and a sensor driver (Sensor Driver) module 712 are provided.
  • the DRM module 711 is responsible for performing copyright verification on the content displayed by the terminal device.
  • the Sensor Driver module 712 is responsible for receiving and transmitting sensor data collected by the sensor.
  • a display screen (Screen) 713 and a local area network service (Local Area Network, LAN) module 714 are provided on the Hardware layer.
  • Screen713 is responsible for displaying screen pictures.
  • the LAN module 714 is responsible for creating and maintaining a local area network connection between the terminal device and other terminal devices.
  • the local area network connection may be a wired communication connection and/or a wireless communication connection.
  • the user can designate the terminal device 601 as the first terminal device, set the display screen 713 of the terminal device 601 as the first display screen, and set the trigger condition for the terminal device 601 to turn on the extended screen mode to the display screen of a terminal device in the local area network. Issue a screen touch event.
  • the terminal device 601 receives sensor data sent by other terminal devices in the local area network through the LAN module 714, and monitors screen touch events that occur in other terminal devices in the local area network.
  • the user When the user wants to use the display screen of the terminal device 602 as an extended screen of the terminal device 601, the user can move the display screen of the terminal device 602 and touch the display screen of the terminal device 602 with the display screen 713 of the terminal device 601.
  • the sensor on the display screen of the terminal device 602 collects corresponding sensor data, and the terminal device 602 transmits the above-mentioned sensor data to the terminal device 601 through the local area network.
  • the LAN module 714 of the terminal device 601 transmits the received sensor data to the Sensor Driver module 712, and the Sensor Driver module 712 transmits the sensor data to the Multi-Display Policy Service module 705 through the Input Manager module 706.
  • the Multi-Display Policy Service module 705 detects that the terminal device 602 has a screen touch event according to the sensor data, turns on the extended screen mode of the terminal device 601, and uses the display screen of the terminal device 602 as the extended screen object.
  • the DMS module 704 After the terminal device 601 opens the extended screen mode, the DMS module 704 performs a redirection operation to register the display screen of the terminal device 602 as an available display screen.
  • graphics rendering components with graphics rendering functions such as GPUs are responsible for rendering graphics data of various applications.
  • the graphics rendering component obtains the display resolution of the display screen of the terminal device 601 and the display resolution of the display screen of the terminal device 602, selects the maximum value among them as the target display resolution, and compares each display resolution according to the target display resolution.
  • the graphics data of the application is rendered.
  • the window data of each application is obtained, and the graphics rendering component transfers the rendered window data of each application to a buffer queue.
  • the SurfaceFlinger module 707 fetches the rendered window data of each application program from the data buffer queue.
  • the buffer queue adopts the producer-consumer model, and the fixed period of the buffer from the production queue to the consumption queue can be adjusted. Its working principle is shown in Figure 8.
  • the producer can create a buffer queue through the dequeue instruction, or the producer can also wake up the buffer queue in the dequeue state.
  • the buffer in the buffer queue is in the Free state; when the buffer queue is created or awakened successfully, the buffer queue passes the buffer to the producer, and the state of the buffer changes to the dequeue state.
  • the producer stores the rendered window data of each application in the buffer, and transfers the buffer to the buffer queue through the queue instruction, and the state of the buffer becomes the queue state.
  • the consumer can send an acquire command to the buffer queue.
  • the buffer queue After the buffer queue receives the acquire command, it passes the buffer to the consumer. At this time, the state of the buffer becomes the acquired state.
  • the consumer finishes processing the data in the buffer, he clears the data cached in the buffer and releases the buffer back to the buffer queue. At this time, the state of the buffer changes to the Free state, and the buffer queue completes the buffering and calling of this data.
  • the producer is a component with graphics rendering functions such as GPU. After the producer finishes rendering the graphics data of each application, each application is obtained. The window data that is rendered by each application is transferred to the buffer of the buffer queue, waiting for the call of the consumer. Consumers are components that need to operate on the data cached in the buffer, such as the SurfaceFlinger module 707. When the producer transfers the rendered window data of each application to the buffer, the SurfaceFlinger module 707 calls the buffer of each buffer queue to obtain each application. The window data of each application program is combined to perform the synthesis operation.
  • the window data of each application program exists in the form of layers.
  • the SurfaceFlinger module 707 fuses the multi-layer window data into a layer of screen display data according to the surface attributes corresponding to the window data of each layer.
  • the SurfaceFlinger module 707 transmits the screen display data to the OnFrameAvailable module 708 in the form of a buffer, and the screen display data exists in the buffer in the form of an array.
  • the total number of screens of the terminal device 601 and the terminal device 602 is 2. Therefore, the OnFrameAvailable module 708 divides the screen display data in the buffer into two equal parts, and transfers the first half of the screen display data in the buffer to the Display710 as the first screen display data. The second half of the screen display data in the buffer is used as the second screen display data and transmitted to the terminal device 602 through the LAN module 714.
  • the screen display data exists in the form of an array in the buffer. Assuming that there are 8 data blocks in the array, the serial numbers are from 000 to 007, then the OnFrameAvailable module 708 divides the array in the buffer into two equal parts, and divides the data from 000 to 003. The block is used as the first screen display data and is transferred to the Display 710, and the data blocks 004 to 007 are used as the second screen display data and transferred to the terminal device 602 through the LAN module 714.
  • the Display 710 transmits the first screen display data to the DRM module 711 for copyright verification. After the copyright verification, the DRM module 711 transmits the first screen display data to the HWC module 709, and the HWC module 709 is responsible for adapting the resolution of the first screen display data, and the first screen display data after the resolution is adjusted It is displayed on the display screen 713 of the terminal 601.
  • Figure 9 is a system architecture diagram of the Android operating system of the terminal device 602.
  • the functions of each module in the Android operating system of the terminal device 602 are the same as those of the corresponding modules in the Android operating system of the terminal device 601. This will not be repeated here.
  • the terminal device 602 After the terminal device 602 receives the second screen display data sent by the terminal device 601 through the LAN module 901, it stores the second screen display data in the buffer queue of the terminal device 602, and the buffer queue is responsible for the buffering of the second screen display data And pass.
  • the buffer queue of the terminal device 602 also adopts a producer-consumer model.
  • the producer is the LAN module 901
  • the consumer is the Display module 902, DRM module 903, HWC module 904, etc., which need to process the display data on the second screen.
  • the producer creates a buffer queue or wakes up a buffer queue in the dequeue state through the dequeue instruction. After successfully creating or waking up the buffer queue, the buffer queue transfers the buffer to the producer, and the state of the buffer changes to the dequeue state at this time.
  • the producer stores the second screen display data in the buffer, and transfers the buffer to the buffer queue through the queue instruction, and the state of the buffer becomes the queue state.
  • the consumer can send an acquire command to the buffer queue.
  • the buffer queue delivers the buffer to the consumer, and at this time the state of the buffer becomes the acquired state.
  • the LAN module stores the second screen display data in the buffer, it transfers the buffer to the buffer queue.
  • the Display module 902 obtains the second screen display data from the buffer queue, and the Display module 902 transmits the second screen display data to the DRM module 903 for copyright verification.
  • the DRM module 903 transmits the second screen display data to the HWC module 904.
  • the HWC module 904 performs resolution adaptation on the second screen display data, and displays the second screen display data after the resolution adaptation on the display screen 905 of the terminal device 602.
  • the main screen device after determining the screen content displayed on each display screen during the virtual display process, the main screen device sends the unrendered graphics data corresponding to the extended screen to the terminal where the extended screen is located. equipment. After the extended screen device receives the unrendered graphics data, it needs to perform rendering operations and compositing operations, and then display the synthesized on-screen display data on the screen. Therefore, when the extended screen displays the screen content, it is prone to screen refresh jams, and the screen picture displayed on the extended screen has a larger time delay compared with the screen picture displayed on the main screen.
  • the first terminal device first performs a rendering operation and a synthesis operation on the graphics data of each application program to obtain the screen display data. Then, the first terminal device divides the screen display data to obtain the first screen display data and the second screen display data, displays the first screen display data on the first display screen and sends the second screen display data to the second screen display data.
  • Terminal Equipment The second screen display data is data that has been rendered and synthesized, that is to say, after the second terminal device receives the second screen display data, it can directly perform the on-screen operation to display the second screen display data on the second display screen.
  • FIG. 11 An embodiment of the present application provides a screen extension device. For ease of description, only the parts related to the present application are shown. As shown in FIG. 11, the screen extension device includes:
  • the rendering synthesis module 1101 is configured to obtain graphic data of each application program, render the graphic data of each application program, and synthesize the rendered graphic data corresponding to each application program into screen display data;
  • the data splitting module 1102 is configured to split the screen display data into first screen display data and second screen display data;
  • the screen expansion module 1103 is configured to display the first screen display data on the display screen of the first terminal device, and send the second screen display data to the second terminal device to instruct the second terminal device
  • the terminal device displays the second screen display data on the display screen of the second terminal device.
  • the device further includes:
  • the event monitoring module is configured to determine the terminal device corresponding to the screen event as the second terminal device when a screen event meeting a preset trigger condition is monitored.
  • the event monitoring module includes:
  • the candidate movement sub-module is used to determine the terminal device corresponding to the screen movement event as a candidate terminal device when a screen movement event is monitored;
  • the distance determination sub-module is configured to determine the candidate terminal device as the second terminal device if the distance between the display screen of the candidate terminal device and the display screen of the first terminal device is less than a preset distance threshold.
  • the event monitoring module includes:
  • the collision determination sub-module is configured to determine the terminal device corresponding to the screen touch event as the second terminal device when the screen touch event is monitored.
  • the device further includes:
  • the near field communication module is configured to determine the terminal device corresponding to the NFC signal as the second terminal device when the first terminal device detects the NFC signal and the signal strength of the NFC signal is greater than or equal to the preset strength threshold.
  • the device further includes:
  • the instruction determination module is configured to determine the terminal device indicated in the screen expansion instruction as the second terminal device when the screen expansion instruction is acquired.
  • the rendering synthesis module 1101 includes:
  • the resolution sub-module is used to obtain the target display resolution
  • the graphics rendering sub-module is used to render the graphics data of each application program according to the target display resolution.
  • the resolution sub-module is specifically configured to obtain the first display resolution corresponding to the first terminal device and the second display resolution corresponding to the second terminal device, and combine the first display resolution with The maximum value of the second display resolution is determined as the target display resolution.
  • the data splitting module 1102 is specifically configured to obtain the sum of the number of display screens of the first terminal device and the number of display screens of the second terminal device, and compare the number of screens according to the sum of the numbers.
  • the display data is divided equally to obtain the first screen display data and the second screen display data.
  • the screen expansion module 1103 is specifically configured to display the first screen display data on the display screen of the first terminal device, and send the second screen display data to the second terminal device To instruct the second terminal device not to render the second screen display data, and display the second screen display data on the display screen of the second terminal device.
  • an embodiment of the present application also provides a terminal device.
  • the terminal device 12 of this embodiment includes: a processor 120, a memory 121, and a computer program 122 stored in the memory 121 and running on the processor 120.
  • the processor 120 executes the computer program 122
  • the steps in the above embodiment of the screen expansion method are implemented, for example, steps S101 to S103 shown in FIG. 1.
  • the processor 120 executes the computer program 122
  • the functions of the modules/units in the foregoing device embodiments, such as the functions of the modules 1101 to 1103 shown in FIG. 11, are realized.
  • the computer program 122 may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 121 and executed by the processor 120 to complete This application.
  • the one or more modules/units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 122 in the terminal device 12.
  • the computer program 122 may be divided into a rendering synthesis module, a data splitting module, and a screen expansion module. The specific functions of each module are as follows:
  • the rendering and synthesis module is used to obtain the graphic data of each application program, render the graphic data of each application program, and synthesize the rendered graphic data corresponding to each application program into screen display data;
  • a data splitting module for splitting the screen display data into first screen display data and second screen display data
  • the screen expansion module is configured to display the first screen display data on the display screen of the first terminal device, and send the second screen display data to the second terminal device to instruct the second terminal
  • the device displays the second screen display data on the display screen of the second terminal device.
  • the terminal device 12 may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server.
  • the terminal device may include, but is not limited to, a processor 120 and a memory 121.
  • FIG. 12 is only an example of the terminal device 12, and does not constitute a limitation on the terminal device 12. It may include more or less components than those shown in the figure, or a combination of certain components, or different components.
  • the terminal device may also include input and output devices, network access devices, buses, and so on.
  • the so-called processor 120 may be a central processing unit (Central Processing Unit, CPU), or other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the memory 121 may be an internal storage unit of the terminal device 12, such as a hard disk or a memory of the terminal device 12.
  • the memory 121 may also be an external storage device of the terminal device 12, for example, a plug-in hard disk equipped on the terminal device 12, a smart memory card (Smart Media Card, SMC), and a Secure Digital (SD). Card, Flash Card, etc. Further, the memory 121 may also include both an internal storage unit of the terminal device 12 and an external storage device.
  • the memory 121 is used to store the computer program and other programs and data required by the terminal device.
  • the memory 121 may also be used to temporarily store data that has been output or will be output.
  • the disclosed device/terminal device and method may be implemented in other ways.
  • the device/terminal device embodiments described above are only illustrative.
  • the division of the modules or units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units.
  • components can be combined or integrated into another system, or some features can be omitted or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
  • the integrated module/unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • this application implements all or part of the processes in the above-mentioned embodiments and methods, and can also be completed by instructing relevant hardware through a computer program.
  • the computer program can be stored in a computer-readable storage medium. When the program is executed by the processor, it can implement the steps of the foregoing method embodiments.
  • the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms.
  • the computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electrical carrier signal, telecommunications signal, and software distribution media, etc.
  • ROM Read-Only Memory
  • RAM Random Access Memory
  • electrical carrier signal telecommunications signal
  • software distribution media etc.
  • the content contained in the computer-readable medium can be appropriately added or deleted according to the requirements of the legislation and patent practice in the jurisdiction.
  • the computer-readable medium Does not include electrical carrier signals and telecommunication signals.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

本申请适用于图像处理领域，提供了一种屏幕扩展方法、装置、终端设备及计算机可读存储介质。在本申请的屏幕扩展方法中，第一终端设备对各个应用程序的图像数据进行渲染和合成，得到可以直接上屏显示的屏幕显示数据。然后，第一终端设备将屏幕显示数据拆分成第一屏幕显示数据和第二屏幕显示数据，将第一屏幕显示数据显示在第一终端设备的显示屏上，并且，将第二屏幕显示数据发送至第二终端设备。由于第二屏幕显示数据为经过渲染与合成的数据，因此，当第二终端设备可以直接对第二屏幕显示数据执行上屏操作，无需对第二屏幕显示数据重新进行渲染和合成的操作，减少了屏幕刷新卡顿情况的发生，并减小第二显示屏显示的屏幕画面的时延。

Description

屏幕扩展方法、装置、终端设备及计算机可读存储介质
本申请要求于2020年05月15日提交国家知识产权局、申请号为202010417816.X、申请名称为“屏幕扩展方法、装置、终端设备及计算机可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请属于图像处理领域，尤其涉及一种屏幕扩展方法、装置、终端设备及计算机可读存储介质。
背景技术
屏幕扩展技术可以让用户同时使用多个显示屏进行协同办公，或者进行娱乐、游戏等，可以极大地提升用户的使用体验。
在当前的屏幕扩展方案中，主屏设备将扩展屏需要显示的显示信息发送至扩展屏所在的终端设备。扩展屏所在的终端设备接收到显示信息后，需要对显示信息执行渲染以及合成的操作，难以快速将显示信息上屏显示，导致扩展屏容易出现刷新卡顿的情况，并且扩展屏显示的屏幕画面与主屏显示的屏幕画面相比，存在较大的时延。
发明内容
本申请实施例提供了一种屏幕扩展方法、装置、终端设备及计算机可读存储介质，可以解决当前的扩展屏方案中，扩展屏在显示屏幕画面时，容易出现刷新卡顿情况，并且扩展屏显示的屏幕画面与主屏显示的屏幕画面相比，存在较大的时延的问题。
本申请实施例的第一方面提供了一种屏幕扩展方法，包括：
第一终端设备获取各个应用程序的图形数据，对所述各个应用程序的图形数据进行渲染，并将各个应用程序对应的渲染后的图形数据合成为屏幕显示数据；
所述第一终端设备将所述屏幕显示数据拆分成第一屏幕显示数据和第二屏幕显示数据；
所述第一终端设备将所述第一屏幕显示数据显示在所述第一终端设备的显示屏上，并将所述第二屏幕显示数据发送至第二终端设备，以指示所述第二终端设备将所述第二屏幕显示数据显示在所述第二终端设备的显示屏上。
需要说明的是，第一终端设备获取到各个应用程序的图形数据后，对各个应用程序的图形数据进行渲染，得到各个应用程序对应的窗口数据，并将各个应用程序对应的窗口数据合成为屏幕显示数据。
然后，第一终端设备再将屏幕显示数据拆分成第一屏幕显示数据和第二屏幕显示数据。
由于第一屏幕显示数据和第二显示数据均为经过渲染和合成的数据，因此，第一终端设备可以直接将第一屏幕显示数据显示在第一显示屏(即第一终端设备的显示屏)上，并将第二屏幕显示数据发送至第二终端设备。
当第一终端设备将第二屏幕显示数据发送至第二终端设备之后，第二终端设备可以直接将第二屏幕显示数据显示在第二显示屏(即第二终端设备的显示屏)上，无需进行渲染和合成的操作，提高了第二屏幕显示数据的上屏速度，减少了屏幕刷新卡顿情况的发生，并减小第二显示屏显示的屏幕画面的时延。
在第一方面的一种可能的实现方式中，在所述第一终端设备获取各个应用程序的图形数据之前，还包括：
当第一终端设备监测到满足预设触发条件的屏幕事件时，所述第一终端设备将所述屏幕事件对应的终端设备确定为第二终端设备。
需要说明的是，当第一终端设备监测到满足预设触发条件的屏幕事件时，第一终端设备进入扩展屏模式，将该屏幕事件对应的终端设备确定为第二终端设备，并将该第二终端设备的显示屏确定为第二显示屏。
在第一方面的一种可能的实现方式中，所述当第一终端设备监测到满足预设触发条件的屏幕事件时，所述第一终端设备将所述屏幕事件对应的终端设备确定为第二终端设备，包括：
当第一终端设备监测到屏幕移动事件时，所述第一终端设备将所述屏幕移动事件对应的终端设备确定为候选终端设备；
若所述候选终端设备的显示屏与所述第一终端设备的显示屏的距离小于预设距离阈值，则所述第一终端设备将所述候选终端设备确定为第二终端设备。
需要说明的是，上述屏幕事件可以为屏幕移动事件，当第一终端设备监测到屏幕移动事件时，将该屏幕移动事件对应的终端设备确定为候选终端设备。
此时，上述预设触发条件可以为候选终端设备的显示屏与第一显示屏的距离小于预设距离阈值。当满足该预设触发条件时，第一终端设备将满足该预设触发条件的候选终端设备确定为第二终端设备，并将该第二终端设备的显示屏确定为第二显示屏。
在第一方面的另一种可能的实现方式中，所述当第一终端设备监测到满足预设触发条件的屏幕事件时，所述第一终端设备将所述屏幕事件对应的终端设备确定为第二终端设备，包括：
当第一终端设备监测到屏幕触碰事件时，所述第一终端设备将所述屏幕触碰事件对应的终端设备确定为第二终端设备。
需要说明的是，上述屏幕事件也可以为屏幕触碰事件。
当第一终端设备监测到屏幕触碰事件时，第一终端设备可以将所述屏幕触碰事件对应的终端设备确定为第二终端设备，并将该第二终端设备的显示屏确定为第二显示屏。
在第一方面的另一种可能的实现方式中，在所述第一终端设备获取各个应用程序的图形数据之前，还包括：
当第一终端设备检测到近场通信NFC信号，且所述NFC信号的信号强度大于或等于预设强度阈值时，所述第一终端设备将所述NFC信号对应的终端设备确定为第二终端设备。
需要说明的是，第一终端设备进入扩展屏模式的触发条件还可以为第一终端设备检测到近场通信(Near Field Communication，NFC)信号，且该NFC信号的信号强度 大于或等于预设强度阈值。
当第一终端设备检测到NFC信号，且NFC信号满足上述触发条件时，第一终端设备可以将产生该NFC信号的终端设备确定为第二终端设备，并将该第二终端设备的显示屏确定为第二显示屏。
在第一方面的另一种可能的实现方式中，在所述第一终端设备获取各个应用程序的图形数据之前，还包括：
当第一终端设备获取到屏幕扩展指令时，所述第一终端设备将所述屏幕扩展指令中指示的终端设备确定为第二终端设备。
需要说明的是，第一终端设备进入扩展屏模式的触发条件还可以为第一终端设备接收到屏幕扩展指令。
当用户想要第一终端设备进入扩展屏模式时，用户可以通过操作第一终端设备触发屏幕扩展指令，屏幕扩展指令中包括屏幕扩展对象，该屏幕扩展对象可以为第二终端设备的标识。
第一终端设备可以将屏幕扩展指令中指示的终端设备确定为第二终端设备，并将第二终端设备的显示屏确定为第二显示屏。
在第一方面的一种可能的实现方式中，所述第一终端设备对所述各个应用程序的图形数据进行渲染，包括：
所述第一终端设备获取目标显示分辨率；
所述第一终端设备根据目标显示分辨率对所述各个应用程序的图形数据进行渲染，得到各个应用程序对应的窗口数据。
需要说明的是，第一终端设备可以将第一显示屏的第一显示分辨率和第二显示屏的第二显示分辨率虚拟至目标显示分辨率，根据目标显示分辨率对各个应用程序的图形数据进行渲染，降低图形数据的渲染难度。
在第一方面的一种可能的实现方式中，所述第一终端设备获取目标显示分辨率，包括：
所述第一终端设备获取所述第一终端设备对应的第一显示分辨率以及所述第二终端设备对应的第二显示分辨率，将所述第一显示分辨率和所述第二显示分辨率中的最大值确定为目标显示分辨率。
需要说明的是，由于使用高分辨率的显示屏显示低分辨率的图像容易出现画面失真或画面撕裂的问题。因此，第一终端设备在确定目标显示分辨率时，可以获取第一终端设备对应的第一显示分辨率以及第二终端设备对应的第二显示分辨率，取第一显示分辨率和第二显示分辨率中的较大值作为目标显示分辨率。
然后，第一终端设备使用目标显示分辨率对各个应用程序的图形数据进行渲染，避免出现画面失真或画面撕裂的问题。
在第一方面的一种可能的实现方式中，所述第一终端设备将所述屏幕显示数据拆分成第一屏幕显示数据和第二屏幕显示数据，包括：
所述第一终端设备获取所述第一终端设备的显示屏数量和所述第二终端设备的显示屏数量的数量总和，根据所述数量总和对所述屏幕显示数据进行等分拆分，得到第一屏幕显示数据和第二屏幕显示数据。
需要说明的是，第一终端设备在拆分屏幕显示数据时，可以先获取第一显示屏和第二显示屏的数量总和，根据数量总和进行拆分。比如，假设存在1个第一显示屏和2个第二显示屏，则数量总和为3，对屏幕显示数据进行3等分拆分，得到1份第一屏幕显示数据和2份第二屏幕显示数据。
在第一方面的一种可能的实现方式中，所述第一终端设备将所述第一屏幕显示数据显示在所述第一终端设备的显示屏上，并将所述第二屏幕显示数据发送至第二终端设备，以指示所述第二终端设备将所述第二屏幕显示数据显示在所述第二终端设备的显示屏上，包括：
所述第一终端设备将所述第一屏幕显示数据显示在所述第一终端设备的显示屏上，并将所述第二屏幕显示数据发送至第二终端设备，以指示所述第二终端设备不对所述第二屏幕显示数据进行渲染，将所述第二屏幕显示数据显示在所述第二终端设备的显示屏上。
需要说明的是，由于第二屏幕显示数据为经过渲染和合成操作的数据，因此，第二终端设备无需对第二屏幕显示数据进行渲染，可以直接对第二屏幕显示数据执行上屏操作，将第二屏幕显示数据显示在第二显示屏上。
本申请实施例的第二方面提供了一种屏幕扩展装置，包括：
渲染合成模块，用于获取各个应用程序的图形数据，对所述各个应用程序的图形数据进行渲染，并将各个应用程序对应的渲染后的图形数据合成为屏幕显示数据；
数据拆分模块，用于将所述屏幕显示数据拆分成第一屏幕显示数据和第二屏幕显示数据；
屏幕扩展模块，用于将所述第一屏幕显示数据显示在所述第一终端设备的显示屏上，并将所述第二屏幕显示数据发送至第二终端设备，以指示所述第二终端设备将所述第二屏幕显示数据显示在所述第二终端设备的显示屏上。
在第二方面的一种可能的实现方式中，所述装置还包括：
事件监测模块，用于当监测到满足预设触发条件的屏幕事件时，将所述屏幕事件对应的终端设备确定为第二终端设备。
在第二方面的一种可能的实现方式中，所述事件监测模块，包括：
移动候选子模块，用于当监测到屏幕移动事件时，将所述屏幕移动事件对应的终端设备确定为候选终端设备；
距离判定子模块，用于若所述候选终端设备的显示屏与所述第一终端设备的显示屏的距离小于预设距离阈值，则将所述候选终端设备确定为第二终端设备。
在第二方面的另一种可能的实现方式中，所述事件监测模块，包括：
碰撞判定子模块，用于当监测到屏幕触碰事件时，将所述屏幕触碰事件对应的终端设备确定为第二终端设备。
在第二方面的另一种可能的实现方式中，所述装置还包括：
近场通信模块，用于当第一终端设备检测到NFC信号，且所述NFC信号的信号强度大于或等于预设强度阈值时，将所述NFC信号对应的终端设备确定为第二终端设备。
在第二方面的另一种可能的实现方式中，所述装置还包括：
指令判定模块，用于当获取到屏幕扩展指令时，将所述屏幕扩展指令中指示的终端设备确定为第二终端设备。
在第二方面的一种可能的实现方式中，所述渲染合成模块，包括：
分辨率子模块，用于获取目标显示分辨率；
图形渲染子模块，用于根据目标显示分辨率对所述各个应用程序的图形数据进行渲染。
在第二方面的一种可能的实现方式中，所述分辨率子模块，具体用于获取第一终端设备对应的第一显示分辨率以及所述第二终端设备对应的第二显示分辨率，将所述第一显示分辨率和所述第二显示分辨率中的最大值确定为目标显示分辨率。
在第二方面的一种可能的实现方式中，所述数据拆分模块，具体用于获取所述第一终端设备的显示屏数量和所述第二终端设备的显示屏数量的数量总和，根据所述数量总和对所述屏幕显示数据进行等分拆分，得到第一屏幕显示数据和第二屏幕显示数据。
在第二方面的一种可能的实现方式中，所述屏幕扩展模块，具体用于将所述第一屏幕显示数据显示在所述第一终端设备的显示屏上，并将所述第二屏幕显示数据发送至第二终端设备，以指示所述第二终端设备不对所述第二屏幕显示数据进行渲染，将所述第二屏幕显示数据显示在所述第二终端设备的显示屏上。
本申请实施例的第三方面提供了一种终端设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时，终端设备实现如上述方法的步骤。
本申请实施例的第四方面提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行时，使得终端设备实现如上述方法的步骤。
本申请实施例的第五方面提供了一种计算机程序产品，当计算机程序产品在终端设备上运行时，使得终端设备实现如上述方法的步骤。
本申请实施例与现有技术相比存在的有益效果是：
在本申请的屏幕扩展方法中，第一终端设备会对各个应用程序的图像数据进行渲染和合成，得到可以直接上屏显示的屏幕显示数据。然后，第一终端设备将屏幕显示数据拆分成第一屏幕显示数据和第二屏幕显示数据，将第一屏幕显示数据显示在第一终端设备的显示屏上，并且，将第二屏幕显示数据发送至第二终端设备。由于第二屏幕显示数据为经过渲染与合成的数据，因此，当第二终端设备接收到第二屏幕显示数据时，可以直接对第二屏幕显示数据执行上屏操作，无需对第二屏幕显示数据重新进行渲染和合成的操作，减少了屏幕刷新卡顿情况的发生，并减小第二显示屏显示的屏幕画面的时延。
附图说明
图1是本申请实施例提供的一种屏幕扩展方法的流程示意图；
图2是本申请实施例提供的一种屏幕移动事件的示意图；
图3是本申请实施例提供的一种屏幕触碰事件的示意图；
图4是本申请实施例提供的一种应用程序跨屏幕显示的示意图；
图5是本申请实施例提供的一种屏幕显示数据拆分方式的示意图；
图6是本申请实施例提供的一种局域网的示意图；
图7是本申请实施例提供的一种安卓操作***的***架构图；
图8是本申请实施例提供的一种生产者-消费者模型的示意图；
图9是本申请实施例提供的另一种安卓操作***的***架构图；
图10是本申请实施例提供的另一种生产者-消费者模型的示意图；
图11是本申请实施例提供的一种屏幕扩展装置的结构示意图；
图12是本申请实施例提供的终端设备的示意图。
具体实施方式
以下描述中，为了说明而不是为了限定，提出了诸如特定***结构、技术之类的具体细节，以便透彻理解本申请实施例。然而，本领域的技术人员应当清楚，在没有这些具体细节的其它实施例中也可以实现本申请。在其它情况中，省略对众所周知的***、装置、电路以及方法的详细说明，以免不必要的细节妨碍本申请的描述。
为了说明本申请所述的技术方案，下面通过具体实施例来进行说明。
应当理解，当在本说明书和所附权利要求书中使用时，术语“包括”指示所描述特征、整体、步骤、操作、元素和/或组件的存在，但并不排除一个或多个其它特征、整体、步骤、操作、元素、组件和/或其集合的存在或添加。
还应当理解，在此本申请说明书中所使用的术语仅仅是出于描述特定实施例的目的而并不意在限制本申请。如在本申请说明书和所附权利要求书中所使用的那样，除非上下文清楚地指明其它情况，否则单数形式的“一”、“一个”及“该”意在包括复数形式。
还应当进一步理解，在本申请说明书和所附权利要求书中使用的术语“和/或”是指相关联列出的项中的一个或多个的任何组合以及所有可能组合，并且包括这些组合。
如在本说明书和所附权利要求书中所使用的那样，术语“如果”可以依据上下文被解释为“当...时”或“一旦”或“响应于确定”或“响应于检测到”。类似地，短语“如果确定”或“如果检测到[所描述条件或事件]”可以依据上下文被解释为意指“一旦确定”或“响应于确定”或“一旦检测到[所描述条件或事件]”或“响应于检测到[所描述条件或事件]”。
本申请实施例提供的屏幕扩展方法可以应用程序于终端设备。该终端设备可以是任意具有显示屏且具备图像处理功能的设备，包括但不限于智能手机、平板电脑、可穿戴设备、车载设备、增强现实(augmented reality，AR)/虚拟现实(virtual reality，VR)设备、笔记本电脑、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本、个人数字助理(personal digital assistant，PDA)、人工智能(Artificial Intelligence，AI)终端等终端设备。本申请实施例对终端设备的具体类型不作任何限制。
例如，所述终端设备可以是WLAN中的站点(STAION，ST)，可以是蜂窝电话、无绳电话、会话启动协议(Session InitiationProtocol，SIP)电话、无线本地环路(Wireless Local Loop，WLL)站、个人数字处理(Personal Digital Assistant，PDA)设备、具有无线通信功能的手持设备、计算设备或连接到无线调制解调器的其它处理设备、车载设备、车联网终端、电脑、膝上型计算机、手持式通信设备、手持式计算设备、卫星无线设 备、用户驻地设备(customer premise equipment，CPE)和/或用于在无线***上进行通信的其它设备以及下一代通信***，例如，5G网络中的移动终端或者未来演进的公共陆地移动网络(Public Land Mobile Network，PLMN)网络中的移动终端等。
另外，在本申请的描述中，术语“第一”、“第二”、“第三”等仅用于区分描述，而不能理解为指示或暗示相对重要性。
屏幕扩展技术可以让用户同时使用多个显示屏进行协同办公，或者进行娱乐、游戏等，可以极大地提升用户的使用体验。
在目前的安卓***上，可以在一个屏幕中实现多窗口技术，比如分屏模式、画中画模式、自由(FreeFrom)模式等。
多窗口技术的核心思想是分栈和设置栈边界。多窗口功能的实现主要依赖于活动管理服务(ActivityManagerService，AMS)以及窗口管理服务(WindowManagerService，WMS)。AMS主要管理窗口的生命周期等参数，WMS主要管理窗口的位置、大小等参数。
多窗口技术是建立在栈(Stack)的数据结构上的，一个栈中包含了多个任务(Task)，一个任务中又包含有多个窗***动(Activity)。
AMS和WMS中的栈和任务的数据结构是一一对应的，AMS中的活动栈(ActivityStack)对应WMS中的任务栈(TaskStack)，AMS中的活动记录(ActivityRecord)对应WMS中的Task。
通过设置不同的多窗口模式下的栈ID和栈边界可以实现上述多窗口模式。例如，目前安卓***中定义了5中栈ID：(1)栈ID＝0，这个栈是主页栈(Home Stack)，是安卓***的桌面启动器(Launcher)所在的栈，除了桌面启动器以外，还有一些***界面也运行在这个栈上，例如近期任务等；(2)栈ID＝1，这个栈是全屏栈(FullScreen Stack)，普通的Activity所在的栈，但是，在分屏模式下，这个栈只占了半个屏幕；(3)栈ID＝2，这个栈是自由模式的Activity所在栈；(4)栈ID＝3，这个栈是对接栈(Docked Stack)，在分屏模式下，屏幕的另一半运行了一个固定的应用程序，这个应用程序的Activity就是在Docked Stack中；(5)栈ID＝4，这个栈是固定栈(Pinned Stack)，是画中画模式的Activity所在的栈。
不同的Activity可能归属于不同的栈，但是可能具有相同的栈ID。通过调节上述栈的边界可以调节栈里面每个任务的大小，最终任务的大小决定了窗口的大小。但是，由于栈的边界只能是当前的屏幕范围，因此，上述多窗口技术解决了将单个终端设备的屏幕切割成多个窗口的问题，比如将屏幕切割成二分屏、三分屏、四分屏等，但是无法实现多屏幕扩展。并且，上述多窗口技术的功能扩展性不好，只能在支持分屏的应用程序上使用。
此外，当前的一些投屏技术可以实现多屏交互，如推送模式、镜像模式。在推送模式中，手机和电视连接同一个局域网，手机和电视通过投屏协议传输数据(投屏协议的通道类似蓝牙的专用传输通道)。用户点击了投屏按钮后，手机搜索局域网内有没有可以应用程序投屏广播服务的设备。当手机搜索到电视后，手机会发送一个视频地址给电视，电视接收到地址后，访问地址中的视频资源进行播放。
在镜像模式中，可以将当前终端设备上的屏幕显示内容投屏到其他终端设备的屏 幕上，即同屏显示。例如，可以将当前终端设备上显示的文件、桌面、游戏画面、软件应用程序等内容投屏到其他终端设备的屏幕上。当前各类手机中的屏幕镜像、多屏互动、镜像开关等功能，都属于镜像模式。
镜像模式的原理是录制当前终端设备的屏幕，发送给其他终端设备。录制的速度通常可以达到每秒60帧以上。其他终端设备接收到录屏画面之后再显示出来，就实现了镜像模式。例如，手机和电视处于相同的局域网内，手机搜索到电视之后，开启镜像模式，手机会不断地截屏(即录屏)。手机一边以特定的频率截屏，一边将截屏图像发送给电视，电视收到后显示出来，这样就可以使用电视查看手机的屏幕画面了。
推送模式和镜像模式可以使用投屏协议进行数据传输，投屏协议包括数字生活网络联盟(Digital Living Network Alliance，DLNA)协议、airplay推送协议、乐联(lelink)协议等。
通过推送模式和镜像模式可以在一定程度上对屏幕进行扩展显示，但是，推送模式和镜像模式都不是真正意义上的屏幕扩展，用户难以对扩展屏进行操作以实现协同办公或娱乐。
对此，有一些软件或服务可以实现扩展屏的功能。例如，太空办公桌(spacedesk)软件以及微软的windows***中的扩展屏服务。
然而，在spacedesk以及windows***中的扩展屏服务中，当主屏设备将扩展屏的显示信息发送至扩展屏所在的终端设备之后，扩展屏所在的终端设备需要对扩展屏的显示信息执行渲染与合成的操作，难以快速上屏，导致屏幕出现刷新卡顿的现象，使得扩展屏显示的屏幕画面与主屏显示的屏幕画面相比，存在较大的时延。
综上，当前的扩展屏方案中，扩展屏在显示屏幕画面时，屏幕会出现刷新卡顿的情况，扩展屏显示的屏幕画面与主屏显示的屏幕画面相比，存在较大的时延。
有鉴于此，本申请实施例提供了一种屏幕扩展方法，当第一终端设备将第二终端设备的第二显示屏作为扩展屏时，会对各个应用程序的图形数据进行渲染和合成，得到可以直接上屏显示的屏幕显示数据。然后，第一终端设备将屏幕显示数据拆分成第一屏幕显示数据和第二屏幕显示数据，对第一屏幕显示数据执行上屏操作，将第一屏幕显示数据显示在第一显示屏上，并且，将第二屏幕显示数据发送至第二终端设备。由于第二屏幕显示数据为经过渲染与合成的数据，因此，当第二终端设备接收到第二屏幕显示数据时，可以直接对第二屏幕显示数据执行上屏操作，无需对第二屏幕显示数据重新进行渲染和合成的操作，减轻屏幕刷新卡顿情况，从而减小第二显示屏显示的屏幕画面的时延，解决当前的扩展屏方案中，扩展屏在显示屏幕画面时，屏幕会出现刷新卡顿的情况，扩展屏显示的屏幕画面与主屏显示的屏幕画面相比，存在较大的时延的问题。
接下来，将从第一终端设备的角度，对本实施例提供的屏幕扩展方法进行描述。请参阅图1所示的屏幕扩展方法的流程图，该方法包括：
S101、第一终端设备获取各个应用程序的图形数据，对各个应用程序的图形数据进行渲染，得到所述各个应用程序对应的窗口数据，并将所述各个应用程序对应的窗口数据合成为屏幕显示数据；
本实施例中的第一终端设备为主显示屏(即第一显示屏)所在的终端设备，第二 终端设备为扩展屏(即第二显示屏)所在的终端设备。
第一终端设备与第二终端设备可以通过有线通信和/或无线通信的方式进行数据传输。例如，第一终端设备和第二终端设备可以通过HDMI线连接、RJ45接口连接、wifi连接等有线的通信方式和/或无线的通信方式进行通信连接。第二终端设备的数量为一个或一个以上，具体数量根据实际情况进行确定。
用户可以在第一终端设备中设置开启扩展屏模式的触发条件。当满足扩展屏模式的触发条件时，第一终端设备可以检测是否存在其他终端设备与第一终端设备通信连接。如果没有其他终端设备与第一终端设备通信连接，则第一终端设备无法开启扩展屏模式。
当存在其他终端设备与第一终端设备通信连接时，第一终端设备可以判断第一显示屏当前显示的屏幕画面对应的应用程序是否支持扩展屏模式。
如果第一显示屏当前显示的屏幕画面对应的应用程序支持扩展屏模式，则第一终端设备开启扩展屏模式，从与第一终端设备通信连接的终端设备中选取第二终端设备，将第二终端设备的第二显示屏作为扩展屏。
如果第一显示屏当前显示的屏幕画面对应的应用程序不支持扩展屏模式，则第一终端设备提示用户当前显示屏显示的屏幕画面中，存在不支持扩展屏模式的应用程序。
在一些可能的实现方式中，第一终端设备所在的局域网中，除了第一终端以外，还存在至少一个终端设备。第一终端设备对局域网内的各个终端设备的屏幕事件进行监测，当监测到满足预设触发条件的屏幕事件时，将该满足预设触发条件的屏幕事件对应的终端设备作为第二终端设备，将第二终端设备的显示屏作为第二显示屏。
屏幕事件可以包括屏幕移动事件和屏幕触碰事件。屏幕移动事件是指终端设备的显示屏被移动的事件。屏幕触碰事件是指终端设备的显示屏发生碰撞的事件。
在一些实施例中，开启扩展屏模式的触发条件可以为某一终端设备发生屏幕移动事件，且该终端设备的显示屏与第一显示屏的距离小于预设距离阈值。因此，当第一终端设备监测到屏幕移动事件时，可以将该屏幕移动事件对应的终端设备作为候选终端设备，如果候选终端设备的显示屏与第一终端设备的第一显示屏的距离小于预设距离阈值时，可以启动扩展屏模式，将该候选终端设备作为第二终端设备，将该第二终端设备的显示屏作为第二显示屏。
例如，如图2所示，用户拥有终端设备1和终端设备2，用户将终端设备1的显示屏201设置为主显示屏。当用户想要将显示屏1的内容扩展显示至终端设备2的显示屏202时，用户可以将显示屏202从位置A移动至显示屏1附近(即位置B)。当终端设备1监测到终端设备2的显示屏202发生屏幕移动事件，且显示屏201与显示屏202之间的距离小于预设距离阈值时，终端设备1开启扩展屏模式，将终端设备2作为第二终端设备，将显示屏202作为第二显示屏(即扩展屏)。
在另一些实施例中，开启扩展屏模式的触发条件可以为某一终端设备的显示屏发出屏幕触碰事件。因此，当第一终端设备在局域网中监测到屏幕触碰事件时，可以将该屏幕触碰事件对应的终端设备作为第二终端设备，将该第二终端设备的显示屏作为第二显示屏。
例如，如图3所示，用户拥有终端设备3和终端设备4，用户将终端设备3的显 示屏301设置为主显示屏。当用户想要将屏幕3的内容扩展显示到终端设备4的显示屏302时，用户可以将显示屏302从位置C搬移至位置D，使显示屏302和显示屏301发生触碰。终端设备3通过局域网监测到终端设备4的运动传感器的传感器数据，当终端设备3通过终端设备4的运动传感器的传感器数据检测到终端设备4的显示屏302发生触碰事件，则终端设备3将终端设备4作为第二终端设备，将显示屏302作为第二显示屏(即扩展屏)。
或者，用户也可以用手或其他物体敲击显示屏302，使得显示屏302发生屏幕触碰事件，终端设备3通过局域网监测到终端设备4的显示屏302发生触碰事件，则将终端设备4作为第二终端设备，将显示屏302作为第二显示屏(即扩展屏)。
需要说明的是，上述屏幕移动事件、屏幕触碰事件以及第一终端设备与其他终端设备的距离可以根据传感器采集到的传感器数据分析得到。前述传感器可以设置在第一终端设备和/或其他终端设备上。传感器的类型根据实际情况进行选择。例如，可以在显示屏上设置加速度传感器，通过加速度传感器检测该显示屏的屏幕移动事件。本实施例不对传感器的类型以及传感器设置的位置进行限制。
在另一些可能的实现方式中，第一终端设备上可以设置有NFC标签，且开启了NFC功能，当第一终端设备检测到其他NFC标签的NFC信号，且该NFC信号的信号强度大于或等于预设强度阈值时，第一终端设备与该NFC信号对应的终端设备进行配对操作。配对成功后，第一终端设备将该NFC信号对应的终端设备作为第二终端设备，将该第二终端设备的显示屏作为第二显示屏。
例如，用户开启终端设备5和终端设备6的NFC功能。然后，终端设备移动终端设备6，使终端设备6的NFC标签与终端设备5的NFC标签接触。此时，终端设备5检测到终端设备6的NFC信号，且该NFC信号的信号强度大于预设强度阈值，所以，终端设备5发送配对请求至终端设备6，用户在终端设备6上同意该配对请求，则终端设备5与终端设备6配对成功。配对成功后，终端设备5将终端设备6作为第二终端设备，将终端设备6的显示屏作为第二显示屏(即扩展屏)。
在另一些可能的实现方式中，第一终端设备开启扩展屏模式的触发条件也可以为用户操作第一终端设备触发的屏幕扩展指令。当第一终端设备检测到屏幕扩展指令时，将该屏幕扩展指令对应的终端设备作为第二终端设备，将该第二终端设备的显示屏作为第二显示屏。
例如，用户将终端设备7的显示屏设置为主显示屏，并且用HDMI线连接终端设备7和终端设备8。此时，用户对终端设备7执行相应的操作，比如，用户可以将终端设备7的应用程序拖拽至终端设备7的屏幕显示范围之外，触发屏幕扩展指令，并指定屏幕扩展指令的操作对象为终端设备8。终端设备7检测到屏幕扩展指令时，将该屏幕扩展指令对应的终端设备8作为第二终端设备，将终端设备8的显示屏作为第二显示屏。
第一终端设备确定了第二终端设备以及第二显示屏后，可以对各个应用程序的图形数据进行渲染和合成。
可以理解的是，显示屏显示的屏幕画面通常由多个应用程序对应的窗口组合而成，第一终端设备通常是分别渲染各个应用程序对应的图形数据，得到各个应用程序对应 的窗口数据，窗口数据为渲染后的图形数据。然后，第一终端设备再将各个应用程序对应的窗口数据合成为屏幕显示数据。屏幕显示数据为可以直接上屏显示的数据，屏幕显示数据在上屏显示的过程中无需再次执行渲染操作。
第一终端设备在渲染各个应用程序的图形数据之前，需要将各个显示屏的显示分辨率均虚拟为目标显示分辨率，根据目标显示分辨率进行渲染操作。
以图4为例，假设显示屏401的显示分辨率为2560×1600，显示屏402的显示分辨率为1280×720。应用程序A的部分窗口界面在显示屏401上，部分窗口界面在显示屏402上。
在渲染应用程序A的图形数据的过程中，如果不将显示屏401的显示分辨率和显示屏402的显示分辨率统一虚拟为目标显示分辨率，则第一终端设备需要先区分应用程序A的图形数据中，哪一部分的图形数据位于显示屏401，需要按照显示屏401的显示分辨率进行渲染，哪一部分图形数据位于显示屏402，需要按照显示屏402的显示分辨率进行渲染，从而将应用程序A的图形数据拆分成图形数据1和图形数据2。然后，第一终端设备再让图形数据1按照显示屏401的显示分辨率2560×1600进行渲染，让图形数据2按照显示屏402的显示分辨率1280×720进行渲染。渲染过程复杂，需要先拆分应用程序的图形数据再按照相应的显示分辨率进行渲染。
如果将显示屏401的显示分辨率和显示屏402的显示分辨率统一虚拟为目标显示分辨率2560×1600，则在渲染过程中，无论是显示在显示屏401对应的图形数据还是显示屏402对应的图形数据，均按照目标显示分辨率2560×1600进行渲染，渲染之前无需拆分应用程序的图形数据，简化渲染的过程。
目标显示分辨率可以根据实际情况进行设置。在一些可能的实现方式中，目标显示分辨率可以为用户预先设置的数值。例如，用户可以预先将目标显示分辨率设置为1280×720，则渲染的过程，不论第一显示屏的第一显示分辨率和第二显示屏的第二显示分辨率是多少，第一终端设备都会将第一显示屏的第一显示分辨率和第二显示屏的第二显示分辨率都虚拟成1280×720，然后执行渲染操作。
此外，屏幕显示数据在上屏显示的过程中，如果目标显示分辨率与显示屏的显示分辨率不一致，需要对屏幕显示数据执行分辨率适配操作，将屏幕显示数据从目标显示分辨率适配至第一显示分辨率或第二显示分辨率。
当目标显示分辨率为用户预先设置的数值时，有可能出现目标显示分辨率高于第一显示分辨率和第二显示分辨率的情况。此时，分辨率适配后的屏幕显示数据可以正常显示在第一显示屏和第二显示屏上。
但是，当目标显示分辨率为用户预先设置的数值时，也有可能出现目标显示分辨率低于第一显示分辨率或目标显示分辨率低于第二显示分辨率的情况。此时，屏幕显示数据需要从分辨率较低的目标显示分辨率适配至分辨率较高的第一显示分辨率或第二显示分辨率，将分辨率适配后的屏幕显示数据显示至显示屏时，显示屏显示的屏幕画面容易出现画面失真或画面撕裂的问题。
因此，在另一些可能的实现方式中，可以获取第一终端设备的第一显示分辨率和第二终端设备的第二显示分辨率，将第一显示分辨率和第二显示分辨率中的最大值作为目标显示分辨率。此时，目标显示分辨率大于或等于各个显示屏的显示分辨率，经 过分辨率适配后，显示屏显示的屏幕画面不会出现画面失真或画面撕裂的问题。
例如，假设第一显示屏的第一显示分辨率为2560×1600，第二显示屏的第二显示分辨率为1280×720。第一显示分辨率大于第二显示分辨率，选取第一显示分辨率作为目标显示分辨率。屏幕显示数据在第一显示屏上屏显示时，第一显示分辨率和目标显示分辨率一致，屏幕显示数据不需要进行分辨率适配，可以直接上屏显示。屏幕显示数据在第二显示屏上屏显示时，第二显示分辨率低于目标显示分辨率，屏幕显示数据从2560×1600的目标显示分辨率适配至1280×720的第二显示分辨率，屏幕显示数据从较高的显示分辨率适配至较低的显示分辨率，可以避免第二显示屏显示的屏幕画面出现画面失真或画面撕裂的问题。
或者，假设第一显示屏的第一显示分辨率为720×480，第二显示屏的第二显示分辨率为1280×720。第二显示分辨率大于第一显示分辨率，选取第二显示分辨率作为目标显示分辨率。屏幕显示数据在第二显示屏上屏显示时，第二显示分辨率和目标显示分辨率一致，屏幕显示数据不需要进行分辨率适配，可以直接上屏显示。屏幕显示数据在第一显示屏上屏显示时，第一显示分辨率低于目标显示分辨率，屏幕显示数据从1280×720的目标显示分辨率适配至720×480的第一显示分辨率，屏幕显示数据从较高的显示分辨率适配至较低的显示分辨率，可以避免第一显示屏显示的屏幕画面出现画面失真或画面撕裂的问题。
分辨率适配的方式可以根据实际情况进行选择。例如，在一些实施例中，可以通过降采样的方式将屏幕显示数据的分辨率由高分辨率适配至低分辨率。
上述图形数据渲染的过程可以由中央处理器(CPU，central processing unit)和/或图形处理器(Graphic Processing Unit，GPU)等具备图形渲染功能的组件执行。
显示屏显示的屏幕画面通常由多个应用程序对应的窗口组合而成，当第一终端设备对各个应用程序的图形数据渲染完成后，可以得到各个应用程序对应的窗口数据，第一终端设备将各个应用程序对应的窗口数据合成为屏幕显示数据。
S102、所述第一终端设备将所述屏幕显示数据拆分成第一屏幕显示数据和第二屏幕显示数据；
第一终端设备得到屏幕显示数据之后，可以将屏幕显示数据拆分成第一屏幕显示数据和第二屏幕显示数据。第一屏幕显示数据为在第一显示屏上显示的数据，第二屏幕显示数据为在第二显示屏上显示的数据。第二屏幕显示数据的份数大于或等于1，第二屏幕显示数据的份数与第二显示屏的数量一致，一个第二显示屏对应一份第二屏幕显示数据。
由于图形数据渲染的过程中，各个显示屏的显示分辨率都被虚拟为目标显示分辨率，因此，渲染完成后，各个显示屏的屏幕画面对应的屏幕显示数据的大小是相等的。在拆分屏幕显示数据时，可以获取第一显示屏和第二显示屏的数量总和，然后根据前述数量总和对屏幕显示数据进行等分拆分，从而得到第一显示屏对应的第一屏幕显示数据以及第二显示屏对应的第二屏幕显示数据。
例如，如图5所示，假设存在1个第一显示屏(显示屏501)和2个第二显示屏(显示屏502和显示屏503)，则第一显示屏和第二显示屏的数量总和为3。屏幕显示数据存储在数组1至数组9中，则第一终端设备在拆分屏幕显示数据时，可以将屏幕 显示数据进行三等分拆分，将数组1至数组3划分为显示屏501对应的第一屏幕显示数据，将数组4至数组6划分为显示屏502的对应的第二屏幕显示数据，将数组7至数组9划分为显示屏503对应的第二屏幕显示数据。从而得到1份第一屏幕显示数据以及2份第二屏幕显示数据。
在拆分第一屏幕显示数据和第二屏幕显示数据时，还可以标记与第一屏幕显示数据相关的应用程序信息以及标记与第二屏幕显示数据相关的应用程序信息。当第一终端设备需要了解第一屏幕显示数据包括哪些应用程序的窗口以及第二屏幕显示数据包括哪些应用程序的窗口时，可以直接调用第一屏幕显示数据对应的应用程序信息以及第二屏幕显示数据对应的应用程序信息。
S103、所述第一终端设备将所述第一屏幕显示数据显示在所述第一终端设备的第一显示屏上，将所述第二屏幕显示数据发送至第二终端设备，以指示所述第二终端设备将所述第二屏幕显示数据显示在所述第二终端设备的第二显示屏上。
第一终端设备将屏幕显示数据拆分成第一屏幕显示数据和第二屏幕显示数据之后，可以对第一屏幕显示数据执行上屏操作，在第一显示屏上显示第一屏幕显示数据。
并且，第一终端设备还可以将第二屏幕显示数据发送至对应的第二终端设备上。第二终端设备接收到第二屏幕显示数据之后，无需重新进行渲染和合成的处理流程，可以直接对第二屏幕显示数据执行上屏操作，将第二屏幕显示数据显示在第二显示屏上。
以下结合具体的应用程序场景对本实施例提供的屏幕扩展方法进行说明：
如图6所示，局域网中存在多个终端设备，如终端设备601、终端设备602和终端设备603，局域网中的各个终端设备相互之间通过有线通信和/或无线通信的方式进行连接。
各个终端设备的操作***为安卓操作***。请参阅图7，图7为终端设备601的安卓操作***的***架构图，安卓操作***的***架构可以包括软件(Application，APP)层、架构(Framework)层、本地(Native)层、核心(Kernel)层、硬件(Hardware)层。
在APP层设置有至少一个应用程序(Application)701，应用程序701可以是***应用程序，也可以是第三方的应用程序。
在Framework层设置有窗口管理服务(Window Manager Service，WMS)模块702、活动管理服务(Activity Manager Service，AMS)模块703、窗口适配服务(Display Manager Srevice，DMS)模块704、多屏显示决策服务(Multi-Display Policy Service)模块705以及输入管理(Input Manager)模块706。
WMS模块702负责为应用程序分配窗口(surface)以及管理surface的属性(例如surface的大小、显示位置、显示区域以及隐藏区域)。
AMS模块703负责管理应用程序的生命周期以及处理事项。
DMS模块704负责管理***当前可用的显示屏。在非扩展屏模式时，DMS模块704将当前终端设备的显示屏登记为可用的显示屏；在扩展屏模式时，DMS模块704将当前终端设备的显示屏和扩展屏登记为可用的显示屏。
Multi-Display Policy Service模块705负责根据预设的扩展屏触发条件决策是否开 启扩展屏模式。
Input Manager模块706负责传递数据。
在Native层设置有图形合成器(SurfaceFlinger)模块707、硬件图形制作器(Hardware composer，HWC)模块709、可用图像管理(OnFrameAvailable)模块708以及送显通道(Display)710。
SurfaceFlinger模块707负责将各个应用程序渲染后的窗口数据合成为屏幕显示数据。
HWC模块709负责将屏幕显示数据上屏显示。
OnFrameAvailable模块708负责拆分屏幕显示数据。
Display710负责传递拆分后的屏幕显示数据。
在Kernel层设置有数字版权管理(Digital rights management，DRM)模块711和传感器驱动(Sensor Driver)模块712。
DRM模块711负责对终端设备显示的内容进行版权校验。
Sensor Driver模块712负责接收和传递传感器采集的传感器数据。
在Hardware层设置有显示屏(Screen)713和局域网服务(Local Area Network，LAN)模块714。
Screen713负责显示屏幕画面。
LAN模块714负责创建与维持终端设备与其他终端设备的局域网连接，该局域网连接可以为有线通信连接和/或无线通信连接。
用户可以将终端设备601指定为第一终端设备，将终端设备601的显示屏713设置为第一显示屏，将终端设备601开启扩展屏模式的触发条件设置为局域网内某一终端设备的显示屏发出屏幕触碰事件。
此时，终端设备601通过LAN模块714接收局域网内其他终端设备发送的传感器数据，监测局域网内其他终端设备发生的屏幕触碰事件。
当用户想要将终端设备602的显示屏作为终端设备601的扩展屏时，用户可以搬动终端设备602的显示屏，将终端设备602的显示屏与终端设备601的显示屏713触碰一下。
终端设备602的显示屏上的传感器采集到相应的传感器数据，终端设备602通过局域网将上述传感器数据传递至终端设备601。
终端设备601的LAN模块714将接收到的传感器数据传入Sensor Driver模块712，Sensor Driver模块712将传感器数据通过Input Manager模块706传入Multi-Display Policy Service模块705。
Multi-Display Policy Service模块705根据传感器数据检测到终端设备602发生了屏幕触碰事件，开启终端设备601的扩展屏模式，将终端设备602的显示屏作为扩展屏对象。
终端设备601开启了扩展屏模式后，DMS模块704执行重定向操作，将终端设备602的显示屏登记为可用的显示屏。
在安卓操作***的图形管道渲染框架中，GPU等具备图形渲染功能的图形渲染组件负责对各个应用程序的图形数据进行渲染。在渲染的过程中，图形渲染组件获取终 端设备601的显示屏的显示分辨率和终端设备602的显示屏的显示分辨率，选取其中的最大值作为目标显示分辨率，根据目标显示分辨率对各个应用程序的图形数据进行渲染。
各个应用程序的图形数据渲染完成后，得到各个应用程序的窗口数据，图形渲染组件将各个应用程序渲染后的窗口数据传入缓存队列(buffer queue)中。
SurfaceFlinger模块707从数据缓冲队列中取出各个应用程序的渲染后的窗口数据。
其中，buffer queue采用生产者-消费者模式，可以调节缓冲区(buffer)从生产队列到消耗队列的固定周期，其工作原理如图8所示。
当生产者需要缓存数据时，生产者可以通过出列(dequeue)指令来创建buffer queue缓存队列，或者，生产者也可以唤醒处于出列(dequeue)状态下的buffer queue。
在buffer queue创建或唤醒成功之前，buffer queue中的buffer处于空闲(Free)状态；当buffer queue创建或唤醒成功后，buffer queue将buffer传递至生产者，此时buffer的状态变为dequeue状态。
生产者将各个应用程序渲染后的窗口数据存入buffer中，通过入列(queue)指令将buffer传递至buffer queue，buffer的状态变为入列(queue)状态。
当消费者需要对buffer内缓存的数据进行处理时，消费者可以发送获取(acquire)指令至buffer queue。
buffer queue接收到acquire指令后，将buffer传递至消费者，此时buffer的状态变为被获取(acquired)状态。
当消费者对buffer内的数据处理完毕后，清除buffer内缓存的数据，将buffer释放(release)回buffer queue，此时buffer的状态变为Free状态，buffer queue完成本次数据的缓存与调用。
由上可知，在终端设备601的buffer queue所采用生产者-消费者模型中，生产者为GPU等具备图形渲染功能的组件，生产者将各个应用程序的图形数据渲染完成后，得到各个应用程序的窗口数据，并将各个应用程序渲染后的窗口数据传入buffer queue的buffer中，等待消费者的调用。消费者为SurfaceFlinger模块707等需要对buffer内缓存的数据进行操作的组件，当生产者将各个应用程序渲染后的窗口数据传入buffer，由SurfaceFlinger模块707调用各个buffer queue的buffer，获取各个应用程序的窗口数据，对各个应用程序的窗口数据执行合成操作。
在SurfaceFlinger模块707中，各个应用程序的窗口数据以层(layer)的形式存在，SurfaceFlinger模块707根据各层窗口数据对应的surface属性，将多层窗口数据融合为一层屏幕显示数据。
之后，SurfaceFlinger模块707将屏幕显示数据以buffer的形式传递至OnFrameAvailable模块708，屏幕显示数据在buffer中以数组的形式存在。
终端设备601和终端设备602的屏幕总数为2，因此，OnFrameAvailable模块708将buffer中的屏幕显示数据两等分分割，将buffer中的前一半屏幕显示数据作为第一屏幕显示数据，传递至Display710，将buffer中的后一半屏幕显示数据作为第二屏幕显示数据，通过LAN模块714传递至终端设备602。
例如，屏幕显示数据在buffer中以数组的形式存在，假设数组中存在8个数据块， 编号依次为000至007，则OnFrameAvailable模块708对buffer中数组进行两等分分割，将000至003号数据块作为第一屏幕显示数据，传递至Display710，将004至007号数据块作为第二屏幕显示数据，通过LAN模块714传递至终端设备602。
Display710将第一屏幕显示数据传递至DRM模块711进行版权校验。DRM模块711在版权校验后，将第一屏幕显示数据传递至HWC模块709，由HWC模块709负责对第一屏幕显示数据进行分辨率适配，将分辨率适配后的第一屏幕显示数据显示在终端601的显示屏713上。
请参阅图9，图9为终端设备602的安卓操作***的***架构图，终端设备602的安卓操作***中的各个模块的功能与终端设备601的安卓操作***中对应的模块的功能相同，在此不重复赘述。
终端设备602通过LAN模块901接收到终端设备601发送的第二屏幕显示数据之后，将第二屏幕显示数据存入终端设备602的buffer queue的buffer中，由buffer queue负责第二屏幕显示数据的缓存与传递。
请参阅图10，终端设备602的buffer queue也是采用生产者-消费者模型。此时，终端设备602的buffer queue的生产者-消费者模型中，生产者为LAN模块901，消费者为Display模块902、DRM模块903、HWC模块904等需要对第二屏幕显示数据进行处理的组件。
生产者通过dequeue指令创建buffer queue或唤醒处于dequeue状态下的buffer queue。成功创建或唤醒buffer queue之后，buffer queue传递buffer至生产者，此时buffer的状态变为dequeue状态。
生产者将第二屏幕显示数据存入buffer中，通过入列(queue)指令将buffer传递至buffer queue，buffer的状态变为入列(queue)状态。
当消费者需要对buffer内的数据进行处理时，消费者可以发送至获取(acquire)指令至buffer queue。
buffer queue将buffer传递至消费者，此时buffer的状态变为被获取(acquired)状态。
结合图9和图10，当LAN模块将第二屏幕显示数据存入buffer之后，将buffer传递至buffer queue中。Display模块902从buffer queue中获取到第二屏幕显示数据，Display模块902将第二屏幕显示数据传递至DRM模块903进行版权校验。版权校验之后，DRM模块903将第二屏幕显示数据传递至HWC模块904中。HWC模块904对第二屏幕显示数据进行分辨率适配，并将分辨率适配后的第二屏幕显示数据显示在终端设备602的显示屏905上。
可以理解的是，在现有的方案中，通常是在虚拟display的过程中确定了各个显示屏显示的屏幕内容之后，主屏设备将扩展屏对应的未渲染的图形数据发送至扩展屏所在的终端设备。扩展屏设备接收到未渲染的图形数据后，需要执行渲染操作和合成操作，然后再将合成后的屏幕显示数据上屏显示。因此，扩展屏在显示屏幕内容时，容易出现屏幕刷新卡顿现象，扩展屏显示的屏幕画面与主屏显示的屏幕画面相比，存在较大的时延。
而本实施例的屏幕扩展方法中，第一终端设备是先对各个应用程序的图形数据执 行渲染操作以及合成操作，得到屏幕显示数据。然后，第一终端设备再对屏幕显示数据进行分割，得到第一屏幕显示数据和第二屏幕显示数据，在第一显示屏上显示第一屏幕显示数据并将第二屏幕显示数据发送至第二终端设备。第二屏幕显示数据为经过渲染与合成的数据，也即是说，第二终端设备接收到第二屏幕显示数据以后，可以直接执行上屏操作，将第二屏幕显示数据显示在第二显示屏上，上屏的过程无需重新对第二屏幕显示数据执行渲染以及合成操作，从而提高数据上屏速度，在一定程度上减少屏幕刷新卡顿现象的发生，减少扩展屏显示的屏幕画面与主屏显示的屏幕画面之间的时延，解决了当前的扩展屏方案中，扩展屏在显示屏幕画面时，屏幕会出现刷新卡顿的情况，扩展屏显示的屏幕画面与主屏显示的屏幕画面相比，存在较大的时延的问题。
应理解，上述实施例中各步骤的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
请参阅图11，本申请实施例提供了一种屏幕扩展装置，为便于说明，仅示出与本申请相关的部分，如图11所示，屏幕扩展装置包括，
渲染合成模块1101，用于获取各个应用程序的图形数据，对所述各个应用程序的图形数据进行渲染，并将各个应用程序对应的渲染后的图形数据合成为屏幕显示数据；
数据拆分模块1102，用于将所述屏幕显示数据拆分成第一屏幕显示数据和第二屏幕显示数据；
屏幕扩展模块1103，用于将所述第一屏幕显示数据显示在所述第一终端设备的显示屏上，并将所述第二屏幕显示数据发送至第二终端设备，以指示所述第二终端设备将所述第二屏幕显示数据显示在所述第二终端设备的显示屏上。
可选地，所述装置还包括：
事件监测模块，用于当监测到满足预设触发条件的屏幕事件时，将所述屏幕事件对应的终端设备确定为第二终端设备。
可选地，所述事件监测模块，包括：
移动候选子模块，用于当监测到屏幕移动事件时，将所述屏幕移动事件对应的终端设备确定为候选终端设备；
距离判定子模块，用于若所述候选终端设备的显示屏与所述第一终端设备的显示屏的距离小于预设距离阈值，则将所述候选终端设备确定为第二终端设备。
可选地，所述事件监测模块，包括：
碰撞判定子模块，用于当监测到屏幕触碰事件时，将所述屏幕触碰事件对应的终端设备确定为第二终端设备。
可选地，所述装置还包括：
近场通信模块，用于当第一终端设备检测到NFC信号，且所述NFC信号的信号强度大于或等于预设强度阈值时，将所述NFC信号对应的终端设备确定为第二终端设备。
可选地，所述装置还包括：
指令判定模块，用于当获取到屏幕扩展指令时，将所述屏幕扩展指令中指示的终 端设备确定为第二终端设备。
可选地，所述渲染合成模块1101，包括：
分辨率子模块，用于获取目标显示分辨率；
图形渲染子模块，用于根据目标显示分辨率对所述各个应用程序的图形数据进行渲染。
可选地，所述分辨率子模块，具体用于获取第一终端设备对应的第一显示分辨率以及所述第二终端设备对应的第二显示分辨率，将所述第一显示分辨率和所述第二显示分辨率中的最大值确定为目标显示分辨率。
可选地，所述数据拆分模块1102，具体用于获取所述第一终端设备的显示屏数量和所述第二终端设备的显示屏数量的数量总和，根据所述数量总和对所述屏幕显示数据进行等分拆分，得到第一屏幕显示数据和第二屏幕显示数据。
可选地，所述屏幕扩展模块1103，具体用于将所述第一屏幕显示数据显示在所述第一终端设备的显示屏上，并将所述第二屏幕显示数据发送至第二终端设备，以指示所述第二终端设备不对所述第二屏幕显示数据进行渲染，将所述第二屏幕显示数据显示在所述第二终端设备的显示屏上。
需要说明的是，上述装置/单元之间的信息交互、执行过程等内容，由于与本申请方法实施例基于同一构思，其具体功能及带来的技术效果，具体可参见方法实施例部分，此处不再赘述。
请参阅图12，本申请实施例还提供了一种终端设备。如图12所示，该实施例的终端设备12包括：处理器120、存储器121以及存储在所述存储器121中并可在所述处理器120上运行的计算机程序122。所述处理器120执行所述计算机程序122时实现上述屏幕扩展方法实施例中的步骤，例如图1所示的步骤S101至S103。或者，所述处理器120执行所述计算机程序122时实现上述各装置实施例中各模块/单元的功能，例如图11所示模块1101至1103的功能。
示例性的，所述计算机程序122可以被分割成一个或多个模块/单元，所述一个或者多个模块/单元被存储在所述存储器121中，并由所述处理器120执行，以完成本申请。所述一个或多个模块/单元可以是能够完成特定功能的一系列计算机程序指令段，该指令段用于描述所述计算机程序122在所述终端设备12中的执行过程。例如，所述计算机程序122可以被分割成渲染合成模块、数据拆分模块以及屏幕扩展模块，各模块具体功能如下：
渲染合成模块，用于获取各个应用程序的图形数据，对所述各个应用程序的图形数据进行渲染，并将各个应用程序对应的渲染后的图形数据合成为屏幕显示数据；
数据拆分模块，用于将所述屏幕显示数据拆分成第一屏幕显示数据和第二屏幕显示数据；
屏幕扩展模块，用于将所述第一屏幕显示数据显示在所述第一终端设备的显示屏上，并将所述第二屏幕显示数据发送至第二终端设备，以指示所述第二终端设备将所述第二屏幕显示数据显示在所述第二终端设备的显示屏上。
所述终端设备12可以是桌上型计算机、笔记本、掌上电脑及云端服务器等计算设备。所述终端设备可包括，但不仅限于，处理器120、存储器121。本领域技术人员可 以理解，图12仅仅是终端设备12的示例，并不构成对终端设备12的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件，例如所述终端设备还可以包括输入输出设备、网络接入设备、总线等。
所称处理器120可以是中央处理单元(Central Processing Unit，CPU)，还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
所述存储器121可以是所述终端设备12的内部存储单元，例如终端设备12的硬盘或内存。所述存储器121也可以是所述终端设备12的外部存储设备，例如所述终端设备12上配备的插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(Secure Digital,SD)卡，闪存卡(Flash Card)等。进一步地，所述存储器121还可以既包括所述终端设备12的内部存储单元也包括外部存储设备。所述存储器121用于存储所述计算机程序以及所述终端设备所需的其他程序和数据。所述存储器121还可以用于暂时地存储已经输出或者将要输出的数据。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，仅以上述各功能单元、模块的划分进行举例说明，实际应用程序中，可以根据需要而将上述功能分配由不同的功能单元、模块完成，即将所述装置的内部结构划分成不同的功能单元或模块，以完成以上描述的全部或者部分功能。实施例中的各功能单元、模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中，上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。另外，各功能单元、模块的具体名称也只是为了便于相互区分，并不用于限制本申请的保护范围。上述***中单元、模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的部分，可以参见其它实施例的相关描述。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用程序和设计约束条件。专业技术人员可以对每个特定的应用程序来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
在本申请所提供的实施例中，应该理解到，所揭露的装置/终端设备和方法，可以通过其它的方式实现。例如，以上所描述的装置/终端设备实施例仅仅是示意性的，例如，所述模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个***，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通讯连接可以是通过一些接口，装置或单元的间接耦合或通讯连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显 示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的模块/单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实现上述实施例方法中的全部或部分流程，也可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一计算机可读存储介质中，该计算机程序在被处理器执行时，可实现上述各个方法实施例的步骤。其中，所述计算机程序包括计算机程序代码，所述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读介质可以包括：能够携带所述计算机程序代码的任何实体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机存储器、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、电载波信号、电信信号以及软件分发介质等。需要说明的是，所述计算机可读介质包含的内容可以根据司法管辖区内立法和专利实践的要求进行适当的增减，例如在某些司法管辖区，根据立法和专利实践，计算机可读介质不包括电载波信号和电信信号。
最后应说明的是：以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何在本申请揭露的技术范围内的变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (22)

  1. 一种屏幕扩展方法，其特征在于，包括：
    第一终端设备获取各个应用程序的图形数据，对所述各个应用程序的图形数据进行渲染，并将各个应用程序对应的渲染后的图形数据合成为屏幕显示数据；
    所述第一终端设备将所述屏幕显示数据拆分成第一屏幕显示数据和第二屏幕显示数据；
    所述第一终端设备将所述第一屏幕显示数据显示在所述第一终端设备的显示屏上，并将所述第二屏幕显示数据发送至第二终端设备，以指示所述第二终端设备将所述第二屏幕显示数据显示在所述第二终端设备的显示屏上。
  2. 如权利要求1所述的屏幕扩展方法，其特征在于，在所述第一终端设备获取各个应用程序的图形数据之前，还包括：
    当第一终端设备监测到满足预设触发条件的屏幕事件时，所述第一终端设备将所述屏幕事件对应的终端设备确定为第二终端设备。
  3. 如权利要求2所述的屏幕扩展方法，其特征在于，所述当第一终端设备监测到满足预设触发条件的屏幕事件时，所述第一终端设备将所述屏幕事件对应的终端设备确定为第二终端设备，包括：
    当第一终端设备监测到屏幕移动事件时，所述第一终端设备将所述屏幕移动事件对应的终端设备确定为候选终端设备；
    若所述候选终端设备的显示屏与所述第一终端设备的显示屏的距离小于预设距离阈值，则所述第一终端设备将所述候选终端设备确定为第二终端设备。
  4. 如权利要求2所述的屏幕扩展方法，其特征在于，所述当第一终端设备监测到满足预设触发条件的屏幕事件时，所述第一终端设备将所述屏幕事件对应的终端设备确定为第二终端设备，包括：
    当第一终端设备监测到屏幕触碰事件时，所述第一终端设备将所述屏幕触碰事件对应的终端设备确定为第二终端设备。
  5. 如权利要求1所述的屏幕扩展方法，其特征在于，在所述第一终端设备获取各个应用程序的图形数据之前，还包括：
    当第一终端设备检测到近场通信NFC信号，且所述NFC信号的信号强度大于或等于预设强度阈值时，所述第一终端设备将所述NFC信号对应的终端设备确定为第二终端设备。
  6. 如权利要求1所述的屏幕扩展方法，其特征在于，在所述第一终端设备获取各个应用程序的图形数据之前，还包括：
    当第一终端设备获取到屏幕扩展指令时，所述第一终端设备将所述屏幕扩展指令中指示的终端设备确定为第二终端设备。
  7. 如权利要求1所述的屏幕扩展方法，其特征在于，所述第一终端设备对所述各个应用程序的图形数据进行渲染，包括：
    所述第一终端设备获取目标显示分辨率；
    所述第一终端设备根据所述目标显示分辨率对所述各个应用程序的图形数据进行渲染。
  8. 如权利要求7所述的屏幕扩展方法，其特征在于，所述第一终端设备获取目标显示分辨率，包括：
    所述第一终端设备获取所述第一终端设备对应的第一显示分辨率以及所述第二终端设备对应的第二显示分辨率，将所述第一显示分辨率和所述第二显示分辨率中的最 大值确定为目标显示分辨率。
  9. 如权利要求1所述的屏幕扩展方法，其特征在于，所述第一终端设备将所述屏幕显示数据拆分成第一屏幕显示数据和第二屏幕显示数据，包括：
    所述第一终端设备获取所述第一终端设备的显示屏数量和所述第二终端设备的显示屏数量的数量总和，根据所述数量总和对所述屏幕显示数据进行等分拆分，得到第一屏幕显示数据和第二屏幕显示数据。
  10. 如权利要求1所述的屏幕扩展方法，其特征在于，所述第一终端设备将所述第一屏幕显示数据显示在所述第一终端设备的显示屏上，并将所述第二屏幕显示数据发送至第二终端设备，以指示所述第二终端设备将所述第二屏幕显示数据显示在所述第二终端设备的显示屏上，包括：
    所述第一终端设备将所述第一屏幕显示数据显示在所述第一终端设备的显示屏上，并将所述第二屏幕显示数据发送至第二终端设备，以指示所述第二终端设备不对所述第二屏幕显示数据进行渲染，将所述第二屏幕显示数据显示在所述第二终端设备的显示屏上。
  11. 一种屏幕扩展装置，其特征在于，包括：
    渲染合成模块，用于获取各个应用程序的图形数据，对所述各个应用程序的图形数据进行渲染，并将各个应用程序对应的渲染后的图形数据合成为屏幕显示数据；
    数据拆分模块，用于将所述屏幕显示数据拆分成第一屏幕显示数据和第二屏幕显示数据；
    屏幕扩展模块，用于将所述第一屏幕显示数据显示在第一终端设备的显示屏上，并将所述第二屏幕显示数据发送至第二终端设备，以指示所述第二终端设备将所述第二屏幕显示数据显示在所述第二终端设备的显示屏上。
  12. 如权利要求11所述的屏幕扩展装置，其特征在于，所述装置还包括：
    事件监测模块，用于当监测到满足预设触发条件的屏幕事件时，将所述屏幕事件对应的终端设备确定为第二终端设备。
  13. 如权利要求12所述的屏幕扩展装置，其特征在于，所述事件监测模块，包括：
    移动候选子模块，用于当监测到屏幕移动事件时，将所述屏幕移动事件对应的终端设备确定为候选终端设备；
    距离判定子模块，用于若所述候选终端设备的显示屏与所述第一终端设备的显示屏的距离小于预设距离阈值，则将所述候选终端设备确定为第二终端设备。
  14. 如权利要求12所述的屏幕扩展装置，其特征在于，所述事件监测模块，包括：
    碰撞判定子模块，用于当监测到屏幕触碰事件时，将所述屏幕触碰事件对应的终端设备确定为第二终端设备。
  15. 如权利要求11所述的屏幕扩展装置，其特征在于，所述装置还包括：
    近场通信模块，用于当第一终端设备检测到NFC信号，且所述NFC信号的信号强度大于或等于预设强度阈值时，将所述NFC信号对应的终端设备确定为第二终端设备。
  16. 如权利要求11所述的屏幕扩展装置，其特征在于，所述装置还包括：
    指令判定模块，用于当获取到屏幕扩展指令时，将所述屏幕扩展指令中指示的终端设备确定为第二终端设备。
  17. 如权利要求11所述的屏幕扩展装置，其特征在于，所述渲染合成模块，包括：
    分辨率子模块，用于获取目标显示分辨率；
    图形渲染子模块，用于根据所述目标显示分辨率对所述各个应用程序的图形数据进行渲染。
  18. 如权利要求17所述的屏幕扩展装置，其特征在于，所述分辨率子模块，具体用于获取第一终端设备对应的第一显示分辨率以及所述第二终端设备对应的第二显示分辨率，将所述第一显示分辨率和所述第二显示分辨率中的最大值确定为目标显示分辨率。
  19. 如权利要求11所述的屏幕扩展装置，其特征在于，所述数据拆分模块，具体用于获取所述第一终端设备的显示屏数量和所述第二终端设备的显示屏数量的数量总和，根据所述数量总和对所述屏幕显示数据进行等分拆分，得到第一屏幕显示数据和第二屏幕显示数据。
  20. 如权利要求11所述的屏幕扩展装置，其特征在于，所述屏幕扩展模块，具体用于将所述第一屏幕显示数据显示在所述第一终端设备的显示屏上，并将所述第二屏幕显示数据发送至第二终端设备，以指示所述第二终端设备不对所述第二屏幕显示数据进行渲染，将所述第二屏幕显示数据显示在所述第二终端设备的显示屏上。
  21. 一种终端设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时，所述终端设备实现如权利要求1至10任一项所述方法的步骤。
  22. 一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其特征在于，所述计算机程序被处理器执行时，终端设备实现如权利要求1至10任一项所述方法的步骤。
PCT/CN2021/084205 2020-05-15 2021-03-30 屏幕扩展方法、装置、终端设备及计算机可读存储介质 WO2021227688A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010417816.X 2020-05-15
CN202010417816.XA CN113672184A (zh) 2020-05-15 2020-05-15 屏幕扩展方法、装置、终端设备及计算机可读存储介质

Publications (1)

Publication Number Publication Date
WO2021227688A1 true WO2021227688A1 (zh) 2021-11-18

Family

ID=78525359

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/084205 WO2021227688A1 (zh) 2020-05-15 2021-03-30 屏幕扩展方法、装置、终端设备及计算机可读存储介质

Country Status (2)

Country Link
CN (1) CN113672184A (zh)
WO (1) WO2021227688A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114691271A (zh) * 2022-03-29 2022-07-01 南京四维智联科技有限公司 一种多屏管理方法及装置、电子设备及***
CN115826898A (zh) * 2023-01-03 2023-03-21 南京芯驰半导体科技有限公司 一种跨屏显示方法、***、装置、设备及存储介质

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114491684B (zh) * 2022-01-07 2022-09-06 广州三七极耀网络科技有限公司 终端设备协同显示方法、***、终端设备及介质
CN114745589B (zh) * 2022-03-23 2023-10-27 青岛海信智慧生活科技股份有限公司 家居设备的投屏方法、目标家居设备及智能设备
CN115857860B (zh) * 2023-03-02 2023-06-02 麒麟软件有限公司 Wayland合成器中的镜像屏优化方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090284538A1 (en) * 2008-05-16 2009-11-19 Asustek Computer Inc. Video streaming data processing method
CN103106679A (zh) * 2013-01-05 2013-05-15 广东威创视讯科技股份有限公司 分布式3d多通道渲染方法、***和平台
CN107678716A (zh) * 2017-09-06 2018-02-09 珠海格力电器股份有限公司 一种图像显示方法、装置及移动终端
CN108108140A (zh) * 2018-01-09 2018-06-01 福建星网视易信息***有限公司 一种多屏协同显示方法及存储装置及支持3d显示的设备
CN110806847A (zh) * 2019-10-30 2020-02-18 支付宝(杭州)信息技术有限公司 一种分布式多屏幕显示方法、装置、设备及***

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110113496A (zh) * 2019-05-13 2019-08-09 Oppo广东移动通信有限公司 屏幕显示模式切换方法、装置及终端设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090284538A1 (en) * 2008-05-16 2009-11-19 Asustek Computer Inc. Video streaming data processing method
CN103106679A (zh) * 2013-01-05 2013-05-15 广东威创视讯科技股份有限公司 分布式3d多通道渲染方法、***和平台
CN107678716A (zh) * 2017-09-06 2018-02-09 珠海格力电器股份有限公司 一种图像显示方法、装置及移动终端
CN108108140A (zh) * 2018-01-09 2018-06-01 福建星网视易信息***有限公司 一种多屏协同显示方法及存储装置及支持3d显示的设备
CN110806847A (zh) * 2019-10-30 2020-02-18 支付宝(杭州)信息技术有限公司 一种分布式多屏幕显示方法、装置、设备及***

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114691271A (zh) * 2022-03-29 2022-07-01 南京四维智联科技有限公司 一种多屏管理方法及装置、电子设备及***
CN115826898A (zh) * 2023-01-03 2023-03-21 南京芯驰半导体科技有限公司 一种跨屏显示方法、***、装置、设备及存储介质
CN115826898B (zh) * 2023-01-03 2023-04-28 南京芯驰半导体科技有限公司 一种跨屏显示方法、***、装置、设备及存储介质

Also Published As

Publication number Publication date
CN113672184A (zh) 2021-11-19

Similar Documents

Publication Publication Date Title
WO2021227688A1 (zh) 屏幕扩展方法、装置、终端设备及计算机可读存储介质
WO2021004489A1 (zh) 显示控制方法及装置
WO2022052772A1 (zh) 多窗口投屏场景下的应用界面显示方法及电子设备
WO2021057830A1 (zh) 一种信息处理方法及电子设备
US20220398059A1 (en) Multi-window display method, electronic device, and system
US10725972B2 (en) Continuous and concurrent device experience in a multi-device ecosystem
WO2021143182A1 (zh) 游戏的处理方法、装置、电子设备及计算机可读存储介质
JP6033854B2 (ja) 大域的コンポジションシステム
TW201024993A (en) Techniques to control self refresh display functionality
JP2013546043A (ja) 即時リモートレンダリング
WO2021213379A1 (zh) 投屏显示方法、***、终端设备和存储介质
JP2013542515A (ja) 異環境間リダイレクション
EP2920693B1 (en) System and method for negotiating control of a shared audio or visual resource
WO2022088974A1 (zh) 一种遥控方法、电子设备及***
CN116821040B (zh) 基于gpu直接存储器访问的显示加速方法、装置及介质
US20100328216A1 (en) Information processing apparatus and input control method
WO2022135210A1 (zh) 一种增强的屏幕共享方法和***、电子设备
WO2019090772A1 (zh) 一种终端的图像处理方法及装置
WO2021248988A1 (zh) 跨终端录屏方法、终端设备及存储介质
EP3262581A1 (en) Opening new application window in response to remote resource sharing
WO2021052488A1 (zh) 一种信息处理方法及电子设备
CN110457522B (zh) 信息分享的方法及装置、终端和存储介质
WO2022242628A1 (zh) 投屏方法、装置、设备及存储介质
CN115150648A (zh) 显示设备及消息传输方法
CN112040149A (zh) 投屏器的输入源切换方法、装置、投屏器和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21803030

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21803030

Country of ref document: EP

Kind code of ref document: A1