WO2022206185A1 - Video stream processing method and apparatus - Google Patents

Video stream processing method and apparatus Download PDF

Info

Publication number
WO2022206185A1
WO2022206185A1 PCT/CN2022/076010 CN2022076010W WO2022206185A1 WO 2022206185 A1 WO2022206185 A1 WO 2022206185A1 CN 2022076010 W CN2022076010 W CN 2022076010W WO 2022206185 A1 WO2022206185 A1 WO 2022206185A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
layout information
video stream
linking
mode
Prior art date
Application number
PCT/CN2022/076010
Other languages
French (fr)
Chinese (zh)
Inventor
王伟
熊丙洋
邵云
Original Assignee
北京达佳互联信息技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京达佳互联信息技术有限公司 filed Critical 北京达佳互联信息技术有限公司
Publication of WO2022206185A1 publication Critical patent/WO2022206185A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/482—End-user interface for program selection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440245—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47205—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/4722—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting additional data associated with the content
    • H04N21/4725—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting additional data associated with the content using interactive regions of the image, e.g. hot spots
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/485—End-user interface for client configuration

Definitions

  • the present disclosure relates to the field of video processing, and in particular, to a video stream processing method, apparatus, electronic device, processor, and storage medium.
  • the present disclosure provides a video stream processing method, apparatus, electronic device, processor and storage medium.
  • a video stream processing method comprising: displaying first video content according to first layout information in response to a first touch operation acting on a graphical user interface, wherein the above-mentioned first layout
  • the information is determined by the currently used first video linking microphone mode; the target video stream is obtained; the second video linking microphone mode corresponding to the above-mentioned target video stream is determined; the second layout information is determined based on the above-mentioned second video linking microphone mode, wherein the above-mentioned
  • the second layout information is used to determine the second video content to be displayed in at least one display area of the target video stream in the GUI; the first video content is switched to the second video content according to the second layout information.
  • the video stream processing method before switching the first video content to the second video content according to the second layout information, further includes: decoding the video key frames in the target video stream to obtain A tuple, wherein the tuple is formed by encapsulating the at least one video stream and the second layout information based on the second video linking mode; the second layout information is determined based on the tuple.
  • the second layout information includes: position information and size information of each display area in the at least one display area.
  • switching the first video content to the second video content according to the second layout information includes: switching the first video content to the second video content based on the position information and size information of each display area in the at least one display area Switch to the above-mentioned second video content.
  • a video stream processing method including: obtaining a currently used first video linking mic mode and a second video linking mic mode to be used, wherein the first video linking mic mode It is used to display the first video content according to the corresponding first layout information, and the above-mentioned second video continuous microphone mode is used to determine the second layout information of each display area in the at least one display area to be displayed in the graphical user interface;
  • the layout rule corresponding to the two-video link-mic mode calculates the second layout information of each display area in the at least one display area; encodes at least one video stream and the second layout information as the target video based on the second video link-mic mode stream; send the above-mentioned target video stream to the target client, wherein the above-mentioned target client is used for decoding the above-mentioned target video stream to obtain the above-mentioned second layout information, and according to the above-mentioned second layout information. Switch to the second video content.
  • the second layout information includes: position information and size information of each display area in the at least one display area.
  • encoding the at least one video stream and the second layout information into the target video stream based on the second video concatenation mode includes: encoding the at least one video stream, the above-mentioned at least one video stream, the above-mentioned video stream based on the second video concatenating mode
  • the location information and the above-mentioned size information are encapsulated into a tuple; the above-mentioned tuple is encoded into a video key frame to obtain the above-mentioned target video stream.
  • a video stream processing apparatus including: a display unit configured to perform a first touch operation on a graphical user interface in response to a first touch operation, and display first video content according to first layout information , wherein, the above-mentioned first layout information is determined by the currently used first video linking mode; the first acquisition unit is configured to execute the acquisition of the target video stream; the first determination unit is configured to execute and determine the corresponding to the above-mentioned target video stream The second video linking microphone mode; the second determining unit is configured to perform determining the second layout information based on the above-mentioned second video linking microphone mode, wherein, the above-mentioned second layout information is used to determine the above-mentioned target video stream in the above-mentioned graphical user interface.
  • the second video content to be displayed in at least one display area in the display area; the switching unit is configured to perform switching the above-mentioned first video content to the above-mentioned second video
  • the above-mentioned video stream processing apparatus includes: a decoding subunit configured to perform decoding processing on video key frames in the above-mentioned target video stream to obtain a tuple, wherein the above-mentioned tuple is based on the above-mentioned second video connection.
  • the microphone mode is formed by encapsulating the above-mentioned at least one video stream and the above-mentioned layout information; the determination subunit is configured to perform the above-mentioned determination of the above-mentioned layout information based on the above-mentioned tuple.
  • the layout information includes: position information and size information of each display area in the at least one display area.
  • the above-mentioned switching unit includes: an updating unit configured to perform an update of the currently displayed content in the above-mentioned graphical user interface based on the position information and size information of each of the above-mentioned at least one display area.
  • a video stream processing apparatus comprising: a second acquisition unit configured to perform acquisition of a currently used first video linking mic mode and a to-be-used second video linking mic mode,
  • the above-mentioned first video linking microphone mode is used to display the first video content according to the corresponding first layout information
  • the above-mentioned second video linking microphone mode is used to determine each display area in the at least one display area to be displayed in the graphical user interface the second layout information
  • the calculation unit is configured to calculate the above-mentioned second layout information of each display area in the above-mentioned at least one display area based on the layout rule corresponding to the above-mentioned second video continuous microphone mode
  • the encoding unit is configured to perform Encode at least one video stream and the second layout information into a target video stream based on the second video linking mode
  • the sending unit is configured to send the target video stream to the target client, wherein the target client uses Decoding the target video stream to obtain the second
  • the second layout information includes: position information and size information of each display area in the at least one display area.
  • the encoding unit includes: an encapsulation subunit, configured to perform encapsulation of the at least one video stream, the location information, and the size information into a tuple based on the second video linking mode; the encoding subunit, is configured to perform encoding of the above-mentioned tuples into video key frames to obtain the above-mentioned target video stream.
  • a computer-readable storage medium When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the above-mentioned video stream processing method.
  • an electronic device comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement any one of the above The above-mentioned video stream processing method.
  • a computer program product including computer programs/instructions, which implement any one of the foregoing video stream processing methods when executed by a processor.
  • the first video content is displayed according to the first layout information, wherein the first layout information is determined by the currently used first video linking mode; obtaining target video stream; determine a second video linking microphone mode corresponding to the above-mentioned target video stream; determine second layout information based on the above-mentioned second video linking microphone mode, wherein the above-mentioned second layout information is used to determine whether the above-mentioned target video stream is in the above-mentioned graphics Second video content to be displayed in at least one display area in the user interface; switching the first video content to the second video content according to the second layout information.
  • the first video content is displayed according to the first layout information in response to the first touch operation acting on the graphical user interface, and received at the viewer end.
  • the corresponding second layout information can be determined based on the microphone connection mode corresponding to the above target video stream, and the second layout information to be displayed in the GUI of the above target video stream can be played based on the second layout information.
  • Video content and then in the process of playing the target video stream on the audience side, the host can change the video linking mode in real time. Since different video linking modes correspond to different layout information, there is no need to configure each linking mode separately. layout information.
  • FIG. 1 is a flowchart of a video stream processing method according to an exemplary embodiment
  • FIG. 2 is a flowchart of an optional video stream processing method according to an exemplary embodiment
  • Fig. 2a is a schematic diagram showing steps performed before step S15 in Fig. 1 is performed according to an exemplary embodiment
  • FIG. 2b is a schematic diagram of step S31 included in step S15 in FIG. 1 according to an exemplary embodiment
  • FIG. 3 is a flowchart of another video stream processing method according to an exemplary embodiment
  • Figure 3a is a schematic diagram illustrating the steps included in Figure 3 according to an exemplary embodiment
  • FIG. 4 is a block diagram of a video stream processing apparatus according to an exemplary embodiment
  • FIG. 5 is a block diagram of another video stream processing apparatus according to an exemplary embodiment
  • Fig. 6 is a block diagram of a computer device according to an exemplary embodiment.
  • the Internet live broadcast technology provides an embodiment of a video stream processing method.
  • different forms such as single-player mic-linking, PK battle, and multi-person mic-linking that support dynamic layout can be realized in the live broadcast room, without It is necessary to reconfigure the layout form separately for each mic connection method.
  • FIG. 1 is a flowchart of a video stream processing method according to an exemplary embodiment.
  • the video stream processing method is applied to an electronic device, and includes the following steps:
  • step S11 in response to the first touch operation acting on the graphical user interface, display the first video content according to the first layout information, wherein the above-mentioned first layout information is determined by the currently used first video linking mode;
  • step S12 obtain the target video stream
  • step S13 determine the second video continuous microphone mode corresponding to the above-mentioned target video stream
  • step S14 second layout information is determined based on the above-mentioned second video linking mode, wherein the above-mentioned second layout information is used to determine the second video content to be displayed in at least one display area of the above-mentioned target video stream in the above-mentioned graphical user interface ;
  • step S15 the first video content is switched to the second video content according to the second layout information.
  • the video linking mode to be used may be any type of video linking mode, such as single-player linking, PK battle, and multi-player linking.
  • the above-mentioned graphical user interface may be a Lianmai video interface, a Lianmai voice chat interface, a Lianmai PK interface, etc.
  • the above-mentioned layout information may include: symmetrical layout information or asymmetrical layout information, for example, upper and lower symmetrical layout information , left-right symmetrical layout information, top-bottom asymmetrical layout information, left-right asymmetrical layout information, and so on.
  • the setting ratio of each display area in the at least one display area to be displayed may be set according to the user's personalized needs, or may be set in equal or non-equal proportions by default, for example , the display area of Lianmai Video PK opponent is set as the auxiliary display area, the user's display area is set as the main display area, and the proportion of the main display area in the graphical user interface is greater than the proportion of the auxiliary display area in the graphical user interface.
  • FIG. 2 is a flowchart of an optional video stream processing method according to an exemplary embodiment.
  • the host, guest and audience are aligned based on video frames.
  • the target video stream is displayed synchronously, and the execution body of the embodiment of the present disclosure is the host. Since there are differences in the layout information of the display areas corresponding to different video linking methods, in the present disclosure, the first touch operation acting on the graphical user interface is used in the present disclosure.
  • display the first video content according to the first layout information after the viewer end receives the target video stream sent by the host end, can determine the corresponding second layout information based on the microphone connection mode corresponding to the above target video stream, and based on the second layout
  • the information plays the second video content to be displayed in the above-mentioned target video stream in the graphical user interface, and then in the process of playing the target video stream on the audience side, the host can change the video linking mode in real time, because different video linking modes correspond to To adapt to different layout information, there is no need to configure layout information separately for each mic connection mode.
  • the video stream processing method before switching the first video content to the second video content according to the second layout information, the video stream processing method further includes:
  • step S21 decoding the video key frames in the target video stream to obtain a tuple, wherein the tuple encapsulates the at least one video stream and the second layout information based on the second video linking mode. to make;
  • step S22 the above-mentioned second layout information is determined based on the above-mentioned tuple.
  • the second layout information includes: position information and size information of each display area in the at least one display area.
  • the host selects the video linking microphone mode to be used, either a user accesses the microphone (increases the connected microphone user) or a user downloads the microphone (reduce the users of connected microphones), the position information (x, y) and size information (w, h) of the display area corresponding to each user can be recalculated according to the layout rules corresponding to the video connected microphone mode, that is, the second layout can be obtained. information, and based on the second layout information, the currently displayed content in the graphical user interface is updated.
  • switching the first video content to the second video content according to the second layout information includes:
  • step S31 the first video content is switched to the second video content based on the position information and size information of each display area in the at least one display area.
  • the anchor terminal encodes the multi-channel video streams of all guest terminals and the local video stream of the anchor terminal based on the above-mentioned second video linking mode, and the above-mentioned layout information as the target video stream; each video key encoded by the anchor.
  • the frame (I frame) is based on the above-mentioned second video linking microphone mode to encapsulate the above-mentioned at least one video stream, the above-mentioned position information and the above-mentioned size information into a tuple, for example, assembled into a quintuple (x, y, w, h, slotId ) and written into the I frame of the target video stream, and sent to the viewer following the target video stream.
  • the viewer performs decoding processing on the video key frame in the target video stream, and obtains the tuple (x) written by the host from the video key frame I frame of the target video stream. , y, w, h, slotId), based on this tuple for determining the second video linking microphone mode corresponding to the above-mentioned target video stream, and then the viewer can determine the second layout information based on the above-mentioned second video linking microphone mode, and The display area corresponding to each guest among all the guests is adjusted according to the second layout information.
  • the mic layout can be updated in real time according to the video mic mode selected by the host at any time, and can be synchronized with the video stream to the viewer, and the layout can be changed in real time in the multi-person video mic and take effect immediately, thereby supporting more Flexible gameplay, and can unify different video linking technology solutions, only need to develop once to achieve the core logic, and can adapt to PK battles, single-player linking, multi-player linking, etc. through different layout configurations.
  • a video connected microphone mode is independently developed, reducing development workload and maintenance costs.
  • FIG. 3 is a flowchart of another video stream processing method according to an exemplary embodiment. As shown in FIG. 3 , the video stream processing method is applied to a server and includes the following steps:
  • step S41 the currently used first video linking microphone mode and the second video linking microphone mode to be used are obtained, wherein the above-mentioned first video linking microphone mode is used to display the first video content according to the corresponding first layout information,
  • the above-mentioned second video continuous microphone mode is used to determine the second layout information of each display area in the at least one display area to be displayed in the graphical user interface;
  • step S42 the above-mentioned second layout information of each display area in the above-mentioned at least one display area is calculated based on the layout rule corresponding to the above-mentioned second video continuous microphone mode;
  • step S43 encoding at least one video stream and the above-mentioned second layout information into a target video stream based on the above-mentioned second video linking mode
  • step S44 the target video stream is sent to the target client, wherein the target client is used for decoding the target video stream to obtain the second layout information, and according to the second layout information A video content is switched to a second video content.
  • the video linking mode to be used may be any type of video linking mode, such as single-player linking, PK battle, and multi-player linking.
  • the host, the guest and the audience display the target video stream synchronously based on video frame alignment.
  • the execution subject of the embodiment of the present disclosure is the audience.
  • the first video content is displayed according to the first layout information in response to the first touch operation acting on the graphical user interface, and the target video sent by the host is received at the viewer end.
  • the corresponding second layout information can be determined based on the microphone connection mode corresponding to the above-mentioned target video stream, and based on the second layout information, the second video content to be displayed in the graphical user interface of the above-mentioned target video stream can be played, and then the second video content to be displayed in the GUI of the above-mentioned target video stream can be played.
  • the host can change the video linking mode in real time. Since different video linking modes correspond to different layout information, there is no need to configure the layout information separately for each linking mode.
  • the above-mentioned graphical user interface may be a Lianmai video interface, a Lianmai voice chat interface, a Lianmai PK interface, etc.
  • the above-mentioned layout information may include: symmetrical layout information or asymmetrical layout information, for example, upper and lower symmetrical layout information , left-right symmetrical layout information, top-bottom asymmetrical layout information, left-right asymmetrical layout information, and so on.
  • the setting ratio of each display area in the at least one display area to be displayed may be set according to the user's personalized needs, or may be set in equal or non-equal proportions by default, for example , the display area of Lianmai Video PK opponent is set as the auxiliary display area, the user's display area is set as the main display area, and the proportion of the main display area in the graphical user interface is greater than the proportion of the auxiliary display area in the graphical user interface.
  • the second layout information includes: position information and size information of each display area in the at least one display area.
  • encoding the at least one video stream and the second layout information into the target video stream based on the second video concatenation mode includes:
  • step S51 the above-mentioned at least one video stream, the above-mentioned position information and the above-mentioned size information are encapsulated into a tuple based on the above-mentioned second video linking microphone mode;
  • step S52 the above-mentioned tuple is encoded into a video key frame to obtain the above-mentioned target video stream.
  • the anchor terminal merges the multi-channel video streams of all guest terminals and the local video stream of the anchor terminal into a new video stream to be displayed based on the above-mentioned second video linking mode.
  • the layout information encodes the video stream; each video key frame (I frame) encoded by the anchor side encapsulates the above-mentioned at least one video stream, the above-mentioned position information and the above-mentioned size information into a tuple based on the above-mentioned second video linking microphone mode, For example, it is assembled into a quintuple (x, y, w, h, slotId) and written into the I frame of the target video stream, and sent to the viewer following the target video stream.
  • quintuple x, y, w, h, slotId
  • the viewer performs decoding processing on the video key frame in the target video stream, and obtains the tuple (x) written by the host from the video key frame I frame of the target video stream. , y, w, h, slotId), based on this tuple for determining the second video linking microphone mode corresponding to the above-mentioned target video stream, and then the viewer can determine the second layout information based on the above-mentioned second video linking microphone mode, and The display area corresponding to each guest among all the guests is adjusted according to the second layout information.
  • the host selects the video linking microphone mode to be used, either a user accesses the microphone (increases the connected microphone user) or a user downloads the microphone (reduce the users of connected microphones), the position information (x, y) and size information (w, h) of the display area corresponding to each user can be recalculated according to the layout rules corresponding to the video connected microphone mode, that is, the second layout can be obtained. information, and based on the second layout information, the currently displayed content in the graphical user interface is updated.
  • the mic layout can be updated in real time according to the video mic mode selected by the host at any time, and can be synchronized with the video stream to the viewer, and the layout can be changed in real time in the multi-person video mic and take effect immediately, thereby supporting more Flexible gameplay, and can unify different video linking technology solutions, only need to develop once to achieve the core logic, and can adapt to PK battles, single-player linking, multi-player linking, etc. through different layout configurations.
  • a video connected microphone mode is independently developed, reducing development workload and maintenance costs.
  • Fig. 4 is a block diagram of a video stream processing apparatus according to an exemplary embodiment.
  • the device includes:
  • the display unit 401 is configured to perform a first touch operation in response to a graphical user interface, and display the first video content according to the first layout information, wherein the above-mentioned first layout information is determined by the currently used first video linking mode ;
  • the first acquisition unit 402 is configured to perform acquisition of the target video stream;
  • the first determination unit 403 is configured to perform determination of the second video linking microphone mode corresponding to the above-mentioned target video stream;
  • the second determination unit 404 is configured as Perform determining the second layout information based on the above-mentioned second video linking mode, wherein the above-mentioned second layout information is used to determine the above-mentioned second video content of the target video stream to be displayed in at least one display area in the above-mentioned graphical user interface;
  • switching unit 405 is configured to perform switching of the first video content to the second video content according to the second layout information.
  • the above-mentioned video stream processing apparatus includes: a decoding subunit configured to perform decoding processing on video key frames in the above-mentioned target video stream to obtain a tuple, wherein the above-mentioned tuple is based on the above-mentioned second video connection.
  • the microphone mode is formed by encapsulating the above-mentioned at least one video stream and the above-mentioned layout information; the determination subunit is configured to perform the above-mentioned determination of the above-mentioned layout information based on the above-mentioned tuple.
  • the layout information includes: position information and size information of each display area in the at least one display area.
  • the above-mentioned switching unit includes: an updating unit configured to perform an update of the currently displayed content in the above-mentioned graphical user interface based on the position information and size information of each of the above-mentioned at least one display area.
  • Fig. 5 is a block diagram of another video stream processing apparatus according to an exemplary embodiment.
  • the device includes:
  • the second acquisition unit 50 is configured to perform acquisition of the currently used first video linking mic mode and the second video linking mic mode to be used, wherein the first video linking mic mode is used for displaying according to the corresponding first layout information
  • the first video content, the above-mentioned second video linking microphone mode is used to determine the second layout information of each display area in the at least one display area to be displayed in the graphical user interface
  • the computing unit 51 is configured to perform an operation based on the above-mentioned second video
  • the layout rule corresponding to the continuous microphone mode calculates the above-mentioned second layout information of each display area in the above-mentioned at least one display area
  • the encoding unit 52 is configured to execute the at least one video stream and the above-mentioned second video stream based on the above-mentioned second video continuous microphone mode.
  • the layout information is encoded into a target video stream; the sending unit 53 is configured to execute sending the above-mentioned target video stream to a target client, wherein the above-mentioned target client is used for decoding the above-mentioned target video stream to obtain the above-mentioned second layout information , and switch the first video content to the second video content according to the second layout information.
  • the above-mentioned second layout information includes: position information and size information of each display area in the above-mentioned at least one display area.
  • the encoding unit includes: an encapsulation subunit, configured to perform encapsulation of the at least one video stream, the location information, and the size information into a tuple based on the second video linking mode; the encoding subunit, is configured to perform encoding of the above-mentioned tuples into video key frames to obtain the above-mentioned target video stream.
  • FIG. 6 is a block diagram of a computer device according to an exemplary embodiment.
  • a storage medium including instructions such as a memory 64 including instructions, is also provided, and the above-mentioned instructions can be processed by the computer device.
  • the controller 62 executes to complete any one of the above-mentioned video stream processing methods.
  • the storage medium may be a non-transitory computer-readable storage medium, for example, the above-mentioned non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device Wait.
  • an embodiment of a computer-readable storage medium is also provided.
  • the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the above-mentioned video stream processing method.
  • an embodiment of an electronic device comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement any of the above video stream processing method.
  • a computer program product comprising a computer program/instruction, and when the computer program/instruction is executed by a processor, any one of the foregoing video stream processing methods is implemented.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Databases & Information Systems (AREA)
  • User Interface Of Digital Computer (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

A video stream processing method and apparatus, an electronic device, a processor, and a storage medium. The video stream processing method comprises: in response to a first touch control operation, which acts on a graphical user interface, displaying first video content according to first layout information, wherein the first layout information is determined according to a currently used first video microphone-connection mode (S11); acquiring a target video stream (S12); determining a second video microphone-connection mode corresponding to the target video stream (S13); determining second layout information on the basis of the second video microphone-connection mode, wherein the second layout information is used for determining second video content to be displayed, of the target video stream, in at least one display region in the graphical user interface (S14); and switching the first video content to the second video content according to the second layout information (S15).

Description

视频流处理方法及装置Video stream processing method and device
相关申请的交叉引用CROSS-REFERENCE TO RELATED APPLICATIONS
本公开基于申请日为2021年04月02日、申请号为202110363735.0号的中国专利申请，并要求该中国专利申请的优先权，在此全文引用上述中国专利申请公开的内容以作为本公开的一部分。This disclosure is based on a Chinese patent application with an application date of April 2, 2021 and an application number of No. 202110363735.0, and claims the priority of the Chinese patent application. The disclosure of the above Chinese patent application is hereby cited in its entirety as a part of this disclosure .
技术领域technical field
本公开涉及视频处理领域，尤其涉及一种视频流处理方法、装置、电子设备、处理器及存储介质。The present disclosure relates to the field of video processing, and in particular, to a video stream processing method, apparatus, electronic device, processor, and storage medium.
背景技术Background technique
相关技术中，随着移动智能设备的发展，在移动直播业务中，用户对于实时视频连麦的需求也日渐丰富多样，例如：单人连麦，双人连麦视频，多人连麦聊天，连麦PK对战，等连麦应用场景层出不穷。In related technologies, with the development of mobile smart devices, in the mobile live broadcast service, users' demands for real-time video linking to the microphone are becoming more and more diverse, for example: single-person linking microphone, two-person linking microphone video, multi-person linking microphone chatting, Mai PK battle, waiting for Lian Mai application scenarios emerge one after another.
发明内容SUMMARY OF THE INVENTION
本公开提供一种视频流处理方法、装置、电子设备、处理器及存储介质。The present disclosure provides a video stream processing method, apparatus, electronic device, processor and storage medium.
本公开的技术方案如下：The technical solutions of the present disclosure are as follows:
根据本公开实施例的第一方面，提供一种视频流处理方法，包括：响应作用于图形用户界面的第一触控操作，按照第一布局信息显示第一视频内容，其中，上述第一布局信息由当前使用的第一视频连麦模式确定；获取目标视频流；确定与上述目标视频流对应的第二视频连麦模式；基于上述第二视频连麦模式确定第二布局信息，其中，上述第二布局信息用于确定上述目标视频流在上述图形用户界面内至少一个显示区域待显示的第二视频内容；按照上述第二布局信息将上述第一视频内容切换为上述第二视频内容。According to a first aspect of the embodiments of the present disclosure, there is provided a video stream processing method, comprising: displaying first video content according to first layout information in response to a first touch operation acting on a graphical user interface, wherein the above-mentioned first layout The information is determined by the currently used first video linking microphone mode; the target video stream is obtained; the second video linking microphone mode corresponding to the above-mentioned target video stream is determined; the second layout information is determined based on the above-mentioned second video linking microphone mode, wherein the above-mentioned The second layout information is used to determine the second video content to be displayed in at least one display area of the target video stream in the GUI; the first video content is switched to the second video content according to the second layout information.
在一些实施例中，在按照上述第二布局信息将上述第一视频内容切换为第二视频内容之前，上述视频流处理方法还包括：对上述目标视频流中的视频关键帧进行解码处理，得到多元组，其中，上述多元组基于上述第二视频连麦模式将上述至少一路视频流和上述第二布局信息封装而成；基于上述多元组确定上述第二布局信息。In some embodiments, before switching the first video content to the second video content according to the second layout information, the video stream processing method further includes: decoding the video key frames in the target video stream to obtain A tuple, wherein the tuple is formed by encapsulating the at least one video stream and the second layout information based on the second video linking mode; the second layout information is determined based on the tuple.
在一些实施例中，上述第二布局信息包括：上述至少一个显示区域中每个显示区域的位置信息和尺寸信息。In some embodiments, the second layout information includes: position information and size information of each display area in the at least one display area.
在一些实施例中，按照上述第二布局信息将上述第一视频内容切换为第二视频内容包括：基于上述至少一个显示区域中每个显示区域的位置信息和尺寸信息，将上述第一视频内容切换为上述第二视频内容。In some embodiments, switching the first video content to the second video content according to the second layout information includes: switching the first video content to the second video content based on the position information and size information of each display area in the at least one display area Switch to the above-mentioned second video content.
根据本公开实施例的第二方面，提供一种视频流处理方法，包括：获取当前使用的第一视频连麦模式和待使用的第二视频连麦模式，其中，上述第一视频连麦模式用于按照对应的第一布局信息显示第一视频内容，上述第二视频连麦模式用于确定图形用户界面内待显示的至少一个显示区域中每个显示区域的第二布局信息；基于上述第二视频连麦模式对应的布局规则计算上述至少一个显示区域中每个显示区域的上述第二布局信息；基于上述第二视频连麦模式将至少一路视频流和上述第二布局信息编码为目标视频流；将上述目标视频流发送至目标客户端，其中，上述目标客户端用于对上述目标视频流进行解码处理，得到上述第二布局信息，并按照上述第二布局信息将上述第一视频内容切换为第二视频内容。According to a second aspect of the embodiments of the present disclosure, there is provided a video stream processing method, including: obtaining a currently used first video linking mic mode and a second video linking mic mode to be used, wherein the first video linking mic mode It is used to display the first video content according to the corresponding first layout information, and the above-mentioned second video continuous microphone mode is used to determine the second layout information of each display area in the at least one display area to be displayed in the graphical user interface; The layout rule corresponding to the two-video link-mic mode calculates the second layout information of each display area in the at least one display area; encodes at least one video stream and the second layout information as the target video based on the second video link-mic mode stream; send the above-mentioned target video stream to the target client, wherein the above-mentioned target client is used for decoding the above-mentioned target video stream to obtain the above-mentioned second layout information, and according to the above-mentioned second layout information. Switch to the second video content.
在一些实施例中，上述第二布局信息包括：上述至少一个显示区域中每个显示区域的位置信息和尺寸信息。In some embodiments, the second layout information includes: position information and size information of each display area in the at least one display area.
在一些实施例中，基于上述第二视频连麦模式将上述至少一路视频流和上述第二布局信息编码为上述目标视频流包括：基于上述第二视频连麦模式将上述至少一路视频流、上述位置信息和上述尺寸信息封装为多元组；将上述多元组编码至视频关键帧，得到上述目标视频流。In some embodiments, encoding the at least one video stream and the second layout information into the target video stream based on the second video concatenation mode includes: encoding the at least one video stream, the above-mentioned at least one video stream, the above-mentioned video stream based on the second video concatenating mode The location information and the above-mentioned size information are encapsulated into a tuple; the above-mentioned tuple is encoded into a video key frame to obtain the above-mentioned target video stream.
根据本公开实施例的第三方面，提供一种视频流处理装置，包括：显示单元，被配置为执行响应作用于图形用户界面的第一触控操作，按照第一布局信息显示第一视频内容，其中，上述第一布局信息由当前使用的第一视频连麦模式确定；第一获取单元，被配置为执行获取目标视频流；第一确定单元，被配置为执行确定与上述目标视频流对应的第二视频连麦模式；第二确定单元，被配置为执行基于上述第二视频连麦模式确定第二布局信息，其中，上述第二布局信息用于确定上述目标视频流在上述图形用户界面内至少一个显示区域待显示的第二视频内容；切换单元，被配置为执行按照上述第二布局信息将上述第一视频内容切换为上述第二视频内容。According to a third aspect of the embodiments of the present disclosure, there is provided a video stream processing apparatus, including: a display unit configured to perform a first touch operation on a graphical user interface in response to a first touch operation, and display first video content according to first layout information , wherein, the above-mentioned first layout information is determined by the currently used first video linking mode; the first acquisition unit is configured to execute the acquisition of the target video stream; the first determination unit is configured to execute and determine the corresponding to the above-mentioned target video stream The second video linking microphone mode; the second determining unit is configured to perform determining the second layout information based on the above-mentioned second video linking microphone mode, wherein, the above-mentioned second layout information is used to determine the above-mentioned target video stream in the above-mentioned graphical user interface. the second video content to be displayed in at least one display area in the display area; the switching unit is configured to perform switching the above-mentioned first video content to the above-mentioned second video content according to the above-mentioned second layout information.
在一些实施例中，上述视频流处理装置包括：解码子单元，被配置为执行对上述目标视频流中的视频关键帧进行解码处理，得到多元组，其中，上述多元组基于上述第二视频连麦模式将上述至少一路视频流和上述布局信息封装而成；确定子单元，被配置为执行基于上述多元组确定上述布局信息。In some embodiments, the above-mentioned video stream processing apparatus includes: a decoding subunit configured to perform decoding processing on video key frames in the above-mentioned target video stream to obtain a tuple, wherein the above-mentioned tuple is based on the above-mentioned second video connection. The microphone mode is formed by encapsulating the above-mentioned at least one video stream and the above-mentioned layout information; the determination subunit is configured to perform the above-mentioned determination of the above-mentioned layout information based on the above-mentioned tuple.
在一些实施例中，上述布局信息包括：上述至少一个显示区域中每个显示区域的位置信息和尺寸信息。In some embodiments, the layout information includes: position information and size information of each display area in the at least one display area.
在一些实施例中，上述切换单元包括：更新单元，被配置为执行基于上述至少一个显示区域中每个显示区域的位置信息和尺寸信息对上述图形用户界面内的当前显示内容进行更新。In some embodiments, the above-mentioned switching unit includes: an updating unit configured to perform an update of the currently displayed content in the above-mentioned graphical user interface based on the position information and size information of each of the above-mentioned at least one display area.
根据本公开实施例的第四方面，提供一种视频流处理装置，包括：第二获取单元，被配置为执行获取当前使用的第一视频连麦模式和待使用的第二视频连麦模式，其中，上述第一视频连麦模式用于按照对应的第一布局信息显示第一视频内容，上述第二视频连麦模 式用于确定图形用户界面内待显示的至少一个显示区域中每个显示区域的第二布局信息；计算单元，被配置为执行基于上述第二视频连麦模式对应的布局规则计算上述至少一个显示区域中每个显示区域的上述第二布局信息；编码单元，被配置为执行基于上述第二视频连麦模式将至少一路视频流和上述第二布局信息编码为目标视频流；发送单元，被配置为执行将上述目标视频流发送至目标客户端，其中，上述目标客户端用于对上述目标视频流进行解码处理，得到上述第二布局信息，并按照上述第二布局信息将上述第一视频内容切换为第二视频内容。According to a fourth aspect of the embodiments of the present disclosure, there is provided a video stream processing apparatus, comprising: a second acquisition unit configured to perform acquisition of a currently used first video linking mic mode and a to-be-used second video linking mic mode, The above-mentioned first video linking microphone mode is used to display the first video content according to the corresponding first layout information, and the above-mentioned second video linking microphone mode is used to determine each display area in the at least one display area to be displayed in the graphical user interface the second layout information; the calculation unit is configured to calculate the above-mentioned second layout information of each display area in the above-mentioned at least one display area based on the layout rule corresponding to the above-mentioned second video continuous microphone mode; the encoding unit is configured to perform Encode at least one video stream and the second layout information into a target video stream based on the second video linking mode; the sending unit is configured to send the target video stream to the target client, wherein the target client uses Decoding the target video stream to obtain the second layout information, and switching the first video content to the second video content according to the second layout information.
在一些实施例中，上述第二布局信息包括：上述至少一个显示区域中每个显示区域的位置信息和尺寸信息。In some embodiments, the second layout information includes: position information and size information of each display area in the at least one display area.
在一些实施例中，上述编码单元包括：封装子单元，被配置为执行基于上述第二视频连麦模式将上述至少一路视频流、上述位置信息和上述尺寸信息封装为多元组；编码子单元，被配置为执行将上述多元组编码至视频关键帧，得到上述目标视频流。In some embodiments, the encoding unit includes: an encapsulation subunit, configured to perform encapsulation of the at least one video stream, the location information, and the size information into a tuple based on the second video linking mode; the encoding subunit, is configured to perform encoding of the above-mentioned tuples into video key frames to obtain the above-mentioned target video stream.
根据本公开实施例的第五方面，提供一种计算机可读存储介质，当上述计算机可读存储介质中的指令由电子设备的处理器执行时，使得电子设备能够执行上述的视频流处理方法。According to a fifth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the above-mentioned video stream processing method.
根据本公开实施例的第六方面，提供一种电子设备，包括：处理器；用于存储上述处理器可执行指令的存储器；其中，上述处理器被配置为执行上述指令，以实现任一项上述的视频流处理方法。According to a sixth aspect of the embodiments of the present disclosure, there is provided an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement any one of the above The above-mentioned video stream processing method.
根据本公开实施例的第七方面，提供一种计算机程序产品，包括计算机程序/指令，上述计算机程序/指令被处理器执行时实现任一项上述的视频流处理方法。According to a seventh aspect of the embodiments of the present disclosure, a computer program product is provided, including computer programs/instructions, which implement any one of the foregoing video stream processing methods when executed by a processor.
本公开实施例，通过响应作用于图形用户界面的第一触控操作，按照第一布局信息显示第一视频内容，其中，上述第一布局信息由当前使用的第一视频连麦模式确定；获取目标视频流；确定与上述目标视频流对应的第二视频连麦模式；基于上述第二视频连麦模式确定第二布局信息，其中，上述第二布局信息用于确定上述目标视频流在上述图形用户界面内至少一个显示区域待显示的第二视频内容；按照上述第二布局信息将上述第一视频内容切换为上述第二视频内容。In this embodiment of the present disclosure, by responding to a first touch operation acting on a graphical user interface, the first video content is displayed according to the first layout information, wherein the first layout information is determined by the currently used first video linking mode; obtaining target video stream; determine a second video linking microphone mode corresponding to the above-mentioned target video stream; determine second layout information based on the above-mentioned second video linking microphone mode, wherein the above-mentioned second layout information is used to determine whether the above-mentioned target video stream is in the above-mentioned graphics Second video content to be displayed in at least one display area in the user interface; switching the first video content to the second video content according to the second layout information.
由于不同的视频连麦方式对应的显示区域的布局信息存在区别，本公开中通过响应作用于图形用户界面的第一触控操作，按照第一布局信息显示第一视频内容，在观众端接收到主播端发送的目标视频流之后，可以基于上述目标视频流对应的连麦模式确定对应的第二布局信息，并基于第二布局信息播放上述目标视频流在该图形用户界面中待显示的第二视频内容，进而在观众端播放目标视频流的过程中，主播端可以实时更改视频连麦模式，由于不同的视频连麦模式对应适配不同的布局信息，无需为每一种连麦模式单独配置布局信息。Due to the differences in the layout information of the display areas corresponding to different video linking methods, in the present disclosure, the first video content is displayed according to the first layout information in response to the first touch operation acting on the graphical user interface, and received at the viewer end. After the target video stream sent by the host, the corresponding second layout information can be determined based on the microphone connection mode corresponding to the above target video stream, and the second layout information to be displayed in the GUI of the above target video stream can be played based on the second layout information. Video content, and then in the process of playing the target video stream on the audience side, the host can change the video linking mode in real time. Since different video linking modes correspond to different layout information, there is no need to configure each linking mode separately. layout information.
附图说明Description of drawings
图1是根据一示例性实施例示出的一种视频流处理方法的流程图；1 is a flowchart of a video stream processing method according to an exemplary embodiment;
图2是根据一示例性实施例示出的一种可选的视频流处理方法的流程图；2 is a flowchart of an optional video stream processing method according to an exemplary embodiment;
图2a是根据一示例性实施例示出的在执行图1中步骤S15之前执行的步骤的示意图；Fig. 2a is a schematic diagram showing steps performed before step S15 in Fig. 1 is performed according to an exemplary embodiment;
图2b是根据一示例性实施例示出的图1中步骤S15所包括的步骤S31的示意图；FIG. 2b is a schematic diagram of step S31 included in step S15 in FIG. 1 according to an exemplary embodiment;
图3是根据一示例性实施例示出的另一种视频流处理方法的流程图；3 is a flowchart of another video stream processing method according to an exemplary embodiment;
图3a是根据一示例性实施例示出的图3中所包括的步骤的示意图；Figure 3a is a schematic diagram illustrating the steps included in Figure 3 according to an exemplary embodiment;
图4是根据一示例性实施例示出的一种视频流处理装置的框图；4 is a block diagram of a video stream processing apparatus according to an exemplary embodiment;
图5是根据一示例性实施例示出的另一种视频流处理装置的框图；5 is a block diagram of another video stream processing apparatus according to an exemplary embodiment;
图6是根据一示例性实施例示出的一种计算机设备的框图。Fig. 6 is a block diagram of a computer device according to an exemplary embodiment.
具体实施方式Detailed ways
为了使本领域普通人员更好地理解本公开的技术方案，下面将结合附图，对本公开实施例中的技术方案进行清楚、完整地描述。In order to make those skilled in the art better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
需要说明的是，本公开的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本公开的实施例能够以除了在这里图示或描述的那些以外的顺序实施。以下示例性实施例中所描述的实施方式并不代表与本公开相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本公开的一些方面相一致的装置和方法的例子。It should be noted that the terms "first", "second" and the like in the description and claims of the present disclosure and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used may be interchanged under appropriate circumstances such that the embodiments of the disclosure described herein can be practiced in sequences other than those illustrated or described herein. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with this disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as recited in the appended claims.
随着移动网络从4G通讯技术、5G通讯技术以及WIFI通讯技术的发展，实时多人视频连麦技术不仅仅可以顺畅互联，还可以进一步灵活支持不同连麦场景玩法的可能，本公开实施例结合互联网直播技术，提供一种视频流处理方法的实施例，通过不同的布局配置可以在直播间内实现了支持动态布局的单人连麦，PK对战，多人连麦等不同的形式，而不需要重新为每一种连麦方式单独配置布局形式。With the development of mobile networks from 4G communication technology, 5G communication technology and WIFI communication technology, the real-time multi-person video linking microphone technology not only enables smooth interconnection, but also further flexibly supports the possibility of playing different microphone linking scenarios. The embodiments of the present disclosure combine The Internet live broadcast technology provides an embodiment of a video stream processing method. Through different layout configurations, different forms such as single-player mic-linking, PK battle, and multi-person mic-linking that support dynamic layout can be realized in the live broadcast room, without It is necessary to reconfigure the layout form separately for each mic connection method.
图1是根据一示例性实施例示出的一种视频流处理方法的流程图，参考图1所示，视频流处理方法应用于电子设备，并且包括以下步骤：FIG. 1 is a flowchart of a video stream processing method according to an exemplary embodiment. Referring to FIG. 1 , the video stream processing method is applied to an electronic device, and includes the following steps:
在步骤S11中，响应作用于图形用户界面的第一触控操作，按照第一布局信息显示第一视频内容，其中，上述第一布局信息由当前使用的第一视频连麦模式确定；In step S11, in response to the first touch operation acting on the graphical user interface, display the first video content according to the first layout information, wherein the above-mentioned first layout information is determined by the currently used first video linking mode;
在步骤S12中，获取目标视频流；In step S12, obtain the target video stream;
在步骤S13中，确定与上述目标视频流对应的第二视频连麦模式；In step S13, determine the second video continuous microphone mode corresponding to the above-mentioned target video stream;
在步骤S14中，基于上述第二视频连麦模式确定第二布局信息，其中，上述第二布局信息用于确定上述目标视频流在上述图形用户界面内至少一个显示区域待显示的第二视频内容；In step S14, second layout information is determined based on the above-mentioned second video linking mode, wherein the above-mentioned second layout information is used to determine the second video content to be displayed in at least one display area of the above-mentioned target video stream in the above-mentioned graphical user interface ;
在步骤S15中，按照上述第二布局信息将上述第一视频内容切换为上述第二视频内 容。In step S15, the first video content is switched to the second video content according to the second layout information.
在一些实施例中，上述待使用的视频连麦模式可以为单人连麦，PK对战，多人连麦等任意一种类型的视频连麦模式。In some embodiments, the video linking mode to be used may be any type of video linking mode, such as single-player linking, PK battle, and multi-player linking.
在一些实施例中，上述图形用户界面可以为连麦视频界面、连麦语音聊天界面、连麦PK界面等，上述布局信息可以包括：对称布局信息或非对称布局信息，例如，上下对称布局信息，左右对称布局信息，上下非对称布局信息，左右非对称布局信息，等等。In some embodiments, the above-mentioned graphical user interface may be a Lianmai video interface, a Lianmai voice chat interface, a Lianmai PK interface, etc., and the above-mentioned layout information may include: symmetrical layout information or asymmetrical layout information, for example, upper and lower symmetrical layout information , left-right symmetrical layout information, top-bottom asymmetrical layout information, left-right asymmetrical layout information, and so on.
在一些实施例中，在本公共实施例中，上述待显示的至少一个显示区域中每个显示区域的设置比例可以根据用户个性化需求进行设置，也可以默认等比例或者非等比例设置，例如，将连麦视频PK对手的显示区域设置为辅显示区域，用户的显示区域设置为主显示区域，并且，主显示区域占图形用户界面中的比例大于辅显示区域占图形用户界面中的比例。In some embodiments, in this public embodiment, the setting ratio of each display area in the at least one display area to be displayed may be set according to the user's personalized needs, or may be set in equal or non-equal proportions by default, for example , the display area of Lianmai Video PK opponent is set as the auxiliary display area, the user's display area is set as the main display area, and the proportion of the main display area in the graphical user interface is greater than the proportion of the auxiliary display area in the graphical user interface.
图2是根据一示例性实施例示出的一种可选的视频流处理方法的流程图，以图2所示的连麦直播应用场景为例，主播端，嘉宾端和观众端基于视频帧对齐同步显示目标视频流，本公开实施例的执行主体为主播端，由于不同的视频连麦方式对应的显示区域的布局信息存在区别，本公开中通过响应作用于图形用户界面的第一触控操作，按照第一布局信息显示第一视频内容，在观众端接收到主播端发送的目标视频流之后，可以基于上述目标视频流对应的连麦模式确定对应的第二布局信息，并基于第二布局信息播放上述目标视频流在该图形用户界面中待显示的第二视频内容，进而在观众端播放目标视频流的过程中，主播端可以实时更改视频连麦模式，由于不同的视频连麦模式对应适配不同的布局信息，无需为每一种连麦模式单独配置布局信息。FIG. 2 is a flowchart of an optional video stream processing method according to an exemplary embodiment. Taking the Lianmai live broadcast application scenario shown in FIG. 2 as an example, the host, guest and audience are aligned based on video frames. The target video stream is displayed synchronously, and the execution body of the embodiment of the present disclosure is the host. Since there are differences in the layout information of the display areas corresponding to different video linking methods, in the present disclosure, the first touch operation acting on the graphical user interface is used in the present disclosure. , display the first video content according to the first layout information, after the viewer end receives the target video stream sent by the host end, can determine the corresponding second layout information based on the microphone connection mode corresponding to the above target video stream, and based on the second layout The information plays the second video content to be displayed in the above-mentioned target video stream in the graphical user interface, and then in the process of playing the target video stream on the audience side, the host can change the video linking mode in real time, because different video linking modes correspond to To adapt to different layout information, there is no need to configure layout information separately for each mic connection mode.
在一些实施例中，如图2a所示，在按照上述第二布局信息将上述第一视频内容切换为第二视频内容之前，上述视频流处理方法还包括：In some embodiments, as shown in FIG. 2a, before switching the first video content to the second video content according to the second layout information, the video stream processing method further includes:
在步骤S21中，对上述目标视频流中的视频关键帧进行解码处理，得到多元组，其中，上述多元组基于上述第二视频连麦模式将上述至少一路视频流和上述第二布局信息封装而成；In step S21, decoding the video key frames in the target video stream to obtain a tuple, wherein the tuple encapsulates the at least one video stream and the second layout information based on the second video linking mode. to make;
在步骤S22中，基于上述多元组确定上述第二布局信息。In step S22, the above-mentioned second layout information is determined based on the above-mentioned tuple.
在一些实施例中，上述第二布局信息包括：上述至少一个显示区域中每个显示区域的位置信息和尺寸信息。In some embodiments, the second layout information includes: position information and size information of each display area in the at least one display area.
在一些实施例中，参考图2所示的多人视频连麦直播的应用场景，当主播选择待使用的视频连麦模式后，或者有用户上麦(增加连麦用户)或者有用户下麦(减少连麦用户)时，可以根据视频连麦模式对应的布局规则，重新计算每个用户对应的显示区域的位置信息(x,y)和尺寸信息(w,h)，即得到第二布局信息，基于上述第二布局信息对上述图形用户界面内的当前显示内容进行更新。In some embodiments, referring to the application scenario of the multi-person video linking microphone live broadcast shown in FIG. 2 , after the host selects the video linking microphone mode to be used, either a user accesses the microphone (increases the connected microphone user) or a user downloads the microphone (reduce the users of connected microphones), the position information (x, y) and size information (w, h) of the display area corresponding to each user can be recalculated according to the layout rules corresponding to the video connected microphone mode, that is, the second layout can be obtained. information, and based on the second layout information, the currently displayed content in the graphical user interface is updated.
在一些实施例中，如图2b所示，按照上述第二布局信息将上述第一视频内容切换为第二视频内容包括：In some embodiments, as shown in FIG. 2b, switching the first video content to the second video content according to the second layout information includes:
在步骤S31中，基于上述至少一个显示区域中每个显示区域的位置信息和尺寸信息，将上述第一视频内容切换为上述第二视频内容。In step S31, the first video content is switched to the second video content based on the position information and size information of each display area in the at least one display area.
仍如图2所示，主播端基于上述第二视频连麦模式将所有嘉宾端的多路视频流和主播端本地的视频流，以及上述布局信息编码为目标视频流；主播编码的每一个视频关键帧(I帧)均基于上述第二视频连麦模式将上述至少一路视频流、上述位置信息和上述尺寸信息封装为多元组，例如，组装为五元组(x,y,w,h,slotId)并写入目标视频流的I帧中，并跟随目标视频流被发送至观众端。Still as shown in Figure 2, the anchor terminal encodes the multi-channel video streams of all guest terminals and the local video stream of the anchor terminal based on the above-mentioned second video linking mode, and the above-mentioned layout information as the target video stream; each video key encoded by the anchor. The frame (I frame) is based on the above-mentioned second video linking microphone mode to encapsulate the above-mentioned at least one video stream, the above-mentioned position information and the above-mentioned size information into a tuple, for example, assembled into a quintuple (x, y, w, h, slotId ) and written into the I frame of the target video stream, and sent to the viewer following the target video stream.
在一些实施例中，在本公开实施例中，观众端对上述目标视频流中的视频关键帧进行解码处理，从目标视频流的视频关键帧I帧中获取主播端写入的多元组(x,y,w,h,slotId)，基于该多元组用于确定与上述目标视频流对应的第二视频连麦模式，进而观众端可以基于上述第二视频连麦模式确定第二布局信息，并依据第二布局信息调整所有嘉宾中的每个嘉宾对应的显示区域。In some embodiments, in the embodiments of the present disclosure, the viewer performs decoding processing on the video key frame in the target video stream, and obtains the tuple (x) written by the host from the video key frame I frame of the target video stream. , y, w, h, slotId), based on this tuple for determining the second video linking microphone mode corresponding to the above-mentioned target video stream, and then the viewer can determine the second layout information based on the above-mentioned second video linking microphone mode, and The display area corresponding to each guest among all the guests is adjusted according to the second layout information.
通过本公开实施例，可以随时依据主播选择的视频连麦模式实时更新连麦布局，并且可以和视频流一起同步到观众端，在多人视频连麦中实时变换布局并立即生效，从而支持更灵活的玩法，并且可以统一不同的视频连麦技术方案，达到核心逻辑只需要开发一次，通过不同的布局配置就可以适配PK对战，单人连麦，多人连麦等形式，无需为每一种视频连麦模式单独开发，减少开发工作量和维护成本。Through the embodiments of the present disclosure, the mic layout can be updated in real time according to the video mic mode selected by the host at any time, and can be synchronized with the video stream to the viewer, and the layout can be changed in real time in the multi-person video mic and take effect immediately, thereby supporting more Flexible gameplay, and can unify different video linking technology solutions, only need to develop once to achieve the core logic, and can adapt to PK battles, single-player linking, multi-player linking, etc. through different layout configurations. A video connected microphone mode is independently developed, reducing development workload and maintenance costs.
图3是根据一示例性实施例示出的另一种视频流处理方法的流程图，如图3所示，视频流处理方法，应用于服务器，并且包括以下步骤：FIG. 3 is a flowchart of another video stream processing method according to an exemplary embodiment. As shown in FIG. 3 , the video stream processing method is applied to a server and includes the following steps:
在步骤S41中，获取当前使用的第一视频连麦模式和待使用的第二视频连麦模式，其中，上述第一视频连麦模式用于按照对应的第一布局信息显示第一视频内容，上述第二视频连麦模式用于确定图形用户界面内待显示的至少一个显示区域中每个显示区域的第二布局信息；In step S41, the currently used first video linking microphone mode and the second video linking microphone mode to be used are obtained, wherein the above-mentioned first video linking microphone mode is used to display the first video content according to the corresponding first layout information, The above-mentioned second video continuous microphone mode is used to determine the second layout information of each display area in the at least one display area to be displayed in the graphical user interface;
在步骤S42中，基于上述第二视频连麦模式对应的布局规则计算上述至少一个显示区域中每个显示区域的上述第二布局信息；In step S42, the above-mentioned second layout information of each display area in the above-mentioned at least one display area is calculated based on the layout rule corresponding to the above-mentioned second video continuous microphone mode;
在步骤S43中，基于上述第二视频连麦模式将至少一路视频流和上述第二布局信息编码为目标视频流；In step S43, encoding at least one video stream and the above-mentioned second layout information into a target video stream based on the above-mentioned second video linking mode;
在步骤S44中，将上述目标视频流发送至目标客户端，其中，上述目标客户端用于对上述目标视频流进行解码处理，得到上述第二布局信息，并按照上述第二布局信息将上述第一视频内容切换为第二视频内容。In step S44, the target video stream is sent to the target client, wherein the target client is used for decoding the target video stream to obtain the second layout information, and according to the second layout information A video content is switched to a second video content.
在一些实施例中，上述待使用的视频连麦模式可以为单人连麦，PK对战，多人连麦等任意一种类型的视频连麦模式。In some embodiments, the video linking mode to be used may be any type of video linking mode, such as single-player linking, PK battle, and multi-player linking.
仍以图2所示的连麦直播应用场景为例，主播端，嘉宾端和观众端基于视频帧对齐同步显示目标视频流，本公开实施例的执行主体为观众端，由于不同的视频连麦方式对应的显示区域的布局信息存在区别，本公开中通过响应作用于图形用户界面的第一触控操作， 按照第一布局信息显示第一视频内容，在观众端接收到主播端发送的目标视频流之后，可以基于上述目标视频流对应的连麦模式确定对应的第二布局信息，并基于第二布局信息播放上述目标视频流在该图形用户界面中待显示的第二视频内容，进而在观众端播放目标视频流的过程中，主播端可以实时更改视频连麦模式，由于不同的视频连麦模式对应适配不同的布局信息，无需为每一种连麦模式单独配置布局信息。Still taking the live broadcast application scenario shown in FIG. 2 as an example, the host, the guest and the audience display the target video stream synchronously based on video frame alignment. The execution subject of the embodiment of the present disclosure is the audience. There are differences in the layout information of the display area corresponding to the mode. In the present disclosure, the first video content is displayed according to the first layout information in response to the first touch operation acting on the graphical user interface, and the target video sent by the host is received at the viewer end. After streaming, the corresponding second layout information can be determined based on the microphone connection mode corresponding to the above-mentioned target video stream, and based on the second layout information, the second video content to be displayed in the graphical user interface of the above-mentioned target video stream can be played, and then the second video content to be displayed in the GUI of the above-mentioned target video stream can be played. In the process of playing the target video stream on the client, the host can change the video linking mode in real time. Since different video linking modes correspond to different layout information, there is no need to configure the layout information separately for each linking mode.
在一些实施例中，上述图形用户界面可以为连麦视频界面、连麦语音聊天界面、连麦PK界面等，上述布局信息可以包括：对称布局信息或非对称布局信息，例如，上下对称布局信息，左右对称布局信息，上下非对称布局信息，左右非对称布局信息，等等。In some embodiments, the above-mentioned graphical user interface may be a Lianmai video interface, a Lianmai voice chat interface, a Lianmai PK interface, etc., and the above-mentioned layout information may include: symmetrical layout information or asymmetrical layout information, for example, upper and lower symmetrical layout information , left-right symmetrical layout information, top-bottom asymmetrical layout information, left-right asymmetrical layout information, and so on.
在一些实施例中，在本公共实施例中，上述待显示的至少一个显示区域中每个显示区域的设置比例可以根据用户个性化需求进行设置，也可以默认等比例或者非等比例设置，例如，将连麦视频PK对手的显示区域设置为辅显示区域，用户的显示区域设置为主显示区域，并且，主显示区域占图形用户界面中的比例大于辅显示区域占图形用户界面中的比例。In some embodiments, in this public embodiment, the setting ratio of each display area in the at least one display area to be displayed may be set according to the user's personalized needs, or may be set in equal or non-equal proportions by default, for example , the display area of Lianmai Video PK opponent is set as the auxiliary display area, the user's display area is set as the main display area, and the proportion of the main display area in the graphical user interface is greater than the proportion of the auxiliary display area in the graphical user interface.
在一些实施例中，上述第二布局信息包括：上述至少一个显示区域中每个显示区域的位置信息和尺寸信息。In some embodiments, the second layout information includes: position information and size information of each display area in the at least one display area.
在一些实施例中，如图3a所示，基于上述第二视频连麦模式将上述至少一路视频流和上述第二布局信息编码为上述目标视频流包括：In some embodiments, as shown in FIG. 3a, encoding the at least one video stream and the second layout information into the target video stream based on the second video concatenation mode includes:
在步骤S51中，基于上述第二视频连麦模式将上述至少一路视频流、上述位置信息和上述尺寸信息封装为多元组；In step S51, the above-mentioned at least one video stream, the above-mentioned position information and the above-mentioned size information are encapsulated into a tuple based on the above-mentioned second video linking microphone mode;
在步骤S52中，将上述多元组编码至视频关键帧，得到上述目标视频流。In step S52, the above-mentioned tuple is encoded into a video key frame to obtain the above-mentioned target video stream.
仍如图2所示，主播端基于上述第二视频连麦模式将所有嘉宾端的多路视频流和主播端本地的视频流汇成新的待展示的视频流，根据待展示的视频流以及相应的布局信息对视频流进行编码；主播端编码的每一个视频关键帧(I帧)均基于上述第二视频连麦模式将上述至少一路视频流、上述位置信息和上述尺寸信息封装为多元组，例如，组装为五元组(x,y,w,h,slotId)并写入目标视频流的I帧中，并跟随目标视频流被发送至观众端。Still as shown in Figure 2, the anchor terminal merges the multi-channel video streams of all guest terminals and the local video stream of the anchor terminal into a new video stream to be displayed based on the above-mentioned second video linking mode. According to the video stream to be displayed and the corresponding The layout information encodes the video stream; each video key frame (I frame) encoded by the anchor side encapsulates the above-mentioned at least one video stream, the above-mentioned position information and the above-mentioned size information into a tuple based on the above-mentioned second video linking microphone mode, For example, it is assembled into a quintuple (x, y, w, h, slotId) and written into the I frame of the target video stream, and sent to the viewer following the target video stream.
在一些实施例中，在本公开实施例中，观众端对上述目标视频流中的视频关键帧进行解码处理，从目标视频流的视频关键帧I帧中获取主播端写入的多元组(x,y,w,h,slotId)，基于该多元组用于确定与上述目标视频流对应的第二视频连麦模式，进而观众端可以基于上述第二视频连麦模式确定第二布局信息，并依据第二布局信息调整所有嘉宾中的每个嘉宾对应的显示区域。In some embodiments, in the embodiments of the present disclosure, the viewer performs decoding processing on the video key frame in the target video stream, and obtains the tuple (x) written by the host from the video key frame I frame of the target video stream. , y, w, h, slotId), based on this tuple for determining the second video linking microphone mode corresponding to the above-mentioned target video stream, and then the viewer can determine the second layout information based on the above-mentioned second video linking microphone mode, and The display area corresponding to each guest among all the guests is adjusted according to the second layout information.
在一些实施例中，参考图2所示的多人视频连麦直播的应用场景，当主播选择待使用的视频连麦模式后，或者有用户上麦(增加连麦用户)或者有用户下麦(减少连麦用户)时，可以根据视频连麦模式对应的布局规则，重新计算每个用户对应的显示区域的位置信息(x,y)和尺寸信息(w,h)，即得到第二布局信息，基于上述第二布局信息对上述图形用户界面内的当前显示内容进行更新。In some embodiments, referring to the application scenario of the multi-person video linking microphone live broadcast shown in FIG. 2 , after the host selects the video linking microphone mode to be used, either a user accesses the microphone (increases the connected microphone user) or a user downloads the microphone (reduce the users of connected microphones), the position information (x, y) and size information (w, h) of the display area corresponding to each user can be recalculated according to the layout rules corresponding to the video connected microphone mode, that is, the second layout can be obtained. information, and based on the second layout information, the currently displayed content in the graphical user interface is updated.
通过本公开实施例，可以随时依据主播选择的视频连麦模式实时更新连麦布局，并且可以和视频流一起同步到观众端，在多人视频连麦中实时变换布局并立即生效，从而支持更灵活的玩法，并且可以统一不同的视频连麦技术方案，达到核心逻辑只需要开发一次，通过不同的布局配置就可以适配PK对战，单人连麦，多人连麦等形式，无需为每一种视频连麦模式单独开发，减少开发工作量和维护成本。Through the embodiments of the present disclosure, the mic layout can be updated in real time according to the video mic mode selected by the host at any time, and can be synchronized with the video stream to the viewer, and the layout can be changed in real time in the multi-person video mic and take effect immediately, thereby supporting more Flexible gameplay, and can unify different video linking technology solutions, only need to develop once to achieve the core logic, and can adapt to PK battles, single-player linking, multi-player linking, etc. through different layout configurations. A video connected microphone mode is independently developed, reducing development workload and maintenance costs.
图4是根据一示例性实施例示出的一种视频流处理装置的框图。参照图4所示，该装置包括：Fig. 4 is a block diagram of a video stream processing apparatus according to an exemplary embodiment. Referring to Figure 4, the device includes:
显示单元401，被配置为执行响应作用于图形用户界面的第一触控操作，按照第一布局信息显示第一视频内容，其中，上述第一布局信息由当前使用的第一视频连麦模式确定；第一获取单元402，被配置为执行获取目标视频流；第一确定单元403，被配置为执行确定与上述目标视频流对应的第二视频连麦模式；第二确定单元404，被配置为执行基于上述第二视频连麦模式确定第二布局信息，其中，上述第二布局信息用于确定上述目标视频流在上述图形用户界面内至少一个显示区域待显示的第二视频内容；切换单元405，被配置为执行按照上述第二布局信息将上述第一视频内容切换为上述第二视频内容。The display unit 401 is configured to perform a first touch operation in response to a graphical user interface, and display the first video content according to the first layout information, wherein the above-mentioned first layout information is determined by the currently used first video linking mode ; The first acquisition unit 402 is configured to perform acquisition of the target video stream; the first determination unit 403 is configured to perform determination of the second video linking microphone mode corresponding to the above-mentioned target video stream; the second determination unit 404 is configured as Perform determining the second layout information based on the above-mentioned second video linking mode, wherein the above-mentioned second layout information is used to determine the above-mentioned second video content of the target video stream to be displayed in at least one display area in the above-mentioned graphical user interface; switching unit 405 is configured to perform switching of the first video content to the second video content according to the second layout information.
在一些实施例中，上述视频流处理装置包括：解码子单元，被配置为执行对上述目标视频流中的视频关键帧进行解码处理，得到多元组，其中，上述多元组基于上述第二视频连麦模式将上述至少一路视频流和上述布局信息封装而成；确定子单元，被配置为执行基于上述多元组确定上述布局信息。In some embodiments, the above-mentioned video stream processing apparatus includes: a decoding subunit configured to perform decoding processing on video key frames in the above-mentioned target video stream to obtain a tuple, wherein the above-mentioned tuple is based on the above-mentioned second video connection. The microphone mode is formed by encapsulating the above-mentioned at least one video stream and the above-mentioned layout information; the determination subunit is configured to perform the above-mentioned determination of the above-mentioned layout information based on the above-mentioned tuple.
在一些实施例中，上述布局信息包括：上述至少一个显示区域中每个显示区域的位置信息和尺寸信息。In some embodiments, the layout information includes: position information and size information of each display area in the at least one display area.
在一些实施例中，上述切换单元包括：更新单元，被配置为执行基于上述至少一个显示区域中每个显示区域的位置信息和尺寸信息对上述图形用户界面内的当前显示内容进行更新。In some embodiments, the above-mentioned switching unit includes: an updating unit configured to perform an update of the currently displayed content in the above-mentioned graphical user interface based on the position information and size information of each of the above-mentioned at least one display area.
图5是根据一示例性实施例示出的另一种视频流处理装置的框图。参照图5所示，该装置包括：Fig. 5 is a block diagram of another video stream processing apparatus according to an exemplary embodiment. Referring to Figure 5, the device includes:
第二获取单元50，被配置为执行获取当前使用的第一视频连麦模式和待使用的第二视频连麦模式，其中，上述第一视频连麦模式用于按照对应的第一布局信息显示第一视频内容，上述第二视频连麦模式用于确定图形用户界面内待显示的至少一个显示区域中每个显示区域的第二布局信息；计算单元51，被配置为执行基于上述第二视频连麦模式对应的布局规则计算上述至少一个显示区域中每个显示区域的上述第二布局信息；编码单元52，被配置为执行基于上述第二视频连麦模式将至少一路视频流和上述第二布局信息编码为目标视频流；发送单元53，被配置为执行将上述目标视频流发送至目标客户端，其中，上述目标客户端用于对上述目标视频流进行解码处理，得到上述第二布局信息，并按照上述第二布局信息将上述第一视频内容切换为第二视频内容。The second acquisition unit 50 is configured to perform acquisition of the currently used first video linking mic mode and the second video linking mic mode to be used, wherein the first video linking mic mode is used for displaying according to the corresponding first layout information The first video content, the above-mentioned second video linking microphone mode is used to determine the second layout information of each display area in the at least one display area to be displayed in the graphical user interface; the computing unit 51 is configured to perform an operation based on the above-mentioned second video The layout rule corresponding to the continuous microphone mode calculates the above-mentioned second layout information of each display area in the above-mentioned at least one display area; the encoding unit 52 is configured to execute the at least one video stream and the above-mentioned second video stream based on the above-mentioned second video continuous microphone mode. The layout information is encoded into a target video stream; the sending unit 53 is configured to execute sending the above-mentioned target video stream to a target client, wherein the above-mentioned target client is used for decoding the above-mentioned target video stream to obtain the above-mentioned second layout information , and switch the first video content to the second video content according to the second layout information.
在一些实施例中，上述第二布局信息包括：上述至少一个显示区域中每个显示区域的 位置信息和尺寸信息。In some embodiments, the above-mentioned second layout information includes: position information and size information of each display area in the above-mentioned at least one display area.
在一些实施例中，上述编码单元包括：封装子单元，被配置为执行基于上述第二视频连麦模式将上述至少一路视频流、上述位置信息和上述尺寸信息封装为多元组；编码子单元，被配置为执行将上述多元组编码至视频关键帧，得到上述目标视频流。In some embodiments, the encoding unit includes: an encapsulation subunit, configured to perform encapsulation of the at least one video stream, the location information, and the size information into a tuple based on the second video linking mode; the encoding subunit, is configured to perform encoding of the above-mentioned tuples into video key frames to obtain the above-mentioned target video stream.
关于上述实施例中的装置，其中各个模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述，此处将不做详细阐述说明。Regarding the apparatus in the above-mentioned embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be described in detail here.
图6是根据一示例性实施例示出的一种计算机设备的框图，在示例性实施例中，还提供了一种包括指令的存储介质，例如包括指令的存储器64，上述指令可由计算机设备的处理器62执行以完成上述任意一种视频流处理方法。可选地，存储介质可以是非临时性计算机可读存储介质，例如，上述非临时性计算机可读存储介质可以是ROM、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。FIG. 6 is a block diagram of a computer device according to an exemplary embodiment. In an exemplary embodiment, a storage medium including instructions, such as a memory 64 including instructions, is also provided, and the above-mentioned instructions can be processed by the computer device. The controller 62 executes to complete any one of the above-mentioned video stream processing methods. Alternatively, the storage medium may be a non-transitory computer-readable storage medium, for example, the above-mentioned non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device Wait.
根据本公开实施例，还提供一种计算机可读存储介质的实施例，当上述计算机可读存储介质中的指令由电子设备的处理器执行时，使得电子设备能够执行上述的视频流处理方法。According to an embodiment of the present disclosure, an embodiment of a computer-readable storage medium is also provided. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the above-mentioned video stream processing method.
根据本公开实施例，提供一种电子设备的实施例，包括：处理器；用于存储上述处理器可执行指令的存储器；其中，上述处理器被配置为执行上述指令，以实现任一项上述的视频流处理方法。According to an embodiment of the present disclosure, an embodiment of an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement any of the above video stream processing method.
根据本公开实施例，还提供一种计算机程序产品的实施例，包括计算机程序/指令，上述计算机程序/指令被处理器执行时实现任一项上述的视频流处理方法。According to an embodiment of the present disclosure, there is also provided an embodiment of a computer program product, comprising a computer program/instruction, and when the computer program/instruction is executed by a processor, any one of the foregoing video stream processing methods is implemented.
本公开所有实施例均可以单独被执行，也可以与其他实施例相结合被执行，均视为本公开要求的保护范围。All the embodiments of the present disclosure can be implemented independently or in combination with other embodiments, which are all regarded as the protection scope required by the present disclosure.
应当理解的是，本公开并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本公开的范围仅由所附的权利要求来限制。It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims (20)

  1. 一种视频流处理方法，其特征在于，包括：A video stream processing method, comprising:
    响应作用于图形用户界面的第一触控操作，按照第一布局信息显示第一视频内容，其中，所述第一布局信息由当前使用的第一视频连麦模式确定；In response to the first touch operation acting on the graphical user interface, display the first video content according to the first layout information, wherein the first layout information is determined by the currently used first video linking mode;
    获取目标视频流；Get the target video stream;
    确定与所述目标视频流对应的第二视频连麦模式；determining the second video linking microphone mode corresponding to the target video stream;
    基于所述第二视频连麦模式确定第二布局信息，其中，所述第二布局信息用于确定所述目标视频流在所述图形用户界面内至少一个显示区域待显示的第二视频内容；determining second layout information based on the second video linking mode, wherein the second layout information is used to determine the second video content to be displayed in at least one display area of the target video stream in the graphical user interface;
    按照所述第二布局信息将所述第一视频内容切换为所述第二视频内容。The first video content is switched to the second video content according to the second layout information.
  2. 根据权利要求1所述的视频流处理方法，其特征在于，所述视频流处理方法还包括：The video stream processing method according to claim 1, wherein the video stream processing method further comprises:
    对所述目标视频流中的视频关键帧进行解码处理，得到多元组，其中，所述多元组基于所述第二视频连麦模式将所述至少一路视频流和所述第二布局信息封装而成；Perform decoding processing on the video key frames in the target video stream to obtain a tuple, wherein the tuple is based on the second video linking mode by encapsulating the at least one video stream and the second layout information to form a tuple. to make;
    基于所述多元组确定所述第二布局信息。The second layout information is determined based on the tuple.
  3. 根据权利要求2所述的视频流处理方法，其特征在于，所述第二布局信息包括：所述至少一个显示区域中每个显示区域的位置信息和尺寸信息。The video stream processing method according to claim 2, wherein the second layout information comprises: position information and size information of each display area in the at least one display area.
  4. 根据权利要求2所述的视频流处理方法，其特征在于，按照所述第二布局信息将所述第一视频内容切换为第二视频内容包括：The video stream processing method according to claim 2, wherein switching the first video content to the second video content according to the second layout information comprises:
    基于所述至少一个显示区域中每个显示区域的位置信息和尺寸信息，将所述第一视频内容切换为所述第二视频内容。The first video content is switched to the second video content based on the position information and size information of each of the at least one display area.
  5. 一种视频流处理方法，其特征在于，包括：A video stream processing method, comprising:
    获取当前使用的第一视频连麦模式和待使用的第二视频连麦模式，其中，所述第一视频连麦模式用于按照对应的第一布局信息显示第一视频内容，所述第二视频连麦模式用于确定图形用户界面内待显示的至少一个显示区域中每个显示区域的第二布局信息；Obtain the currently used first video linking microphone mode and the second video linking microphone mode to be used, wherein the first video linking microphone mode is used to display the first video content according to the corresponding first layout information, and the second video linking microphone mode is used to display the first video content. The video connected microphone mode is used to determine the second layout information of each display area in the at least one display area to be displayed in the graphical user interface;
    基于所述第二视频连麦模式对应的布局规则计算所述至少一个显示区域中每个显示区域的所述第二布局信息；Calculate the second layout information of each display area in the at least one display area based on the layout rule corresponding to the second video continuous microphone mode;
    基于所述第二视频连麦模式将至少一路视频流和所述第二布局信息编码为目标视频流；encoding at least one video stream and the second layout information into a target video stream based on the second video linking mode;
    将所述目标视频流发送至目标客户端，其中，所述目标客户端用于对所述目标视频流进行解码处理，得到所述第二布局信息，并按照所述第二布局信息将所述第一视频内容切换为第二视频内容。Send the target video stream to the target client, wherein the target client is used to decode the target video stream to obtain the second layout information, and according to the second layout information The first video content is switched to the second video content.
  6. 根据权利要求5所述的视频流处理方法，其特征在于，所述第二布局信息包括：所述至少一个显示区域中每个显示区域的位置信息和尺寸信息。The video stream processing method according to claim 5, wherein the second layout information comprises: position information and size information of each display area in the at least one display area.
  7. 根据权利要求6所述的视频流处理方法，其特征在于，基于所述第二视频连麦模式将所述至少一路视频流和所述第二布局信息编码为所述目标视频流包括：The video stream processing method according to claim 6, wherein encoding the at least one video stream and the second layout information into the target video stream based on the second video linking mode comprises:
    基于所述第二视频连麦模式将所述至少一路视频流、所述位置信息和所述尺寸信息封 装为多元组；Encapsulating the at least one video stream, the location information and the size information into a tuple based on the second video link-to-mic mode;
    将所述多元组编码至视频关键帧，得到所述目标视频流。The tuple is encoded into video key frames to obtain the target video stream.
  8. 一种视频流处理装置，其特征在于，包括：A video stream processing device, comprising:
    显示单元，被配置为执行响应作用于图形用户界面的第一触控操作，按照第一布局信息显示第一视频内容，其中，所述第一布局信息由当前使用的第一视频连麦模式确定；a display unit configured to perform a first touch operation in response to a graphical user interface, and display the first video content according to first layout information, wherein the first layout information is determined by the currently used first video linking mode ;
    第一获取单元，被配置为执行获取目标视频流；a first acquisition unit, configured to perform acquisition of the target video stream;
    第一确定单元，被配置为执行确定与所述目标视频流对应的第二视频连麦模式；a first determination unit, configured to perform determination of a second video connection mode corresponding to the target video stream;
    第二确定单元，被配置为执行基于所述第二视频连麦模式确定第二布局信息，其中，所述第二布局信息用于确定所述目标视频流在所述图形用户界面内至少一个显示区域待显示的第二视频内容；a second determining unit, configured to perform determining second layout information based on the second video linking mode, wherein the second layout information is used to determine at least one display of the target video stream in the graphical user interface the second video content to be displayed in the area;
    切换单元，被配置为执行按照所述第二布局信息将所述第一视频内容切换为所述第二视频内容。A switching unit configured to perform switching of the first video content to the second video content according to the second layout information.
  9. 根据权利要求8所述的视频流处理装置，其特征在于，所述视频流处理装置包括：The video stream processing device according to claim 8, wherein the video stream processing device comprises:
    解码子单元，被配置为执行对所述目标视频流中的视频关键帧进行解码处理，得到多元组，其中，所述多元组基于所述第二视频连麦模式将所述至少一路视频流和所述布局信息封装而成；A decoding subunit, configured to perform decoding processing on the video key frames in the target video stream, to obtain a tuple, wherein the tuple is based on the second video linking mode to combine the at least one video stream and the The layout information is packaged;
    确定子单元，被配置为执行基于所述多元组确定所述布局信息。A determination subunit configured to perform determining the layout information based on the tuple.
  10. 根据权利要求8所述的视频流处理装置，其特征在于，所述布局信息包括：所述至少一个显示区域中每个显示区域的位置信息和尺寸信息。The video stream processing apparatus according to claim 8, wherein the layout information comprises: position information and size information of each display area in the at least one display area.
  11. 根据权利要求9所述的视频流处理装置，其特征在于，所述切换单元包括：The video stream processing apparatus according to claim 9, wherein the switching unit comprises:
    更新单元，被配置为执行基于所述至少一个显示区域中每个显示区域的位置信息和尺寸信息对所述图形用户界面内的当前显示内容进行更新。An updating unit configured to perform updating of currently displayed content in the graphical user interface based on position information and size information of each of the at least one display area.
  12. 一种视频流处理装置，其特征在于，包括：A video stream processing device, comprising:
    第二获取单元，被配置为执行获取当前使用的第一视频连麦模式和待使用的第二视频连麦模式，其中，所述第一视频连麦模式用于按照对应的第一布局信息显示第一视频内容，所述第二视频连麦模式用于确定图形用户界面内待显示的至少一个显示区域中每个显示区域的第二布局信息；The second acquiring unit is configured to perform acquisition of the currently used first video linking mic mode and the second video linking mic mode to be used, wherein the first video linking mic mode is used for displaying according to the corresponding first layout information the first video content, the second video linking mode is used to determine the second layout information of each display area in the at least one display area to be displayed in the graphical user interface;
    计算单元，被配置为执行基于所述第二视频连麦模式对应的布局规则计算所述至少一个显示区域中每个显示区域的所述第二布局信息；a computing unit, configured to calculate the second layout information of each display area in the at least one display area based on the layout rule corresponding to the second video continuous microphone mode;
    编码单元，被配置为执行基于所述第二视频连麦模式将至少一路视频流和所述第二布局信息编码为目标视频流；an encoding unit, configured to encode at least one video stream and the second layout information into a target video stream based on the second video linking mode;
    发送单元，被配置为执行将所述目标视频流发送至目标客户端，其中，所述目标客户端用于对所述目标视频流进行解码处理，得到所述第二布局信息，并按照所述第二布局信息将所述第一视频内容切换为第二视频内容。a sending unit, configured to execute sending the target video stream to a target client, wherein the target client is configured to perform decoding processing on the target video stream to obtain the second layout information, and execute the process according to the The second layout information switches the first video content to the second video content.
  13. 根据权利要求12所述的视频流处理装置，其特征在于，所述第二布局信息包括： 所述至少一个显示区域中每个显示区域的位置信息和尺寸信息。The video stream processing apparatus according to claim 12, wherein the second layout information comprises: position information and size information of each display area in the at least one display area.
  14. 根据权利要求13所述的视频流处理装置，其特征在于，所述编码单元包括：The video stream processing apparatus according to claim 13, wherein the encoding unit comprises:
    封装子单元，被配置为执行基于所述第二视频连麦模式将所述至少一路视频流、所述位置信息和所述尺寸信息封装为多元组；an encapsulation subunit, configured to perform encapsulation of the at least one video stream, the location information and the size information into a tuple based on the second video link-to-mic mode;
    编码子单元，被配置为执行将所述多元组编码至视频关键帧，得到所述目标视频流。An encoding subunit configured to perform encoding of the tuple into video key frames to obtain the target video stream.
  15. 一种非易失性计算机可读存储介质，其特征在于，当所述计算机可读存储介质中的指令由电子设备的处理器执行时，使得电子设备能够执行一种视频流处理方法，所述方法包括：A non-volatile computer-readable storage medium, characterized in that, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute a video stream processing method, and the Methods include:
    响应作用于图形用户界面的第一触控操作，按照第一布局信息显示第一视频内容，其中，所述第一布局信息由当前使用的第一视频连麦模式确定；In response to the first touch operation acting on the graphical user interface, display the first video content according to the first layout information, wherein the first layout information is determined by the currently used first video linking mode;
    获取目标视频流；Get the target video stream;
    确定与所述目标视频流对应的第二视频连麦模式；determining a second video linking microphone mode corresponding to the target video stream;
    基于所述第二视频连麦模式确定第二布局信息，其中，所述第二布局信息用于确定所述目标视频流在所述图形用户界面内至少一个显示区域待显示的第二视频内容；determining second layout information based on the second video linking mode, wherein the second layout information is used to determine the second video content to be displayed in at least one display area of the target video stream in the graphical user interface;
    按照所述第二布局信息将所述第一视频内容切换为所述第二视频内容。The first video content is switched to the second video content according to the second layout information.
  16. 一种非易失性计算机可读存储介质，其特征在于，当所述计算机可读存储介质中的指令由电子设备的处理器执行时，使得电子设备能够执行一种视频流处理方法，所述方法包括：A non-volatile computer-readable storage medium, characterized in that, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute a video stream processing method, and the Methods include:
    获取当前使用的第一视频连麦模式和待使用的第二视频连麦模式，其中，所述第一视频连麦模式用于按照对应的第一布局信息显示第一视频内容，所述第二视频连麦模式用于确定图形用户界面内待显示的至少一个显示区域中每个显示区域的第二布局信息；Obtain the currently used first video linking microphone mode and the second video linking microphone mode to be used, wherein the first video linking microphone mode is used to display the first video content according to the corresponding first layout information, and the second video linking microphone mode The video connected microphone mode is used to determine the second layout information of each display area in the at least one display area to be displayed in the graphical user interface;
    基于所述第二视频连麦模式对应的布局规则计算所述至少一个显示区域中每个显示区域的所述第二布局信息；Calculate the second layout information of each display area in the at least one display area based on the layout rule corresponding to the second video continuous microphone mode;
    基于所述第二视频连麦模式将至少一路视频流和所述第二布局信息编码为目标视频流；Encoding at least one video stream and the second layout information into a target video stream based on the second video linking mode;
    将所述目标视频流发送至目标客户端，其中，所述目标客户端用于对所述目标视频流进行解码处理，得到所述第二布局信息，并按照所述第二布局信息将所述第一视频内容切换为第二视频内容。Send the target video stream to the target client, wherein the target client is used to decode the target video stream to obtain the second layout information, and according to the second layout information The first video content is switched to the second video content.
  17. 一种电子设备，其特征在于，包括：An electronic device, comprising:
    处理器；processor;
    用于存储所述处理器可执行指令的存储器；a memory for storing the processor-executable instructions;
    其中，所述处理器被配置为执行所述指令，以实现一种视频流处理方法，所述方法包括：Wherein, the processor is configured to execute the instructions to implement a video stream processing method, the method comprising:
    响应作用于图形用户界面的第一触控操作，按照第一布局信息显示第一视频内容，其中，所述第一布局信息由当前使用的第一视频连麦模式确定；In response to the first touch operation acting on the graphical user interface, display the first video content according to the first layout information, wherein the first layout information is determined by the currently used first video linking mode;
    获取目标视频流；Get the target video stream;
    确定与所述目标视频流对应的第二视频连麦模式；determining a second video linking microphone mode corresponding to the target video stream;
    基于所述第二视频连麦模式确定第二布局信息，其中，所述第二布局信息用于确定所述目标视频流在所述图形用户界面内至少一个显示区域待显示的第二视频内容；determining second layout information based on the second video linking mode, wherein the second layout information is used to determine the second video content to be displayed in at least one display area of the target video stream in the graphical user interface;
    按照所述第二布局信息将所述第一视频内容切换为所述第二视频内容。The first video content is switched to the second video content according to the second layout information.
  18. 一种电子设备，其特征在于，包括：An electronic device, comprising:
    处理器；processor;
    用于存储所述处理器可执行指令的存储器；a memory for storing the processor-executable instructions;
    其中，所述处理器被配置为执行所述指令，以实现一种视频流处理方法，所述方法包括：Wherein, the processor is configured to execute the instructions to implement a video stream processing method, the method comprising:
    获取当前使用的第一视频连麦模式和待使用的第二视频连麦模式，其中，所述第一视频连麦模式用于按照对应的第一布局信息显示第一视频内容，所述第二视频连麦模式用于确定图形用户界面内待显示的至少一个显示区域中每个显示区域的第二布局信息；Obtain the currently used first video linking microphone mode and the second video linking microphone mode to be used, wherein the first video linking microphone mode is used to display the first video content according to the corresponding first layout information, and the second video linking microphone mode The video connected microphone mode is used to determine the second layout information of each display area in the at least one display area to be displayed in the graphical user interface;
    基于所述第二视频连麦模式对应的布局规则计算所述至少一个显示区域中每个显示区域的所述第二布局信息；Calculate the second layout information of each display area in the at least one display area based on the layout rule corresponding to the second video continuous microphone mode;
    基于所述第二视频连麦模式将至少一路视频流和所述第二布局信息编码为目标视频流；encoding at least one video stream and the second layout information into a target video stream based on the second video linking mode;
    将所述目标视频流发送至目标客户端，其中，所述目标客户端用于对所述目标视频流进行解码处理，得到所述第二布局信息，并按照所述第二布局信息将所述第一视频内容切换为第二视频内容。Send the target video stream to the target client, wherein the target client is used to decode the target video stream to obtain the second layout information, and according to the second layout information The first video content is switched to the second video content.
  19. 一种计算机程序产品，包括计算机程序/指令，其特征在于，所述计算机程序/指令被处理器执行时实现一种视频流处理方法，所述方法包括：A computer program product, comprising a computer program/instruction, characterized in that, when the computer program/instruction is executed by a processor, a video stream processing method is implemented, and the method includes:
    响应作用于图形用户界面的第一触控操作，按照第一布局信息显示第一视频内容，其中，所述第一布局信息由当前使用的第一视频连麦模式确定；In response to the first touch operation acting on the graphical user interface, display the first video content according to the first layout information, wherein the first layout information is determined by the currently used first video linking mode;
    获取目标视频流；Get the target video stream;
    确定与所述目标视频流对应的第二视频连麦模式；determining a second video linking microphone mode corresponding to the target video stream;
    基于所述第二视频连麦模式确定第二布局信息，其中，所述第二布局信息用于确定所述目标视频流在所述图形用户界面内至少一个显示区域待显示的第二视频内容；determining second layout information based on the second video linking mode, wherein the second layout information is used to determine the second video content to be displayed in at least one display area of the target video stream in the graphical user interface;
    按照所述第二布局信息将所述第一视频内容切换为所述第二视频内容。The first video content is switched to the second video content according to the second layout information.
  20. 一种计算机程序产品，包括计算机程序/指令，其特征在于，所述计算机程序/指令被处理器执行时实现一种视频流处理方法，所述方法包括：A computer program product, comprising a computer program/instruction, characterized in that, when the computer program/instruction is executed by a processor, a video stream processing method is implemented, and the method includes:
    获取当前使用的第一视频连麦模式和待使用的第二视频连麦模式，其中，所述第一视频连麦模式用于按照对应的第一布局信息显示第一视频内容，所述第二视频连麦模式用于确定图形用户界面内待显示的至少一个显示区域中每个显示区域的第二布局信息；Obtain the currently used first video linking microphone mode and the second video linking microphone mode to be used, wherein the first video linking microphone mode is used to display the first video content according to the corresponding first layout information, and the second video linking microphone mode The video connected microphone mode is used to determine the second layout information of each display area in the at least one display area to be displayed in the graphical user interface;
    基于所述第二视频连麦模式对应的布局规则计算所述至少一个显示区域中每个显示 区域的所述第二布局信息；Calculate the second layout information of each display area in the at least one display area based on the layout rule corresponding to the second video continuous microphone mode;
    基于所述第二视频连麦模式将至少一路视频流和所述第二布局信息编码为目标视频流；encoding at least one video stream and the second layout information into a target video stream based on the second video linking mode;
    将所述目标视频流发送至目标客户端，其中，所述目标客户端用于对所述目标视频流进行解码处理，得到所述第二布局信息，并按照所述第二布局信息将所述第一视频内容切换为第二视频内容。Send the target video stream to the target client, wherein the target client is used to decode the target video stream to obtain the second layout information, and according to the second layout information The first video content is switched to the second video content.
PCT/CN2022/076010 2021-04-02 2022-02-11 Video stream processing method and apparatus WO2022206185A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110363735.0A CN113115119A (en) 2021-04-02 2021-04-02 Video stream processing method, device and storage medium
CN202110363735.0 2021-04-02

Publications (1)

Publication Number Publication Date
WO2022206185A1 true WO2022206185A1 (en) 2022-10-06

Family

ID=76713846

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/076010 WO2022206185A1 (en) 2021-04-02 2022-02-11 Video stream processing method and apparatus

Country Status (2)

Country Link
CN (1) CN113115119A (en)
WO (1) WO2022206185A1 (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113115119A (en) * 2021-04-02 2021-07-13 北京达佳互联信息技术有限公司 Video stream processing method, device and storage medium
CN114125486B (en) * 2021-12-01 2023-11-07 杭州米络星科技(集团)有限公司 Wheat connecting scheduling method and device and electronic equipment
CN114327732B (en) * 2021-12-31 2024-04-19 北京五八信息技术有限公司 Page configuration method, page configuration device, electronic equipment and computer readable medium
CN115334353B (en) * 2022-08-11 2024-03-12 北京达佳互联信息技术有限公司 Information display method, device, electronic equipment and storage medium

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108574879A (en) * 2018-03-30 2018-09-25 乐蜜有限公司 Joint live broadcasting method, device and electronic equipment
CN108900921A (en) * 2018-07-20 2018-11-27 广州酷狗计算机科技有限公司 Even wheat live broadcasting method, device and storage medium
CN109688419A (en) * 2018-12-27 2019-04-26 北京潘达互娱科技有限公司 Company's wheat method, apparatus and server in a kind of live streaming
US10530731B1 (en) * 2016-03-28 2020-01-07 Snap Inc. Systems and methods for chat with audio and video elements
CN111083507A (en) * 2019-12-09 2020-04-28 广州酷狗计算机科技有限公司 Method and system for connecting to wheat, first main broadcasting terminal, audience terminal and computer storage medium
CN111711833A (en) * 2020-07-28 2020-09-25 广州华多网络科技有限公司 Live video stream push control method, device, equipment and storage medium
CN112243137A (en) * 2019-07-17 2021-01-19 北京密境和风科技有限公司 Live broadcast interface updating method, device, server and system
CN113115119A (en) * 2021-04-02 2021-07-13 北京达佳互联信息技术有限公司 Video stream processing method, device and storage medium

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109640191A (en) * 2017-10-09 2019-04-16 武汉斗鱼网络科技有限公司 A kind of method and apparatus of even wheat live streaming
CN110401820A (en) * 2019-08-15 2019-11-01 北京迈格威科技有限公司 Multipath video processing method, device, medium and electronic equipment

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10530731B1 (en) * 2016-03-28 2020-01-07 Snap Inc. Systems and methods for chat with audio and video elements
CN108574879A (en) * 2018-03-30 2018-09-25 乐蜜有限公司 Joint live broadcasting method, device and electronic equipment
CN108900921A (en) * 2018-07-20 2018-11-27 广州酷狗计算机科技有限公司 Even wheat live broadcasting method, device and storage medium
CN109688419A (en) * 2018-12-27 2019-04-26 北京潘达互娱科技有限公司 Company's wheat method, apparatus and server in a kind of live streaming
CN112243137A (en) * 2019-07-17 2021-01-19 北京密境和风科技有限公司 Live broadcast interface updating method, device, server and system
CN111083507A (en) * 2019-12-09 2020-04-28 广州酷狗计算机科技有限公司 Method and system for connecting to wheat, first main broadcasting terminal, audience terminal and computer storage medium
CN111711833A (en) * 2020-07-28 2020-09-25 广州华多网络科技有限公司 Live video stream push control method, device, equipment and storage medium
CN113115119A (en) * 2021-04-02 2021-07-13 北京达佳互联信息技术有限公司 Video stream processing method, device and storage medium

Also Published As

Publication number Publication date
CN113115119A (en) 2021-07-13

Similar Documents

Publication Publication Date Title
WO2022206185A1 (en) Video stream processing method and apparatus
US11701587B2 (en) Methods and systems for rendering and encoding content for online interactive gaming sessions
JP5896389B2 (en) Graphics rendering alignment for displays
US9549152B1 (en) Application content delivery to multiple computing environments using existing video conferencing solutions
US9272220B2 (en) System and method for improving the graphics performance of hosted applications
US9682318B2 (en) System and method for improving the graphics performance of hosted applications
CN110267064B (en) Audio playing state processing method, device, equipment and storage medium
US8961316B2 (en) System and method for improving the graphics performance of hosted applications
JP2012521721A5 (en)
US20220193540A1 (en) Method and system for a cloud native 3d scene game
JP5913554B2 (en) System and method for transmitting media over a network
KR20120119504A (en) System for servicing game streaming according to game client device and method
US20220210484A1 (en) Method for processing live broadcast data, system, electronic device, and storage medium
US8845434B2 (en) System and method for improving the graphics performance of hosted applications
JP6306089B2 (en) System and method for transmitting media over a network
US20220369000A1 (en) Split rendering of extended reality data over 5g networks
US8851999B2 (en) System and method for improving the graphics performance of hosted applications
EP4338424A1 (en) Split rendering of extended reality data over 5g networks
WO2016107174A1 (en) Method and system for processing multimedia file data, player and client
CN113747181A (en) Network live broadcast method, live broadcast system and electronic equipment based on remote desktop
KR102545468B1 (en) IPTV set-top box capable of providing cloud game service and method of providing cloud game service in IPTV set-top box
TWI615803B (en) System and method for improving the graphics performance of hosted applications
US20180361236A1 (en) Method for playing back applications from a cloud, telecommunication network for streaming and for replaying applications (apps) via a specific telecommunication system, and use of a telecommunication network for streaming and replaying applications (apps)
WO2013040261A1 (en) System and method for improving the graphics performance of hosted applications

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22778371

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22778371

Country of ref document: EP

Kind code of ref document: A1