CN113382280B - Method and device for interactive live broadcast system and electronic equipment - Google Patents
Method and device for interactive live broadcast system and electronic equipment Download PDFInfo
- Publication number
- CN113382280B CN113382280B CN202110922055.8A CN202110922055A CN113382280B CN 113382280 B CN113382280 B CN 113382280B CN 202110922055 A CN202110922055 A CN 202110922055A CN 113382280 B CN113382280 B CN 113382280B
- Authority
- CN
- China
- Prior art keywords
- resolution
- guest
- video picture
- main
- pushing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000002452 interceptive effect Effects 0.000 title claims abstract description 103
- 238000000034 method Methods 0.000 title claims abstract description 60
- 230000011664 signaling Effects 0.000 claims description 16
- 230000006854 communication Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 8
- 230000005540 biological transmission Effects 0.000 description 7
- 238000001514 detection method Methods 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000003993 interaction Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000009977 dual effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234363—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234381—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
- H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The application provides a method and a device for an interactive live broadcast system and electronic equipment, wherein the interactive live broadcast system comprises at least one main speaking end, a first guest end, an interactive live broadcast server and a watching end, and the method and the device are used for the interactive live broadcast server and comprise the steps of receiving a main speaking and streaming video picture of the main speaking end, wherein the main speaking and streaming video picture comprises a main speaking and streaming video picture with a first resolution and a main speaking and streaming video picture with a second resolution, and the first resolution is greater than the second resolution; receiving a first guest push streaming video picture of the first guest end, wherein the first guest push streaming video picture comprises a guest push streaming video picture of a third resolution and a guest push streaming video picture of a fourth resolution, and the third resolution is greater than the fourth resolution; pushing the guest at the fourth resolution to the main speaking end to push the video picture; pushing a main speaking video picture with a second resolution to the first guest terminal; and pushing the main speaking push streaming video picture with the first resolution and the guest push streaming video picture with the third resolution to a viewing end.
Description
Technical Field
The application relates to the field of live video, in particular to a method and a device for an interactive live broadcast system and electronic equipment.
Background
The traditional audio and video communication conversation process comprises the steps that a main speaking end and a guest end respectively carry out signaling transmission with a server, an initiating end audio and video coding, an audio and video data transmission, a receiving end audio and video decoding and the communication session of the main speaking end and the guest end is respectively ended. The main steps influencing the audio and video quality can be seen from the whole transmission process to be in the audio and video coding, transmission and decoding processes, and the factors influencing the three aspects comprise the equipment performance, the stream pushing resolution and the network bandwidth of the main speaking end and the guest end.
A data flow diagram of a conventional audio-video interactive communication process is shown in fig. 1. Audio is usually ignored because it occupies a low bandwidth during transmission. Taking the video stream 480p resolution as an example, each participant will stream one path of 480p video data and receive two paths of 480p video data during the communication process. If a participant is added, each receiver adds a new path of 480p video data. Therefore, the video data is received (N-1) paths 480P and transmitted 1 paths 480P, decoded (N-1) paths 480P and encoded 1 paths 480P when participating in the communication.
Thus, if more participants are involved, the greater the network bandwidth required. Under the condition of a certain bandwidth, the conversation quality of a video receiver is affected along with the increase of the participants. Meanwhile, as the amount of received video data increases, the requirements on the performance of the devices at the main speaking end and the guest end are improved during video decoding, and the decoding efficiency is influenced, so that the fluency of live sound broadcasting is influenced.
Disclosure of Invention
The application provides a method and a device for an interactive live broadcast system and electronic equipment, which are used for solving the influence of equipment performance and network bandwidth on audio and video communication in the audio and video interactive communication process.
According to an aspect of the present application, a method for an interactive live broadcast system is provided, where the interactive live broadcast system includes at least one main speaking end, a first guest end, an interactive live broadcast server, and a viewing end, and the method is used for the interactive live broadcast server, where the method includes receiving main speaking and streaming video pictures of the main speaking end, where the main speaking and streaming video pictures include a main speaking and streaming video picture with a first resolution and a main speaking and streaming video picture with a second resolution, and the first resolution is greater than the second resolution; receiving a first guest push streaming video picture of the first guest terminal, wherein the first guest push streaming video picture comprises a guest push streaming video picture of a third resolution and a guest push streaming video picture of a fourth resolution, and the third resolution is greater than the fourth resolution; pushing the guest at the fourth resolution to the main speaking end to push and stream the video picture; pushing the main speaking push streaming video picture with the second resolution to the first guest terminal; and pushing the main speaking push streaming video picture with the first resolution and the guest push streaming video picture with the third resolution to the viewing end.
According to some embodiments, the interactive live broadcasting system further comprises a second guest terminal, the method further comprises receiving a second guest push stream video picture of the second guest terminal, the second guest push stream video picture comprising a guest push stream video picture of a fifth resolution and a guest push stream video picture of a sixth resolution, the fifth resolution being greater than the sixth resolution; pushing and streaming the guest push streaming video picture with the sixth resolution to the first guest terminal; pushing the main speaking push-streaming video picture with the second resolution to the second guest terminal; and pushing and streaming the guest push streaming video picture of the fourth resolution to the second guest terminal.
According to some embodiments, the first resolution is greater than the third resolution; the second resolution is greater than the fourth resolution.
According to some embodiments, the third resolution and the fifth resolution are the same; the fourth resolution and the sixth resolution are the same.
According to some embodiments, the first resolution comprises a 480p resolution, and the second resolution comprises a 240p resolution; or the first resolution comprises a 360p resolution and the second resolution comprises a 180p resolution; or the first resolution comprises a 240p resolution and the second resolution comprises a 120p resolution.
According to some embodiments, the third resolution comprises a 240p resolution and the fourth resolution comprises a 120p resolution.
According to some embodiments, the method further comprises sending first role switching information to the master speaker, the first role switching information comprising switching the master speaker to the first guest; and/or sending second role switching information to the first guest terminal, wherein the second role switching information comprises switching the first guest terminal into the main speaking terminal.
According to some embodiments, before streaming the guest push streaming video picture of the fourth resolution to the master speaker, the method further comprises receiving first signaling information of the master speaker, the first signaling information comprising a resolution of the guest push streaming video picture received by the master speaker.
According to some embodiments, before streaming the main push streaming video picture of the second resolution to the first guest, the method further comprises receiving second signaling information of the first guest, the second signaling information comprising the resolution of the main push streaming video picture received by the first guest.
According to some embodiments, the streaming the main push streaming video picture of the first resolution and the guest push streaming video picture of the third resolution to the viewing end comprises mixing the main push streaming video picture of the first resolution and the guest push streaming video picture of the third resolution to obtain a bypass mixed stream video picture; and pushing and flowing the bypass mixed flow video picture to the watching end.
According to an aspect of the present application, a method for an interactive live broadcast system is provided, where the interactive live broadcast system includes at least one main speaking end, a first guest end, an interactive live broadcast server, and a viewing end, and the method is used for the main speaking end, where the method includes streaming a main speaking streaming video picture to the interactive live broadcast server, where the main speaking streaming video picture includes a main speaking streaming video picture with a first resolution and a main speaking streaming video picture with a second resolution, and the first resolution is greater than the second resolution; and receiving a first guest push stream video picture of the first guest end push stream from the interactive live broadcast server.
According to some embodiments, the method further comprises periodically detecting a risk value of the device performance of the main speaking end, wherein the risk value comprises a CPU occupancy rate; if the risk value is higher than a risk threshold value, the resolution of the main speaking end is reduced; and if the current resolution of the main speaking end is the resolution threshold, carrying out risk early warning.
According to an aspect of the application, a device for an interactive live broadcast system is provided, where the interactive live broadcast system includes at least one main speaking end, a guest end, an interactive live broadcast server, and a viewing end, the device includes a main speaking end stream pushing receiving module configured to receive main speaking stream pushing video pictures of the main speaking end, where the main speaking stream pushing video pictures include a main speaking stream pushing video picture with a first resolution and a main speaking stream pushing video picture with a second resolution, and the first resolution is greater than the second resolution; the guest end video pushing module is used for receiving guest pushing video pictures of the guest end, wherein the guest pushing video pictures comprise a guest pushing video picture with a third resolution and a guest pushing video picture with a fourth resolution, and the third resolution is higher than the fourth resolution; the stream pushing main speaking end module is used for pushing the guest stream pushing video picture with the fourth resolution to the main speaking end; the stream pushing guest end module is used for pushing the main speaking stream video picture with the second resolution to the guest end; and the stream pushing and watching end module is used for pushing and streaming the main speaking stream pushing video picture with the first resolution and the guest stream pushing video picture with the third resolution to the watching end.
According to an aspect of the application, an electronic device is presented, comprising one or more processors; storage means for storing one or more programs; when executed by the one or more processors, cause the one or more processors to implement a method as in any one of the preceding claims.
According to some example embodiments of the application, the main speaking end and the guest end respectively push two paths of audio and video data with different stream resolutions to the interactive live broadcast system server. The interactive live broadcast server pushes the low-resolution push stream video picture of the guest end and the low-resolution push stream video picture of the guest end to the main speaking end respectively, the push stream data and the received data in the interactive live broadcast system are reduced, the decoding workload of the main speaking end and the guest end is reduced, the power consumption of equipment is reduced, the main speaking end and the guest end are pushed with large stream mixed stream video pictures, and the definition of the push stream video picture of the viewing end is ensured.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments will be briefly introduced below.
Fig. 1 shows a data flow diagram of a conventional audio-video interactive communication process.
Fig. 2 shows a data flow diagram of an audio-video interactive communication process according to an example embodiment of the present application.
Fig. 3 shows a flow chart of a method for an interactive live system according to an example embodiment of the present application.
Fig. 4 shows a flowchart of a method for automatically switching a stream-pushing resolution according to a timing detection CPU occupancy according to an example embodiment of the present application.
Fig. 5 shows a flow chart of a method for an interactive live system according to an example embodiment of the present application.
Fig. 6 shows a block diagram of an apparatus for an interactive live system according to an example embodiment of the present application.
Fig. 7 shows a block diagram of an electronic device according to an exemplary embodiment of the present application.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The same reference numerals denote the same or similar parts in the drawings, and thus, a repetitive description thereof will be omitted.
The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the embodiments of the disclosure can be practiced without one or more of the specific details, or with other means, components, materials, devices, or operations. In such cases, well-known structures, methods, devices, implementations, materials, or operations are not shown or described in detail.
The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
The terms "first," "second," and the like in the description and claims of the present application and in the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
Fig. 2 shows a data flow diagram of an audio-video interactive communication process according to an example embodiment of the present application. Referring to fig. 2, a data flow diagram of an audio-video interactive communication process according to an exemplary embodiment of the present application will be described in detail.
As shown in fig. 2, the main terminal pushes two paths of video frames, one path of 480P video frame and one path of 1 path of 240P video frame, to the interactive live broadcast server. The guest terminal comprises a guest A and a guest B, and the guest A and the guest B respectively push and stream two paths of video pictures to the interactive live broadcast server. Among them, 1 way 240P video picture and 1 way 120P video picture.
When the main speaking end and the guest end receive plug flow video data, the main speaking end and the interactive live broadcast server only receive 120P video pictures of the guests A and the guests B through signaling interaction, and the guest end and the interactive live broadcast server receive 240P video pictures of the main speaking and 120P video pictures of other guests through signaling interaction.
In order to ensure the definition of the watching end, the interactive live broadcast server mixes the 480P video picture of the main speaking end with the 240P video pictures of the guest A and the guest B, and sends the mixed pictures to the watching end through the CDN in a bypass mixed flow or RTMP mode.
As can be seen from comparing fig. 1 and fig. 2, when receiving data, assuming that N parties (including all the speakers at the speaker end and all the guests at the guest end) participate in the interaction, the total received data in the conventional scheme shown in fig. 1 is (N-1) × 480P, and the total received data in the dual-stream scheme according to the present application shown in fig. 2 is (N-1) × 120P. The amount of data received is reduced by (N-1) × (480P-120P) for the same maximum resolution.
When pushing streaming data, the speaker identity pushes one path of 240P picture. However, since the amount of received data is reduced, the CPU resources consumed for video decoding are reduced while the bandwidth requirements are reduced. Meanwhile, the push stream resolution of the guest end is only about 1/4 of the conventional scheme shown in fig. 1.
According to the technical scheme shown in fig. 2, due to the reduction of the total received data of the system, the decoding workload of the main speaker end and the guest end is reduced, and the power consumption of the device is reduced. Since the display area is small, the user does not feel that the image quality is significantly deteriorated although the resolution of the received data becomes small.
For the watching end, the interactive live broadcast server pushes and streams all pushed and streamed high-resolution video pictures of the main speaking end and the guest end in a mixed mode. Therefore, the technical scheme shown in fig. 2 does not affect the watching definition of the watching end.
Fig. 3 shows a flow chart of a method for an interactive live system according to an example embodiment of the present application. A method for an interactive live system according to an exemplary embodiment of the present application is described in detail below with reference to fig. 3.
According to some embodiments of the present application, an interactive live broadcast system includes at least one main speaker terminal, a first guest terminal, an interactive live broadcast server, and a viewing terminal.
The method shown in fig. 3 is used for an interactive live server.
Referring to fig. 3, in step S301, the interactive live server receives a main talk stream video frame of a main talk end.
According to some embodiments of the application, the main speaking end adopts a double-flow mechanism to push two paths of video pictures with different resolution ratios to the interactive live broadcast server. For example, the main streamlet video pictures include a main streamlet video picture of a first resolution and a main streamlet video picture of a second resolution. Wherein the first resolution is greater than the second resolution.
According to some embodiments, the main speaking end supports three definitions of high definition, standard definition and fluency, and each definition comprises a first resolution and a second resolution. For example, the first resolution of high definition is 480p and the second resolution is 240 p; the first resolution of standard definition is 360p, and the second resolution is 180 p; the first resolution for fluency is 240p and the second resolution is 120 p.
In step S303, the interactive live broadcast server receives a first guest push stream video frame of the first guest terminal.
According to some embodiments of the application, the first guest terminal pushes two paths of video pictures with different resolution ratios to the interactive live broadcast server by adopting a double-stream mechanism. For example, the first guest streamlet video picture includes a guest streamlet video picture of a third resolution and a guest streamlet video picture of a fourth resolution. Wherein the third resolution is greater than the fourth resolution.
The windows of the images of the main speaking end displayed on the main speaking end and the guest end are larger than the window of the image of the guest end. Therefore, the resolution of the video picture of the main speaking end plug flow is larger than that of the video picture of the guest end plug flow. According to some embodiments, the first resolution is greater than the third resolution and the second resolution is greater than the fourth resolution.
Because the main speaker and the guests are both displayed on the main speaker end and the guests end through the small windows, the main speaker end only needs to receive the plug-flow video picture with the fourth resolution of the guest end plug flow, and the guests end only needs to receive the plug-flow video picture with the second resolution of the main speaker end plug flow, so that the plug-flow data transmitted in the interactive live broadcast system are reduced.
Because guests are displayed in the small windows at the main speaking end and the guests, the requirement on the resolution is not high, and therefore the resolution of the plug-flow video picture transmitted by the guests can be lower than that of the plug-flow video picture transmitted by the main speaking end. According to some embodiments, the third resolution comprises a 240p resolution and the fourth resolution comprises 120 p.
In step S305, the main talk push streaming video frame of the second resolution is pushed to the first guest.
According to some embodiments, the main speaking end picture and the guest end picture are respectively displayed by small windows, so that the quality of the main speaking end picture is not obviously deteriorated by using the second resolution in the main speaking plug flow video picture.
According to some embodiments, before the main talk stream pushed video picture with the second resolution is pushed to the first guest terminal, the first guest terminal and the interactive live broadcast server perform signaling transmission to inform the interactive live broadcast server that the first guest terminal receives the resolution of the main talk stream pushed video picture of the main talk stream pushed by the first guest terminal.
In step S307, the guest at the fourth resolution is pushed to the main speaking end.
According to some embodiments, the guest side pictures are displayed in the small windows at the main speaker side and the guest side respectively, so that the fourth resolution in the guest push streaming video pictures is not perceived to be significantly degraded.
According to some embodiments, the window displaying the main speaking end in the main speaking end and the guest end is larger than the window displaying the guest end, so that the second resolution of the main speaking push streaming video picture pushed to the first guest end is larger than the fourth resolution of the guest push streaming video picture pushed to the main speaking end.
According to some embodiments, before the guest push streaming video picture with the fourth resolution is pushed to the main speaking end, the main speaking end and the interactive live broadcasting server perform signaling transmission to inform the interactive live broadcasting server of the resolution of the guest push streaming video picture received by the main speaking end.
In step S309, the main speaking push streaming video picture at the first resolution and the guest push streaming video picture at the third resolution are pushed to the viewing end.
According to some embodiments, the interactive live broadcast server mixes the main speaking push stream video picture of the first resolution at the main speaking end and the guest push stream video picture of the third resolution at the first guest end into a bypass mixed stream video picture, and pushes the mixed stream to the watching end.
According to some embodiments of the application, the main speaker receives all the push stream information of the guest terminals in the interactive live broadcast system, and the first guest terminal receives the push stream information of the main speaker and other guest terminals in the interactive live broadcast system.
According to some embodiments, the interactive live system further comprises a second guest terminal. And the interactive live broadcast server also receives a second guest push stream video picture of a second guest terminal. The second guest push stream video picture comprises a guest push stream video picture with a fifth resolution and a guest push stream video picture with a sixth resolution, and the fifth resolution is greater than the sixth resolution.
According to some embodiments, the interactive live broadcast server pushes and streams the guest push and stream video picture of the sixth resolution of the second guest to the first guest.
According to some embodiments, the interactive live broadcast server pushes the main speaking push streaming video picture of the second resolution of the main speaking end to the second guest end.
According to some embodiments, the interactive live broadcast server pushes and streams the guest push and stream video picture of the fourth resolution of the first guest to the second guest.
According to some embodiments of the present application, the master terminal and the first guest terminal may perform role exchange. According to some embodiments, the interactive live broadcast server sends a first role switching message to the main speaker to notify the main speaker to switch to the first guest. According to some embodiments, the interactive live server sends a second role switching message to the first guest, and notifies the first guest to switch to the master speaker.
According to some example embodiments of the application, the main speaking end and the guest end respectively push two paths of audio and video data with different stream resolutions to the interactive live broadcast system server. The interactive live broadcast server pushes the low-resolution push stream video picture of the guest end and the low-resolution push stream video picture of the guest end to the main speaking end and pushes the high-resolution push stream video picture of the main speaking end and the guest end to the watching end respectively, thereby reducing the push stream data and the received data in the interactive live broadcast system, reducing the decoding workload of the main speaking end and the guest end equipment, reducing the power consumption of the equipment and simultaneously ensuring the definition of the push stream video picture of the watching end.
Fig. 4 shows a flowchart of a method for automatically switching a stream-pushing resolution according to a timing detection CPU occupancy according to an example embodiment of the present application. A method for automatically switching the stream-pushing resolution according to the timing detection CPU occupancy will be described in detail with reference to fig. 4.
According to some embodiments of the present application, an interactive live broadcast system includes at least one main speaker terminal, a first guest terminal, an interactive live broadcast server, and a viewing terminal. The method shown in fig. 4 is used for the main speaking end.
According to some embodiments, the main speaker shown in fig. 4 streams the main speaker streaming video picture to the interactive live server. The main stream pushing video picture comprises a main stream pushing video picture with a first resolution and a main stream pushing video picture with a second resolution, and the first resolution is greater than the second resolution.
According to some embodiments, the master speaker further receives a first guest push streaming video picture of the first guest push streaming from the interactive live server.
According to some embodiments, when pushing streaming to the interactive live broadcast server, the main speaker detects the risk value of the device performance at regular time, and if the risk value is higher than the risk threshold, the resolution of the main speaker is reduced. And if the current resolution of the main speaking end is the resolution threshold, carrying out risk early warning.
Now, a method for automatically switching the stream pushing resolution according to the CPU occupancy by the main terminal will be described in detail, taking the CPU occupancy detected at a fixed time as an example.
According to some embodiments, three plug-flow resolution gears are set for the main speaking end, namely high definition, standard definition and fluency. Wherein the first and second resolutions of high definition are 480p resolution and 240p, respectively; the first resolution and the second resolution of the standard definition are respectively 360p resolution and 180p resolution; the fluent first and second resolutions are 240p resolution and 120p resolution, respectively.
In step S401, the main terminal periodically detects whether the current CPU occupancy is higher than the risk threshold. For example, every 5 seconds, it is checked whether the CPU occupancy is higher than 80%.
If the detected CPU occupancy is not greater than the risk threshold, timing detection continues.
If the detected CPU occupancy is higher than the risk threshold, step S403 is executed to detect whether the current resolution is the resolution threshold.
If the current resolution is detected to be the resolution threshold, for example, the current resolution is smooth, step S407 is executed to perform an early warning.
If the current resolution is not detected to be the resolution threshold, step S405 is executed to switch the resolution. According to some embodiments, if the current resolution is high definition, the current resolution is switched to standard definition; and if the current resolution is standard definition, switching the current resolution to be smooth.
As shown in fig. 4, after step S405 or step S407 is executed, the loop continues to execute step S401.
According to some embodiments of the present application, the push stream resolution is automatically switched through a timed detection of the performance of the main talker device to reduce the impact of the device performance on the encoding.
According to some embodiments, the talk-around end may also manually switch the push-stream resolution.
Fig. 5 shows a flow chart of a method for an interactive live system according to an example embodiment of the present application. The method shown in fig. 5 is used for the main speaker of the interactive live system.
As shown, in step S501, the main speaker pushes a stream to the interactive live server.
According to some embodiments of the present application, in step S501, the main speaker uses a dual stream mechanism to push two channels of video frames to the interactive live broadcast service. For example, one path of 480p video pictures and one path of 240p video pictures are pushed.
In step S503, the master speaker sends signaling to the interactive live broadcast server.
According to some embodiments, the signaling sent by the master terminal includes the resolution of the video picture of the guest terminal received by the master terminal. Through step 503, the host informs the interactive live broadcast server that the host receives only one video frame with lower resolution from the two video frames pushed to the interactive live broadcast server by the guest.
In step S505, the master terminal receives the guest push stream video frame.
According to some embodiments, the interactive live broadcast server receives the signaling sent in step S503, determines resolution information of the guest push stream video picture that the anchor terminal needs to receive, and the live broadcast terminal pushes one video picture with lower resolution in two video pictures of the guest push stream.
According to some embodiments of the present application, the master speaker uses a dual stream mechanism to push two channels of video frames to the interactive live broadcast server. Wherein, the video picture of one path is higher than the video picture of the other path.
Fig. 6 shows a block diagram of an apparatus for an interactive live system according to an example embodiment of the present application. An apparatus for an interactive live system according to an exemplary embodiment of the present application will be described in detail below with reference to fig. 6.
According to some embodiments, the interactive live broadcast system comprises at least one main speaker terminal, a guest terminal, an interactive live broadcast server and a watching terminal.
As shown in fig. 6, an apparatus for an interactive live broadcasting system includes a receiving main speaking end stream pushing module 601, a receiving guest end stream pushing module 603, a stream pushing main speaking end module 605, a stream pushing guest end module 607, and a stream pushing watching end module 609, where:
the receiving main speaking end stream pushing module 601 is configured to receive a main speaking stream pushing video picture of the main speaking end. The main conversational streamlet video picture comprises a main conversational streamlet video picture of a first resolution and a main conversational streamlet video picture of a second resolution. Wherein the first resolution is greater than the second resolution.
The guest end video stream receiving module 603 is configured to receive a guest video stream of a guest end. The guest push stream video pictures comprise a guest push stream video picture of a third resolution and a guest push stream video picture of a fourth resolution. Wherein the third resolution is greater than the fourth resolution.
The stream pushing main speaking end module 605 is configured to push the guest stream video picture of the fourth resolution to the main speaking end. The stream pushing guest end module 607 is configured to push the main speaking stream video picture with the second resolution to the guest end. The stream pushing and viewing end module 609 is configured to push the main speaking stream pushing video picture with the first resolution and the guest stream pushing video picture with the third resolution to the viewing end.
An electronic device 200 according to this embodiment of the present application is described below with reference to fig. 7. The electronic device 200 shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 7, the electronic device 200 is embodied in the form of a general purpose computing device. The components of the electronic device 200 may include, but are not limited to: at least one processing unit 210, at least one memory unit 220, a bus 230 connecting different system components (including the memory unit 220 and the processing unit 210), a display unit 240, and the like. Wherein the storage unit stores program code that can be executed by the processing unit 210 such that the processing unit 210 performs the methods according to various exemplary embodiments of the present application described herein. For example, the processing unit 210 may perform a method as shown in fig. 1.
The storage unit 220 may include readable media in the form of volatile storage units, such as a random access memory unit (RAM) 2201 and/or a cache memory unit 2202, and may further include a read only memory unit (ROM) 2203.
The storage unit 220 may also include a program/utility 2204 having a set (at least one) of program modules 2205, such program modules 2205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 200 may also communicate with one or more external devices 300 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 200, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 200 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 250. Also, the electronic device 200 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 260. The network adapter 260 may communicate with other modules of the electronic device 200 via the bus 230. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. The technical solution according to the embodiments of the present application may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, or a network device, etc.) to execute the above method according to the embodiments of the present application.
The software product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
The computer readable medium carries one or more programs which, when executed by a device, cause the computer readable medium to perform the functions described above.
Those skilled in the art will appreciate that the modules described above may be distributed in the apparatus according to the description of the embodiments, or may be modified accordingly in one or more apparatuses unique from the embodiments. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.
According to some example embodiments of the application, the main speaking end and the guest end respectively push two paths of audio and video data with different stream resolutions to the interactive live broadcast system server. The interactive live broadcast server pushes the low-resolution push stream video picture of the guest end and the low-resolution push stream video picture of the guest end to the main speaking end and pushes the high-resolution push stream video picture of the main speaking end and the guest end to the watching end respectively, thereby reducing the push stream data and the received data in the interactive live broadcast system, reducing the decoding workload of the main speaking end and the guest end equipment, reducing the power consumption of the equipment and simultaneously ensuring the definition of the push stream video picture of the watching end.
The foregoing detailed description of the embodiments of the present application has been presented to illustrate the principles and implementations of the present application, and the description of the embodiments is only intended to facilitate the understanding of the methods and their core concepts of the present application. Meanwhile, a person skilled in the art should, according to the idea of the present application, change or modify the embodiments and applications of the present application based on the scope of the present application. In view of the above, the description should not be taken as limiting the application.
Claims (14)
1. A method for an interactive live system, wherein the interactive live system comprises at least one main speaker terminal, a first guest terminal, an interactive live server and a viewing terminal, and the method is used for the interactive live server, and the method comprises the following steps:
receiving a main speaking plug flow video picture of the main speaking end,
the main stream pushing video picture comprises a main stream pushing video picture with a first resolution and a main stream pushing video picture with a second resolution, and the first resolution is greater than the second resolution;
receiving a first guest push streaming video picture of the first guest terminal,
the first guest push streaming video picture comprises a guest push streaming video picture of a third resolution and a guest push streaming video picture of a fourth resolution, and the third resolution is greater than the fourth resolution;
pushing the guest at the fourth resolution to the main speaking end to push and stream the video picture;
pushing the main speaking push streaming video picture with the second resolution to the first guest terminal;
and pushing the main speaking push streaming video picture with the first resolution and the guest push streaming video picture with the third resolution to the viewing end.
2. The method of claim 1, wherein the interactive live broadcast system further comprises a second guest terminal, and wherein the method further comprises:
receiving a second guest push streaming video picture of the second guest terminal,
the second guest push stream video picture comprises a guest push stream video picture of a fifth resolution and a guest push stream video picture of a sixth resolution,
the fifth resolution is greater than the sixth resolution;
pushing and streaming the guest push streaming video picture with the sixth resolution to the first guest terminal;
pushing the main speaking push-streaming video picture with the second resolution to the second guest terminal;
and pushing and streaming the guest push streaming video picture of the fourth resolution to the second guest terminal.
3. The method of claim 1, wherein:
the first resolution is greater than the third resolution;
the second resolution is greater than the fourth resolution.
4. The method of claim 2, wherein:
the third resolution and the fifth resolution are the same;
the fourth resolution and the sixth resolution are the same.
5. The method of claim 1, wherein:
the first resolution comprises a 480p resolution and the second resolution comprises a 240p resolution; or
The first resolution comprises a 360p resolution and the second resolution comprises a 180p resolution; or
The first resolution comprises a 240p resolution and the second resolution comprises a 120p resolution.
6. The method of claim 1, wherein:
the third resolution comprises a 240p resolution and the fourth resolution comprises a 120p resolution.
7. The method of claim 1, further comprising:
sending first role switching information to the main speaking end, wherein the first role switching information comprises switching the main speaking end into the first guest end; and/or
And sending second role switching information to the first guest terminal, wherein the second role switching information comprises switching the first guest terminal into the master speaker terminal.
8. The method of claim 1, wherein prior to streaming the fourth resolution guest streamlet video picture to the master end, the method further comprises:
and receiving first signaling information of the main speaking end, wherein the first signaling information comprises the resolution of the guest push stream video picture received by the main speaking end.
9. The method of claim 1, wherein prior to streaming the main speaking video picture at the second resolution to the first guest end, the method further comprises:
and receiving second signaling information of the first guest terminal, wherein the second signaling information comprises the resolution of the main talk push stream video picture received by the first guest terminal.
10. The method of claim 1, wherein said streaming a main conversational streaming video picture of the first resolution and a guest streaming video picture of the third resolution to the viewing end comprises:
mixing the main speaking plug flow video picture with the first resolution and the guest plug flow video picture with the third resolution to obtain a bypass mixed flow video picture;
and pushing and flowing the bypass mixed flow video picture to the watching end.
11. A method for an interactive live broadcast system is characterized in that the interactive live broadcast system comprises at least one main speaking end, a first guest end, an interactive live broadcast server and a watching end, wherein the first guest end pushes and streams a first guest push and stream video picture to the interactive live broadcast server, the first guest push and stream video picture comprises a guest push and stream video picture with a third resolution and a guest push and stream video picture with a fourth resolution, the third resolution is higher than the fourth resolution, and the method is used for the main speaking end and comprises the following steps:
pushing a main stream pushing video picture to the interactive live broadcast server, wherein the main stream pushing video picture comprises a main stream pushing video picture with a first resolution and a main stream pushing video picture with a second resolution, and the first resolution is higher than the second resolution, so that the interactive live broadcast server pushes the main stream pushing video picture with the second resolution to the first guest end;
and receiving a first guest push streaming video picture of a fourth resolution of the first guest push streaming from the interactive live broadcast server.
12. The method of claim 11, further comprising:
detecting a risk value of the equipment performance of the main speaking end at regular time, wherein the risk value comprises a CPU occupancy rate;
if the risk value is higher than a risk threshold value, the resolution of the main speaking end is reduced;
and if the current resolution of the main speaking end is the resolution threshold, carrying out risk early warning.
13. An apparatus for an interactive live system, the interactive live system comprising at least one main speaker, a guest, an interactive live server and a viewer, the apparatus comprising:
a receiving main speaking end plug flow module, configured to receive a main speaking plug flow video picture of the main speaking end, where the main speaking plug flow video picture includes a main speaking plug flow video picture with a first resolution and a main speaking plug flow video picture with a second resolution, and the first resolution is greater than the second resolution;
the guest end video pushing module is used for receiving guest pushing video pictures of the guest end, wherein the guest pushing video pictures comprise a guest pushing video picture with a third resolution and a guest pushing video picture with a fourth resolution, and the third resolution is higher than the fourth resolution;
the stream pushing main speaking end module is used for pushing the guest stream pushing video picture with the fourth resolution to the main speaking end;
the stream pushing guest end module is used for pushing the main speaking stream video picture with the second resolution to the guest end;
and the stream pushing and watching end module is used for pushing and streaming the main speaking stream pushing video picture with the first resolution and the guest stream pushing video picture with the third resolution to the watching end.
14. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-12.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110922055.8A CN113382280B (en) | 2021-08-12 | 2021-08-12 | Method and device for interactive live broadcast system and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110922055.8A CN113382280B (en) | 2021-08-12 | 2021-08-12 | Method and device for interactive live broadcast system and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113382280A CN113382280A (en) | 2021-09-10 |
| CN113382280B true CN113382280B (en) | 2021-11-09 |
Family
ID=77576882
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110922055.8A Active CN113382280B (en) | 2021-08-12 | 2021-08-12 | Method and device for interactive live broadcast system and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113382280B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106412621A (en) * | 2016-09-28 | 2017-02-15 | 广州华多网络科技有限公司 | Video display method and device of network studio, control method and related equipment |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101778259A (en) * | 2009-12-25 | 2010-07-14 | 深圳中兴力维技术有限公司 | Network video monitoring system and method for improving video display efficiency thereof |
| KR102069695B1 (en) * | 2013-10-11 | 2020-01-23 | 한국전자통신연구원 | Method and apparatus of providing a distributed telepresense service |
| CN104602044B (en) * | 2015-02-05 | 2019-02-15 | 秦永红 | A kind of RTMP Streaming Media public network live broadcast system and its design method |
| US9538139B2 (en) * | 2015-04-24 | 2017-01-03 | Avaya Inc. | Multi-stream video switching with selective optimized composite |
| CN107241564B (en) * | 2016-03-29 | 2020-09-18 | 华为技术有限公司 | Multi-stream video conference method, device and system based on IMS network architecture |
| CN106454404B (en) * | 2016-09-29 | 2019-05-31 | 广州华多网络科技有限公司 | A kind of methods, devices and systems playing live video |
| CN107750014B (en) * | 2017-09-25 | 2020-10-16 | 迈吉客科技（北京）有限公司 | Live wheat-connecting method and system |
| CN112019927B (en) * | 2020-09-23 | 2023-01-06 | Oppo广东移动通信有限公司 | Video live broadcast method, microphone connecting equipment, live broadcast system and storage medium |
-
2021
- 2021-08-12 CN CN202110922055.8A patent/CN113382280B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106412621A (en) * | 2016-09-28 | 2017-02-15 | 广州华多网络科技有限公司 | Video display method and device of network studio, control method and related equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113382280A (en) | 2021-09-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9462228B2 (en) | Distributed real-time media composer | |
| CN112543297B (en) | Video conference live broadcast method, device and system | |
| US8976220B2 (en) | Devices and methods for hosting a video call between a plurality of endpoints | |
| CN101895718B (en) | Video conference system multi-image broadcast method, and device and system thereof | |
| US9961303B2 (en) | Video conference virtual endpoints | |
| CN101437140B (en) | Multi-picture transmission method and multi-point control unit | |
| CN107770600A (en) | Transmission method, device, equipment and the storage medium of stream medium data | |
| CN110913165A (en) | Video stream carousel method and device of video conference system based on cascade framework | |
| CN114546308A (en) | Application interface screen projection method, device, equipment and storage medium | |
| CN113542660A (en) | Method, system and storage medium for realizing conference multi-picture high-definition display | |
| CN112911383A (en) | Multipath screen projection method, device and system under local area network | |
| KR20160099977A (en) | Data processing apparatus and data processing method for videoconferencing service | |
| CN112468763B (en) | Video transmission and display method, device and equipment for conference television and storage medium | |
| CN102572367B (en) | Multiple auxiliary stream control method, device and network system | |
| CN103957445A (en) | Video redirecting system and method based on application virtualization technology | |
| CN113382280B (en) | Method and device for interactive live broadcast system and electronic equipment | |
| CN109309805B (en) | Multi-window display method, device, equipment and system for video conference | |
| CN112804471A (en) | Video conference method, conference terminal, server and storage medium | |
| CN111385590A (en) | Live broadcast data processing method and device and terminal | |
| US20120075408A1 (en) | Technique for providing in-built audio/video bridge on endpoints capable of video communication over ip | |
| US11916982B2 (en) | Techniques for signaling multiple audio mixing gains for teleconferencing and telepresence for remote terminals using RTCP feedback | |
| KR100747895B1 (en) | Multipoint image communication method | |
| CN113727183B (en) | Live push method, apparatus, device, storage medium and computer program product | |
| CN117714734A (en) | Video processing method, device, equipment and medium | |
| CN115623069A (en) | Multi-screen projection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |