CN114449344B - Video stream transmission method, device, electronic equipment and storage medium - Google Patents
Video stream transmission method, device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114449344B CN114449344B CN202210115104.1A CN202210115104A CN114449344B CN 114449344 B CN114449344 B CN 114449344B CN 202210115104 A CN202210115104 A CN 202210115104A CN 114449344 B CN114449344 B CN 114449344B
- Authority
- CN
- China
- Prior art keywords
- video stream
- video
- jitter buffer
- frame
- switching
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 230000005540 biological transmission Effects 0.000 title description 9
- 239000000872 buffer Substances 0.000 claims abstract description 117
- 230000003139 buffering effect Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 3
- 238000009877 rendering Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 18
- 230000006870 function Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/239—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests
- H04N21/2393—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests involving handling client requests
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
- H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/437—Interfacing the upstream path of the transmission network, e.g. for transmitting client requests to a VOD server
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
- H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
- H04N21/4621—Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The present disclosure provides a video streaming method, apparatus, electronic device, and storage medium, the method comprising: receiving a video stream switching request for switching a first video stream to a second video stream; the video stream switching request carries a switching time point; responding to the video stream switching request, acquiring the first video stream and the second video stream sent by a server, caching the first video stream to a first jitter buffer area, and caching the second video stream to a second jitter buffer area; the second video stream takes the first frame of the current picture group period of the switching time point as a starting frame, and the server stops sending the first video stream under the condition that the set condition is met; and continuing to display the video frames of the second video stream in the second jitter buffer area under the condition that the video frame display of the first video stream in the first jitter buffer area is finished. By the method, the switching time of the video stream can be reduced.
Description
Technical Field
The present disclosure relates to the field of communication technologies, and more particularly, to a video streaming method, apparatus, electronic device, and computer-readable storage medium.
Background
In a scene of a multi-person room, in order to adapt to the requirement of a spectator terminal for playing pictures with different definition, a main broadcasting terminal generally sends multiple paths of video streams with different definition at the same time, so that the spectator terminal subscribes to a certain path of video stream according to the actual situation of the spectator terminal. When the situation changes, the viewer end subscribes to another video stream for playing, however, because of the dependency relationship during video frame decoding, the subscribed new stream must start from the first frame of the group of pictures period (Group of Pictures, GOP) of the video stream. In the related art, the new stream is generally started from the next frame group period at the switching time point, so that the switching time is long, and the user experience is affected.
Disclosure of Invention
An object of the present disclosure is to provide a video stream transmission method, apparatus, electronic device, and storage medium, so as to reduce a switching duration of a video stream.
According to a first aspect of embodiments of the present disclosure, there is provided a video streaming method, the method comprising:
receiving a video stream switching request for switching a first video stream to a second video stream; wherein, the video stream switching request carries a switching time point;
responding to the video stream switching request, acquiring the first video stream and the second video stream sent by a server, caching the first video stream to a first jitter buffer area, and caching the second video stream to a second jitter buffer area; the second video stream takes the first frame of the current picture group period of the switching time point as a starting frame, and the server stops sending the first video stream under the condition that the set condition is met;
and continuing to display the video frames of the second video stream in the second jitter buffer area under the condition that the video frame display of the first video stream in the first jitter buffer area is finished.
According to a second aspect of the present disclosure, there is provided a video transmission method, the method comprising:
receiving a video stream switching request for switching a first video stream to a second video stream; wherein, the video stream switching request carries a switching time point;
according to the video stream switching request, the first video stream and the second video stream are sent to terminal equipment so that the terminal equipment can buffer the first video stream in a first jitter buffer area and buffer the second video stream in a second jitter buffer area; the second video stream takes a first frame of a current picture group period at the switching time point as a starting frame;
and stopping sending the first video stream to the terminal equipment and continuing sending the second video stream to the terminal equipment under the condition that the set condition is met.
According to a third aspect of the present disclosure, there is provided a video streaming apparatus, the apparatus comprising:
the first receiving module is used for receiving a video stream switching request for switching the first video stream to the second video stream; wherein, the video stream switching request carries a switching time point;
the acquisition module is used for responding to the video stream switching request, acquiring the first video stream and the second video stream sent by the server, caching the first video stream into a first jitter buffer area and caching the second video stream into a second jitter buffer area; the second video stream takes the first frame of the current picture group period of the switching time point as a starting frame, and the server stops sending the first video stream under the condition that the set condition is met;
and the display module is used for continuously displaying the video frames of the second video stream in the second jitter buffer area under the condition that the video frames of the first video stream in the first jitter buffer area are displayed.
According to a fourth aspect of the present disclosure, there is provided a video streaming apparatus, the apparatus comprising:
the second receiving module is used for receiving a video stream switching request for switching the first video stream to the second video stream; wherein, the video stream switching request carries a switching time point;
the first sending module is used for sending the first video stream and the second video stream to the terminal equipment according to the video stream switching request so that the terminal equipment can buffer the first video stream in a first jitter buffer area and buffer the second video stream in a second jitter buffer area; the second video stream takes a first frame of a current picture group period at the switching time point as a starting frame;
and the second sending module is used for stopping sending the first video stream to the terminal equipment and continuing sending the second video stream to the terminal equipment under the condition that the set condition is met.
According to a fifth aspect of the present disclosure there is provided an electronic device comprising the apparatus of the above second aspect; alternatively, an apparatus as described in the third aspect above is included; or comprises a memory for storing executable instructions and a processor; the processor is configured to perform the video streaming method according to the first or second aspect above under control of the instructions.
According to a sixth aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method according to the first aspect of the present disclosure.
According to the video stream transmission method, the video stream transmission device, the electronic equipment and the storage medium, when receiving a video stream switching request for switching a first video stream to a second video stream, the audience terminal equipment acquires and caches the first video stream and the second video stream simultaneously, and the second video stream is cached from the first frame of the current picture group period at the switching time point as the initial frame, so that after the video frame of the first video stream is displayed, the video frame of the second video stream is directly switched and displayed. Namely, the second video stream is buffered to the second jitter buffer area from the current picture group period at the switching time point, and the smooth playing of the two paths of video streams is realized by the double jitter buffer area mode, so that the switching time of the video streams is reduced.
Other features of the present disclosure and its advantages will become apparent from the following detailed description of exemplary embodiments of the disclosure, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description, serve to explain the principles of the disclosure.
Fig. 1 is a schematic flow chart of a video streaming method according to an embodiment of the present disclosure;
FIG. 2 is a schematic diagram of an interface display of an electronic device according to an embodiment of the present disclosure;
FIG. 3 is a schematic diagram of a system architecture according to an embodiment of the present disclosure;
FIG. 4 is an interactive schematic diagram according to an embodiment of the present disclosure;
fig. 5a is a schematic diagram of a video streaming process according to an embodiment of the present disclosure;
fig. 5b is a second schematic diagram of a video streaming process according to an embodiment of the present disclosure;
fig. 6 is a schematic flow chart diagram of a video streaming method according to another embodiment of the present disclosure;
fig. 7 is a functional block diagram of a video streaming apparatus according to an embodiment of the present disclosure;
fig. 8 is a functional block diagram of a video streaming apparatus according to another embodiment of the present disclosure;
fig. 9 is a block diagram of a hardware configuration of an electronic device according to an embodiment of the present disclosure.
Detailed Description
Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless it is specifically stated otherwise.
The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses.
Techniques, methods, and apparatus known to one of ordinary skill in the relevant art may not be discussed in detail, but are intended to be part of the specification where appropriate.
In all examples shown and discussed herein, any specific values should be construed as merely illustrative, and not a limitation. Thus, other examples of exemplary embodiments may have different values.
It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further discussion thereof is necessary in subsequent figures.
< method example one >
Fig. 1 is a flow diagram of a video streaming method implemented by a terminal device, which may be an electronic product having a processor and memory, according to an embodiment of the present disclosure. For example, the video streaming method may be a smart phone, a portable computer, a desktop computer, a tablet computer, etc., in a network live broadcast scenario, the terminal device may be referred to as a live broadcast device, and the live broadcast device includes a main broadcast end device and an audience end device, where the video streaming method in the embodiment of the disclosure is implemented by the audience end device.
As shown in fig. 1, the method of the present embodiment may include the following steps S1100-S1300, which are described in detail below.
Step S1100, receiving a video stream switching request for switching a first video stream to a second video stream; wherein, the video stream switching request carries a switching time point.
The first video stream is a video stream currently played by the audience terminal equipment, and the first video stream can be a low-resolution low-code-rate video stream or a high-resolution high-code-rate video stream. The second video stream may be a switched video stream, and the second video stream may be a low resolution, low bitrate video stream, or a high resolution, high bitrate video stream. Illustratively, in the case where the first video stream is a low resolution, low rate video stream 1, the corresponding second video stream is a high resolution, high rate video stream 2.
The video stream may be video stream data collected in real time by the webcam at the anchor side after the anchor logs in the anchor side device, and the content of the video stream is various, such as eating, sleeping, singing, playing, and the like. As shown in fig. 3, after the video stream data is collected by the anchor terminal device, the video stream data is sent to the viewer terminal device through the anchor terminal connected background program, i.e., the anchor terminal background program, and the viewer terminal connected background program, i.e., the viewer terminal background program, where the viewer terminal device can receive the video stream sent by the anchor terminal device, and decode, render, and display the video frames of the video stream. Both the anchor-side daemon and the viewer-side daemon run in a server, which may be a monolithic server or a decentralized server across multiple computers or computer data centers.
The video stream switching request may be a double click operation, a single click operation, etc. implemented by the user on the video playing interface, which is not limited herein, and the video stream switching request carries a switching time point, where the switching time point may be a timestamp carried by a video frame of the video stream when the video stream switching is triggered.
As shown in fig. 3 and 4, the user enters a multi-person room, and the anchor device transmits a low-resolution low-bitrate video stream 1 and a high-resolution high-bitrate video stream 2, and the user subscribes to the low-resolution low-bitrate video stream 1 by default, where the anchor device transmits the video stream 1 and the video stream 2 to the anchor daemon simultaneously, and the anchor daemon transmits the video stream 1 to the viewer device through the viewer daemon. The viewer-side device buffers the video stream 1 in a first jitter buffer, and the first jitter buffer decodes, renders and displays the video frames of the video stream 1 to obtain a display interface as shown in the left diagram of fig. 2.
When the user double-clicks the display interface of the audience terminal equipment, the audience terminal equipment sends a video stream switching request (subscribing to the video stream 2) for switching the video stream 1 to the video stream 2 to the audience terminal background program, and the audience terminal background program sends the video stream switching request to the anchor terminal background program, wherein the video stream switching request carries a switching time point.
Step S1200, in response to the video stream switching request, acquires the first video stream and the second video stream sent by the server, and buffers the first video stream in a first jitter buffer, and buffers the second video stream in a second jitter buffer.
The second video stream takes the first frame of the current picture group period at the switching time point as a starting frame. A group of pictures period, which may be 1.95s (typically approximately 2 s), is typically a set of consecutive pictures, consisting of instantaneous decoding refresh (Instantaneous Decoding Refresh, IDR) frames and P frames, where an IDR frame is the first frame of a group of pictures period and an IDR frame is intra-frame decoding; p frames rely on previous IDR frames or P frame decoding.
In this embodiment, after receiving the video switching request, the audience terminal device sends the video stream switching request to the audience terminal daemon, and the audience terminal daemon sends the video switching request to the anchor terminal daemon, and the anchor terminal daemon continues to send the first video stream to the audience terminal daemon. At the same time, the anchor end background program acquires a switching time point carried in the video stream switching request, and starts to send a second video stream to the audience end background program from the first frame of the picture group period at the switching time point as a starting frame.
Continuing with the above example, as shown in fig. 4, after the anchor-side daemon receives the video stream switching request for switching the video stream 1 to the video stream 2 sent by the audience-side equipment through the audience-side daemon, the anchor-side daemon will send the video stream 1 and the video stream 2 to the audience-side daemon simultaneously. Specifically, after receiving the video stream switching request from the video stream 1 to the video stream 2, the anchor end background program will continue to send the video stream 1 to the viewer end background program, and start to send the video stream 2 to the viewer end background program from the first frame of the frame group period at the switching time point carried in the video stream switching request.
In an alternative embodiment, after the viewer-side daemon receives the first video stream and the second video stream, the first video stream and the second video stream are sent to the viewer-side device for the viewer-side device to buffer the first video stream to a first jitter buffer and the second video stream to a second jitter buffer. Specifically, after receiving the first video stream and the second video stream sent by the anchor end background program, the audience end background program may acquire a first data amount of the first video stream and a second data amount of the second video stream, and acquire a total data amount of the first data amount and the second data amount, so as to send the first video stream and the second video stream to the audience end device according to the total data amount and a bandwidth of a network where the first video stream and the second video stream are located.
Continuing with the above example, as shown in fig. 5a, the viewer-side daemon puts video stream 1 sent by the anchor-side daemon into a wait queue, wherein each video frame of video stream 1 contains a switch sequence number 0 (SwitchSeq 0) field. And placing video stream 2 into a switch queue, wherein each video frame of video stream 2 includes a switch sequence number 1 (SwitchSeq 1) field. For each switch the switch sequence number (SwitchSeq) field is accumulated once, such as the switch sequence number 2 (SwitchSeq 2) field, the switch sequence number 3 (SwitchSeq 3) field, etc., it is understood that the switch sequence number field can be used to distinguish the switch times of different video streams. At the same time, the viewer-side daemon will send the video stream 1 and video stream 2 to the viewer-side device. As shown in fig. 5b, the viewer-side device buffers the video frame of the SwitchSeq0 field, i.e., the video frame of the video stream 1, into the first jitter buffer, and buffers the video frame of the SwitchSeq1 field, i.e., the video frame of the video stream 2, into the second jitter buffer.
For example, when the viewer-side daemon transmits a video stream to the viewer-side device, the bandwidth of the network is detected in real time, for example, the bandwidth of the network is 500Mbps, the video frame of the video stream 1 is 100M, and the video frame of the video stream 2 is 300M, and then the viewer-side daemon will preferentially transmit one video frame of the video stream 1 and then immediately transmit one video frame of the video stream 2. And after one video frame of the video stream 2 is transmitted, the next video frame of the video stream 1 is transmitted again, so as to cycle.
In this embodiment, the server stops sending the first video stream when the set condition is satisfied. Specifically, the viewer-side daemon stops acquiring the first video stream from the anchor-side daemon and stops sending the first video stream to the viewer-side device, and continues sending the second video stream to the viewer-side device when the setting condition is satisfied.
In an alternative embodiment, the viewer-side daemon acquires a first timestamp carried by a video frame of the first video stream and acquires a second timestamp carried by a video frame of the second video stream, and if a time difference between the first timestamp and the second timestamp meets a set time threshold, the viewer-side daemon unsubscribes from the first video stream, stops sending the first video stream to the viewer-side equipment, and continues sending the second video stream to the viewer-side equipment.
The set time threshold may be a value set according to an actual application scenario and an actual requirement, and the set time threshold may indicate that the first timestamp and the second timestamp are very close to each other to some extent.
The first timestamp and the second timestamp are both a predetermined time system (Predetermined Time System, PTS).
Continuing with the above example, if the absolute value of the difference between the timestamp carried by the video frame of the video stream 1 and the timestamp carried by the video frame of the video stream 2 is determined to be less than 100ms, the viewer-side daemon indicates that the video frames of the first video stream and the second video stream overlap, where the viewer-side daemon clears the data in the waiting queue, transfers the data in the switching queue to the waiting queue, and unsubscribes from the video stream 1.
After responding to the video stream switching request, acquiring the first video stream and the second video stream sent by a server, caching the first video stream in a first jitter buffer area, and caching the second video stream in a second jitter buffer area, entering:
step S1300, when the display of the video frames of the first video stream in the first jitter buffer is finished, continuing to display the video frames of the second video stream in the second jitter buffer.
In this embodiment, after buffering the first video stream in the first jitter buffer according to step S1200, the viewer-side device decodes the video frames of the first video stream in the first jitter buffer to obtain the decoded video frames of the first video stream, and renders and displays the decoded video frames of the first video stream.
In this embodiment, after the second video stream is buffered in the second jitter buffer according to the above step S1200, the viewer device further decodes the video frame of the second video stream in the second jitter buffer, obtains the video frame of the decoded second video stream, obtains the timestamp carried by the video frame of the decoded first video stream, and discards the video frame of the decoded second video stream corresponding to the timestamp in the second jitter buffer according to the timestamp carried by the video frame of the decoded first video stream.
Continuing with the example above, as shown in fig. 5b, as the viewer-side device receives the data frames, each video frame of the switch sequence number field enters a jitter buffer. The first jitter buffer normally receives, decodes and renders video frames of video stream 1 and synchronizes the time stamps of the rendered frames in real time to the second jitter buffer. The second jitter buffer normally receives and decodes video frames of video stream 2 and discards video frames of video stream 2 earlier than the time stamp and decoded according to the time stamp synchronized in real time by the first jitter buffer.
In this embodiment, in the case where the display of the video frames of the first video stream in the first jitter buffer is completed in step S1300, continuing to display the video frames of the second video stream in the second jitter buffer further includes: destroying a first jitter buffer area under the condition that video frame display of the first video stream in the first jitter buffer area is finished; and continuing to render and display the remaining video frames of the decoded second video stream in the second jitter buffer.
Continuing with the above example, when there is no video frame in the first jitter buffer, the viewer device destroys the first jitter buffer, and switches to the second jitter buffer to start rendering and display the remaining video frames of the decoded second video stream, so as to obtain the display interface shown in the right diagram of fig. 2.
According to the method of the embodiment, when receiving a video stream switching request for switching a first video stream to a second video stream, the viewer device acquires and caches the first video stream and the second video stream at the same time, and the second video stream is cached starting from the first frame of the current frame group period at the switching time point as the start frame, and then after the video frame of the first video stream is displayed, the video frame of the second video stream is directly switched and displayed. Namely, the second video stream is buffered to the second jitter buffer area from the current picture group period at the switching time point, and the smooth playing of the two paths of video streams is realized by the double jitter buffer area mode, so that the switching time of the video streams is reduced.
< method example two >
Fig. 6 is a flow diagram of a video streaming method implemented by a server, which may be a monolithic server or a decentralized server across multiple computers or computer data centers, running a viewer-side daemon and a host-side daemon, according to an embodiment of the present disclosure.
As shown in fig. 6, the method of the present embodiment may include the following steps S6100 to S6300, which are described in detail below.
Step S6100, receiving a video stream switching request for switching a first video stream to a second video stream; wherein, the video stream switching request carries a switching time point.
In this embodiment, after receiving a video stream switching request from a user to switch a first video stream to a second video stream, the viewer-side device sends the video stream switching request to a viewer-side daemon, and the viewer-side daemon sends the video stream switching request to a anchor-side daemon.
As shown in fig. 3 and 4, the user enters a multi-person room, and the anchor device transmits a low-resolution low-bitrate video stream 1 and a high-resolution high-bitrate video stream 2, and the user subscribes to the low-resolution low-bitrate video stream 1 by default, where the anchor device transmits the video stream 1 and the video stream 2 to the anchor daemon simultaneously, and the anchor daemon transmits the video stream 1 to the viewer device through the viewer daemon. The viewer-side device buffers the video stream 1 in a first jitter buffer, and the first jitter buffer decodes, renders and displays the video frames of the video stream 1 to obtain a display interface as shown in the left diagram of fig. 2.
When the user double-clicks the display interface of the audience terminal equipment, the audience terminal equipment sends a video stream switching request (subscribing to the video stream 2) for switching the video stream 1 to the video stream 2 to the audience terminal background program, and the audience terminal background program sends the video stream switching request to the anchor terminal background program, wherein the video stream switching request carries a switching time point.
Step S6200, according to the video stream switching request, sending the first video stream and the second video stream to a terminal device, so that the terminal device buffers the first video stream in a first jitter buffer and buffers the second video stream in a second jitter buffer.
In this embodiment, sending the first video stream and the second video stream to the terminal device in the step S6200 may further include: acquiring a first data volume of the first video stream and a second data volume of the second video stream; acquiring the total data volume of the first data volume and the second data volume; and sending the first video stream and the second video stream to terminal equipment according to the total data volume and the bandwidth of the network.
Specifically, after receiving the video stream switching request, the anchor end background program continues to send the first video stream to the viewer end background program. At the same time, the anchor end background program acquires a switching time point carried in the video stream switching request, and starts to send a second video stream to the audience end background program from the first frame of the picture group period at the switching time point as a starting frame. After receiving the first video stream and the second video stream, the background program at the audience end detects the bandwidth of the network, the total data amount of the first video stream and the second data amount of the second video stream in real time, and then sends the first video stream and the second video stream to the audience end equipment according to the total data amount and the bandwidth of the network.
Continuing with the above example, as shown in fig. 4, after the anchor-side daemon receives the video stream switching request for switching the video stream 1 to the video stream 2, the anchor-side daemon will send the video stream 1 and the video stream 2 to the viewer-side daemon simultaneously. Specifically, after receiving the video stream switching request from the video stream 1 to the video stream 2, the anchor end background program will continue to send the video stream 1 to the viewer end background program, and start to send the video stream 2 to the viewer end background program from the first frame of the frame group period at the switching time point carried in the video stream switching request. As shown in fig. 5a, the viewer-side daemon puts video stream 1 in a waiting queue, wherein each video frame of video stream 1 contains a switch sequence number 0 (SwitchSeq 0) field. And placing video stream 2 into a switch queue, wherein each video frame of video stream 2 includes a switch sequence number 1 (SwitchSeq 1) field. At the same time, as shown in fig. 5b, the viewer-side daemon will send the video stream 1 and the video stream 2 to the viewer-side device, and the viewer-side device will buffer the video frame of the field SwitchSeq0, i.e. the video frame of the video stream 1, to the first jitter buffer, and buffer the video frame of the field SwitchSeq1, i.e. the video frame of the video stream 2, to the second jitter buffer.
For example, when the viewer-side daemon transmits a video stream to the viewer-side device, the bandwidth of the network is detected in real time, for example, the bandwidth of the network is 500Mbps, the video frame of the video stream 1 is 100M, and the video frame of the video stream 2 is 300M, and then the viewer-side daemon will preferentially transmit one video frame of the video stream 1 and then immediately transmit one video frame of the video stream 2. And after one video frame of the video stream 2 is transmitted, the next video frame of the video stream 1 is transmitted again, so as to cycle.
Step S6300, stopping sending the first video stream to the terminal device and continuing to send the second video stream to the terminal device if the setting condition is satisfied.
In this embodiment, when the setting condition is satisfied, the step S6300 of stopping sending the first video stream to the terminal device and continuing to send the second video stream to the terminal device may further include: acquiring a first timestamp carried by a video frame of the first video stream; acquiring a second timestamp carried by a video frame of the second video stream; and stopping sending the first video stream to the terminal equipment and continuing sending the second video stream to the terminal equipment under the condition that the time difference between the first time stamp and the second time stamp meets a set time threshold.
Specifically, the background program at the viewer end can acquire a first timestamp carried by the video frame of the first video stream and acquire a second timestamp carried by the video frame of the second video stream, and cancel subscribing to the first video stream and stop sending the first video stream to the viewer end device and continue sending the second video stream to the viewer end device when the time difference between the first timestamp and the second timestamp meets a set time threshold.
Continuing with the above example, if the absolute value of the difference between the timestamp carried by the video frame of the video stream 1 and the timestamp carried by the video frame of the video stream 2 is determined to be less than 100ms, the viewer-side daemon indicates that the video frames of the first video stream and the second video stream overlap, where the viewer-side daemon clears the data in the waiting queue, transfers the data in the switching queue to the waiting queue, and unsubscribes from the video stream 1. Here, the anchor-side daemon stops transmitting the video stream 1 to the viewer-side daemon, and transmits the video stream 2 only to the viewer-side daemon. Correspondingly, the viewer-side daemon stops transmitting video stream 1 to the viewer-side device and transmits video stream 2 only to the viewer-side device.
According to the method of the embodiment, when the server receives the video stream switching request from the first video stream to the second video stream, the first video stream and the second video stream are simultaneously transmitted to the terminal device, and the second video stream is transmitted from the first frame of the current frame group period at the switching time point as the start frame, and when the set condition is met, only the second video stream is transmitted to the terminal device, that is, the server can realize normal transmission of multiple streams.
Next, an example video streaming method is shown, which, in combination with fig. 2, 3, 4, 5a and 5b, may include:
step S4100: initially, the anchor end device sends video stream 1 and video stream 2, and the viewer end device subscribes to video stream 1.
Step S4200: the audience terminal equipment receives a video stream switching request for switching the video stream 1 to the video stream 2; the video stream switching request carries a switching time point.
In step S4300, the viewer-side device transmits a video stream switching request carrying a switching time point to the viewer-side daemon.
In step S4400, the viewer-side daemon sends a video stream switching request carrying a switching time point to the anchor-side daemon.
In step S4500, the anchor-side daemon transmits the video stream 1, and starts transmitting the video stream 2 to the viewer-side daemon from the first frame of the frame group period where the switching time point is located, i.e., the anchor-side daemon transmits the video stream 1 and the video stream 2 simultaneously.
In step S4600, the viewer-side daemon receives video stream 1 and video stream 2 and places video stream 1 in a wait queue while marking video frames in video stream 1 to contain a SwitchSeq0 field. Video stream 2 is placed in the switch queue while the video frames in marked video stream 2 contain a SwitchSeq1 field.
In step S4700, the viewer-side daemon obtains the first data amount of the video stream 1 and the second data amount of the video stream 2, obtains the total data amount of the first data amount and the second data amount, and sends the video stream 1 and the video stream 2 to the viewer-side device according to the total data amount and the bandwidth of the network in which the total data amount and the second data amount are located.
In step S4800, the viewer-side daemon clears the data in the waiting queue, transfers the data in the switching queue to the waiting queue, and unsubscribes the video stream 1 from the anchor-side daemon if the difference between the time stamp carried by the video frame of the video stream 1 and the time stamp carried by the data frame of the video stream 2 is less than 100 ms. Here, the anchor-side daemon stops transmitting the video stream 1 to the viewer-side daemon and transmits the video stream 2 only to the viewer-side daemon, so that the viewer-side daemon stops transmitting the video stream 1 to the viewer-side device and transmits the video stream 2 only to the viewer-side device.
In step S4900, when the viewer-side device receives a video frame, each video frame of the SwitchSeq field enters a jitter buffer. The first jitter buffer normally receives, decodes and renders video frames of video stream 1 and synchronizes the time stamps of the rendered frames in real time to the second jitter buffer. The second jitter buffer normally receives and decodes video frames of video stream 2 and discards frames earlier than the time stamp and decoded according to the time stamp synchronized in real time. When the first jitter buffer has no video frame, destroying the first jitter buffer and switching to the second jitter buffer to start rendering.
According to the example, the host-side background program and the audience-side background program can realize normal transmission of multiple paths of streams through subscription and transmission strategies, and the audience-side equipment can realize smooth playing of multiple paths of video streams through a double jitter buffer zone mode, so that the switching time of the video streams can be reduced, for example, the switching time can be reduced from 200ms to 2s to 200ms to 300ms.
< device example one >
In this embodiment, a video streaming apparatus 7000 is also provided, and as shown in fig. 7, the apparatus 7000 may include a first receiving module 7100, an acquiring module 7200 and a display module 7300.
A first receiving module 7100, configured to receive a video stream switching request for switching a first video stream to a second video stream; wherein, the video stream switching request carries a switching time point.
An acquiring module 7200, configured to respond to the video stream switching request, acquire the first video stream and the second video stream sent by the server, buffer the first video stream to a first jitter buffer, and buffer the second video stream to a second jitter buffer; and the second video stream takes the first frame of the current picture group period of the switching time point as a starting frame, and the server stops sending the first video stream under the condition that the set condition is met.
A display module 7300, configured to continue displaying the video frames of the second video stream in the second jitter buffer when the video frames of the first video stream in the first jitter buffer are displayed.
< device example two >
In this embodiment, there is also provided a video streaming apparatus 8000, as shown in fig. 8, the apparatus 8000 may include a second receiving module 8100, a first transmitting module 8200, and a second transmitting module 8300.
A second receiving module 8100, configured to receive a video stream switching request for switching the first video stream to the second video stream; wherein, the video stream switching request carries a switching time point.
A first sending module 8200, configured to send the first video stream and the second video stream to a terminal device according to the video stream switching request, so that the terminal device buffers the first video stream in a first jitter buffer and buffers the second video stream in a second jitter buffer; the second video stream takes the first frame of the current picture group period at the switching time point as a starting frame.
A second sending module 8300, configured to stop sending the first video stream to the terminal device and continue sending the second video stream to the terminal device, where the second sending module is configured to receive a video stream switching request for switching the first video stream to the second video stream; wherein, the video stream switching request carries a switching time point.
< device example >
The disclosed embodiments provide an electronic device 9000.
In one embodiment, the electronic device 9000 may comprise a video streaming apparatus 7000 according to any embodiment of the disclosure, or a video streaming apparatus 8000 for implementing a video streaming method of any embodiment of the disclosure.
In another embodiment, as shown in fig. 9, an electronic device 9000 comprises a processor 9100 and a memory 9200, the memory 9100 to store executable instructions; the processor 9200 is configured to perform a method according to any of the embodiments of the present disclosure under control of instructions.
< example of Medium >
An embodiment of the present invention provides a computer-readable storage medium having stored thereon computer instructions which, when executed by a processor, implement a video streaming method provided by any of the foregoing embodiments.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, implementation by software, and implementation by a combination of software and hardware are all equivalent.
The foregoing description of the embodiments of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the technical improvements in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present disclosure is defined by the appended claims.
Claims (11)
1. A method of video streaming, the method comprising:
receiving a video stream switching request for switching a first video stream to a second video stream; wherein, the video stream switching request carries a switching time point;
responding to the video stream switching request, acquiring the first video stream and the second video stream which are simultaneously transmitted by a server, caching the first video stream into a first jitter buffer area, and caching the second video stream into a second jitter buffer area; the second video stream takes the first frame of the current picture group period of the switching time point as a starting frame, and the server stops sending the first video stream under the condition that the set condition is met;
and continuing to display the video frames of the second video stream in the second jitter buffer area under the condition that the video frame display of the first video stream in the first jitter buffer area is finished.
2. The method of claim 1, further comprising, after buffering the first video stream in a first jitter buffer and buffering the second video stream in a second jitter buffer:
decoding the video frames of the first video stream in the first jitter buffer to obtain decoded video frames of the first video stream;
and rendering and displaying the decoded video frames of the first video stream.
3. The method of claim 2, further comprising, after buffering the first video stream in a first jitter buffer and buffering the second video stream in a second jitter buffer:
decoding the video frames of the second video stream in the second jitter buffer to obtain decoded video frames of the second video stream;
acquiring a time stamp carried by a video frame of the decoded first video stream;
and discarding the video frames of the decoded second video stream corresponding to the time stamp in the second jitter buffer according to the time stamp carried by the video frames of the decoded first video stream.
4. A method according to claim 3, wherein said continuing to display video frames of said second video stream in said second jitter buffer in the event that the display of video frames of said first video stream in said first jitter buffer is completed, comprises:
destroying the first jitter buffer area under the condition that the video frame of the first video stream in the first jitter buffer area is displayed;
and continuing to render and display the remaining video frames of the second video stream decoded in the second jitter buffer.
5. A method of video streaming, the method comprising:
receiving a video stream switching request for switching a first video stream to a second video stream; wherein, the video stream switching request carries a switching time point;
according to the video stream switching request, the first video stream and the second video stream are simultaneously sent to a terminal device so that the terminal device can buffer the first video stream in a first jitter buffer area and buffer the second video stream in a second jitter buffer area; the second video stream takes a first frame of a current picture group period at the switching time point as a starting frame;
and stopping sending the first video stream to the terminal equipment and continuing sending the second video stream to the terminal equipment under the condition that the set condition is met.
6. The method of claim 5, wherein the sending the first video stream and the second video stream to a terminal device comprises:
acquiring a first data volume of the first video stream and a second data volume of the second video stream;
acquiring the total data volume of the first data volume and the second data volume;
and sending the first video stream and the second video stream to terminal equipment according to the total data volume and the bandwidth of the network.
7. The method according to claim 5, wherein stopping sending the first video stream to the terminal device and continuing to send the second video stream to the terminal device in the case where a set condition is satisfied, comprises:
acquiring a first timestamp carried by a video frame of the first video stream;
acquiring a second timestamp carried by a video frame of the second video stream;
and stopping sending the first video stream to the terminal equipment and continuing sending the second video stream to the terminal equipment under the condition that the time difference between the first time stamp and the second time stamp meets a set time threshold.
8. A video streaming apparatus, the apparatus comprising:
the first receiving module is used for receiving a video stream switching request for switching the first video stream to the second video stream; wherein, the video stream switching request carries a switching time point;
the acquisition module is used for responding to the video stream switching request, acquiring the first video stream and the second video stream which are simultaneously transmitted by the server, buffering the first video stream into a first jitter buffer area and buffering the second video stream into a second jitter buffer area; the second video stream takes the first frame of the current picture group period of the switching time point as a starting frame, and the server stops sending the first video stream under the condition that the set condition is met;
and the display module is used for continuously displaying the video frames of the second video stream in the second jitter buffer area under the condition that the video frames of the first video stream in the first jitter buffer area are displayed.
9. A video streaming apparatus, the apparatus comprising:
the second receiving module is used for receiving a video stream switching request for switching the first video stream to the second video stream; wherein, the video stream switching request carries a switching time point;
the first sending module is used for simultaneously sending the first video stream and the second video stream to the terminal equipment according to the video stream switching request so that the terminal equipment can buffer the first video stream in a first jitter buffer area and buffer the second video stream in a second jitter buffer area; the second video stream takes a first frame of a current picture group period at the switching time point as a starting frame;
and the second sending module is used for stopping sending the first video stream to the terminal equipment and continuing sending the second video stream to the terminal equipment under the condition that the set condition is met.
10. An electronic device, comprising:
the apparatus of claim 8; or,
the apparatus of claim 9; or,
a memory and a processor, the memory to store executable instructions; the processor is configured to perform the video streaming method according to any of claims 1-7 under control of the instructions.
11. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the video streaming method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210115104.1A CN114449344B (en) | 2022-02-03 | 2022-02-03 | Video stream transmission method, device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210115104.1A CN114449344B (en) | 2022-02-03 | 2022-02-03 | Video stream transmission method, device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114449344A CN114449344A (en) | 2022-05-06 |
| CN114449344B true CN114449344B (en) | 2024-02-09 |
Family
ID=81370826
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210115104.1A Active CN114449344B (en) | 2022-02-03 | 2022-02-03 | Video stream transmission method, device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114449344B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115997384A (en) * | 2022-10-28 | 2023-04-21 | 广州酷狗计算机科技有限公司 | Live stream switching method, device, server, terminal and program product |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101370139A (en) * | 2007-08-17 | 2009-02-18 | 华为技术有限公司 | Method and device for switching channels |
| CN101778426A (en) * | 2010-01-21 | 2010-07-14 | 深圳市同洲电子股份有限公司 | Method and equipment for video data stream transmission in mobile wireless network |
| CN102932676A (en) * | 2012-11-14 | 2013-02-13 | 武汉烽火众智数字技术有限责任公司 | Self-adaptive bandwidth transmitting and playing method based on audio and video frequency synchronization |
| CN106162235A (en) * | 2016-08-17 | 2016-11-23 | 北京百度网讯科技有限公司 | Method and apparatus for Switch Video stream |
| CN109168078A (en) * | 2018-09-10 | 2019-01-08 | 苏宁智能终端有限公司 | A kind of video definition switching method and device |
| CN110139148A (en) * | 2019-05-22 | 2019-08-16 | 武汉瓯越网视有限公司 | A kind of method and relevant apparatus of video switching clarity |
| CN110582012A (en) * | 2018-06-11 | 2019-12-17 | 腾讯科技（深圳）有限公司 | Video switching method, video processing device and storage medium |
| CN111866433A (en) * | 2020-07-31 | 2020-10-30 | 腾讯科技（深圳）有限公司 | Video source switching method, video source playing method, video source switching device, video source playing device, video source equipment and storage medium |
| CN113225585A (en) * | 2020-01-21 | 2021-08-06 | 腾讯科技（深圳）有限公司 | Video definition switching method and device, electronic equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101938456B (en) * | 2009-06-30 | 2014-03-12 | 华为技术有限公司 | Method, device and system for reducing media delays |
-
2022
- 2022-02-03 CN CN202210115104.1A patent/CN114449344B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101370139A (en) * | 2007-08-17 | 2009-02-18 | 华为技术有限公司 | Method and device for switching channels |
| CN101778426A (en) * | 2010-01-21 | 2010-07-14 | 深圳市同洲电子股份有限公司 | Method and equipment for video data stream transmission in mobile wireless network |
| CN102932676A (en) * | 2012-11-14 | 2013-02-13 | 武汉烽火众智数字技术有限责任公司 | Self-adaptive bandwidth transmitting and playing method based on audio and video frequency synchronization |
| CN106162235A (en) * | 2016-08-17 | 2016-11-23 | 北京百度网讯科技有限公司 | Method and apparatus for Switch Video stream |
| CN110582012A (en) * | 2018-06-11 | 2019-12-17 | 腾讯科技（深圳）有限公司 | Video switching method, video processing device and storage medium |
| CN109168078A (en) * | 2018-09-10 | 2019-01-08 | 苏宁智能终端有限公司 | A kind of video definition switching method and device |
| CN110139148A (en) * | 2019-05-22 | 2019-08-16 | 武汉瓯越网视有限公司 | A kind of method and relevant apparatus of video switching clarity |
| CN113225585A (en) * | 2020-01-21 | 2021-08-06 | 腾讯科技（深圳）有限公司 | Video definition switching method and device, electronic equipment and storage medium |
| CN111866433A (en) * | 2020-07-31 | 2020-10-30 | 腾讯科技（深圳）有限公司 | Video source switching method, video source playing method, video source switching device, video source playing device, video source equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 自适应动态网络的流媒体传输策略;洪煌辉;袁清珂;;计算机工程与设计(12);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114449344A (en) | 2022-05-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7110234B2 (en) | Video playback methods, devices and systems | |
| US8205236B2 (en) | Contents distribution system, contents distribution method thereof and contents distribution program thereof | |
| US20230256345A1 (en) | Game data processing method and apparatus, and storage medium | |
| US11025982B2 (en) | System and method for synchronizing content and data for customized display | |
| US11228764B2 (en) | Streaming multiple encodings encoded using different encoding parameters | |
| US20150208103A1 (en) | System and Method for Enabling User Control of Live Video Stream(s) | |
| US20030002578A1 (en) | System and method for timeshifting the encoding/decoding of audio/visual signals in real-time | |
| EP1775953A1 (en) | Switching between digital video streams using buffering of second digital video stream | |
| CN110582012B (en) | Video switching method, video processing device and storage medium | |
| CN104918137A (en) | Method enabling spliced screen system to play videos | |
| JP2002033771A (en) | Media data processor | |
| US11445229B2 (en) | Managing deep and shallow buffers in a thin-client device of a digital media distribution network | |
| CN110913165A (en) | Video stream carousel method and device of video conference system based on cascade framework | |
| CN114449344B (en) | Video stream transmission method, device, electronic equipment and storage medium | |
| US9226003B2 (en) | Method for transmitting video signals from an application on a server over an IP network to a client device | |
| CN111669605B (en) | Method and device for synchronizing multimedia data and associated interactive data thereof | |
| CN101262600B (en) | A quick forward and backward playing system and method for network video | |
| CN111385590A (en) | Live broadcast data processing method and device and terminal | |
| US7890651B2 (en) | Sending content from multiple content servers to clients at time reference points | |
| CN113630575A (en) | Method, system and storage medium for displaying multi-person online video conference image | |
| WO2017096377A1 (en) | Managing deep and shallow buffers in a thin-client device of a digital media distribution network | |
| CN116708860A (en) | Live broadcast system | |
| CN115767115A (en) | Live broadcast picture processing method, device, equipment and storage medium | |
| CN117729392A (en) | Commodity information display method, commodity information display device, computer equipment and medium | |
| JP2001016570A (en) | Video information acquisition device and video distributor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |