CN112822519A - Video stream streaming method, device and equipment for sharing video pictures by multiple users - Google Patents
Video stream streaming method, device and equipment for sharing video pictures by multiple users Download PDFInfo
- Publication number
- CN112822519A CN112822519A CN202011620987.9A CN202011620987A CN112822519A CN 112822519 A CN112822519 A CN 112822519A CN 202011620987 A CN202011620987 A CN 202011620987A CN 112822519 A CN112822519 A CN 112822519A
- Authority
- CN
- China
- Prior art keywords
- video
- client
- slave
- quality parameter
- clients
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 68
- 230000008569 process Effects 0.000 claims description 28
- 238000012508 change request Methods 0.000 claims description 13
- 238000009877 rendering Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 6
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 230000009471 action Effects 0.000 description 3
- 230000000903 blocking effect Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011112 process operation Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000002045 lasting effect Effects 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/238—Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
- H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
- H04N21/4621—Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses a video stream streaming method for sharing video pictures by multiple users, which comprises the steps of dividing a plurality of clients into a master client and a slave client in advance; acquiring a video frame image in real time; based on a first video quality parameter determined according to hardware configuration and network conditions of a host client, encoding a video frame image by using a first encoder to obtain a first video stream, and sending the first video stream to the host client; and coding the video frame images by using a second coder based on the second video quality parameter of the slave client to obtain a second video stream, and sending the second video stream to the slave client. According to the method and the device, the plurality of clients are divided into the master client and the slave clients, and more appropriate video streams are provided for different types of clients, so that the problems of the clients such as jamming and the like are solved, and the watching experience of watching video pictures by a user is improved. The application also provides a video stream streaming device and equipment for sharing the video pictures by multiple users, and the beneficial effects are achieved.
Description
Technical Field
The invention relates to the technical field of sharing video pictures by multiple users, in particular to a method, a device and equipment for video stream streaming of sharing video pictures by multiple users.
Background
The WebRTC can support a plurality of users to share video pictures, and the same video stream can be simultaneously streamed to the clients of the plurality of users by using the host. However, because the network condition and hardware configuration of the client of each user are different, when the same video stream is streamed to the clients of the users at the same time, if the network condition of some clients is poor or the hardware configuration performance is insufficient, the video stream display and play process of the clients may be jammed or fuzzy.
In the prior art, a Scalable Video Coding (SVC) scheme may be adopted, and different Video streams are coded by using one coder, although this Coding scheme can code a plurality of different Video streams, the Coding process is too complex, and how to distribute streaming streams of each coded Video stream among clients is also complex.
Disclosure of Invention
The invention aims to provide a video stream streaming method, a device and equipment for sharing video pictures by multiple users, which can meet different requirements of different clients on video quality to a certain extent.
In order to solve the above technical problem, the present invention provides a video stream streaming method for sharing video pictures by multiple users, comprising:
dividing a plurality of clients into a master client and slave clients according to different video stream quality requirements in advance;
acquiring a video frame image in real time;
based on a first video quality parameter, encoding the video frame image by using a first encoder to obtain a first video stream, and sending the first video stream to the main client; the first video quality parameter is a parameter determined according to the hardware configuration and the network condition of the host client;
based on a second video quality parameter, encoding the video frame image by using a second encoder to obtain a second video stream, and sending the second video stream to the slave client; wherein the second video quality parameter is a parameter determined according to the hardware configuration and the network condition of the slave client.
Optionally, the process of determining the first video quality parameter according to the hardware configuration and the network condition of the host client includes:
detecting and obtaining network condition information at least comprising network bandwidth, packet loss proportion, delay time, image rendering time, image coding time and I frame request frequency corresponding to a main client;
when the delay time, the image rendering time and the image coding time are not greater than the corresponding time thresholds, the frame number of the first video quality parameter is the theoretical decodable frame number of the host client;
when the packet loss proportion is smaller than a preset proportion and the I frame request frequency is smaller than a preset frequency threshold, the size of the video code rate of the first video quality parameter is equal to the size of the network bandwidth;
setting an encoding type in the first video quality parameter according to the primary client decodable encoding format.
Optionally, the process of determining the first video quality parameter according to the hardware configuration and the network condition of the host client includes:
when any one of the delay time length, the image rendering time and the image encoding time is greater than a corresponding time threshold, the frame number of the first video quality parameter is half of the theoretical decodable frame number of the main client, and after a first preset time length, the frame number of the first video quality parameter is adjusted to the theoretical decodable frame number;
when the I frame request frequency is not less than a preset frequency threshold, setting the video code rate of the first video quality parameter to be less than the current preset code rate of the network bandwidth, and after a second preset duration, adjusting the video code rate of the first video quality parameter to be equal to the current network bandwidth.
Optionally, after sending the first video stream to the host client, the method further includes:
receiving decoding time and a decoding frame sequence number of the first video stream decoded by the main client;
and if the decoding time exceeds a preset duration or the number of the decoded frame numbers is less than a preset number, reducing the video code rate of the first video quality parameter.
Optionally, dividing the multiple clients into a master client and a slave client according to different video stream quality requirements includes:
and setting the first accessed client as a master client with high video stream quality requirement, and setting the clients accessed after the master client as slave clients with low video stream quality requirement.
Optionally, the determining a second video quality parameter according to the hardware configuration of the slave client and the network condition includes:
determining a video quality parameter corresponding to each slave client according to the hardware configuration and the network condition of each slave client;
and respectively taking the minimum frame number, the minimum video code rate and the minimum performance coding type in the video quality parameters corresponding to each slave client as the frame number, the video code rate and the coding type of the second video quality parameter.
Optionally, the method further comprises:
and when a new slave client access exists or a slave client exit exists, re-determining the second video quality parameter according to the video quality parameters corresponding to all currently accessed slave clients.
Optionally, the method further comprises:
when a video code rate change request input from the client is received, judging whether the video code rate corresponding to the client is the minimum video code rate in the video code rates of all the clients, if so, taking the video code rate input from the client as the video code rate of the video quality parameters of the client, and re-determining the video code rate in the second video quality parameters of each client;
when the frame number change request input by the slave client is received, judging whether the frame number corresponding to the slave client is the minimum frame number in the frame numbers of all the slave clients, if so, taking the frame number input by the slave client as the frame number of the video quality parameters of the slave client, and re-determining the frame number in the second video quality parameters of each slave client.
The present application further provides a video streaming device for sharing video frames among multiple users, comprising:
the client dividing module is used for dividing a plurality of clients into a master client and slave clients in advance according to different video stream quality requirements;
the frame image module is used for acquiring a video frame image in real time;
a first encoding module, configured to encode the video frame image by using a first encoder based on a first video quality parameter to obtain a first video stream, and send the first video stream to the host client; the first video quality parameter is a parameter determined according to the hardware configuration and the network condition of the host client;
the second coding module is used for coding the video frame image by using a second coder based on a second video quality parameter to obtain a second video stream and sending the second video stream to the slave client; wherein the second video quality parameter is a parameter determined according to the hardware configuration and the network condition of the slave client.
The present application further provides a video streaming device for sharing video frames among multiple users, including:
a memory for storing a computer program;
a processor for executing the computer program to implement the steps of the video stream streaming method for multi-user shared video pictures as described in any of the above.
The video stream streaming method for sharing video pictures by multiple users comprises the steps of dividing multiple clients into a master client and slave clients according to different video stream quality requirements in advance; acquiring a video frame image in real time; based on the first video quality parameter, a first encoder is used for encoding the video frame image to obtain a first video stream, and the first video stream is sent to a main client; the first video quality parameter is a parameter determined according to hardware configuration and network conditions of a host client; based on the second video quality parameter, encoding the video frame image by using a second encoder to obtain a second video stream, and sending the second video stream to the slave client; wherein the second video quality parameter is a parameter determined according to a hardware configuration and a network condition from the client.
The method comprises the steps of dividing clients of a plurality of users into two different clients of a main client and a slave client, setting two groups of different video quality parameters based on actual conditions of the two clients, respectively encoding two video streams by adopting two encoders, correspondingly, the video streams obtained by encoding each group of video quality parameters can meet the requirements of the corresponding client on video quality, and further provide more suitable video streams for the clients of different types under the condition of not increasing complexity of an encoding process, thereby improving the problems of the clients such as blocking and the like, and improving the viewing experience of the users for viewing video pictures.
The application also provides a video stream streaming device and equipment for sharing the video pictures by multiple users, and the beneficial effects are achieved.
Drawings
In order to more clearly illustrate the embodiments or technical solutions of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.
Fig. 1 is a schematic flowchart of a video stream streaming method for sharing video pictures by multiple users according to an embodiment of the present application;
fig. 2 is a block diagram of a video streaming apparatus for sharing video frames among multiple users according to an embodiment of the present invention.
Detailed Description
Svc (scalable Video coding) is a Video coding method that can code Video streams of various formats for the same frame picture. When multiple users share a video picture, the requirements of the client of each user on the quality and format of the video stream are different based on the difference of the hardware configuration and the network state of the client computer used by different users. The SVC coding can obtain video streams with different formats, and thus, suitable video streams can be provided for different users.
However, in the SVC, video streams of various formats need to be coded simultaneously, the coding process is often complicated, and the coding process occupies more operation space at the host to a certain extent, which affects the operation performance at the host to a certain extent; and the process of encoding which format of video stream should be allocated to each client after obtaining a plurality of video streams is also relatively complicated.
Therefore, the technical scheme capable of realizing video sharing by multiple users is provided in the application, and different video stream images are provided for different users on the basis that the video coding process is not too complicated.
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, fig. 1 is a schematic flowchart of a video stream streaming method for sharing a video picture by multiple users according to an embodiment of the present application, where the video stream streaming method may include:
s11: the method comprises the steps of dividing a plurality of clients into a master client and a slave client according to different video stream quality requirements in advance.
The client in this embodiment is a computer client of each user sharing a video. The division principle of the master client and the slave client can be various. For example, a first client accessing the host side may be used as a master client, and a second client accessing the host side may be used as a slave client. Generally, the first client accessing the host is a user for performing operation control on video images. For example, when a user performs a game process operation on the host computer through the client, a plurality of other users watch the video image of the game process at the same time, and at this time, the quality of the video displayed by the client of the user performing the game process operation is the highest.
In addition, hardware configuration and network state among the clients can be evaluated, and the client with high hardware configuration and good network state can obviously display video stream with better video quality and can be used as a master client, otherwise, the client with relatively low hardware configuration and relatively poor network state can be used as a slave client.
The division can also be performed according to encoding formats decodable by clients, for example, clients capable of decoding videos in the H.265 format are all divided into master clients, and clients capable of decoding videos in the H.264 format are divided into slave clients.
There are many other ways to divide the master client and the slave client, which are not listed in this embodiment.
S12: video frame images are obtained in real time.
The source of the video frame image can be a video frame image obtained by screen capture of a display picture on the host side, or a video frame image generated in other manners.
S13: and based on the first video quality parameter, encoding the video frame image by using a first encoder to obtain a first video stream, and sending the first video stream to the main client.
The first video quality parameter is a parameter determined according to the hardware configuration and the network condition of the host client.
Optionally, the process of setting the first video quality parameter may include:
detecting and obtaining network condition information at least comprising network bandwidth, packet loss proportion, delay time, image rendering time, image coding time and I frame request frequency corresponding to a main client;
when the delay time, the image rendering time and the image coding time are not greater than the corresponding time threshold, the frame number of the first video quality parameter is the theoretical decodable frame number of the host client;
when the packet loss proportion is smaller than the preset proportion and the I frame request frequency is smaller than the preset frequency threshold, the video code rate of the first video quality parameter is equal to the network bandwidth;
the encoding type in the first video quality parameter is set according to a host-client decodable encoding format.
It should be noted that the delay time is a time between a time when the host sends the video stream to the client and a time when the client receives feedback information of the video stream, and the image rendering time is a time required by the client to perform image processing such as denoising on the video frame image; and the image coding time refers to the time required for coding the video frame images into a video stream. The image rendering time and the image coding time can be directly obtained through detection, and the delay time can be determined through real-time interactive data between a host side and a client side.
The theoretical decodable frame number of the host client is related to the hardware configuration of the host client, the decoding software used and other factors. Similarly, the theoretical network bandwidth for the host client may be the network bandwidth provided by the provider under normal conditions, or the average bandwidth of the network operation, etc.
Under the condition that an interactive network between a client and a host is normal, the theoretical decodable frame number of a main client is equal to the actual decodable frame number of the main client, the size of a video code rate is set according to a network bandwidth and fluctuates along with the fluctuation of the network bandwidth, and if the network state or the operation speed of the main client is slow, the actual decodable frame number and the network bandwidth of the main client can fluctuate.
When setting the video quality parameter, the frame number and the video bitrate of the first video quality parameter are set according to the decodable frame number and the network bandwidth of the host client, and the decodable frame number should be understood as a theoretical decodable frame number of the host client corresponding to the hardware configuration and the decoding software of the host client.
In addition, the encoding type is mainly related to the encoding format decodable by the client, and specifically, the hardware configuration of the client. For example, h.264 and h.265 formats may be included.
As mentioned above, the division between the master client and the slave client may be divided according to the coding format decodable by the client, and if the master client is a client capable of decoding the video in the h.265 format and the slave client is a client capable of decoding the video in the h.264 format, obviously, the coding type in the video quality parameter of the master client is the h.265 format.
S14: and based on the second video quality parameter, encoding the video frame image by using a second encoder to obtain a second video stream, and sending the second video stream to the slave client.
Wherein the second video quality parameter is a parameter determined according to a hardware configuration and a network condition from the client.
Although the plurality of clients are divided into the master client and the slave clients, the manner of determining the second video quality parameter is similar to the process of determining the first video quality parameter, and is also set according to factors such as the corresponding delay time, packet loss ratio, network bandwidth and I-positive request frequency of the slave clients.
However, based on the above-mentioned division manner between the master client and the slave clients, the master client may only have one or may include a plurality of the master clients, and the slave clients generally include a plurality of the slave clients.
When only one host client exists, based on the information such as the delay time length and the packet loss condition corresponding to the host client, a unique group of video quality parameters can be determined to serve as the first video quality parameters. If the number of the host clients is multiple, each host client can obtain one video quality parameter, and it is necessary to comprehensively set a reasonable first video quality parameter based on the video quality parameters of all the host clients.
Similarly, the number of slave clients is also plural, and there is a problem that the second video quality parameter is set based on the video quality parameters corresponding to the plural slave clients.
Therefore, in an optional embodiment of the present application, the process of setting the second video quality parameter may include:
determining a video quality parameter corresponding to each slave client according to the hardware configuration and the network condition of each slave client; this process is similar to the process described above for setting video quality parameters for the host client.
And respectively taking the minimum frame number, the minimum video code rate and the minimum performance coding type in the video quality parameters corresponding to each slave client as the frame number, the video code rate and the coding type of the second video quality parameter.
In order to ensure that the slave client with the worst configuration network condition among the plurality of clients can receive the video stream, and avoid the problem of blocking, in this embodiment, the second video quality parameter is set by the video stream with the worst acceptable quality among the plurality of slave clients.
For example, when the number of frames from the client a is the smallest, the video bitrate from the client B is the smallest, and the encoding format that can be encoded from the client C is the lowest-performance encoding format, the video bitrate corresponding to the client B and the encoding type corresponding to the client C are used as the second video quality parameter according to the number of frames corresponding to the client a. Of course, the slave client A, B, C may be the same slave client, or two of them may be the same slave client.
Of course, for the master client, when there are a plurality of master clients, the process of determining the first video quality parameter based on the video quality parameter of each master client is similar to the process of determining the second video quality parameter based on the video quality parameters of the plurality of slave clients, and thus will not be described in detail.
In addition, in the actual application process, the client accessing the host is not fixed, and in the process of video streaming, there are new clients that are continuously accessed and exited, so in an optional embodiment of the present application, the method may further include:
and when a new slave client access exists or a slave client exit exists, re-determining the second video quality parameter according to the corresponding video quality parameters of all currently accessed slave clients.
It can be understood that, when there is a new slave client access to obtain that there is one or more slave clients to exit, the minimum frame number, the minimum video bitrate, and the minimum performance coding type in the video quality parameters corresponding to the slave clients currently in the access state with the host may all change, so that the second video quality parameter needs to be updated again according to the video quality parameters of the currently accessed slave clients.
Of course, if there are multiple host clients, or only one host client but the host client is changed, the first video quality parameter also needs to be determined again.
It should be noted that, in general, the master client refers to a client with higher video quality requirement, and the slave client refers to a client with relatively lower video quality requirement. In the embodiment, the clients are only divided into two types, and the clients with approximate video quality requirements or capable of receiving and displaying video quality are divided into the clients of the same type, so that video quality parameters determined for hardware configuration and network conditions of the clients of the same type can meet the quality requirements of the clients on the video streams to the maximum extent, and different requirements of different clients on the video stream quality are met to a certain extent.
Obviously, compared with the SVC coding method, in this embodiment, only two formats of video stream coding are performed, so that the complexity of the coding process is greatly reduced, and the problem of video blocking occurring in the client with poor network condition and low hardware configuration can be avoided on the basis of not reducing the video quality of the client with good network condition and high hardware configuration.
To sum up, in the present application, the requirements of each client for video quality are considered for the clients of multiple users sharing video according to the network state, hardware configuration, and other factors, the multiple clients are divided into two types, i.e., a master client and a slave client, and for the specific network state and hardware configuration of the master client and the slave client, a first video quality parameter satisfying the video quality requirements of the master client and a second video quality parameter satisfying the video quality requirements of the slave client are set, and therefore two different sets of video streams are obtained by using two different encoders according to the first video quality parameter and the second video quality parameter, under the condition of avoiding the encoding process being too complicated, the different requirements of different types of clients for video streams are satisfied, on the basis of not reducing the video quality of the clients with good network condition and high hardware configuration, the problem that video blockage occurs to a client side with poor network condition and low hardware configuration is avoided, and the use experience of sharing video pictures by a plurality of users is further improved.
Based on any of the above embodiments, as shown in the foregoing, the state of the communication network may not always be stable during the process of video streaming between the client and the host. For this reason, for the first video quality parameter of the master client and the second video quality parameter of the slave client to be appropriately adjusted according to the actual video streaming situation, in an optional embodiment of the present application, the method may further include:
when any one of the delay time length, the image rendering time and the image coding time is greater than the corresponding time threshold, the frame number of the first video quality parameter is half of the theoretical decodable frame number of the main client, and after a first preset time length, the frame number of the first video quality parameter is adjusted to the theoretical decodable frame number;
when the I frame request frequency is not less than the preset frequency threshold, the video code rate of the first video quality parameter is set to be less than the preset code rate of the current network bandwidth, and after a second preset time, the video code rate of the first video quality parameter is adjusted to be equal to the current network bandwidth.
As mentioned above, the frame number of the first video quality parameter is normally the theoretical decodable frame number of the host client. When the network condition is bad or the rendering image and the encoding image time are prolonged, it is necessary to appropriately reduce the number of frames in the first video quality parameter, which may be a half of the theoretical decodable number of frames of the main client, or two thirds of the theoretical decodable number of frames, and so on.
For example, for the case that the number of frames of the video frame image obtained per second is equal to the theoretical decodable number of frames of the main client, if the number of frames in the first video quality parameter is reduced to half of the theoretical decodable number of frames, one frame of video frame image is selected every other frame of video frame image to carry out video stream coding, and thus the reduction of the number of frames by half can be realized; and similarly, if the frame number needs to be reduced to two thirds of the theoretical decodable frame number of the main client, selecting two frames of video frame images at intervals of one frame of video image for video stream coding.
Of course, the instability of the network status generally does not last for a long time, and therefore, after the frame number of the first video quality parameter is reduced by half, the frame number can be restored to the theoretical decodable frame number of the client again after the first preset duration. The first preset time period may be 10s, 20s, 30s, etc.
Similarly, when the I-frame request is too frequent, the video bitrate of the first video quality parameter also needs to be adjusted down, for example, to 2M, and then adjusted to be equal to the current network bandwidth after lasting for a second preset duration, which may be equal to the first preset duration.
Optionally, besides the frame number, the video code rate, and the like need to be adjusted due to the change of the I-frame request frequency, the delay, the image rendering time, the image encoding time, and the like, the decoding time and the decoded frame number for decoding the first video stream by the primary client also need to be received in real time;
and if the decoding time exceeds the preset duration or the number of the decoded frame numbers is less than the preset number, reducing the video code rate of the first video quality parameter.
The decoding duration is related to the working performance of the client, and the longer the decoding duration is, the worse the performance of the current client is, so that the client is not suitable for sending a video stream with high quality, and the video code rate should be adjusted downward to relieve the decoding pressure of the client.
The ratio between the decoded frame number and the frame number of the video stream image sent by the host reflects the packet loss rate of the client to a certain extent, and the decoded frame number refers to the already decoded frame image, and the greater the number, the less the packet loss.
In both the master client and the slave client, if the decoding time is too long or the packet loss rate is too high, the video bitrate should be reduced, and similarly, the video bitrate may be reduced to a preset bitrate, for example, 2M, and then recovered after a certain time.
In addition to the above changing the video quality parameter based on the status of the streaming video stream between the master client and the slave client, in another optional embodiment of the present application, the method may further include:
when a video code rate change request input from the client is received, judging whether the video code rate corresponding to the client is the minimum video code rate in the video code rates of all the clients, if so, taking the video code rate input from the client as the video code rate of the video quality parameter of the client, and re-determining the video code rate in the second video quality parameter of each client;
when a frame number change request input from the client is received, judging whether the frame number corresponding to the client is the minimum frame number in all the frame numbers of the slave clients, if so, taking the frame number input from the client as the frame number of the video quality parameters of the slave clients, and re-determining the frame number in the second video quality parameters of each slave client.
As described above, the second video quality parameter is set based on the minimum frame number, the minimum video bitrate, and the minimum performance coding type in the video quality parameters corresponding to all the slave clients, and in practical applications, some users may consider that the video quality is too poor, and require to obtain a higher-quality video stream. At this time, the user can input a change request for changing the frame number and the video code rate required to be changed through the client.
If the frame number or the video code rate in the video quality parameter corresponding to the slave client sending the change request is the minimum of all the slave clients, the video code rate and the frame number set in the change request of the slave client can be used as the video code rate and the frame number of the slave client, the minimum video code rate and the minimum frame number of all the slave clients are determined again, and the second video quality parameter is determined according to the determined minimum video code rate and the minimum frame number.
Of course, if the frame number and the video code rate of the video quality parameter corresponding to the slave client of the change request are not all the minimum in all the slave clients, the video code rate and the frame number in the change request can be recorded first, and when the client exits, whether the video code rate or the frame number in the change request is the minimum in all the slave clients is judged again, and if so, the second video quality parameter is adjusted based on the video code rate and the frame number in the change request.
Similarly, if there are a plurality of host clients, the first video quality parameter may be changed in the same manner.
Further, although the first video quality parameter and the second video quality parameter need to be frequently adjusted according to the actual condition in the streaming process of the video stream, if the adjustment is too frequent, the normal streaming of the video stream is also affected, therefore, when the first video quality parameter and the second video quality parameter are adjusted, whether the time length from the last adjustment reaches the set time length or not can be firstly judged, if so, the adjustment is performed, and if not, the adjustment is not performed.
In the following, the video streaming device for sharing video pictures by multiple users according to the embodiments of the present invention is introduced, and the video streaming device for sharing video pictures by multiple users described below and the video streaming method for sharing video pictures by multiple users described above may be referred to correspondingly.
Fig. 2 is a block diagram of a video streaming device for sharing video pictures among multiple users according to an embodiment of the present invention, and the video streaming device for sharing video pictures among multiple users shown in fig. 2 may include:
the client dividing module 100 is configured to divide a plurality of clients into a master client and a slave client according to different video stream quality requirements in advance;
a frame image module 200, configured to obtain a video frame image in real time;
a first encoding module 300, configured to encode the video frame image by using a first encoder based on a first video quality parameter to obtain a first video stream, and send the first video stream to the host client; the first video quality parameter is a parameter determined according to the hardware configuration and the network condition of the host client;
a second encoding module 400, configured to encode the video frame image by using a second encoder based on a second video quality parameter to obtain a second video stream, and send the second video stream to the slave client; wherein the second video quality parameter is a parameter determined according to the hardware configuration and the network condition of the slave client.
The video stream streaming apparatus for sharing video pictures by multiple users of this embodiment is used to implement the video stream streaming method for sharing video pictures by multiple users, and therefore specific embodiments of the video stream streaming apparatus for sharing video pictures by multiple users can be seen in the foregoing embodiments of the video stream streaming method for sharing video pictures by multiple users, for example, the client partitioning module 100, the frame image module 200, the first encoding module 300, and the second encoding module 400 are respectively used to implement steps S11, S12, S13, and S14 in the video stream streaming method for sharing video pictures by multiple users.
The present application further provides an embodiment of a video streaming device for sharing video pictures among multiple users, where the device may include:
a memory for storing a computer program;
a processor for executing a computer program to implement the steps of the video stream streaming method for multi-user sharing of video pictures as described in any of the above.
The steps of the processor executing the video stream streaming method for sharing video pictures by multiple users may include:
dividing a plurality of clients into a master client and slave clients according to different video stream quality requirements in advance;
acquiring a video frame image in real time;
based on a first video quality parameter, encoding the video frame image by using a first encoder to obtain a first video stream, and sending the first video stream to the main client; the first video quality parameter is a parameter determined according to the hardware configuration and the network condition of the host client;
based on a second video quality parameter, encoding the video frame image by using a second encoder to obtain a second video stream, and sending the second video stream to the slave client; wherein the second video quality parameter is a parameter determined according to the hardware configuration and the network condition of the slave client.
The memory may include Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The device in this embodiment divides the clients into two types, and sets reasonable video quality parameters for the hardware status, the network device, and other conditions of each type of client, so that two different encoding processes are performed simultaneously based on different video quality parameters to obtain two different video streams to be provided to the two different types of clients, and under the condition that the complexity of the encoding process is not increased as much as possible, the requirements of all clients on the video streams are met as much as possible, the problem of video blockage is avoided, and the user experience is improved.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include elements inherent in the list. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element. In addition, parts of the above technical solutions provided in the embodiments of the present application, which are consistent with the implementation principles of corresponding technical solutions in the prior art, are not described in detail so as to avoid redundant description.
The principles and embodiments of the present invention are explained herein using specific examples, which are presented only to assist in understanding the method and its core concepts. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
Claims (10)
1. A method for streaming video streams of a video picture shared by multiple users is characterized by comprising the following steps:
dividing a plurality of clients into a master client and slave clients according to different video stream quality requirements in advance;
acquiring a video frame image in real time;
based on a first video quality parameter, encoding the video frame image by using a first encoder to obtain a first video stream, and sending the first video stream to the main client; the first video quality parameter is a parameter determined according to the hardware configuration and the network condition of the host client;
based on a second video quality parameter, encoding the video frame image by using a second encoder to obtain a second video stream, and sending the second video stream to the slave client; wherein the second video quality parameter is a parameter determined according to the hardware configuration and the network condition of the slave client.
2. The method of claim 1, wherein the step of determining the first video quality parameter based on the hardware configuration and network conditions of the host client comprises:
detecting and obtaining network condition information at least comprising network bandwidth, packet loss proportion, delay time, image rendering time, image coding time and I frame request frequency corresponding to the main client;
when the delay time, the image rendering time and the image coding time are not greater than the corresponding time thresholds, the frame number of the first video quality parameter is the theoretical decodable frame number of the host client;
when the packet loss proportion is smaller than a preset proportion and the I frame request frequency is smaller than a preset frequency threshold, the size of the video code rate of the first video quality parameter is equal to the size of the network bandwidth;
setting an encoding type in the first video quality parameter according to the primary client decodable encoding format.
3. The method as claimed in claim 2, wherein the step of determining the first video quality parameter according to the hardware configuration and the network condition of the host client comprises:
when any one of the delay time length, the image rendering time and the image encoding time is greater than a corresponding time threshold, the frame number of the first video quality parameter is half of the theoretical decodable frame number of the main client, and after a first preset time length, the frame number of the first video quality parameter is adjusted to the theoretical decodable frame number;
when the I frame request frequency is not less than a preset frequency threshold, setting the video code rate of the first video quality parameter to be less than the current preset code rate of the network bandwidth, and after a second preset duration, adjusting the video code rate of the first video quality parameter to be equal to the current network bandwidth.
4. The method for streaming video streams of a video picture shared by multiple users according to claim 2, wherein after transmitting the first video stream to the primary client, further comprising:
receiving decoding time and a decoding frame sequence number of the first video stream decoded by the main client;
and if the decoding time exceeds a preset duration or the number of the decoded frame numbers is less than a preset number, reducing the video code rate of the first video quality parameter.
5. The method of claim 1, wherein the dividing of the plurality of clients into the master client and the slave clients according to different video stream quality requirements comprises:
and setting the first accessed client as a master client with high video stream quality requirement, and setting the clients accessed after the master client as slave clients with low video stream quality requirement.
6. The method for streaming video streams of a multi-user shared video picture according to claim 1, wherein the process of determining the second video quality parameter according to the hardware configuration and the network condition of the slave client comprises:
determining a video quality parameter corresponding to each slave client according to the hardware configuration and the network condition of each slave client;
and respectively taking the minimum frame number, the minimum video code rate and the minimum performance coding type in the video quality parameters corresponding to each slave client as the frame number, the video code rate and the coding type of the second video quality parameter.
7. The method for streaming video of a video picture shared by multiple users of claim 6, further comprising:
and when a new slave client access exists or a slave client exit exists, re-determining the second video quality parameter according to the video quality parameters corresponding to all currently accessed slave clients.
8. The method for streaming video of a video picture shared by multiple users of claim 6, further comprising:
when a video code rate change request input from the client is received, judging whether the video code rate corresponding to the client is the minimum video code rate in the video code rates of all the clients, if so, taking the video code rate input from the client as the video code rate of the video quality parameters of the client, and re-determining the video code rate in the second video quality parameters of each client;
when the frame number change request input by the slave client is received, judging whether the frame number corresponding to the slave client is the minimum frame number in the frame numbers of all the slave clients, if so, taking the frame number input by the slave client as the frame number of the video quality parameters of the slave client, and re-determining the frame number in the second video quality parameters of each slave client.
9. A video streaming apparatus for sharing video frames among multiple users, comprising:
the client dividing module is used for dividing a plurality of clients into a master client and slave clients in advance according to different video stream quality requirements;
the frame image module is used for acquiring a video frame image in real time;
a first encoding module, configured to encode the video frame image by using a first encoder based on a first video quality parameter to obtain a first video stream, and send the first video stream to the host client; the first video quality parameter is a parameter determined according to the hardware configuration and the network condition of the host client;
the second coding module is used for coding the video frame image by using a second coder based on a second video quality parameter to obtain a second video stream and sending the second video stream to the slave client; wherein the second video quality parameter is a parameter determined according to the hardware configuration and the network condition of the slave client.
10. A video streaming apparatus for sharing video pictures among multiple users, comprising:
a memory for storing a computer program;
a processor for executing said computer program to implement the steps of the method for streaming a video stream of multi-user shared video pictures according to any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011620987.9A CN112822519A (en) | 2020-12-30 | 2020-12-30 | Video stream streaming method, device and equipment for sharing video pictures by multiple users |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011620987.9A CN112822519A (en) | 2020-12-30 | 2020-12-30 | Video stream streaming method, device and equipment for sharing video pictures by multiple users |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112822519A true CN112822519A (en) | 2021-05-18 |
Family
ID=75854538
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011620987.9A Pending CN112822519A (en) | 2020-12-30 | 2020-12-30 | Video stream streaming method, device and equipment for sharing video pictures by multiple users |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112822519A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113891153A (en) * | 2021-09-30 | 2022-01-04 | 杭州雾联科技有限公司 | Cloud game streaming processing method, device and medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101365125A (en) * | 2008-09-27 | 2009-02-11 | 腾讯科技（深圳）有限公司 | Multipath video communication method and system |
| CN102045312A (en) * | 2009-10-21 | 2011-05-04 | 三星电子（中国）研发中心 | Self-adaptative transcoding method and system |
| CN108093294A (en) * | 2016-11-21 | 2018-05-29 | ***通信有限公司研究院 | A kind of terminal screen live broadcasting method and terminal |
| CN109640116A (en) * | 2018-12-24 | 2019-04-16 | 广州微算互联信息技术有限公司 | The coding method of cloud mobile phone network video and server |
| CN109831637A (en) * | 2017-11-23 | 2019-05-31 | 中兴通讯股份有限公司 | Data transmission method, server and system in a kind of video conference |
| WO2020015678A1 (en) * | 2018-07-19 | 2020-01-23 | 北京达佳互联信息技术有限公司 | Media code rate self-adaptation method and apparatus, computer device, and storage medium |
-
2020
- 2020-12-30 CN CN202011620987.9A patent/CN112822519A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101365125A (en) * | 2008-09-27 | 2009-02-11 | 腾讯科技（深圳）有限公司 | Multipath video communication method and system |
| CN102045312A (en) * | 2009-10-21 | 2011-05-04 | 三星电子（中国）研发中心 | Self-adaptative transcoding method and system |
| CN108093294A (en) * | 2016-11-21 | 2018-05-29 | ***通信有限公司研究院 | A kind of terminal screen live broadcasting method and terminal |
| CN109831637A (en) * | 2017-11-23 | 2019-05-31 | 中兴通讯股份有限公司 | Data transmission method, server and system in a kind of video conference |
| WO2020015678A1 (en) * | 2018-07-19 | 2020-01-23 | 北京达佳互联信息技术有限公司 | Media code rate self-adaptation method and apparatus, computer device, and storage medium |
| CN109640116A (en) * | 2018-12-24 | 2019-04-16 | 广州微算互联信息技术有限公司 | The coding method of cloud mobile phone network video and server |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113891153A (en) * | 2021-09-30 | 2022-01-04 | 杭州雾联科技有限公司 | Cloud game streaming processing method, device and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Petrangeli et al. | An http/2-based adaptive streaming framework for 360 virtual reality videos | |
| US9445136B2 (en) | Signaling characteristics of segments for network streaming of media data | |
| US8635357B2 (en) | Dynamic selection of parameter sets for transcoding media data | |
| TWI511544B (en) | Techniques for adaptive video streaming | |
| CN102625106B (en) | Scene self-adaptive screen encoding rate control method and system | |
| US20140348246A1 (en) | Video Conversion Method and Apparatus | |
| EP2785068B1 (en) | Video image data delivery device, method and system | |
| US8842159B2 (en) | Encoding processing for conferencing systems | |
| US20180077385A1 (en) | Data, multimedia & video transmission updating system | |
| WO2008045963A2 (en) | Digital content buffer for adaptive streaming | |
| US11856191B2 (en) | Method and system for real-time content-adaptive transcoding of video content on mobile devices to save network bandwidth during video sharing | |
| EP2710808A1 (en) | Distributing audio video content | |
| US10869048B2 (en) | Method, device and system for transmitting and receiving pictures using a hybrid resolution encoding framework | |
| KR20150131175A (en) | Resilience in the presence of missing media segments in dynamic adaptive streaming over http | |
| CN110956683A (en) | Information processing system, terminal device, and computer-readable medium | |
| CN112866746A (en) | Multi-path streaming cloud game control method, device, equipment and storage medium | |
| US10708667B1 (en) | Combining fragments with different encodings | |
| CN113068001A (en) | Data processing method, device, equipment and medium based on cascade camera | |
| CN111263243A (en) | Video coding method and device, computer readable medium and electronic equipment | |
| CN112822519A (en) | Video stream streaming method, device and equipment for sharing video pictures by multiple users | |
| JP4447443B2 (en) | Image compression processor | |
| CN116962613A (en) | Data transmission method and device, computer equipment and storage medium | |
| CN115379235A (en) | Image decoding method and device based on buffer pool, readable medium and electronic equipment | |
| CN114007137A (en) | ROI-based video hierarchical coding method, device and medium | |
| KR20120012089A (en) | System and method for proving video using scalable video coding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210518 |
|
| RJ01 | Rejection of invention patent application after publication |