CN116320641B - Video data transmission method and system - Google Patents
Video data transmission method and system Download PDFInfo
- Publication number
- CN116320641B CN116320641B CN202310564881.9A CN202310564881A CN116320641B CN 116320641 B CN116320641 B CN 116320641B CN 202310564881 A CN202310564881 A CN 202310564881A CN 116320641 B CN116320641 B CN 116320641B
- Authority
- CN
- China
- Prior art keywords
- communication channel
- predicted
- data packet
- video
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47217—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/80—Responding to QoS
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
- H04N21/23605—Creation or processing of packetized elementary streams [PES]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/238—Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
- H04N21/2385—Channel allocation; Bandwidth allocation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
- H04N21/2402—Monitoring of the downstream path of the transmission network, e.g. bandwidth available
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
- H04N21/44245—Monitoring the upstream path of the transmission network, e.g. its availability, bandwidth
Abstract
The invention discloses a video data transmission method and a video data transmission system, and relates to the field of streaming media communication. The invention comprises the following steps: sequentially acquiring a key frame and a predicted frame in a video; acquiring a grouping sequence of key frames and predicted frames; packaging the key frames and the predicted frames into key data packets and predicted data packets respectively according to the grouping sequence of the key frames and the predicted frames; acquiring the grouping sequence of the key data packet and the predicted data packet according to the grouping sequence of the key frame and the predicted frame; establishing a bidirectional communication channel and a unidirectional communication channel which are communicated with a client; transmitting critical data packets and/or predicted data packets via a bi-directional communication channel and/or a uni-directional communication channel; acquiring video buffer time length of a client through a bidirectional communication channel and packet loss rate of the unidirectional communication channel; and adjusting the transmission channels of the key data packets and the predicted data packets according to the video buffering time length of the client and the packet loss rate of the unidirectional communication channel. The invention effectively improves the integrity and the rate of receiving video data by the client.
Description
Technical Field
The invention belongs to the technical field of streaming media communication, and particularly relates to a video data transmission method and a video data transmission system.
Background
In the process of providing streaming media services for clients such as a browser, a server may cause packet loss of video data in the transmission process due to unstable network transmission and the like, so that the clients cannot analyze stable and smooth pictures.
In order to cope with the problem of packet loss in video data network transmission, a transmission control protocol (TCP, transmission Control Protocol) is generally adopted to transmit video data at the network layer, but because the TCP protocol requires three handshakes and four wavings to complete one communication in the communication process, the transmission response speed is slow, and the video watching experience of the client is also affected.
Disclosed in the patent with publication number CN105430320A is an audio/video media data transmission method based on UDP and TCP, which is applied to VOIP, audio/video conference service, and comprises the following steps: before the terminal or the server sends the RTP packet, an RTP/UDP channel is created, and a TCP connection is additionally established; during normal transmission, RTP packets are sent and received by using an RTP/UDP channel; when the receiving end finds that the packet is lost, a NACK request is generated and sent to the sending end through TCP connection, and retransmission of the lost packet is requested; after receiving the NACK request, the sending end analyzes the NACK request, and sends the RTP packet to be retransmitted to the receiving end through TCP connection. However, the scheme realizes the monitoring and packet loss retransmission of the RTP/UDP channel through the TCP connection, so that the advantage of high response speed of the UDP protocol is lost, and the advantage of stability and reliability of the TCP protocol is lost.
Disclosure of Invention
The invention aims to provide a video data transmission method and a video data transmission system, which are used for distributing different data packets transmitted by different data transmission channels through analysis of video playing states of a client, so that the integrity and the rate of receiving video data of the client are effectively improved.
In order to solve the technical problems, the invention is realized by the following technical scheme:
the present invention provides a video data transmission method for execution in a server of a streaming media service, comprising:
sequentially acquiring a key frame and a predicted frame in a video;
acquiring a grouping sequence of the key frames and the predicted frames;
packaging the key frame and the predicted frame into a key data packet and a predicted data packet according to the grouping sequence of the key frame and the predicted frame;
acquiring the grouping sequence of the key data packet and the predicted data packet according to the grouping sequence of the key frame and the predicted frame;
establishing a bidirectional communication channel and a unidirectional communication channel which are communicated with a client;
transmitting the critical data packet and/or the predicted data packet via the bidirectional communication channel and the unidirectional communication channel;
acquiring video buffering time length of the client and packet loss rate of the unidirectional communication channel through the bidirectional communication channel;
and adjusting the transmission channels of the key data packet and the predicted data packet according to the video buffering duration of the client and the packet loss rate of the unidirectional communication channel.
The present invention also provides a video data transmission method for execution in a client of a streaming media service, comprising,
receiving the key data packet and the prediction data packet generated according to the video data transmission method;
performing video decoding according to the key data packet and the prediction data packet to obtain a decoded video at the client;
acquiring the packet loss rate of the unidirectional communication channel according to the grouping sequence of the key data packet and the predicted data packet received by the client;
acquiring the playing progress of the decoded video of the client;
acquiring video buffer time length of the client according to the decoded video and the playing progress of the decoded video of the client;
and transmitting the video buffer time length of the client and the packet loss rate of the unidirectional communication channel through the bidirectional communication channel.
The invention also discloses a video data transmission system, which comprises,
the server is used for sequentially acquiring key frames and predicted frames in the video;
acquiring a grouping sequence of the key frames and the predicted frames;
packaging the key frame and the predicted frame into a key data packet and a predicted data packet according to the grouping sequence of the key frame and the predicted frame;
acquiring the grouping sequence of the key data packet and the predicted data packet according to the grouping sequence of the key frame and the predicted frame;
establishing a bidirectional communication channel and a unidirectional communication channel which are communicated with a client;
transmitting the critical data packet and/or the predicted data packet via the bidirectional communication channel and the unidirectional communication channel;
acquiring video buffering time length of the client and packet loss rate of the unidirectional communication channel through the bidirectional communication channel;
adjusting the sending channels of the key data packet and the predicted data packet according to the video buffering duration of the client and the packet loss rates of the key data packet and the unidirectional communication channel;
the system also comprises a client, a server and a client, wherein the client is used for receiving the key data packet and the prediction data packet;
performing video decoding according to the key data packet and the prediction data packet to obtain a decoded video at the client;
acquiring the packet loss rate of the unidirectional communication channel according to the grouping sequence of the key data packet and the predicted data packet received by the client;
acquiring the playing progress of the decoded video of the client;
acquiring video buffer time length of the client according to the decoded video and the playing progress of the decoded video of the client;
and transmitting the video buffer time length of the client and the packet loss rate of the unidirectional communication channel through the bidirectional communication channel.
According to the invention, the picture frames in the video are divided into the key frames and the predicted frames, the key frames and the predicted frames are respectively packed into the key data packets and the predicted data packets of the network layer, and finally, the video watching experience of the client is ensured by adjusting the transmission channels of the key data packets and the predicted data packets.
Of course, it is not necessary for any one product to practice the invention to achieve all of the advantages set forth above at the same time.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a video data transmission method according to the present invention;
fig. 2 is a schematic diagram of functional modules and information flow of a video data transmission system according to the present invention;
FIG. 3 is a schematic diagram illustrating the structure of a key frame, a predicted frame and a bi-directional predicted frame according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of the communication of the bi-directional communication channel according to the present invention;
FIG. 5 is a schematic diagram illustrating the implementation of step S6 according to the present invention;
FIG. 6 is a schematic diagram showing the implementation steps of step S65 according to the present invention;
FIG. 7 is a schematic diagram illustrating the implementation of step S8 according to the present invention;
FIG. 8 is a schematic diagram illustrating the implementation of step S81 according to the present invention;
FIG. 9 is a schematic diagram illustrating the implementation of step S82 according to the present invention;
FIG. 10 is a schematic diagram showing the implementation steps of step S85 according to the present invention;
fig. 11 is a second flowchart of a video data transmission method according to the present invention.
In the drawings, the list of components represented by the various numbers is as follows:
1-server, 2-client.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In the process that the server provides the streaming media data service for the client, the video data resource is required to be firstly called from the database of the server, but because the volume of the original video is too large, streaming media service providers such as an online video website can compress the video and store the compressed video, and then the compressed data stream is sent to the client to be decoded and played.
The open system interconnection communication reference model (Open System Interconnection Reference Model, abbreviated OSI) may be employed for analysis during video data streaming. The video pushing process of the server is used for transmitting the video stream in a transmission layer in the process of packaging the video stream into data packets for transmission, and the main transmission protocols are TCP (transmission control protocol) and UDP (user datagram protocol). Among other things, the TCP protocol provides a reliable data streaming service. When data interfered by errors is transmitted, or network faults are raised, or network load is too heavy to make the basic transmission system of the internet work normally, the reliability of communication needs to be ensured through other protocols. The UDP protocol is used to process packets as the TCP protocol, and in the OSI model, both are located at the transport layer, at the upper layer of the IP protocol. The UDP protocol has the disadvantage of not providing packet packets, assembling and not ordering packets, i.e. it is not known whether it arrives safely and completely after the message is sent, but also because it has a small response delay, it is suitable for transmitting video streams.
In summary, the bidirectional communication channel established according to the TCP protocol has high reliability, and the unidirectional communication channel established according to the UDP protocol has low latency. In order to transmit stable and reliable video pictures in an unreliable network environment, the present invention provides the following scheme.
Referring to fig. 1 and 2, the present invention provides a video data transmission system, in which video is extracted from a database of a server before transmission, and the server performs step S1 to extract key frames and predicted frames according to a compression encoding mode of the video. As shown in fig. 3, after the frames in the video are compressed, a plurality of groups of pictures (GOP) are formed, and each group of pictures includes a key frame (I) and a plurality of predicted frames (P), where the key frame is obtained by static compression of the original frames in the video, and generally only details that are difficult to be identified by human eyes are lost. The prediction frame is not, but mainly records the difference between the corresponding original picture frame and the key frame, so the data compression in the prediction frame is higher, but the decoding of the prediction frame depends on the corresponding key frame. The bi-directional predicted frame (B) is more specific, and also has a higher compression ratio, but it can refer to the key frame and the predicted frame before and after the key frame and the predicted frame at the same time, and has higher recoverability. With this feature, reliable and stable video pictures can be transmitted in an unreliable network. Step S2 may then be performed to obtain a grouping sequence of key frames and predicted frames to facilitate subsequent sequential transmissions. Step S3 may be executed to package the key frame and the predicted frame into a key packet and a predicted packet according to the grouping sequence of the key frame and the predicted frame, where the corresponding grouping sequence is described in the key packet and the predicted packet. Step S4 may be performed to obtain the grouping sequence of the key data packets and the predicted data packets according to the grouping sequence of the key frames and the predicted frames, and the transmission is performed according to the grouping sequence at the time of video transmission.
Step S5 may be performed to establish a bidirectional communication channel and a unidirectional communication channel that communicate with the client, where the bidirectional communication channel may be established based on the TCP protocol, and the unidirectional communication channel may be established based on the UDP protocol. Step S6 may then be performed to transmit critical data packets and/or predictive data packets via the bi-directional communication channel and the uni-directional communication channel. Step S7 may be performed to obtain the video buffering duration of the client and the packet loss rate of the unidirectional communication channel via the bidirectional communication channel, that is, the information of the client is returned to the server by using the bidirectional connectivity coordinated by the TCP. And finally, step S8 can be executed to adjust the transmission channels of the key data packet and the predicted data packet according to the video buffer time length of the client and the packet loss rate of the unidirectional communication channel, and the stability of the picture frame corresponding to the data packet received by the client is maintained by adjusting the transmission channels of the key data packet and the predicted data packet.
Referring to fig. 4, a complex process of the server and the client in the process of establishing the bidirectional communication channel is specifically shown, and when handshaking (sending a data request or response) each time, the sent data is a TCP packet, and the TCP segment includes a source/destination address, a port number, an initial sequence number, a sliding window size, a window expansion factor, a maximum packet segment length, and the like. There are also some flag bits. From the figure, it can be seen that the basic steps of the three-way handshake are:
first handshake: the client transmits a connection request packet SYN (syn=j) to the server, and waits for a server response.
Second handshake: after receiving the client connection request packet SYN (syn=j), the server puts the client request packet SYN (syn=j) into its own unconnected queue, and at this time, the server needs to send two packets to the client.
(1) An acknowledgement packet ACK (ack=j+1) acknowledging its own receipt of the connection request is sent to the client, indicating to the client that its connection request is known.
(2) A connection inquiry request packet SYN (syn=k) is sent to the client, inquiring whether the client is ready to establish a connection, and data communication is performed. That is, during the second handshake, the server sends ACK (ack=j+1) and SYN (syn=k) packets to the client, and at this time, the server enters a syn_recv state, where the syn_recv state refers to a state when the server receives the SYN of the client and sends ACK after the server is passively opened, and specifically refers to a state when the server receives a connection request and waits for the peer to answer after sending the connection request in the network application.
Third handshake: after receiving the ACK (ack=j+1) and SYN (syn=k) packets from the server, the client knows that the server agrees to establish a connection, and needs to send a message that the connection has been established to the server. An acknowledgement packet ACK (ack=k+1) of connection establishment is sent to the server, and the server is informed of the SYN (syn=k) of the server that a connection has been established between us, and data communication can be performed.
After the ACK (ack=k+1) packet is sent, the server and the client enter an ESTABLISHED state at this time after receiving the ACK (ack=k+1), and data transmission starts, where the ESTABLISHED state means that the TCP connection is successful.
Next we discuss the present technical solution from the Client side, where the Client side in the present solution refers to a Server-Client side, i.e. a Client side in a Client-Server (C/S) structure, which may be a browser, or an application of a mobile terminal or a desktop terminal. After establishing the bi-directional communication channel and the uni-directional communication channel with the server, the client may first perform step S9 to receive the critical data packet and the predicted data packet, and after receiving, buffer the critical data packet and the predicted data packet for subsequent decoding. Step S10 may be performed to decode video according to the key packet and the predicted packet, to obtain decoded video at the client, and not decode the picture frame corresponding to the predicted packet that is not received. Step S11 may be executed to obtain the packet loss rate of the unidirectional communication channel according to the grouping sequence of the key data packet and the predicted data packet received by the client, where the packet loss rate is an important parameter reflecting the network status, so that the packet loss rate needs to be transmitted back to the server for later adjustment. Next, step S12 may be performed to obtain the playing progress of the decoded video of the client, and then step S13 may be performed to obtain the video buffering duration of the client according to the decoded video and the playing progress of the decoded video of the client, where the video buffering duration is not only an important parameter reflecting the network state, but also an important parameter affecting the user' S video watching experience, so that the video buffering duration needs to be transmitted back to the server. And finally, step S14 can be executed to send the video buffer time of the client and the packet loss rate of the unidirectional communication channel through the bidirectional communication channel, so that the rear server can adjust the communication channels of the key data packet and the predicted data packet.
From the video information density of the data packet, the key data frames in the key data packet can affect the decoding imaging of the whole group of pictures (GOP), while the predicted data packet and the picture frames in the bi-directional predicted data packet only affect the decoding imaging of the predicted data packet, so that the importance of the key data packet is significantly higher than that of the predicted data packet and the bi-directional predicted data packet. Meanwhile, because the data content part capacity of the data packets is the same or similar, the number of picture frames corresponding to each predicted data packet and the bi-directional predicted data packet is larger than that of the key data packet. In view of this, the use of the bi-directional communication channel to transmit the critical packets can ensure the picture decoding effect of the client. The use of unidirectional communication channels to transmit the predicted data packets and bi-directional predicted data packets can increase the smoothness of the client video picture.
Referring to fig. 5, since there is only one key data frame in each group of pictures (GOP), but the number of predicted data frames and bi-directional predicted data frames may be plural, and the communication rates of the bi-directional communication channel and the uni-directional communication channel are different, in order to ensure the video viewing synchronization rate of the client, step S6 may be performed first, where step S61 sends the key data packets via the bi-directional communication channel according to the grouping sequence of the key data packets. Step S62 may be performed next to acquire the time schedule of the transmitted critical packets in the video, and step S63 may be performed next to transmit the predicted packets via the bi-directional communication channel or the uni-directional communication channel in the order of grouping the predicted packets. Step S64 may be performed next to obtain the time schedule of the transmitted predicted data packet in the video, and step S65 may be performed finally to control the time schedules of the transmitted critical data packet and the transmitted predicted data packet to coincide in the video. And the key frames and the predicted frames received by the client are kept synchronous in a time control mode, so that the influence on the video watching experience of a user is avoided.
Referring to fig. 6, since the key frames, the predicted frames and the bi-directional predicted frames in the same group of pictures (GOP) need to be decoded simultaneously to be completely played, it needs to be ensured that the corresponding data packets are also synchronously and completely transmitted to the client, specifically, in the process of executing step S65, step S651 may be executed first, according to the temporal sequence of the key frames and the predicted frames in the video, to take the predicted frames between two adjacent key frames in the temporal sequence as a prediction unit group. Step S652 may be performed next to take the temporally previous key frame of the prediction unit group as the key frame to which the prediction unit group depends, and step S653 may be performed next to pack the key frame into a number of key packets. Step S654 may then be performed to package each predicted frame in the group of prediction units to which the key frame belongs into a number of predicted data packets. Step S655 may be performed to split the key data packet corresponding to each key frame and the prediction data packet corresponding to each prediction frame in the prediction unit group to which the key frame belongs into a partition to be transmitted. Step S656 may be performed to sort each partition to be sent according to the temporal sequence of the key frames in the video in each partition to be sent, to obtain the temporal sequence of the partition to be sent. Finally, step S657 may be performed to transmit the critical data packet and the predicted data packet in each partition to be transmitted according to the time sequence of the partitions to be transmitted.
Referring to fig. 7, because the packet loss rate and the video buffer duration have a great influence on the video viewing experience of the client, which is also a core index for embodying the viewing experience of the user, the communication channel of the data packet can be adjusted accordingly, and in the implementation process of step S8, step S81 can be executed first to obtain the corresponding relationship of the data packet sending rate ratio of the bidirectional communication channel and the unidirectional communication channel with respect to time as the corresponding relationship of the channel sending rate ratio with respect to time. Step S82 may be performed next to obtain, according to the correspondence between the channel transmission rate ratio and the time, the correspondence between the packet loss rate of the unidirectional communication channel and the channel transmission rate ratio, and the correspondence between the video buffering duration of the client and the channel transmission rate ratio. Step S83 may be performed next to acquire the set packet loss rate fault tolerance range, and step S84 may be performed next to acquire the set video buffer duration range. And finally, step S85 can be executed to adjust the transmission channels of the key data packets and the predicted data packets according to the corresponding relation of the packet loss rate of the unidirectional communication channel with respect to the transmission rate ratio of the channel, the corresponding relation of the video buffer time length of the client with respect to the transmission rate ratio of the channel, the set packet loss rate fault tolerance range and the set video buffer time length range. It should be noted that, the foregoing set fault tolerance range of packet loss rate is adjusted and set by a technician, and the set video buffer duration range is usually set by the user of the client.
Referring to fig. 8, since the network status is an objective factor, the main adjustment mode for the video watching experience of the client is to adjust the ratio of the sending rates of the packets of the bidirectional communication channel to the unidirectional communication channel, and in order to quantitatively analyze the ratio, step S81 may be executed to first obtain the time of the critical packets and the predicted packets sent by the bidirectional communication channel in step S811. Step S812 may be performed to obtain a correspondence of the data packet transmission rate of the bidirectional communication channel with respect to time according to the time of the critical data packet and the predicted data packet transmitted by the bidirectional communication channel. Step S813 may be performed next to acquire the timing of the predicted packet transmitted by the unidirectional communication channel, and step S814 may be performed next to acquire the correspondence of the packet transmission rate of the unidirectional communication channel with respect to time based on the timing of the predicted packet transmitted by the unidirectional communication channel. Finally, step S815 may be executed to obtain the corresponding relationship of the ratio of the data packet sending rates of the bidirectional communication channel and the unidirectional communication channel with respect to time according to the corresponding relationship of the data packet sending rate of the bidirectional communication channel with respect to time and the corresponding relationship of the data packet sending rate of the unidirectional communication channel with respect to time. The subsequent accurate quantitative analysis is facilitated by establishing a corresponding relation about time, and the prediction judgment can be performed according to the time trend.
Referring to fig. 9, in order to quantitatively analyze the packet loss rate of the unidirectional communication channel and the video buffer duration of the client, a correspondence relationship between the two and the channel transmission rate ratio needs to be established, so step S82 may be executed first to obtain the correspondence relationship of the packet loss rate of the unidirectional communication channel with respect to time in the execution process. Step S822 may be performed to obtain a correspondence of the packet loss rate of the unidirectional communication channel with respect to the channel transmission rate ratio according to the correspondence of the channel transmission rate ratio with respect to time and the correspondence of the packet loss rate of the unidirectional communication channel with respect to time. Step S823 may be performed next to acquire the correspondence of the video buffering duration of the client with respect to time. Finally, step S824 may be executed to obtain the corresponding relationship of the video buffer duration of the client with respect to the channel sending rate ratio according to the corresponding relationship of the channel sending rate ratio with respect to time and the corresponding relationship of the video buffer duration of the client with respect to time, thereby establishing a quantitative relationship of the packet loss rate, the video buffer duration and the channel sending rate ratio, so as to facilitate subsequent analysis and adjustment.
Referring to fig. 10, since the video buffering duration and the packet loss rate both affect the video viewing experience of the client, as long as the video buffering duration and the packet loss rate are all enough in the set range, in the process of quantitative analysis and adjustment, step S851 may be executed to obtain the first calculation range of the channel sending rate ratio according to the correspondence of the packet loss rate of the unidirectional communication channel with respect to the channel sending rate ratio and the set fault tolerance range of the packet loss rate. Step S852 may be performed to obtain a second calculation range of the channel transmission rate ratio according to the correspondence of the video buffer duration of the client with respect to the channel transmission rate ratio and the set video buffer duration range. Next, step S853 may be performed to intercept an intersection of the first calculation range and the second calculation range of the packet loss rate of the unidirectional communication channel as a planning range of the channel transmission rate ratio. Finally, step S854 may be performed to adjust the transmission channels of the critical packets and the predicted packets according to the planned range of the channel transmission rate ratio.
Referring to fig. 11, the different video compression encoding modes also make the internal structure of the video different, for example, the h.261 encoding mode includes a key frame and a predicted frame, but in the case of the following bi-directional predicted frame introduced in the MPEG-1 encoding mode, step S15 may be executed first to obtain the bi-directional predicted frame in the video. Step S16 may be performed next to acquire the grouping order of the bi-directional predicted frames, and step S17 may be performed next to pack the bi-directional predicted frames into bi-directional predicted data packets according to the grouping order of the bi-directional predicted frames. The same processing flow as that of steps S1 to S6 is maintained in the process of steps S15 to S17. Step S18 is then performed to obtain the grouping order of the bi-directional predicted data packets according to the grouping order of the bi-directional predicted frames. However, in the packet transmission flow of step S19, since the bi-directional predicted packet has a higher picture frame information density, in order to achieve both the transmission efficiency and the integrity of the client video picture, the bi-directional predicted packet may be simultaneously transmitted on the bi-directional communication channel and the uni-directional communication channel, specifically, the bi-directional predicted packet may be transmitted via the bi-directional communication channel and the uni-directional communication channel according to the channel transmission rate ratio along the grouping order of the bi-directional predicted packet. Of course, for the compressed picture frames included in other video compression encoding modes, the corresponding data packets can also be sent in the same mode, so that the transmission efficiency of video data and the integrity of video pictures are both considered.
In summary, in the implementation process of the scheme, the compression mode of the video data is combined, the data source requested by the client is respectively packed into the key data packet and the prediction data packet according to the compressed picture frame, and the transmission communication channel of the data packet is adjusted according to the watching return data of the client as the adjustment parameter, so that the smoothness of video transmission and playing is improved on the premise of guaranteeing the integrity of the video picture of the client, and the watching experience of the video is improved.
The above description of illustrated embodiments of the invention, including what is described in the abstract, is not intended to be exhaustive or to limit the invention to the precise forms disclosed herein. Although specific embodiments of, and examples for, the invention are described herein for illustrative purposes only, various equivalent modifications are possible within the spirit and scope of the present invention, as those skilled in the relevant art will recognize and appreciate. As noted, these modifications can be made to the present invention in light of the foregoing description of illustrated embodiments of the present invention and are to be included within the spirit and scope of the present invention.
The systems and methods have been described herein in general terms as being helpful in understanding the details of the present invention. Furthermore, various specific details have been set forth in order to provide a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that an embodiment of the invention can be practiced without one or more of the specific details, or with other apparatus, systems, assemblies, methods, components, materials, parts, and/or the like. In other instances, well-known structures, materials, and/or operations are not specifically shown or described in detail to avoid obscuring aspects of embodiments of the invention.
Thus, although the invention has been described herein with reference to particular embodiments thereof, a latitude of modification, various changes and substitutions are intended in the foregoing disclosures, and it will be appreciated that in some instances some features of the invention will be employed without a corresponding use of other features without departing from the scope and spirit of the invention as set forth. Therefore, many modifications may be made to adapt a particular situation or material to the essential scope and spirit of the present invention. It is intended that the invention not be limited to the particular terms used in following claims and/or to the particular embodiment disclosed as the best mode contemplated for carrying out this invention, but that the invention will include any and all embodiments and equivalents falling within the scope of the appended claims. Accordingly, the scope of the invention should be determined only by the following claims.
Claims (9)
1. A video data transmission method, characterized by comprising,
sequentially acquiring a key frame and a predicted frame in a video;
acquiring a grouping sequence of the key frames and the predicted frames;
packaging the key frame and the predicted frame into a key data packet and a predicted data packet according to the grouping sequence of the key frame and the predicted frame;
acquiring the grouping sequence of the key data packet and the predicted data packet according to the grouping sequence of the key frame and the predicted frame;
establishing a bidirectional communication channel and a unidirectional communication channel which are communicated with a client;
transmitting the critical data packet and/or the predicted data packet via the bidirectional communication channel and the unidirectional communication channel;
acquiring video buffering time length of the client and packet loss rate of the unidirectional communication channel through the bidirectional communication channel;
adjusting the transmission channels of the key data packet and the predicted data packet according to the video buffer time length of the client and the packet loss rate of the unidirectional communication channel;
wherein said step of transmitting said critical data packets and/or said predicted data packets via said bi-directional communication channel and said uni-directional communication channel comprises,
transmitting the key data packets through the bidirectional communication channel according to the grouping sequence of the key data packets;
acquiring the time schedule of the sent key data packet in the video;
transmitting the predicted data packets via the bidirectional communication channel or the unidirectional communication channel according to the grouping sequence of the predicted data packets;
acquiring the time schedule of the sent prediction data packet in the video;
and controlling the time schedule of the transmitted key data packet and the transmitted predicted data packet in the video to be consistent.
2. The method of claim 1, wherein the step of controlling the time schedule in video of the transmitted critical data packets and the transmitted predicted data packets to coincide comprises,
taking the predicted frame between two temporally adjacent key frames as a prediction unit group according to the temporal sequence of the key frames and the predicted frame in video;
taking the key frame of the prediction unit group before in time sequence as the key frame to which the prediction unit group depends;
packaging the key frames into a plurality of key data packets;
packaging each predicted frame in the prediction unit group to which the key frame belongs into a plurality of predicted data packets;
subpackaging the key data packet corresponding to each key frame and the predicted data packet corresponding to each predicted frame in the prediction unit group to which the key frame belongs into a partition to be transmitted;
sequencing each partition to be sent according to the time sequence of the key frames in the video in each partition to be sent to obtain the time sequence of the partition to be sent;
and sending the key data packets and the predicted data packets in each partition to be sent according to the sequence of the time sequence of the partition to be sent.
3. The method of claim 1, wherein said step of adjusting transmission channels of said critical data packets and said predicted data packets based on a video buffer duration of said client and a packet loss rate of said unidirectional communication channel comprises,
acquiring the corresponding relation of the data packet sending rate ratio of the bidirectional communication channel and the unidirectional communication channel with respect to time as the corresponding relation of the channel sending rate ratio with respect to time;
acquiring a corresponding relation of the packet loss rate of the unidirectional communication channel with respect to the channel transmission rate ratio and a corresponding relation of the video buffer duration of the client with respect to the channel transmission rate ratio according to the corresponding relation of the channel transmission rate ratio with respect to time;
acquiring a set packet loss rate fault tolerance range;
acquiring a set video buffer duration range;
and adjusting the transmission channels of the key data packets and the predicted data packets according to the corresponding relation of the packet loss rate of the unidirectional communication channel with respect to the channel transmission rate ratio, the corresponding relation of the video buffer time length of the client with respect to the channel transmission rate ratio, the set packet loss rate fault tolerance range and the set video buffer time length range.
4. The method of claim 3, wherein the step of obtaining the correspondence of the packet transmission rate ratio of the bidirectional communication channel and the unidirectional communication channel with respect to time comprises,
acquiring the time of the key data packet and the predicted data packet sent by the bidirectional communication channel;
acquiring a corresponding relation of the data packet sending rate of the bidirectional communication channel with respect to time according to the time of the key data packet and the predicted data packet sent by the bidirectional communication channel;
acquiring the time of the predicted data packet sent by the unidirectional communication channel;
acquiring a corresponding relation of the data packet sending rate of the unidirectional communication channel with respect to time according to the time of the predicted data packet sent by the unidirectional communication channel;
and acquiring the corresponding relation of the ratio of the data packet sending rates of the bidirectional communication channel and the unidirectional communication channel with respect to time according to the corresponding relation of the data packet sending rate of the bidirectional communication channel with respect to time and the corresponding relation of the data packet sending rate of the unidirectional communication channel with respect to time.
5. The method of claim 3, wherein the step of obtaining the correspondence of the packet loss rate of the unidirectional communication channel with respect to the channel transmission rate ratio and the correspondence of the video buffering duration of the client with respect to the channel transmission rate ratio based on the correspondence of the channel transmission rate ratio with respect to time comprises,
acquiring a corresponding relation of the packet loss rate of the unidirectional communication channel with respect to time;
obtaining the corresponding relation of the packet loss rate of the unidirectional communication channel with respect to the channel sending rate ratio according to the corresponding relation of the channel sending rate ratio with respect to time and the corresponding relation of the packet loss rate of the unidirectional communication channel with respect to time;
acquiring a corresponding relation of video buffering duration of the client with respect to time;
and obtaining the corresponding relation of the video buffer duration of the client side with respect to the channel sending rate ratio according to the corresponding relation of the channel sending rate ratio with respect to time and the corresponding relation of the video buffer duration of the client side with respect to time.
6. The method of claim 3, wherein said step of adjusting the transmission channels of said critical packets and said predicted packets based on the correspondence of the packet loss rate of said unidirectional communication channel with respect to the channel transmission rate ratio, the correspondence of the video buffer duration of said client with respect to the channel transmission rate ratio, the set packet loss rate tolerance range, and the set video buffer duration range, comprises,
acquiring a first calculation range of the channel transmission rate ratio according to the corresponding relation of the packet loss rate of the unidirectional communication channel with respect to the channel transmission rate ratio and the set packet loss rate fault tolerance range;
acquiring a second calculation range of the channel transmission rate ratio according to the corresponding relation of the video buffer time length of the client with respect to the channel transmission rate ratio and the set video buffer time length range;
intercepting an intersection of a first calculation range and a second calculation range of the packet loss rate of the unidirectional communication channel as a planning range of a channel transmission rate ratio;
and adjusting the transmission channels of the key data packet and the predicted data packet according to the planning range of the channel transmission rate ratio.
7. The method of claim 3, further comprising,
acquiring a bidirectional prediction frame in a video;
acquiring a grouping sequence of the bidirectional predicted frames;
packaging the bidirectional predicted frames into bidirectional predicted data packets according to the grouping sequence of the bidirectional predicted frames;
acquiring the grouping sequence of the bidirectional predictive data packet according to the grouping sequence of the bidirectional predictive frame;
and transmitting the bidirectional predicted data packets through the bidirectional communication channel and/or the unidirectional communication channel according to a channel transmission rate ratio along the grouping sequence of the bidirectional predicted data packets.
8. A video data transmission method, characterized by comprising,
receiving a critical data packet and a predictive data packet generated by the video data transmission method according to any one of claims 1 to 7;
video decoding is carried out according to the key data packet and the prediction data packet, so that decoded video at the client side is obtained;
acquiring the packet loss rate of a one-way communication channel according to the grouping sequence of the key data packet and the predicted data packet received by the client;
acquiring the playing progress of the decoded video of the client;
acquiring video buffer time length of the client according to the decoded video and the playing progress of the decoded video of the client;
and sending the video buffer time of the client through the bidirectional communication channel and the packet loss rate of the unidirectional communication channel.
9. A video data transmission system, comprising,
the server is used for sequentially acquiring key frames and predicted frames in the video;
acquiring a grouping sequence of the key frames and the predicted frames;
packaging the key frame and the predicted frame into a key data packet and a predicted data packet according to the grouping sequence of the key frame and the predicted frame;
acquiring the grouping sequence of the key data packet and the predicted data packet according to the grouping sequence of the key frame and the predicted frame;
establishing a bidirectional communication channel and a unidirectional communication channel which are communicated with a client;
transmitting the critical data packet and/or the predicted data packet via the bidirectional communication channel and the unidirectional communication channel;
acquiring video buffering time length of the client and packet loss rate of the unidirectional communication channel through the bidirectional communication channel;
adjusting the sending channels of the key data packet and the predicted data packet according to the video buffering duration of the client and the packet loss rates of the key data packet and the unidirectional communication channel;
the system also comprises a client, a server and a client, wherein the client is used for receiving the key data packet and the prediction data packet;
performing video decoding according to the key data packet and the prediction data packet to obtain a decoded video at the client;
acquiring the packet loss rate of the unidirectional communication channel according to the grouping sequence of the key data packet and the predicted data packet received by the client;
acquiring the playing progress of the decoded video of the client;
acquiring video buffer time length of the client according to the decoded video and the playing progress of the decoded video of the client;
transmitting the video buffer time length of the client and the packet loss rate of the unidirectional communication channel through the bidirectional communication channel;
wherein said step of transmitting said critical data packets and/or said predicted data packets via said bi-directional communication channel and said uni-directional communication channel comprises,
transmitting the key data packets through the bidirectional communication channel according to the grouping sequence of the key data packets;
acquiring the time schedule of the sent key data packet in the video;
transmitting the predicted data packets via the bidirectional communication channel or the unidirectional communication channel according to the grouping sequence of the predicted data packets;
acquiring the time schedule of the sent prediction data packet in the video;
and controlling the time schedule of the transmitted key data packet and the transmitted predicted data packet in the video to be consistent.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310564881.9A CN116320641B (en) | 2023-05-19 | 2023-05-19 | Video data transmission method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310564881.9A CN116320641B (en) | 2023-05-19 | 2023-05-19 | Video data transmission method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116320641A CN116320641A (en) | 2023-06-23 |
| CN116320641B true CN116320641B (en) | 2023-08-04 |
Family
ID=86801711
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310564881.9A Active CN116320641B (en) | 2023-05-19 | 2023-05-19 | Video data transmission method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116320641B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007036160A1 (en) * | 2005-09-29 | 2007-04-05 | Tencent Technology (Shenzhen) Company Limited | An apparatus, system and method for realizing communication between the client and the server |
| CN101729898A (en) * | 2009-11-16 | 2010-06-09 | 中国人民解放军国防科学技术大学 | Video coding and decoding methods and devices |
| CN102694791A (en) * | 2012-04-17 | 2012-09-26 | 清华大学 | Real-time TCP (transmission control protocol) transmission protocol based on fountain code |
| CN105704580A (en) * | 2016-01-21 | 2016-06-22 | 深圳比特新技术有限公司 | Video transmission method |
| CN106209683A (en) * | 2016-06-20 | 2016-12-07 | 中国科学院上海高等研究院 | Data transmission method based on data center's wide area network and system |
| CN107707933A (en) * | 2017-10-18 | 2018-02-16 | 中国联合网络通信集团有限公司 | Send, receive the method and device of video flowing |
| CN109729439A (en) * | 2019-01-11 | 2019-05-07 | 北京三体云联科技有限公司 | Method for real-time video transmission |
| CN110049058A (en) * | 2019-04-25 | 2019-07-23 | 深圳市云语科技有限公司 | A kind of Transmission system bottom method for selecting transmission protocol |
| WO2022041282A1 (en) * | 2020-08-31 | 2022-03-03 | 华为技术有限公司 | Video data transmission method and apparatus |
| CN114844959A (en) * | 2022-04-29 | 2022-08-02 | 深圳市云语科技有限公司 | Method for automatically switching transmission protocol |
| CN116112535A (en) * | 2023-02-03 | 2023-05-12 | 北京字跳网络技术有限公司 | Data transmission method, device, computer equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9386127B2 (en) * | 2011-09-28 | 2016-07-05 | Open Text S.A. | System and method for data transfer, including protocols for use in data transfer |
-
2023
- 2023-05-19 CN CN202310564881.9A patent/CN116320641B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007036160A1 (en) * | 2005-09-29 | 2007-04-05 | Tencent Technology (Shenzhen) Company Limited | An apparatus, system and method for realizing communication between the client and the server |
| CN101729898A (en) * | 2009-11-16 | 2010-06-09 | 中国人民解放军国防科学技术大学 | Video coding and decoding methods and devices |
| CN102694791A (en) * | 2012-04-17 | 2012-09-26 | 清华大学 | Real-time TCP (transmission control protocol) transmission protocol based on fountain code |
| CN105704580A (en) * | 2016-01-21 | 2016-06-22 | 深圳比特新技术有限公司 | Video transmission method |
| CN106209683A (en) * | 2016-06-20 | 2016-12-07 | 中国科学院上海高等研究院 | Data transmission method based on data center's wide area network and system |
| CN107707933A (en) * | 2017-10-18 | 2018-02-16 | 中国联合网络通信集团有限公司 | Send, receive the method and device of video flowing |
| CN109729439A (en) * | 2019-01-11 | 2019-05-07 | 北京三体云联科技有限公司 | Method for real-time video transmission |
| CN110049058A (en) * | 2019-04-25 | 2019-07-23 | 深圳市云语科技有限公司 | A kind of Transmission system bottom method for selecting transmission protocol |
| WO2022041282A1 (en) * | 2020-08-31 | 2022-03-03 | 华为技术有限公司 | Video data transmission method and apparatus |
| CN114844959A (en) * | 2022-04-29 | 2022-08-02 | 深圳市云语科技有限公司 | Method for automatically switching transmission protocol |
| CN116112535A (en) * | 2023-02-03 | 2023-05-12 | 北京字跳网络技术有限公司 | Data transmission method, device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116320641A (en) | 2023-06-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2019144836A1 (en) | Data transmission method, apparatus and system | |
| CA2442439C (en) | Packet transmission system and packet reception system | |
| US7191242B1 (en) | Methods and apparatuses for transferring data | |
| EP1449331B1 (en) | Data transmission | |
| US6018516A (en) | Method for minimizing unneeded retransmission of packets in a packet communication environment supporting a plurality of data link rates | |
| CN1906906B (en) | Optimized radio bearer configuration for voice over IP | |
| CN112804711B (en) | Data transmission method, data transmission device, computer readable medium and electronic equipment | |
| US8694609B2 (en) | Method and apparatus for improving wireless data networks performance | |
| CN113037440B (en) | Data retransmission processing method and device, computer equipment and storage medium | |
| US10862945B2 (en) | Adaptive restful real-time live media streaming | |
| Raman et al. | ITP: An image transport protocol for the internet | |
| CN112953850B (en) | Data transmission method and device, computer readable medium and electronic equipment | |
| US20180026811A1 (en) | Efficient transport of encapsulated media traffic over restrictive networks | |
| BR112014023248B1 (en) | METHOD FOR TRANSMITTING DATA, METHOD FOR RECEIVING DATA AND COMMUNICATIONS DEVICE | |
| CN116320641B (en) | Video data transmission method and system | |
| US7000024B1 (en) | Systems and methods for providing transmission control protocol communications | |
| CN110233856A (en) | Message processing method, device and computer readable storage medium | |
| JP2005051299A (en) | Packet transmission apparatus, packet reception apparatus, packet transmission method and packet reception method | |
| WO2023217188A1 (en) | Livestream data transmission method, apparatus and system, device and medium | |
| CN105611424B (en) | Audio-video reliable transmission QOS method, receiving end and system based on RUDP | |
| WO2023213089A1 (en) | Data processing method and apparatus, computer readable medium, and terminal device | |
| US9762412B2 (en) | Redundant traffic encoding of encapsulated real time communications | |
| US20170033946A1 (en) | Negative acknowledgment of tunneled encapsulated media | |
| Bhattacharyya et al. | Improving live-streaming experience for delay-sensitive iot applications: A restful approach | |
| Shamieh et al. | Dynamic cross-layer signaling exchange for real-time and on-demand multimedia streams |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |