CN108696772B

CN108696772B - Real-time video transmission method and device

Info

Publication number: CN108696772B
Application number: CN201710233377.5A
Authority: CN
Inventors: 袁荣喜; 周巍巍; 张凯磊
Original assignee: Suzhou Qianwen Wandaba Education Technology Co Ltd
Current assignee: Suzhou Qianwen Wandaba Education Technology Co ltd
Priority date: 2017-04-11
Filing date: 2017-04-11
Publication date: 2021-07-16
Anticipated expiration: 2037-04-11
Also published as: CN108696772A

Abstract

The embodiment of the invention discloses a real-time video transmission method and a real-time video transmission device. The method comprises the following steps: the video sending end sends a connection request to a video server and receives connection confirmation information returned by the video server so as to establish connection with the video server; the connection request comprises the identity identification information of the video sending end; after receiving the connection request, the video server activates at least one associated video playing end according to the identity identification information to open a transmission channel corresponding to the video sending end; and the video sending end sends the video frame fragments of the video coding frames generated in real time to the video server according to the set data transmission parameters, so that the video server forwards the video frame fragments to the video playing end which opens the corresponding transmission channel for real-time video playing. By the technical scheme of the embodiment of the invention, the video transmission delay can be reduced, and the user experience can be improved.

Description

Real-time video transmission method and device

Technical Field

The embodiment of the invention relates to computer technology, in particular to a real-time video transmission method and device.

Background

With the continuous popularization and development of the internet, live video broadcast of the mobile internet is in the fierce blowout development. Most of traditional live broadcasting is one-way, such as live broadcasting of a television station or an operator, a user only needs to open a terminal to watch the live broadcasting, and the requirement on real-time performance is not high. The mobile internet live video often needs interaction between a recording end and a playing end in function, the interaction is not limited to character interaction, and also comprises video interaction, so that the real-time requirement of the mobile internet live video is high.

At present, a Transmission method of a Real-Time video in live video broadcast of a mobile internet includes acquiring video data by a recording end, performing video coding according to an X264 format, pushing video stream data to a CDN (Content Delivery Network) server for distribution based on a Real Time Messaging Protocol (RTMP) on a Transmission Control Protocol (TCP) through a Quality of Service (QoS) algorithm, and pulling a stream from the CDN server by a playing end for playing.

The real-time live video broadcast delay of the prior art is represented by that video stream data is pushed to and played from the CDN based on RTMP based stream pulling cache, and the data transmission process based on TCP is too complex, so that the video transmission delay of the whole network link is usually 1-3 seconds or worse.

Disclosure of Invention

The embodiment of the invention provides a real-time video transmission method and device, which are used for reducing video transmission delay and improving user experience.

In a first aspect, an embodiment of the present invention provides a method for transmitting a real-time video, including:

the video sending end sends a connection request to a video server and receives connection confirmation information returned by the video server so as to establish connection with the video server;

the connection request comprises the identity identification information of the video sending end; after receiving the connection request, the video server activates at least one associated video playing end according to the identity identification information to open a transmission channel corresponding to the video sending end;

and the video sending end sends the video frame fragments of the video coding frames generated in real time to the video server according to the set data transmission parameters, so that the video server forwards the video frame fragments to the video playing end which opens the corresponding transmission channel for real-time video playing.

In a second aspect, an embodiment of the present invention further provides a method for transmitting a real-time video, including:

the video server returns corresponding connection confirmation information to the video sending end according to a connection request sent by the video sending end so as to establish connection with the video sending end, wherein the connection request comprises identity identification information of the video sending end;

the video server sends the identity identification information to at least one associated video playing end so as to activate the video playing end to open a transmission channel corresponding to the video sending end;

and the video server receives video frame fragments of the video coding frames which are sent by the video sending end according to the set data transmission parameters and generated in real time, and forwards the video frame fragments to the video playing end which opens the corresponding transmission channel for real-time video playing.

In a third aspect, an embodiment of the present invention further provides a method for transmitting a real-time video, where the method includes:

the video playing end receives the identity identification information of the video sending end sent by the video server;

the video playing end inquires a transmission channel list according to the identity identification information, and determines and opens a transmission channel corresponding to the video sending end, wherein the transmission channel list records the corresponding relation between the identity identification information and the transmission channel;

the video player receives video frame fragments of a video coding frame which are transmitted by the video transmitting end according to set data transmission parameters and are generated in real time, wherein the video frame fragments are transmitted by the video transmitting end through the transmission channel;

and the video playing end plays the video in real time according to the received video frame fragments.

In a fourth aspect, an embodiment of the present invention further provides a real-time video transmission apparatus, configured at a video sending end, where the apparatus includes:

the connection request sending module is used for sending a connection request to a video server and receiving connection confirmation information returned by the video server so as to establish connection with the video server;

and the video fragment sending module is used for sending the video frame fragments of the video coding frames generated in real time to the video server by the video sending end according to the set data transmission parameters so as to enable the video server to forward the video frame fragments to the video playing end which opens the corresponding transmission channel for real-time video playing.

In a fifth aspect, an embodiment of the present invention further provides a real-time video transmission apparatus, configured on a video server, where the apparatus includes:

a connection confirmation returning module, configured to return corresponding connection confirmation information to the video sending end according to a connection request sent by the video sending end, so as to establish a connection with the video sending end, where the connection request includes identity information of the video sending end;

the identification information sending module is used for sending the identification information to at least one associated video playing end so as to activate the video playing end to open a transmission channel corresponding to the video sending end;

and the video fragment forwarding module is used for receiving the video frame fragments of the video coding frames which are sent by the video sending end according to the set data transmission parameters and generated in real time, and forwarding the video frame fragments to the video playing end which opens the corresponding transmission channel for real-time video playing.

In a sixth aspect, an embodiment of the present invention further provides a real-time video transmission device, configured at a video playing end, where the device includes:

the identification information receiving module is used for receiving the identity identification information of the video sending end sent by the video server;

a transmission channel opening module, configured to query a transmission channel list according to the identity information, determine and open a transmission channel corresponding to the video sending end, where a correspondence relationship between the identity information and the transmission channel is recorded in the transmission channel list;

the video fragment receiving module is used for receiving the video frame fragments of the video coding frames which are transmitted by the video transmitting end according to the set data transmission parameters and are generated in real time and transmitted by the video server through the transmission channel;

and the real-time video playing module is used for the video playing end to play the real-time video according to the received video frame fragments.

In the embodiment of the invention, the video sending end sends the connection request to the video server so as to establish connection with the video server, and then sends the video frame fragments of the video coding frames generated in real time to the video server according to the set data transmission parameters, and then the video server forwards the video frame fragments to the video playing end to play the video in real time. The method has the advantages that the connection is established by only sending the request once, and the video coding frame is divided into the plurality of video frame fragments to be sent after the connection is established, so that the problem of high video transmission delay caused by a complex process of three-way handshake before the connection is performed based on TCP in the prior art and a method of directly transmitting and playing based on the video data stream is solved, and the effects of reducing the video transmission delay and improving the user experience are realized.

Drawings

Fig. 1 is a schematic flowchart of a real-time video transmission method according to an embodiment of the present invention;

fig. 2 is a schematic flowchart of a real-time video transmission method according to a second embodiment of the present invention;

fig. 3 is a schematic flowchart of a real-time video transmission method according to a third embodiment of the present invention;

fig. 4 is a schematic flowchart of a real-time video transmission method according to a fourth embodiment of the present invention;

fig. 5 is a schematic structural diagram of a real-time video transmission apparatus according to a fifth embodiment of the present invention;

fig. 6 is a schematic structural diagram of a real-time video transmission apparatus according to a sixth embodiment of the present invention;

fig. 7 is a schematic structural diagram of a real-time video transmission apparatus according to a seventh embodiment of the present invention.

Detailed Description

The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.

Example one

Fig. 1 is a flowchart illustrating a real-time video transmission method according to an embodiment of the present invention. The method is applicable to the transmission situation of real-time videos, and can be executed by a real-time video transmission device which can be composed of hardware and/or software and can be generally integrated in a video sending end and all intelligent terminals with video sending functions. The method specifically comprises the following steps:

s110, the video sending end sends a connection request to the video server and receives connection confirmation information returned by the video server so as to establish connection with the video server.

The video sending end may be, for example, a group video initiator, or a new group video adder. For example, when a client newly initiates a group video session to the video server, the client is a video sending end relative to other clients participating in the group video; when a client needs to newly join a group video, the client is a video sending end relative to other clients already participating in the group video.

Optionally, the connection request may include a User Identification (UID) of the video sending end, and may also include an Internet Protocol (IP) address of the video sending end.

Compared with a three-way handshake connection mode adopted by a connection-oriented TCP, the connection mode adopted by the embodiment of the invention reduces the complexity of the connection process and shortens the connection establishment time, and for a transmission mode oriented to a connection-free UDP (User Datagram Protocol), the connection-based transmission mode adopted by the embodiment of the invention provides a reliable transmission mechanism, so that the subsequent data transmission process is more reliable. Therefore, the advantage of sending a connection request to the video server by the video sender to establish a connection with the video server is that the reliability of subsequent data transmission can be improved while reducing delay.

S120, the connection request comprises the identity identification information of the video sending end; and after receiving the connection request, the video server activates at least one associated video playing end to open a transmission channel corresponding to the video sending end according to the identity identification information.

The identification information may be a UID of the video sender. Illustratively, the video server is connected with a plurality of video playing terminals, and when a video sending terminal is connected with the video server, the video server needs to send the identification information of the video sending terminal to all the video playing terminals connected with the video server, so that the video playing terminals open transmission channels corresponding to the video sending terminal. The transmission channel may specifically be a network port. The purpose of opening the transmission channel corresponding to the video sending end is to provide a special data transmission channel corresponding to the video sending end for the transmission of video frame fragments in the subsequent steps, so that when a plurality of video sending ends transmit data to the same video playing end at the same time, the data transmission channels can not be influenced mutually.

S130, the video sending end sends the video frame fragments of the video coding frames generated in real time to the video server according to the set data transmission parameters, so that the video server forwards the video frame fragments to the video playing end which opens the corresponding transmission channel to play the video in real time.

Optionally, setting the data transmission parameter may include setting a resolution of the video encoder, and/or setting a data transmission code rate. Preferably, the video coding frame may be obtained by a video encoder encoding the video data acquired in real time in the form of h264 according to a preset resolution.

The video frame slices can be obtained by slicing the video coding frame. Illustratively, when a video encoder encodes a complete h264 video encoded frame, a video sending end needs to send the video encoded frame, and for a high-resolution video encoded frame, the size of the frame is often higher than the maximum network transmission unit in a UDP-based transmission mode, so that the video sending end needs to perform fragmentation processing on the video encoded frame and then send the video encoded frame, and send video data by taking video frame fragments as units each time, thereby effectively solving the problem of high video delay caused by an overlarge transmission unit and reducing the transmission delay of the video.

Preferably, after the video sending end sends the video frame fragments of the video coding frame generated in real time to the video server according to the set data transmission parameters, the method further includes: the video sending end sends a disconnection request to the video server and receives disconnection confirmation information returned by the video server so as to be disconnected with the video server; and after receiving the disconnection request, the video server controls at least one associated video playing end to close a transmission channel corresponding to the video sending end.

For example, when a client in a group video needs to quit the group video, the client needing to quit the group video may be regarded as a video sending end, and other clients participating in the group video may be regarded as video playing ends, at this time, the video sending end may disconnect the connection with the video server by sending a disconnection request, and further release related connection resources, and the video server also needs to notify all the video playing ends to close a transmission channel corresponding to the video sending end after receiving the disconnection request, so as to save channel resources.

According to the technical scheme of the embodiment, the video sending end sends the connection request to the video server so as to establish connection with the video server, and then sends the video frame fragments of the video coding frames generated in real time to the video server according to the set data transmission parameters, and then the video server forwards the video frame fragments to the video playing end to play the video in real time. The method has the advantages that the connection is established by only sending the request once, and the video coding frame is divided into the plurality of video frame fragments to be sent after the connection is established, so that the problem of high video transmission delay caused by a complex process of three-way handshake before the connection is performed based on TCP in the prior art and a method of directly transmitting and playing based on the video data stream is solved, and the effects of reducing the video transmission delay and improving the user experience are realized.

Example two

Fig. 2 is a flowchart illustrating a real-time video transmission method according to a second embodiment of the present invention. The present embodiment is optimized based on the above embodiments, and provides a preferable real-time video transmission method, specifically, after a video sending end sends a connection request to a video server and receives connection confirmation information returned by the video server, further optimization is performed. The method specifically comprises the following steps:

s210, the video sending end sends a connection request to the video server and receives connection confirmation information returned by the video server so as to establish connection with the video server.

S220, the connection request comprises the identity identification information of the video sending end; and after receiving the connection request, the video server activates at least one associated video playing end to open a transmission channel corresponding to the video sending end according to the identity identification information.

And S230, the video sending end returns corresponding video synchronization confirmation information to the video server according to the video synchronization request sent by the video server.

Wherein the video synchronization confirmation information includes: the sequence number of the starting video frame fragment and the sequence number of the current sending video frame fragment; when confirming that a target video playing end opens a transmission channel corresponding to the video sending end, the video server sends a video synchronization request to the video sending end, so that the target video playing end obtains the playing progress of the real-time video.

Optionally, the synchronization confirmation information may be sent to the video server through a handshake signal, and the video server forwards the synchronization confirmation information to the video sending end through the handshake signal. Wherein the starting video frame slice may be the first video frame slice corresponding to a key frame (I-frame) in the video picture group. The purpose of providing the sequence number of the starting video frame fragment is to enable the video playing end to play the video from the key frame where the starting video frame fragment is located, thereby avoiding the screen splash phenomenon during playing. The purpose of providing the sequence number of the currently transmitted video frame fragment is to enable the video playing end to update the current playing progress according to the sequence number of the currently transmitted video frame fragment. Therefore, the advantage that the video sending end returns the corresponding video synchronization confirmation information to the video server is that the target video playing end can obtain the playing progress of the real-time video in time so as to reduce the video delay.

Optionally, the execution order of S220 and S230 may be exchanged or performed simultaneously.

S240, the video sending end calculates the network delay time according to the current time for sending the video frame fragments and the last time for receiving the receiving confirmation response.

Due to unpredictability of a network, jitter, congestion or a large amount of packet loss may occur, and if video frame fragments are transmitted according to a fixed code rate and parameters, a transmission channel between a video transmitting end and a video playing end may be more congested, so that the playing delay of the playing end is higher or a large amount of mosaics occur. Therefore, it is necessary to detect the current network situation by estimating the current bandwidth at the video transmitting end in real time, so as to adjust the frame rate or code rate of the video encoder according to the current network situation in real time. Optionally, a timer may be set at the video sending end, and bandwidth statistics may be performed every predetermined time.

Preferably, the network delay time may be calculated according to the formula delay _ ts _ delta ═ cur _ ts-acked _ ts, where delay _ ts _ delta represents the network delay time; cur _ ts represents the current time of sending a video frame slice; acked _ ts represents the time when the reception acknowledgement response was last received. Illustratively, the time when a video frame slice is currently transmitted, i.e. the time when a video frame slice is currently being transmitted, is t₁The time of receiving the last acknowledgement response, i.e. the time of receiving the acknowledgement response last time, is t₂Then the current network is delayedThe delay time is t₁-t₂。

And S250, the video sending end determines the current bandwidth jitter correction value according to the bandwidth receiving the acknowledgement response in the current unit time and the bandwidth jitter correction value in the previous stage.

For example, the corresponding bandwidth value may be estimated according to the condition that the reception acknowledgement response is received in the current unit time, that is, the bandwidth for receiving the reception acknowledgement response is received in the current unit time. The bandwidth jitter correction value of the previous stage refers to a bandwidth jitter correction value calculated in the last bandwidth measurement.

Preferably, the video sending end calculates the current bandwidth jitter correction value according to the bandwidth receiving the acknowledgment response in the current unit time and the bandwidth jitter correction value in the previous stage, and the method includes: video transmitting end according to formula

Calculating a current bandwidth jitter correction value; wherein acked _ bw is the current bandwidth jitter correction value, acked _ bw₀Bw is the bandwidth jitter correction value of the previous stage, and bw is the bandwidth receiving the acknowledgement response in the current unit time. As can be seen from the calculation formula of acked _ bw, the value of acked _ bw is an integer value. When acked _ bw is first calculated, acked _ bw₀Is 0.

And S260, the video sending end updates the data transmission parameters according to the network delay time and the current bandwidth jitter correction value.

Preferably, the bandwidth is counted every 10 seconds, and the data transmission parameters are updated. The advantage of periodically updating the data transmission parameters is that the data transmission parameters can be adjusted in real time according to the current network conditions, so that the transmission efficiency is improved to the maximum extent, the video delay is reduced, and the user experience is improved.

Preferably, updating the data transmission parameters according to the network delay time and the current bandwidth jitter correction value includes: if the retransmitted video frame fragments exist at present and the network delay time is greater than a preset threshold value, setting the expected bandwidth value as a current bandwidth jitter correction value, and reducing the value of a data transmission parameter according to the expected bandwidth value, wherein the data transmission parameter comprises: resolution of a video encoder, and/or data transmission rate; if the retransmitted video frame fragments do not exist at present and the current bandwidth jitter correction value is larger than 0, setting the expected bandwidth value as a preset multiple of the current bandwidth jitter correction value, and increasing the value of the data transmission parameter according to the expected bandwidth value; and if the current bandwidth jitter correction value is equal to 0, keeping the value of the data transmission parameter.

Exemplarily, if delay _ ts _ delta is greater than 8 × MAX (rtt + rtt _ var,100), dst _ bw is acked _ bw, and then the resolution, the frame rate, and/or the data transmission rate of the video encoder are reduced according to the updated value of dst _ bw to ensure the real-time performance and the fluency of the playing; if acked _ bw is greater than 0, dst _ bw is acked _ bw 9/8, and then the resolution, the frame rate and/or the data transmission code rate of the video encoder are/is improved according to the updated dst _ bw value, so that the definition is restored and the user playing experience is improved; if acked _ bw is equal to 0, the value of dst _ bw is not updated, i.e. the resolution, frame rate and/or data transmission rate of the current video encoder are kept unchanged. Wherein delay _ ts _ delta represents a network delay time; rtt represents the round-trip transmission time of the current video frame slice; the corrected value of rtt is rtt _ var which can be obtained according to the existing rtt correction algorithm; dst _ bw represents the desired bandwidth value; acked _ bw is the current bandwidth jitter correction value.

And S270, the video sending end sends the video frame fragments of the video coding frames generated in real time to the video server according to the set data transmission parameters, so that the video server forwards the video frame fragments to the video playing end which opens the corresponding transmission channel for real-time video playing.

Optionally, S240 to S260 in this embodiment may be executed simultaneously with S270.

According to the technical scheme of the embodiment, the video synchronization confirmation information is sent to the video server through the video sending end, so that the target video playing end is finally enabled to perform synchronization confirmation, in addition, the data transmission parameters are updated according to the network delay time and the current bandwidth jitter correction value obtained through calculation, and then the video frame fragments are sent according to the data transmission parameters, so that the video transmission and playing delay is reduced, and the user playing experience is improved.

EXAMPLE III

Fig. 3 is a flowchart illustrating a real-time video transmission method according to a third embodiment of the present invention. The method is applicable to the transmission of real-time video, and can be executed by a real-time video transmission device, which can be composed of hardware and/or software and can be generally integrated in a video server and all intelligent terminals containing video transmission functions. The method specifically comprises the following steps:

s310, the video server returns corresponding connection confirmation information to the video sending end according to a connection request sent by the video sending end so as to establish connection with the video sending end, wherein the connection request comprises identity identification information of the video sending end.

Illustratively, after receiving a connection request sent by a video sending end, a video server returns corresponding connection confirmation information to the video sending end, and establishes connection with the video sending end through the returned connection confirmation information.

The identification information may be a UID of the video sender. Preferably, the connection request may include the UID of the video sender and the IP address of the video sender. In order to manage the connection information of each client uniformly, a corresponding information storage table can be set on the video server to store the UID and IP address of each client, so that the video information or video data sent by the video sending end can be forwarded through the IP address stored correspondingly by the information storage table.

S320, the video server sends the identification information to the associated at least one video playing end to activate the video playing end to open a transmission channel corresponding to the video sending end.

Optionally, the at least one video playing terminal associated with the video server may be the at least one video playing terminal that establishes a connection with the video server.

For example, when a client is newly added to a group video, or when an IP address of a client participating in the group video changes, the video server needs to send the UID of the client to other clients participating in the group video, so that the other clients record the UID of the client and allocate and open a transmission channel corresponding to the UID.

Preferably, after the video server sends the identification information to the associated at least one video playing end, the method further includes: the video server sends a video synchronization request to a video sending end when confirming that a target video playing end opens a transmission channel corresponding to the video sending end; the video server receives video synchronization confirmation information returned by the video sending end, wherein the video synchronization confirmation information comprises: the sequence number of the starting video frame fragment and the sequence number of the current sending video frame fragment; and forwarding the video synchronization confirmation information to a target video playing end so that the target video playing end acquires the playing progress of the real-time video.

Optionally, when the target video playing end opens the transmission channel corresponding to the video sending end, the video server is informed of the opening of the transmission channel through handshake confirmation information, and the video server can confirm that the target video playing end opens the transmission channel corresponding to the video sending end after receiving the handshake confirmation information. The purpose of sending the video synchronization request to the video sending end is to acquire and forward video synchronization confirmation information so that the data sending condition of the video sending end is consistent with the data receiving condition of the video playing end, and therefore video transmission delay is reduced.

S330, the video server receives video frame fragments of the video coding frames which are sent by the video sending end according to the set data transmission parameters and generated in real time, and forwards the video frame fragments to the video playing end which opens the corresponding transmission channel for real-time video playing.

Optionally, the video server serves as a relay to forward the video frame fragments sent by the video sending end to the associated video playing end. For example, each client connected to the video server in the group video may be regarded as a video sending end as compared with other clients, and when a certain client sends a video frame fragment of a video encoded frame acquired and encoded in real time to the video server, the video server sends the video frame fragment to other clients to perform real-time video playing on other clients.

Preferably, after the video server receives the video frame fragments of the video coding frame which are sent by the video sending end according to the set data transmission parameters and are generated in real time, the method further includes: the video server updates and generates a video coding frame group in a set time interval in real time according to the received video frame fragments, wherein the video coding frame group comprises at least one key frame; forwarding the video synchronization confirmation information to the target video playing terminal, further comprising: and forwarding the video synchronization confirmation information and the video coding frame group to the target video playing end so that the video playing end performs receiving-playing display on the video coding frame group.

Optionally, the video coded frame Group within the set time interval may specifically include at least one complete GOP (Group of Pictures). The key frame may be an I frame. The video coding frame group in the set time interval is cached in the video server in real time, and the video coding frame group is taken as the first screen picture and forwarded to the video playing end together with the video synchronization confirmation information for receiving and playing immediately.

Preferably, after the video server receives the video frame fragments of the video encoded frames which are sent by the video sending end according to the set data transmission parameters and generated in real time, and forwards the video frame fragments to the video playing end which opens the corresponding transmission channel for real-time video playing, the method further includes: the video server receives a disconnection request sent by a video sending end; the video server controls at least one associated video playing end to close a transmission channel corresponding to the video sending end according to the disconnection request; and the video server returns corresponding disconnection confirmation information to the video sending end so as to disconnect with the video sending end.

For example, if the video server receives a disconnection request sent from a certain client in the group video, the video server sends a video disconnection request including the client UID to other clients in the group video, so that the other clients close a transmission channel corresponding to the client, and send disconnection confirmation information to the client to confirm that the disconnection is completed.

According to the technical scheme, the video server responds to the connection request of the video sending end to establish reliable connection, the identification information of the video sending end is sent to the video playing end, so that the video playing end opens the corresponding transmission channel, and finally the video sending end is matched to forward the video frame fragments of the video coding frames generated in real time to the associated video playing end, so that the video playing end carries out real-time video playing, video transmission delay is reduced, the reliability of video transmission is improved, and user experience is improved.

Example four

Fig. 4 is a flowchart illustrating a real-time video transmission method according to a fourth embodiment of the present invention. The method is applicable to the transmission situation of real-time video, and can be executed by a real-time video transmission device, which can be composed of hardware and/or software and can be generally integrated in a video playing end and all intelligent terminals containing video receiving and playing functions. The method specifically comprises the following steps:

and S410, the video player end receives the identity identification information of the video sending end sent by the video server.

The identification information of the video sender may be a UID of the video sender.

For example, each time the UID of the video sender sent by the video server is received, it may be determined that the video sender has joined the group video, and therefore, a corresponding transmission channel needs to be allocated and opened to transmit the video data.

S420, the video playing end inquires a transmission channel list according to the identity identification information, determines and opens a transmission channel corresponding to the video sending end, wherein the transmission channel list records the corresponding relation between the identity identification information and the transmission channel.

For example, for a client newly joining a group video, other clients already participating in the group video are video playing terminals, and the client newly joining the group video is a video sending terminal. If the transmission channel lists stored on other clients in the group video do not contain the UID of the client newly added to the group video and the transmission channel number corresponding to the UID, the UID of the client and the transmission channel number allocated to the UID are added to each transmission channel list, and when the client corresponding to the UID is added to the group video again next time, the client only needs to inquire each transmission channel list to find the corresponding UID and then open the transmission channel of the transmission channel number recorded correspondingly.

Preferably, after the video playing end queries the transmission channel list according to the identity information, and determines and opens the transmission channel corresponding to the video sending end, the method further includes: the video playing end sends transmission channel opening confirmation information to the video server so that the video server sends a video synchronization request to the video sending end; the video playing end receives the video synchronization confirmation information forwarded by the video server to acquire the playing progress of the real-time video; wherein the video synchronization confirmation information includes: the sequence number of the starting video frame fragment and the sequence number of the current sending video frame fragment; the video synchronization confirmation information is information returned by the video sending end according to the video synchronization request.

Preferably, when the video player receives the video synchronization confirmation information forwarded by the video server, the method further includes: the video playing end receives a video coding frame group which is sent by a video server and corresponds to the video sending end, and performs receiving-playing display on the video coding frame group; the video server updates and generates a video coding frame group in a set time interval in real time according to the received video frame fragments, wherein the video coding frame group comprises at least one key frame.

Alternatively, the set of video coding frames within the set time interval may be at least one GOP. The advantage of displaying the video coded frame groups in a receiving and playing manner is that the playing latency can be reduced, thereby reducing the playing delay of the video.

And S430, the video player receives the video frame fragments of the video coding frames which are transmitted by the video transmitting end according to the set data transmission parameters and are generated in real time and transmitted by the video server through the transmission channel.

Illustratively, according to the relationship between the UID of each video sending end and the transmission channel stored in the local transmission channel list, the video player may receive the video frame fragments sent by different video sending ends forwarded by the video server through different transmission channels, so as to implement a transmission mode without mutual interference between video data, improve data transmission efficiency, and reduce video transmission delay.

And S440, the video playing end plays the video in real time according to the received video frame fragments.

Optionally, the video playing end combines the video frame fragments corresponding to different video coding frames according to the received video frame fragments, and then performs real-time video playing according to the frames.

Preferably, after the video playing end performs real-time video playing according to the received video frame segment, the method further includes: the video playing end receives a disconnection instruction sent by a video server, wherein the disconnection instruction comprises the identity identification information of the video sending end; and the video playing end closes the transmission channel corresponding to the video sending end according to the identity identification information in the disconnection instruction.

Illustratively, when a client in the group video receives a disconnection instruction about the UID of another client sent by the video server, the client closes the transmission channel corresponding to the UID of the client in the transmission channel list, so as to release the channel resource.

According to the technical scheme, the video playing end is matched with the video sending end to perform related operations with the video server, so that the video transmission delay is reduced, and the user experience effect is improved.

EXAMPLE five

Fig. 5 is a schematic structural diagram of a real-time video transmission apparatus according to a fifth embodiment of the present invention. The device can be suitable for the condition of transmitting real-time video, can be composed of hardware and/or software, and can be generally integrated in a video transmitting terminal and all intelligent terminals with video transmitting functions. Referring to fig. 5, the transmission apparatus of real-time video includes: a connection request sending module 510 and a video fragment sending module 520, which are described in detail below.

A connection request sending module 510, configured to send a connection request to a video server, and receive connection confirmation information returned by the video server, so as to establish a connection with the video server;

the video fragment sending module 520 is configured to send a video frame fragment of a video encoded frame generated in real time to the video server according to a set data transmission parameter, so that the video server forwards the video frame fragment to a video playing end that opens a corresponding transmission channel for real-time video playing.

In the transmission device for real-time video provided by this embodiment, a connection request is sent to a video server through a video sending end, so that after connection is established with the video server, the video sending end sends video frame fragments of video coding frames generated in real time to the video server according to set data transmission parameters, and then the video server forwards the video frame fragments to a video playing end for real-time video playing. The method has the advantages that the connection is established by only sending the request once, and the video coding frame is divided into the plurality of video frame fragments to be sent after the connection is established, so that the problem of high video transmission delay caused by a complex process of three-way handshake before the connection is performed based on TCP in the prior art and a method of directly transmitting and playing based on the video data stream is solved, and the effects of reducing the video transmission delay and improving the user experience are realized.

On the basis of the above embodiments, the method may further include:

the synchronous confirmation returning module is used for returning corresponding video synchronous confirmation information to the video server according to the video synchronous request sent by the video server after sending a connection request to the video server and receiving the connection confirmation information returned by the video server;

wherein the video synchronization confirmation information includes: the sequence number of the starting video frame fragment and the sequence number of the current sending video frame fragment; and when confirming that a target video playing end opens a transmission channel corresponding to the video sending end, the video server sends the video synchronization request to the video sending end, so that the target video playing end obtains the playing progress of the real-time video.

On the basis of the above embodiments, the method may further include:

the network delay calculation module is used for calculating network delay time according to the current time for sending the video frame fragments and the time for receiving the receiving confirmation response at last while sending the video frame fragments of the video coding frame generated in real time to the video server according to the set data transmission parameters;

the bandwidth jitter correction module is used for determining a current bandwidth jitter correction value according to the bandwidth receiving the acknowledgement response in the current unit time and the bandwidth jitter correction value in the previous stage;

and the transmission parameter updating module is used for updating the data transmission parameters according to the network delay time and the current bandwidth jitter correction value.

On the basis of the foregoing embodiments, the bandwidth jitter correction module may be specifically configured to:

the video sending end is according to the formula

Calculating a current bandwidth jitter correction value;

wherein acked _ bw is the current bandwidth jitter correction value, acked _ bw₀Bw is the bandwidth jitter correction value of the previous stage, and bw is the bandwidth receiving the acknowledgement response in the current unit time.

On the basis of the foregoing embodiments, the transmission parameter updating module may be specifically configured to:

if the retransmitted video frame fragments exist at present and the network delay time is greater than a preset threshold value, setting an expected bandwidth value as the current bandwidth jitter correction value, and reducing the value of the data transmission parameter according to the expected bandwidth value, wherein the data transmission parameter comprises: resolution of a video encoder, and/or data transmission rate;

if the retransmitted video frame fragments do not exist at present and the current bandwidth jitter correction value is larger than 0, setting an expected bandwidth value as a preset multiple of the current bandwidth jitter correction value, and increasing the value of the data transmission parameter according to the expected bandwidth value;

and if the current bandwidth jitter correction value is equal to 0, keeping the value of the data transmission parameter.

On the basis of the above embodiments, the method may further include:

the server connection disconnection module is used for sending a disconnection request to the video server after sending the video frame fragments of the video coding frames generated in real time to the video server according to the set data transmission parameters, receiving disconnection confirmation information returned by the video server and disconnecting the video server;

and after receiving the disconnection request, the video server controls the associated at least one video playing end to close a transmission channel corresponding to the video sending end.

The product can execute the method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.

EXAMPLE six

Fig. 6 is a schematic structural diagram of a real-time video transmission apparatus according to a sixth embodiment of the present invention. The device can be suitable for the real-time video transmission, can be composed of hardware and/or software, and can be generally integrated in a video server and all intelligent terminals containing video transmission functions. Referring to fig. 6, the transmission apparatus of real-time video includes: a connection confirmation returning module 610, an identification information sending module 620, and a video fragment forwarding module 630, which are described in detail below.

A connection confirmation returning module 610, configured to return, according to a connection request sent by the video sending end, corresponding connection confirmation information to the video sending end to establish a connection with the video sending end, where the connection request includes identity information of the video sending end;

an identification information sending module 620, configured to send the identity identification information to at least one associated video playing end, so as to activate the video playing end to open a transmission channel corresponding to the video sending end;

the video fragment forwarding module 630 is configured to receive a video frame fragment of a video encoded frame generated in real time and sent by the video sending end according to the set data transmission parameter, and forward the video frame fragment to a video playing end that opens a corresponding transmission channel for real-time video playing.

The real-time video transmission device provided by this embodiment responds to a connection request of a video sending end through a video server to establish a reliable connection, and sends identification information of the video sending end to a video playing end, so that the video playing end opens a corresponding transmission channel, and finally, the video sending end is matched to forward video frame fragments of a video coding frame generated in real time to an associated video playing end, so that the video playing end performs real-time video playing, thereby reducing video transmission delay, improving reliability of video transmission, and improving user experience.

On the basis of the above embodiments, the method may further include:

a synchronization request sending module, configured to send the video synchronization request to the video sending end after the identity information is sent to the associated at least one video playing end and when it is determined that a target video playing end opens a transmission channel corresponding to the video sending end;

a synchronization confirmation receiving module, configured to receive video synchronization confirmation information returned by the video sending end, where the video synchronization confirmation information includes: the sequence number of the starting video frame fragment and the sequence number of the current sending video frame fragment;

and the synchronous confirmation forwarding module is used for forwarding the video synchronous confirmation information to a target video playing end so as to enable the target video playing end to acquire the playing progress of the real-time video.

On the basis of the above embodiments, the method may further include:

the video coding system comprises a coding frame group generating module, a video coding frame group generating module and a video coding frame group updating module, wherein the coding frame group generating module is used for updating and generating a video coding frame group in a set time interval in real time according to a received video frame fragment after receiving the video frame fragment of a video coding frame which is sent by a video sending end according to set data transmission parameters and generated in real time;

the synchronization confirmation forwarding module is further configured to: and forwarding the video synchronization confirmation information and the video coding frame group to the target video playing end so that the video playing end performs receiving-playing display on the video coding frame group.

On the basis of the above embodiments, the method may further include:

a disconnection request receiving module, configured to receive a disconnection request sent by the video sending end after receiving a video frame fragment of a video encoded frame, which is sent by the video sending end according to a set data transmission parameter and generated in real time, and forwarding the video frame fragment to a video playing end that opens a corresponding transmission channel for real-time video playing;

a transmission channel closing module, configured to control the associated at least one video playing end to close a transmission channel corresponding to the video sending end according to the disconnection request;

and the disconnection confirmation returning module is used for returning corresponding disconnection confirmation information to the video sending end so as to disconnect with the video sending end.

EXAMPLE seven

Fig. 7 is a schematic structural diagram of a real-time video transmission apparatus according to a seventh embodiment of the present invention. The device can be suitable for the condition of transmitting real-time video, can be composed of hardware and/or software, and can be generally integrated in a video playing end and all intelligent terminals containing video receiving and playing functions. Referring to fig. 7, the transmission apparatus of real-time video includes: an identification information receiving module 710, a transmission channel opening module 720, a video fragment receiving module 730, and a real-time video playing module 740, which are described in detail below.

An identification information receiving module 710, configured to receive identity identification information of a video sending end sent by a video server;

a transmission channel opening module 720, configured to query a transmission channel list according to the identity information, determine and open a transmission channel corresponding to the video sending end, where a correspondence relationship between the identity information and the transmission channel is recorded in the transmission channel list;

a video fragment receiving module 730, configured to receive, through the transmission channel, a video frame fragment of a video encoded frame that is generated in real time and is transmitted by the video transmitting end according to a set data transmission parameter, where the video frame fragment is forwarded by the video server;

and the real-time video playing module 740 is configured to play a real-time video according to the received video frame segment.

The transmission device of the real-time video provided by the embodiment realizes the reduction of video transmission delay and improves the effect of user experience by the video playing end cooperating with the related operations of the video sending end and the video server.

On the basis of the above embodiments, the method may further include:

a channel opening confirmation module, configured to query a transmission channel list according to the identity information, determine and open a transmission channel corresponding to the video sending end, and send transmission channel opening confirmation information to the video server, so that the video server sends the video synchronization request to the video sending end;

the synchronous confirmation receiving module is used for receiving the video synchronous confirmation information forwarded by the video server so as to acquire the playing progress of the real-time video;

wherein the video synchronization confirmation information includes: the sequence number of the starting video frame fragment and the sequence number of the current sending video frame fragment; and the video synchronization confirmation information is information returned by the video sending end according to the video synchronization request.

On the basis of the above embodiments, the method may further include:

the receiving and playing display module is used for receiving the video synchronization confirmation information forwarded by the video server, receiving the video coding frame group which is sent by the video server and corresponds to the video sending end by the video playing end, and performing receiving and playing display on the video coding frame group;

and the video server updates and generates the video coding frame group in a set time interval in real time according to the received video frame fragments, wherein the video coding frame group comprises at least one key frame.

On the basis of the above embodiments, the method may further include:

a disconnection instruction receiving module, configured to receive a disconnection instruction sent by the video server after real-time video playing is performed according to the received video frame fragment, where the disconnection instruction includes identity identification information of the video sending end;

and the transmission channel closing module is used for closing the transmission channel corresponding to the video sending end according to the identity identification information in the disconnection instruction.

It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for transmitting real-time video, comprising:

the video sending end sends video frame fragments of a video coding frame generated in real time to the video server according to set data transmission parameters, so that the video server forwards the video frame fragments to a video playing end which opens a corresponding transmission channel for real-time video playing;

after the video sending end sends a connection request to the video server and receives connection confirmation information returned by the video server, the method further comprises the following steps:

the video sending end returns corresponding video synchronization confirmation information to the video server according to the video synchronization request sent by the video server;

the video synchronization confirmation information includes: the sequence number of the starting video frame fragment and the sequence number of the current sending video frame fragment; when confirming that a target video playing end opens a transmission channel corresponding to the video sending end, the video server sends the video synchronization request to the video sending end so that the target video playing end obtains the playing progress of a real-time video;

after the video sending end sends the video frame fragments of the video coding frames generated in real time to the video server according to the set data transmission parameters, the method further comprises the following steps:

the video server updates and generates a video coding frame group in a set time interval in real time according to the received video frame fragments, wherein the video coding frame group comprises at least one key frame;

and the video server forwards the video synchronization confirmation information and the video coding frame group to a target video playing end so that the video playing end performs receiving and playing display on the video coding frame group.

2. The method according to claim 1, wherein while the video sending end sends video frame fragments of video encoded frames generated in real time to the video server according to the set data transmission parameters, the method further comprises:

the video sending end calculates the network delay time according to the current time for sending the video frame fragments and the time for receiving the receiving confirmation response;

the video sending end determines a current bandwidth jitter correction value according to the bandwidth receiving the acknowledgement response in the current unit time and the bandwidth jitter correction value of the previous stage;

and the video sending end updates the data transmission parameters according to the network delay time and the current bandwidth jitter correction value.

3. The method of claim 2, wherein the video transmitting end calculates the current bandwidth jitter correction value according to the bandwidth receiving the acknowledgement response in the current unit time and the bandwidth jitter correction value of the previous stage, and comprises:

the video sending end is according to the formula

Calculating a current bandwidth jitter correction value;

4. The method of claim 3, wherein updating the data transmission parameters based on the network delay time and the current bandwidth jitter correction value comprises:

5. The method according to claim 1, wherein after the video sending end sends the video frame fragments of the video coding frames generated in real time to the video server according to the set data transmission parameters, the method further comprises:

the video sending end sends a disconnection request to the video server and receives disconnection confirmation information returned by the video server so as to be disconnected with the video server;

6. A method for transmitting real-time video, comprising:

the video server receives video frame fragments of a video coding frame which are sent by the video sending end according to set data transmission parameters and generated in real time, and forwards the video frame fragments to a video playing end which opens a corresponding transmission channel for real-time video playing;

after the video server sends the identification information to the associated at least one video playing end, the method further includes:

the video server sends the video synchronization request to the video sending terminal when confirming that a target video playing terminal opens a transmission channel corresponding to the video sending terminal;

the video server receives video synchronization confirmation information returned by the video sending end, wherein the video synchronization confirmation information comprises: the sequence number of the starting video frame fragment and the sequence number of the current sending video frame fragment;

the video synchronization confirmation information is forwarded to a target video playing end, so that the target video playing end obtains the playing progress of the real-time video;

after the video server receives the video frame fragments of the video coding frames which are sent by the video sending end according to the set data transmission parameters and generated in real time, the method further comprises the following steps:

the video server updates and generates a video coding frame group in a set time interval in real time according to the received video frame fragment, wherein the video coding frame group comprises at least one key frame;

forwarding the video synchronization confirmation information to a target video playing end, further comprising: and forwarding the video synchronization confirmation information and the video coding frame group to the target video playing end so that the video playing end performs receiving-playing display on the video coding frame group.

7. The method according to claim 6, wherein after the video server receives the video frame fragments of the video encoded frames which are sent by the video sending end according to the set data transmission parameters and generated in real time, and forwards the video frame fragments to the video playing end which opens the corresponding transmission channel for real-time video playing, the method further comprises:

the video server receives a disconnection request sent by the video sending end;

the video server controls the associated at least one video playing end to close a transmission channel corresponding to the video sending end according to the disconnection request;

and the video server returns corresponding disconnection confirmation information to the video sending end so as to disconnect with the video sending end.

8. A method for transmitting real-time video, comprising:

the video playing end plays the video in real time according to the received video frame fragments;

after the video playing end queries a transmission channel list according to the identity information and determines and opens a transmission channel corresponding to the video sending end, the method further comprises the following steps:

the video playing end sends transmission channel opening confirmation information to the video server so that the video server sends the video synchronization request to the video sending end;

the video playing end receives the video synchronization confirmation information forwarded by the video server to acquire the playing progress of the real-time video;

wherein the video synchronization confirmation information includes: the sequence number of the starting video frame fragment and the sequence number of the current sending video frame fragment; the video synchronization confirmation information is information returned by the video sending end according to the video synchronization request;

wherein, when the video playing end receives the video synchronization confirmation information forwarded by the video server, the method further comprises:

the video playing end receives a video coding frame group which is sent by the video server and corresponds to the video sending end, and the video coding frame group is displayed in a receiving and playing mode;

9. The method according to claim 8, wherein after the video playing end performs real-time video playing according to the received video frame segment, the method further comprises:

the video playing end receives a disconnection instruction sent by the video server, wherein the disconnection instruction comprises the identity identification information of the video sending end;

and the video playing end closes a transmission channel corresponding to the video sending end according to the identity identification information in the disconnection instruction.

10. A real-time video transmission apparatus configured at a video transmitting end, comprising:

the video fragment sending module is used for sending the video frame fragments of the video coding frames generated in real time to the video server according to the set data transmission parameters so that the video server forwards the video frame fragments to the video playing end which opens the corresponding transmission channel for real-time video playing;

wherein the apparatus further comprises:

after transmitting the video frame fragments of the video coding frame generated in real time to the video server according to the set data transmission parameters, the method further comprises the following steps:

11. A real-time video transmission apparatus configured in a video server, comprising:

the video fragment forwarding module is used for receiving video frame fragments of a video coding frame which are sent by the video sending end according to set data transmission parameters and generated in real time, and forwarding the video frame fragments to a video playing end which opens a corresponding transmission channel for real-time video playing;

wherein the apparatus further comprises:

the synchronous confirmation forwarding module is used for forwarding the video synchronous confirmation information to a target video playing end so that the target video playing end can obtain the playing progress of a real-time video;

after receiving the video frame fragments of the video coding frame which is sent by the video sending end according to the set data transmission parameters and generated in real time, the method further comprises the following steps:

updating and generating a video coding frame group in a set time interval in real time according to the received video frame fragment, wherein the video coding frame group comprises at least one key frame;

12. A real-time video transmission device configured at a video playing end, comprising:

the real-time video playing module is used for playing a real-time video according to the received video frame fragments;

wherein the apparatus further comprises:

the video synchronization confirmation information includes: the sequence number of the starting video frame fragment and the sequence number of the current sending video frame fragment; the video synchronization confirmation information is information returned by the video sending end according to the video synchronization request;

wherein, when receiving the video synchronization confirmation information forwarded by the video server, the method further comprises:

receiving a video coding frame group which is sent by the video server and corresponds to the video sending end, and carrying out receiving-and-playing display on the video coding frame group;