CN108234997B - Time-delay measuring method, equipment, system and the computer-readable medium of live video - Google Patents

Time-delay measuring method, equipment, system and the computer-readable medium of live video Download PDF

Info

Publication number
CN108234997B
CN108234997B CN201711320836.XA CN201711320836A CN108234997B CN 108234997 B CN108234997 B CN 108234997B CN 201711320836 A CN201711320836 A CN 201711320836A CN 108234997 B CN108234997 B CN 108234997B
Authority
CN
China
Prior art keywords
video frame
target video
acquisition time
code stream
time stamp
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201711320836.XA
Other languages
Chinese (zh)
Other versions
CN108234997A (en
Inventor
李永兴
沈慧锋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN201711320836.XA priority Critical patent/CN108234997B/en
Publication of CN108234997A publication Critical patent/CN108234997A/en
Application granted granted Critical
Publication of CN108234997B publication Critical patent/CN108234997B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N17/00—Diagnosis, testing or measuring for television systems or their details
    • H04N17/004—Diagnosis, testing or measuring for television systems or their details for digital television systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)

Abstract

The present invention provides time-delay measuring method, equipment, system and the computer-readable medium of a kind of live video.Its method includes: the incidence relation of the acquisition time stamp according to the PTS of the target video frame after the coding and PTS of the target video frame pre-established and acquisition target video frame, obtains the acquisition time stamp of target video frame；Acquisition time stamp is encapsulated in code stream；The code stream that the data of target video frame after being packaged with from coding to the transmission of the end CDN and corresponding acquisition time stab, for the end CDN to play end transmission objectives video frame data while, the acquisition time of transmission objectives video frame stabs, by playing end when playing target video frame, the time delay of live streaming is measured.Technical solution of the present invention can save manual operation, effectively improve the precision of measurement.And technical solution of the present invention, the various protocols such as flv, rtmp and hls can be supported simultaneously, and flexibility is very high, very wide using field.

Description

Time-delay measuring method, equipment, system and the computer-readable medium of live video
[technical field]
The present invention relates to computer application technology more particularly to a kind of time-delay measuring method of live video, equipment, System and computer-readable medium.
[background technique]
With the development of internet, occur various services along with internet.For example, live streaming is by mutual A kind of Video service of networking and appearance.
Existing live broadcast system can specifically include plug-flow end, content distributing network (Content Delivery Network；CDN) end is cloud and broadcasting end.Wherein plug-flow end acquires video frame, by video frame after coded treatment Initial data occurs to give the end CDN；It is a variety of resolution ratio by video frame transcoding after CDN terminates the initial data for receiving video frame Format；Then when the live streaming of certain format is requested at broadcasting end, the video requency frame data of corresponding format is transferred to and plays end, with The video frame is played for playing end.Each video frame is handled by using aforesaid way, the straight of video may be implemented It broadcasts.Since the process played belongs to live streaming, then time delay is to measure an important reference indicator of live streaming quality, which refers to Be to acquire video frame from plug-flow end to play time delay the video frame to end is played.In the prior art, in live streaming Latency measurement, can by way of manual setting clock, respectively record acquisition time stamp and play time stamp, with to time delay into Row measurement.
But the latency measurement mode in above-mentioned live streaming needs manual operations, causes the precision of measurement lower.
[summary of the invention]
The present invention provides a kind of time-delay measuring method of live video, equipment, system and computer-readable mediums, are used for Improve the measurement accuracy of time delay in live streaming.
The present invention provides a kind of time-delay measuring method of live video, which comprises
According to the PTS of the PTS of the target video frame after coding and the target video frame pre-established and acquisition institute The incidence relation for stating the acquisition time stamp of target video frame obtains the acquisition time stamp of the target video frame；
Acquisition time stamp is encapsulated in code stream；
The data of the target video frame after being packaged with from coding to the transmission of the end CDN and the corresponding acquisition time stamp Code stream, for the end CDN to play end transmit the data of the target video frame while, transmit the target video frame The acquisition time stamp, by the broadcasting end when playing the target video frame, measure the time delay of live streaming.
Still optionally further, in method as described above, acquisition time stamp is encapsulated in code stream, is specifically included:
Acquisition time stamp is encapsulated in the SEI of code stream.
Still optionally further, in method as described above, further includes:
The presence mark of the acquisition time stamp is provided in the SEI of the code stream.
Still optionally further, it in method as described above, builds according to the PTS of the target video frame after coding and in advance The incidence relation that the PTS of the vertical target video frame and the acquisition time for acquiring the target video frame stab, obtains the mesh Before the acquisition time stamp for marking video frame, the method also includes:
When acquiring the target video frame, the acquisition time stamp when target video frame is obtained；
Establish the incidence relation that the PTS of the target video frame is stabbed with the corresponding acquisition time.
The present invention also provides a kind of time-delay measuring methods of live video, which comprises
Before the data of the target video frame in the code stream transmitted to plug-flow end carry out transcoding, according to the institute in the code stream State the data of target video frame and the acquisition time stamp of the target video frame, establish the PTS of the target video frame with it is right The incidence relation for the acquisition time stamp answered；
After the data to the target video frame carry out transcoding, according to the incidence relation, the acquisition time is stabbed It is added in the code stream of the data of the target video frame after being packaged with transcoding；
To the data and the corresponding acquisition time for playing the target video frame after end transmission is packaged with transcoding The code stream of stamp measures the time delay of live streaming so that the broadcasting end is when playing the target video frame.
Still optionally further, in method as described above, according to the data of the target video frame in the code stream with And the acquisition time of the target video frame stabs, what the PTS and the corresponding acquisition time for establishing the target video frame were stabbed Incidence relation specifically includes:
The PTS of the target video frame is obtained from the data of the target video frame in the code stream；
The acquisition time stamp of the target video frame is obtained from the SEI in the code stream；
Establish the incidence relation that the PTS of the target video frame is stabbed with the corresponding acquisition time.
Still optionally further, in method as described above, the target video frame is obtained from the SEI in the code stream Before the acquisition time stamp, the method also includes:
According to the presence mark in the SEI in the code stream, determines and carry the acquisition time in the SEI Stamp.
The present invention also provides a kind of time-delay measuring methods of live video, which comprises
Before the target video frame in the code stream transmitted to the end CDN plays out, according to the target in the code stream The acquisition time of the data of video frame and target video frame stamp, establishes the PTS of the target video frame and corresponding institute State the incidence relation of acquisition time stamp；
When the target video frame is played, the PTS of the target video frame and broadcasting for the target video frame are established Put the corresponding relationship of timestamp；
According to the incidence relation and the corresponding relationship, the live streaming time delay for playing the target video frame is obtained.
Still optionally further, in method as described above, according to the data of the target video frame in the code stream with And the acquisition time of the target video frame stabs, what the PTS and the corresponding acquisition time for establishing the target video frame were stabbed Incidence relation specifically includes:
The PTS of the target video frame is obtained from the data of the target video frame in the code stream；
The acquisition time stamp of the target video frame is obtained from the SEI in the code stream；
Establish the incidence relation that the PTS of the target video frame is stabbed with the corresponding acquisition time.
Still optionally further, in method as described above, the target video frame is obtained from the SEI in the code stream Before the acquisition time stamp, the method also includes:
According to the presence mark in the SEI in the code stream, determines and carry the acquisition time in the SEI Stamp.
The present invention provides a kind of plug-flow end equipment, and the equipment includes:
Module is obtained, for according to the PTS of the target video frame after coding and the target video frame pre-established PTS and acquire the target video frame acquisition time stamp incidence relation, obtain the acquisition of the target video frame Timestamp；
Package module, for acquisition time stamp to be encapsulated in code stream；
Transmission module, for the data of the target video frame after being packaged with from coding to the transmission of the end CDN and corresponding The code stream of the acquisition time stamp, while transmitting the data of the target video frame to broadcasting end for the end CDN, transmission The acquisition time of the target video frame stabs, and by the broadcasting end when playing the target video frame, measures live streaming Time delay.
Still optionally further, in equipment as described above, the package module is specifically used for stabbing the acquisition time and seal In the SEI of code stream.
Still optionally further, in equipment as described above, the equipment further include:
Setup module, for being provided with the presence mark of the acquisition time stamp in the SEI of the code stream.
It still optionally further, further include establishing module in equipment as described above；
The acquisition module is also used to obtain the acquisition when target video frame when acquiring the target video frame Timestamp；
The association described to establish module, that the PTS for establishing the target video frame is stabbed with the corresponding acquisition time Relationship.
The present invention also provides a kind of CDN end equipments, which is characterized in that the equipment includes:
Establish module, for the data of the target video frame in the code stream transmitted to plug-flow end carry out transcoding before, according to The data of the target video frame in the code stream and the acquisition time stamp of the target video frame, establish the target view The incidence relation of the PTS of frequency frame and the corresponding acquisition time stamp；
Adding module, for the target video frame data carry out transcoding after, according to the incidence relation, by institute In the code stream for stating the data that acquisition time stamp is added to the target video frame after being packaged with transcoding；
Transmission module, for being packaged with the data of the target video frame after transcoding and corresponding to playing end transmission The code stream of the acquisition time stamp measures the time delay of live streaming so that the broadcasting end is when playing the target video frame.
Still optionally further, described to establish module in equipment as described above, it is specifically used for:
The PTS of the target video frame is obtained from the data of the target video frame in the code stream；
The acquisition time stamp of the target video frame is obtained from the SEI in the code stream；
Establish the incidence relation that the PTS of the target video frame is stabbed with the corresponding acquisition time.
Still optionally further, in equipment as described above, further includes:
Determining module is determined and is carried in the SEI for being identified according to the presence in the SEI in the code stream The acquisition time stamp.
The present invention also provides a kind of broadcasting end equipment, the equipment includes:
Module is established, for before the target video frame in the code stream transmitted to the end CDN plays out, according to the code stream In the target video frame data and the target video frame acquisition time stamp, establish the target video frame The incidence relation of PTS and the corresponding acquisition time stamp；
It is described to establish module, be also used to when the target video frame is played, establish the PTS of the target video frame with The corresponding relationship of the play time stamp of the target video frame；
Module is obtained, for obtaining and playing the target video frame according to the incidence relation and the corresponding relationship Live streaming time delay.
Still optionally further, described to establish module in equipment as described above, it is specifically used for:
The PTS of the target video frame is obtained from the data of the target video frame in the code stream；
The acquisition time stamp of the target video frame is obtained from the SEI in the code stream；
Establish the incidence relation that the PTS of the target video frame is stabbed with the corresponding acquisition time.
Still optionally further, in equipment as described above, further includes:
Determining module is determined and is carried in the SEI for being identified according to the presence in the SEI in the code stream The acquisition time stamp.
The present invention also provides a kind of video living transmission systems, including plug-flow end equipment as described above, the end CDN as described above Equipment and end equipment is played as described above；The CDN end equipment is set with the plug-flow end equipment and the broadcasting end respectively Standby communication connection.
The present invention also provides a kind of computer equipment, the equipment includes:
One or more processors；
Memory, for storing one or more programs；
When one or more of programs are executed by one or more of processors, so that one or more of processing Device realizes the time-delay measuring method of live video as described above.
The present invention also provides a kind of computer-readable mediums, are stored thereon with computer program, which is held by processor The time-delay measuring method of live video as described above is realized when row.
Time-delay measuring method, equipment, system and the computer-readable medium of live video of the invention, by using above-mentioned The timestamp of target video frame can be encapsulated in code stream and transmit by technical solution, to guarantee that acquisition time stamp can be by It is successfully transferred to and plays end, enable and play the survey that end effectively completes live streaming time delay when playing according to acquisition time stamp Amount.The measurement that live streaming time delay is realized relative to existing manual operation, can save manual operation, effectively improve the essence of measurement Degree.And technical solution of the present invention, the various protocols such as flv, rtmp and hls can be supported simultaneously, and flexibility is very high, uses Field is very wide.
[Detailed description of the invention]
Fig. 1 is the flow chart of the time-delay measuring method embodiment one of live video of the invention.
Fig. 2 is the flow chart of the time-delay measuring method embodiment two of live video of the invention.
Fig. 3 is the flow chart of the time-delay measuring method embodiment three of live video of the invention.
Fig. 4 is the signaling diagram of the time-delay measuring method example IV of live video of the invention.
Fig. 5 is the structure chart of plug-flow end equipment embodiment of the invention.
Fig. 6 is the structure chart of CDN end equipment embodiment of the invention.
Fig. 7 is the structure chart of broadcasting end equipment embodiment of the invention.
Fig. 8 is the structure chart of video living transmission system embodiment of the invention.
Fig. 9 is the structure chart of computer equipment embodiment of the invention.
Figure 10 is a kind of exemplary diagram of computer equipment provided by the invention.
[specific embodiment]
To make the objectives, technical solutions, and advantages of the present invention clearer, right in the following with reference to the drawings and specific embodiments The present invention is described in detail.
Fig. 1 is the flow chart of the time-delay measuring method embodiment one of live video of the invention.As shown in Figure 1, this implementation The time-delay measuring method of the live video of example, can specifically include following steps:
100, according to Presentation Time Stamp (the Presentation Time Stamp of the target video frame after coding；PTS), And the incidence relation of the PTS of the target video frame pre-established and the acquisition time stamp of acquisition target video frame, obtain target The acquisition time of video frame stabs；
The executing subject of the time-delay measuring method of the live video of the present embodiment is that the plug-flow end in video living transmission system is set Standby, which for acquiring the image of each video frame in live video, and video frame is led in net cast Cross content distributing network (Content Delivery Network；CDN) end is that cloud is sent to broadcasting end, is broadcast for playing end Each video frame is put, realizes net cast.In the present embodiment by taking a target video frame as an example, to introduce the live streaming of the present embodiment The latency measurement of video.
The target video frame of the present embodiment is specifically as follows the image of any frame in live video；In target video frame PTS can be carried in target video frame, for identifying the target video frame.It acquires available to the mesh when target video frame The PTS of video frame is marked, related art can be referred in detail, no longer repeated herein.
It optionally, can also include: to obtain adopting for target video frame when acquiring target video frame before step 100 Collect timestamp；Establish the incidence relation that the PTS of target video frame is stabbed with corresponding acquisition time.
Specifically, the present embodiment is when acquiring target video frame, it is also necessary to acquire the acquisition time of target video frame The stamp i.e. acquisition moment.The purpose of the present embodiment is finally to be transmitted to acquisition time stamp by the end CDN to play end, for playing end When playing, corresponding play time is stabbed when stabbing and play according to the acquisition time, when measuring the live streaming of the target video frame Prolong.It can be assumed in the present embodiment under normal circumstances, the live streaming time delay of each of live video video frame is all identical, can be with Using the time delay of either objective video frame as the time delay of live video, to obtain the time delay of live video.
It, can be by target video frame by the PTS unique identification of the target video frame in this present embodiment target video frame PTS and corresponding acquisition time stab and establish association, associate to realize and stab acquisition time with target video frame.Plug-flow Target video frame feeding encoder is first encoded before sending target video frame to the end CDN, obtains and compile from encoder by end The data of target video frame after code；Then according to the PTS of the target video frame after coding, from the target video frame pre-established PTS and acquisition time stamp incidence relation, the available acquisition time stamp to the target video frame.
101, acquisition time stamp is encapsulated in code stream；
Specifically, the acquisition time stamp that will acquire is encapsulated in code stream, is encapsulated in for example, acquisition time can specifically be stabbed Supplemental enhancement information (the Supplemental enhancement information of code stream；SEI in), it is further preferred that The SEI can be inserted in data packet i.e. network abstract layer (the Network Abstract of the target video frame in code stream Layer；NAL before) wrapping.In practical application, acquisition time stabs the other positions that can also be encapsulated in code stream, no longer superfluous herein It states.
In the present embodiment, being stabbed by the acquisition time of SEI transmission objectives video frame, SEI is a part of H.264 standard, It can support flash video (abbreviation flv) agreement, http live stream (http live streaming；Hls) agreement and Real-time messages transport protocol (real time messaging protocol；rtmp).Therefore, the present embodiment is by regarding target The acquisition time stamp of frequency frame is carried to be transmitted in SEI, even if carrying out transcoding frame rate conversion at the end CDN, can also take one Fixed measure guarantees that the acquisition time stamp in SEI is not lost, so as to complete the measurement of live video time delay very well.
In addition, still optionally further, in the present embodiment, depositing for acquisition time stamp is also provided in the SEI of code stream It is identifying, such as when in SEI, there are when acquisition time stamp, 1 can be set by the numerical value of some predeterminated position, when as acquisition Between stab presence mark.And 0 is set by the numerical value of the predeterminated position there is no when acquisition time stamp in SEI.Make in this way Used time, such as the end CDN can first detect whether the numerical value of the predeterminated position is 1, if 1, it is determined that the numerical value is acquisition time The presence of stamp identifies, and can stab position acquisition acquisition time of preset storage acquisition time stamp from SEI.And if should The numerical value of predeterminated position is 0, it is determined that without carrying acquisition time stamp in SEI.
102, the code that the data of the target video frame after being packaged with coding to the transmission of the end CDN and corresponding acquisition time stab Stream, for the end CDN to play end transmission objectives video frame data while, the acquisition time of transmission objectives video frame, by broadcasting End is put when playing target video frame, measures the time delay of live streaming.
By above-mentioned processing, after target video frame is encapsulated in code stream, plug-flow end can be transmitted to the end CDN to be packaged with The data of target video frame after coding and corresponding acquisition time stamp code stream, for the end CDN to play end transmission objectives While the data of video frame, the acquisition time of transmission objectives video frame, by playing end when playing target video frame, measurement is straight The time delay broadcast.
In the present embodiment, the bit stream between plug-flow end and the end CDN can use flv agreement, rtmp agreement or hls Agreement come realize data transmit.
It should be noted that be realized in the present embodiment live streaming time delay measurement, must be requested that plug-flow end, the end CDN and The clock for playing end is synchronous, and the synchronous method of specific clock can refer to related art, does not do in the present invention in detail It repeats.
By using the time-delay measuring method of the live video of the present embodiment, may be implemented to carry out the time delay of live video Accurate measurement optimizes the time delay of the live streaming in live broadcast system in order to the time delay of the subsequent live video according to measurement, improves straight Broadcast quality and live streaming efficiency.
The time-delay measuring method of the live video of the present embodiment, by according to the PTS of the target video frame after coding and The incidence relation of the acquisition time stamp of the PTS and acquisition target video frame of the target video frame pre-established, obtains target video The acquisition time of frame stabs；Acquisition time stamp is encapsulated in code stream；The target video frame after encoding is packaged with to the transmission of the end CDN Data and corresponding acquisition time stamp code stream, for the end CDN to play end transmission objectives video frame data while, pass The acquisition time of defeated target video frame measures the time delay of live streaming by playing end when playing target video frame.The skill of the present embodiment It in art scheme, is transmitted by the way that the timestamp of target video frame to be encapsulated in code stream, to guarantee that acquisition time stamp can be by It is successfully transferred to and plays end, enable and play the survey that end effectively completes live streaming time delay when playing according to acquisition time stamp Amount.The measurement that live streaming time delay is realized relative to existing manual operation, can save manual operation, effectively improve the essence of measurement Degree.
In addition, in the prior art also by the way of transmitting the acquisition time stamp of video frame in flv agreement, to broadcast It is stabbed when putting the video frame according to play time stamp and acquisition time, realizes the measurement of time delay；But this kind of mode does not support code stream Conversion, flexibility is poor.It is also provided with a kind of timestamp command by increasing rtmp in the prior art, transmits video frame Acquisition time stamp realizes the measurement of time delay to stab when playing the video frame according to play time stamp and acquisition time.This side Formula needs for rtmp order to be associated with video requency frame data by force when realizing, the error for not only resulting in latency measurement is larger, simultaneously should Mode does not support flv agreement and hls agreement, flexibility poor yet.
And the time-delay measuring method of the live video of the present embodiment can be supported simultaneously compared with above-mentioned existing scheme The various protocols such as flv, rtmp and hls, flexibility is very high, very wide using field.
Fig. 2 is the flow chart of the time-delay measuring method embodiment two of live video of the invention.As shown in Fig. 2, this implementation The time-delay measuring method of the live video of example, can specifically include following steps:
200, before the data of the target video frame in the code stream transmitted to plug-flow end carry out transcoding, according to the mesh in code stream The data of video frame and the acquisition time stamp of target video frame are marked, the PTS of target video frame and corresponding acquisition time are established The incidence relation of stamp；
The executing subject of the time-delay measuring method of the live video of the present embodiment is CDN end equipment or is referred to as cloud End, i.e., the present embodiment describes technical solution of the present invention at the end CDN.Specifically, plug-flow end can pass through flv, rtmp or hls etc. Any agreement will be packaged with the bit stream of the acquisition time stamp of the data and acquisition target video frame of target video frame extremely The end CDN.After CDN termination receives code stream, need for target video frame to be converted into the format of various resolution ratio, to support height respectively Clearly, the broadcasting of standard and low clear video.For example, can specifically be realized respectively by different encoders to target video frame Data carry out transcoding.If but directly will lead to code stream progress transcoding with realizing that the data to target video frame carry out transcoding The acquisition time of the target video frame encapsulated in code stream, which stabs, to be lost.It therefore, can be in the number to target video frame in the present embodiment Before transcoding, it can specifically include following steps:
(a) PTS of target video frame is obtained from the data of the target video frame in code stream；
(b) the acquisition time stamp of target video frame is obtained from the SEI in code stream；
(c) incidence relation that the PTS of target video frame is stabbed with corresponding acquisition time is established.
It had not only included the data of target video frame in code stream in the present embodiment, but also the acquisition time including target video frame stabs, Specifically, acquisition time stamp can be encapsulated in the SEI of code stream.Specifically, which can be located at encapsulation target view in code stream Before the NAL packet of the data of frequency frame.Since the PTS of target video frame is used for the unique identification target video frame, specifically may be used To obtain the PTS of target video frame from the data of the target video frame in code stream；Target video is obtained from the SEI in code stream The acquisition time of frame stabs；Then the incidence relation that the PTS of target video frame is stabbed with corresponding acquisition time is established.
The step of the present embodiment (a)-(c) is a kind of specific implementation of the step 200 in the present embodiment, practical application In, the acquisition time in the present embodiment stabs the other positions that can also be encapsulated in code stream, and details are not described herein.
Still optionally further, the presence mark for identifying acquisition time stamp can also be carried in the SEI of the present embodiment, If carrying presence mark in SEI, then it represents that carry acquisition time stamp in the SEI；Otherwise acquisition time stamp is not carried. It therefore, can also include: from code before step (b) " from the acquisition time stamp for obtaining target video frame in the SEI in code stream " The numerical value of the position where the presence mark of acquisition time stamp is obtained in the SEI of stream, then according to the numerical value judge in SEI whether There are acquisition time stamps.If the numerical value is 1, the presence mark in the SEI in code stream there are acquisition time stamp, the presence are identified The position of mark both sides can be set in advance.Then it can determine according to the presence mark 1 in the SEI in code stream, it can be with It determines and carries acquisition time stamp in SEI.Otherwise such as numerical value is 0, then identifies and acquisition time stamp is not present in the SEI in code stream Presence mark.
201, after the data to target video frame carry out transcoding, according to incidence relation, acquisition time stamp is added to envelope In the code stream of data equipped with the target video frame after transcoding；
In the end CDN, the data of target video frame can be subjected to transcoding using the relevant technologies, obtain various resolution ratio The data of target video frame after transcoding.Then when playing user's request target video frame at end, being further continued for will be after transcoding The data of target video frame, which are transferred to, plays end.It, can be with before transmission such as after the data to target video frame carry out transcoding According to the incidence relation that the PTS of the target video frame after transcoding and the PTS pre-established are stabbed with corresponding acquisition time, obtain The corresponding acquisition time stamp of the target video frame, is then added to the target video frame after being packaged with transcoding for acquisition time stamp In the code stream of data.Such as can specifically acquisition time stamp be added in the SEI of the code stream, specifically, which can be located at code Before the NAL packet for finally encapsulating target video frame in stream.In practical application, acquisition time stabs other that can also be encapsulated in code stream Position, details are not described herein.
202, it is stabbed to the data and corresponding acquisition time for playing the target video frame after end transmission is packaged with transcoding Code stream measures the time delay of live streaming for playing end when playing target video frame.
By above-mentioned processing, after target video frame is encapsulated in code stream, the end CDN is packaged with transcoding to end transmission is played The code stream of the data of target video frame afterwards and corresponding acquisition time stamp.The available target video frame in end is played in this way Acquisition time stamp, and play when record the target video frame play time stamp, in this way it is available arrive target video frame Time delay, and then get the time delay of live streaming.
The time-delay measuring method of the live video of the present embodiment, compared with the technical solution of above-mentioned embodiment illustrated in fig. 1 Difference is: embodiment illustrated in fig. 1 describes technical solution of the present invention at plug-flow end；And the present embodiment describes this hair at the end CDN Bright technical solution, realization principle is similar, in detail can also be with reference to the record of embodiment described in above-mentioned Fig. 1, and details are not described herein.
Similarly, the measurement that live streaming time delay is realized in the present embodiment needs plug-flow end, the end CDN and the clock for playing end It must synchronize.
The time-delay measuring method of the live video of the present embodiment passes through the target video in the code stream transmitted to plug-flow end Before the data of frame carry out transcoding, is stabbed, built according to the acquisition time of the data of the target video frame in code stream and target video frame The incidence relation that the PTS of vertical target video frame is stabbed with corresponding acquisition time；After the data to target video frame carry out transcoding, According to incidence relation, in the code stream for the data that acquisition time stamp is added to the target video frame after being packaged with transcoding；To broadcasting End transmission is packaged with the data of the target video frame after transcoding and the code stream of corresponding acquisition time stamp, is broadcasting for playing end When putting target video frame, the time delay of live streaming is measured.In the technical solution of the present embodiment, by the way that the timestamp of target video frame is sealed It is transmitted in code stream, end is played to guarantee that acquisition time stamp can be delivered successfully, so that playing end when playing The measurement for effectively completing live streaming time delay can be stabbed according to acquisition time.Live streaming time delay is realized relative to existing manual operation Measurement, can save manual operation, effectively improve the precision of measurement.And the latency measurement side of the live video of the present embodiment Method can support the various protocols such as flv, rtmp and hls simultaneously, and flexibility is very high, very wide using field.
Fig. 3 is the flow chart of the time-delay measuring method embodiment three of live video of the invention.As shown in figure 3, this implementation The time-delay measuring method of the live video of example, can specifically include following steps:
300, before the target video frame in the code stream transmitted to the end CDN plays out, according to the target video in code stream The data of frame and the acquisition time stamp of target video frame, establish the pass that the PTS of target video frame is stabbed with corresponding acquisition time Connection relationship；
The executing subject of the time-delay measuring method of the live video of the present embodiment is to play end equipment, that is, is playing end description Technical solution of the present invention.Specifically, the end CDN can will be packaged with target view by any agreement such as flv, rtmp or hls The bit stream of the acquisition time stamp of the data of frequency frame and acquisition target video frame is to playing end.Play termination receive code stream it Afterwards, for the ease of measuring time delay, before can playing out with the target video frame in code stream, target video frame is obtained from code stream The acquisition time of data and target video frame stamp, and then the pass that the PTS for establishing target video frame is stabbed with corresponding acquisition time Connection relationship.Specific implementation can be with reference to step (a)-(c) in above-described embodiment, in detail can be with reference in above-described embodiment It is related record, details are not described herein.
It similarly, optionally, also may include: to be identified according to the presence in the SEI in code stream before executing step (a), It determines and carries acquisition time stamp in SEI.In detail can also be with reference to the related record in above-described embodiment, details are not described herein.
301, when target video frame is played, the play time of the PTS and target video frame that establish target video frame are stabbed Corresponding relationship；
301, according to incidence relation and corresponding relationship, the live streaming time delay for playing target video frame is obtained.
Specifically, end is played when playing target video frame, obtains the play time stamp of target video frame, play time stamp The as playing time of target video frame, because in video living transmission system, the clock at plug-flow end, the end CDN and broadcasting end is It is synchronous.So herein can by record target video frame play time stab, then according to transmission come acquisition time stamp with Play time stamp, to measure the time delay of live streaming.But due to net cast during, each target video frame is continuously It is transferred to from plug-flow end by CDN sections and plays end.For the ease of being monitored and processed to each target video frame, regarded in target When frequency frame is played, it is also necessary to the PTS that target video frame is obtained from target video frame, since PTS is used for the unique identification mesh Video frame is marked, the corresponding relationship that the play time of the PTS and target video frame that then establish target video frame are stabbed just obtains target The corresponding relationship that video frame is stabbed with corresponding play time.Then it is associated with according to the PTS of target video frame with what acquisition time stabbed The corresponding relationship that the play time of the PTS of relationship and target video frame and target video frame is stabbed, just obtains the target video frame Corresponding acquisition time stamp and play time are stabbed, and so as to calculate the time difference of the two, obtain the straight of the target frame video The time delay broadcast.It is assumed that the time delay of the live streaming of each video frame is identical in the video living transmission system, to obtain the straight of live video The time delay broadcast.It is subsequent the time delay of live streaming in live broadcast system to be optimized according to the time delay of the live video of measurement, improve live streaming Quality and live streaming efficiency.
The time-delay measuring method of the live video of the present embodiment, compared with the technical solution of above-mentioned embodiment illustrated in fig. 2 Difference is: embodiment illustrated in fig. 2 describes technical solution of the present invention at the end CDN；And the present embodiment describes this hair at broadcasting end Bright technical solution, realization principle is similar, in detail can also be with reference to the record of embodiment described in above-mentioned Fig. 2, and details are not described herein.
The time-delay measuring method of the live video of the present embodiment passes through the target video in the code stream transmitted to the end CDN Before frame plays out, is stabbed according to the acquisition time of the data of the target video frame in code stream and target video frame, establish target The incidence relation that the PTS of video frame is stabbed with corresponding acquisition time；When target video frame is played, target video frame is obtained The corresponding relationship that the play time of PTS and target video frame is stabbed；According to incidence relation and corresponding relationship, obtains and play target view The live streaming time delay of frequency frame.In the technical solution of the present embodiment, carried out by the way that the timestamp of target video frame to be encapsulated in code stream Transmission plays end to guarantee that acquisition time stamp can be delivered successfully, when enabling broadcasting end when playing according to acquisition Between stamp effectively complete live streaming time delay measurement.The measurement that live streaming time delay is realized relative to existing manual operation, can save Manual operation effectively improves the precision of measurement.And the time-delay measuring method of the live video of the present embodiment, it can prop up simultaneously The various protocols such as flv, rtmp and hls are held, flexibility is very high, very wide using field.
Fig. 4 is the signaling diagram of the time-delay measuring method example IV of live video of the invention.The live streaming of the present embodiment regards The time-delay measuring method of frequency is further situated between in further detail on the basis of the technical solution of above-mentioned Fig. 1-embodiment illustrated in fig. 3 Continue technical solution of the present invention.As shown in figure 4, the time-delay measuring method of the live video of the present embodiment, can specifically include as Lower step:
400, plug-flow end obtains acquisition time stamp when target video frame when acquiring target video frame；
401, plug-flow end obtains the PTS of target video frame；
402, the incidence relation that the PTS of target video frame is stabbed with corresponding acquisition time is established at plug-flow end；
403, plug-flow end encoding to target video frame；According to the PTS and correspondence of the target video frame after coding Incidence relation, the acquisition time stamp of target video frame is obtained, by the SEI that is encapsulated in code stream of acquisition time stamp, and in SEI The presence mark that middle setting acquisition time stabs；
404, the data of the target video frame after plug-flow end is packaged with coding to the transmission of the end CDN and corresponding acquisition time The code stream of stamp；
405, the end CDN obtains the PTS of target video frame from the data of the target video frame in the code stream that plug-flow end is transmitted；
406, the end CDN judges and determines the presence mark in SEI there are acquisition time stamp, and obtains from the SEI in code stream The acquisition time of target video frame is taken to stab；
407, the incidence relation that the PTS of target video frame is stabbed with corresponding acquisition time is established at the end CDN；
408, the end CDN carries out transcoding to the data of the target video frame in code stream；And according to incidence relation and transcoding after Target video frame in PTS, obtain acquisition time stamp；Acquisition time stamp is added to the target video frame after being packaged with transcoding Data code stream in；
409, the data of the target video frame after the end CDN is packaged with transcoding to the transmission of broadcasting end and corresponding acquisition time The code stream of stamp；
410, it plays end and obtains target video frame from the data of the target video frame in the code stream that plug-flow end is transmitted PTS；
411, it plays end to judge and determine the presence mark in SEI there are acquisition time stamp, and is obtained from the SEI in code stream The acquisition time of target video frame is taken to stab；
412, it plays end and establishes the incidence relation that the PTS of target video frame is stabbed with corresponding acquisition time；
413, the target video frame in the broadcast code flow of end is played；Record the play time stamp of target video frame；Establish target The corresponding relationship that the PTS of video frame and the play time of target video frame are stabbed；
414, it plays end to broadcast according to incidence relation and corresponding relationship, obtains the live streaming time delay for playing target video frame.
Plug-flow end in the framework of live broadcast system applied by the time-delay measuring method of the live video of the present embodiment, CDN End and the clock for playing end are all synchronous.
The present embodiment is that technical solution of the present invention is described along the trend of signaling, realization principle and above-mentioned Fig. 1-Fig. 3 institute Show the identical of embodiment, the record of above-mentioned Fig. 1-embodiment illustrated in fig. 3 can be referred in detail, details are not described herein.
The time-delay measuring method of the live video of the present embodiment, by using above-mentioned technical proposal, by by target video The timestamp of frame, which is encapsulated in code stream, to be transmitted, and can be played end to guarantee that acquisition time stamp can be delivered successfully, be made End must be played can stab the measurement for effectively completing live streaming time delay when playing according to acquisition time.Relative to existing artificial behaviour Make to realize the measurement that time delay is broadcast live, manual operation can be saved, effectively improve the precision of measurement.And the live streaming of the present embodiment The time-delay measuring method of video can support the various protocols such as flv, rtmp and hls simultaneously, and flexibility is very high, uses field It is very wide.
Fig. 5 is the structure chart of plug-flow end equipment embodiment of the invention.As shown in figure 5, the plug-flow end equipment of the present embodiment, It can specifically include:
Module 10 is obtained to be used for according to the PTS of the target video frame after coding and the target video frame pre-established The incidence relation of the acquisition time stamp of PTS and acquisition target video frame, obtains the acquisition time stamp of target video frame；
The acquisition time stamp that package module 11 is used to will acquire the acquisition of module 10 is encapsulated in code stream；
Transmission module 12 is used for that treated is packaged with the target video frame after coding to the end CDN transmission package module 11 Data and corresponding acquisition time stamp code stream, for the end CDN to play end transmission objectives video frame data while, pass The acquisition time of defeated target video frame measures the time delay of live streaming by playing end when playing target video frame.
Still optionally further, in the plug-flow end equipment of the present embodiment, package module 11 is obtained specifically for will acquire module 10 The acquisition time stamp taken is encapsulated in the SEI of code stream.
Still optionally further, as shown in figure 5, in the plug-flow end equipment of the present embodiment further include:
Setup module 13 be used for be provided in the SEI of code stream obtain module 10 obtain acquisition time stamp there are marks Know.
Still optionally further, as shown in figure 5, in the plug-flow end equipment of the present embodiment further include: establish module 14；
It obtains module 10 to be also used to when acquiring target video frame, obtains acquisition time stamp when target video frame；
Establish what module 14 was stabbed for establishing the PTS of target video frame with the corresponding acquisition time for obtaining the acquisition of module 10 Incidence relation.
The plug-flow end equipment of the present embodiment realizes the realization principle of the latency measurement of live video by using above-mentioned module And technical effect is identical as the realization of above-mentioned related method embodiment, can refer to the note of above-mentioned related method embodiment in detail It carries, details are not described herein.
Fig. 6 is the structure chart of CDN end equipment embodiment of the invention.As shown in fig. 6, the CDN end equipment of the present embodiment, tool Body may include:
Module 20 is established to be used for before the data of the target video frame in the code stream for transmitting plug-flow end carry out transcoding, according to The acquisition time of the data of target video frame in code stream and target video frame stamp, establish the PTS of target video frame with it is corresponding Acquisition time stamp incidence relation；
Adding module 21 is used for after the data to target video frame carry out transcoding, according to the association for establishing the foundation of module 20 Relationship, by acquisition time stamp be added to the target video frame after being packaged with transcoding data code stream in；
Transmission module 22 is used to be packaged with the view of the target after transcoding to playing the addition of end transmission adding module 21 treated The code stream of the data of frequency frame and corresponding acquisition time stamp measures live streaming for playing end when playing target video frame Time delay.
Still optionally further, it in the CDN end equipment of the present embodiment, establishes module 20 and is specifically used for:
The PTS of target video frame is obtained from the data of the target video frame in code stream；
The acquisition time stamp of target video frame is obtained from the SEI in code stream；
Establish the incidence relation that the PTS of target video frame is stabbed with corresponding acquisition time.
Still optionally further, as shown in fig. 6, in the CDN end equipment of the present embodiment further include:
Determining module 23 is used to be identified according to the presence in the SEI in code stream, determines and carries acquisition time stamp in SEI. Then just the acquisition time stamp that module 20 obtains target video frame from the SEI in code stream is established in triggering.
The CDN end equipment of the present embodiment realizes the realization principle of the latency measurement of live video by using above-mentioned module And technical effect is identical as the realization of above-mentioned related method embodiment, can refer to the note of above-mentioned related method embodiment in detail It carries, details are not described herein.
Fig. 7 is the structure chart of broadcasting end equipment embodiment of the invention.As shown in fig. 7, the broadcasting end equipment of the present embodiment, It can specifically include:
It establishes module 30 to be used for before the target video frame in the code stream for transmitting the end CDN plays out, according in code stream Target video frame data and target video frame acquisition time stamp, establish the PTS of target video frame and corresponding acquisition The incidence relation of timestamp；
Establish PTS and target video frame that module 30 is also used to establish target video frame when target video frame is played Play time stamp corresponding relationship；
Module 31 is obtained to be used to obtain according to the incidence relation and corresponding relationship for establishing the foundation of module 30 and play target view The live streaming time delay of frequency frame.
Still optionally further, it in the broadcasting end equipment of the present embodiment, establishes module 30 and is specifically used for:
The PTS of target video frame is obtained from the data of the target video frame in code stream；
The acquisition time stamp of target video frame is obtained from the SEI in code stream；
Establish the incidence relation that the PTS of target video frame is stabbed with corresponding acquisition time.
Still optionally further, as shown in fig. 7, in the broadcasting end equipment of the present embodiment, further includes:
Determining module 32 is used to be identified according to the presence in the SEI in code stream, determines and carries acquisition time stamp in SEI. Then just the acquisition time stamp that module 30 obtains target video frame from the SEI in code stream is established in triggering.
The broadcasting end equipment of the present embodiment realizes the realization principle of the latency measurement of live video by using above-mentioned module And technical effect is identical as the realization of above-mentioned related method embodiment, can refer to the note of above-mentioned related method embodiment in detail It carries, details are not described herein.
Fig. 8 is the structure chart of video living transmission system embodiment of the invention.As shown in figure 8, the net cast of the present embodiment System includes: plug-flow end equipment 40, CDN end equipment 50 and broadcasting end equipment 60.
Specifically, CDN end equipment 50 is communicated to connect with plug-flow end equipment 40 and broadcasting end equipment 60 respectively.Specifically, it pushes away The plug-flow end equipment of above-mentioned embodiment illustrated in fig. 5 can be used by flowing end equipment 40 specifically；CDN end equipment 50 can specifically use upper State the CDN end equipment of embodiment illustrated in fig. 6；The broadcasting end of above-mentioned embodiment illustrated in fig. 7 can be used by playing end equipment 60 specifically Equipment.
Plug-flow end equipment 40, CDN end equipment 50 and broadcasting end equipment 60, can when realizing the latency measurement of live video The latency measurement of live video is realized with the record with reference to above-mentioned Fig. 1-Fig. 4 any embodiment.
Fig. 9 is the structure chart of computer equipment embodiment of the invention.As shown in figure 9, the computer equipment of the present embodiment, It include: one or more processors 70 and memory 80, memory 80 works as memory for storing one or more programs The one or more programs stored in 80 are executed by one or more processors 70, so that one or more processors 70 are real respectively Now such as the time-delay measuring method of the live video of figure 1 above-Fig. 3 any embodiment.That is, the computer equipment of the present embodiment It is specifically as follows plug-flow end equipment, CDN end equipment or plays end equipment.To include multiple processors in embodiment illustrated in fig. 9 For 70.
For example, Figure 10 is a kind of exemplary diagram of computer equipment provided by the invention.Figure 10, which is shown, to be suitable for being used to realizing The block diagram of the exemplary computer device 12a of embodiment of the present invention.The computer equipment 12a that Figure 10 is shown is only one and shows Example, should not function to the embodiment of the present invention and use scope bring any restrictions.Similarly, the computer equipment tool of the present embodiment Body can be plug-flow end equipment, CDN end equipment or broadcasting end equipment.
As shown in Figure 10, computer equipment 12a is showed in the form of universal computing device.The component of computer equipment 12a Can include but is not limited to: one or more processor 16a, system storage 28a, connecting different system components (including is Unite memory 28a and processor 16a) bus 18a.
Bus 18a indicates one of a few class bus structures or a variety of, including memory bus or Memory Controller, Peripheral bus, graphics acceleration port, processor or the local bus using any bus structures in a variety of bus structures.It lifts For example, these architectures include but is not limited to industry standard architecture (ISA) bus, microchannel architecture (MAC) Bus, enhanced isa bus, Video Electronics Standards Association (VESA) local bus and peripheral component interconnection (PCI) bus.
Computer equipment 12a typically comprises a variety of computer system readable media.These media can be it is any can The usable medium accessed by computer equipment 12a, including volatile and non-volatile media, moveable and immovable Jie Matter.
System storage 28a may include the computer system readable media of form of volatile memory, such as deposit at random Access to memory (RAM) 30a and/or cache memory 32a.Computer equipment 12a may further include it is other it is removable/ Immovable, volatile/non-volatile computer system storage medium.Only as an example, storage system 34a can be used for reading Write immovable, non-volatile magnetic media (Figure 10 do not show, commonly referred to as " hard disk drive ").Although not showing in Figure 10 Out, the disc driver for reading and writing to removable non-volatile magnetic disk (such as " floppy disk ") can be provided, and to removable The CD drive of anonvolatile optical disk (such as CD-ROM, DVD-ROM or other optical mediums) read-write.In these cases, Each driver can be connected by one or more data media interfaces with bus 18a.System storage 28a may include At least one program product, the program product have one group of (for example, at least one) program module, these program modules are configured To execute the function of the above-mentioned each embodiment of Fig. 1-Fig. 8 of the present invention.
Program with one group of (at least one) program module 42a/utility 40a, can store and deposit in such as system In reservoir 28a, such program module 42a include --- but being not limited to --- operating system, one or more application program, It may include the reality of network environment in other program modules and program data, each of these examples or certain combination It is existing.Program module 42a usually executes the function and/or method in above-mentioned each embodiment of Fig. 1-Fig. 8 described in the invention.
Computer equipment 12a can also be with one or more external equipment 14a (such as keyboard, sensing equipment, display 24a etc.) communication, the equipment interacted with computer equipment 12a communication can be also enabled a user to one or more, and/or (such as network interface card is adjusted with any equipment for enabling computer equipment 12a to be communicated with one or more of the other calculating equipment Modulator-demodulator etc.) communication.This communication can be carried out by input/output (I/O) interface 22a.Also, computer equipment 12a can also by network adapter 20a and one or more network (such as local area network (LAN), wide area network (WAN) and/or Public network, such as internet) communication.As shown, network adapter 20a passes through its of bus 18a and computer equipment 12a The communication of its module.It should be understood that although not shown in the drawings, other hardware and/or software can be used in conjunction with computer equipment 12a Module, including but not limited to: microcode, device driver, redundant processor, external disk drive array, RAID system, tape Driver and data backup storage system etc..
Processor 16a by the program that is stored in system storage 28a of operation, thereby executing various function application and Data processing, such as realize the time-delay measuring method of live video shown in above-described embodiment.
The present invention also provides a kind of computer-readable mediums, are stored thereon with computer program, which is held by processor The time-delay measuring method of the live video as shown in above-described embodiment is realized when row.
The computer-readable medium of the present embodiment may include in the system storage 28a in above-mentioned embodiment illustrated in fig. 10 RAM30a, and/or cache memory 32a, and/or storage system 34a.
With the development of science and technology, the route of transmission of computer program is no longer limited by tangible medium, it can also be directly from net Network downloading, or obtained using other modes.Therefore, the computer-readable medium in the present embodiment not only may include tangible Medium can also include invisible medium.
The computer-readable medium of the present embodiment can be using any combination of one or more computer-readable media. Computer-readable medium can be computer-readable signal media or computer readable storage medium.Computer-readable storage medium Matter for example may be-but not limited to-system, device or the device of electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor, or Any above combination of person.The more specific example (non exhaustive list) of computer readable storage medium includes: with one Or the electrical connections of multiple conducting wires, portable computer diskette, hard disk, random access memory (RAM), read-only memory (ROM), Erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light Memory device, magnetic memory device or above-mentioned any appropriate combination.In this document, computer readable storage medium can With to be any include or the tangible medium of storage program, the program can be commanded execution system, device or device use or Person is in connection.
Computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal, Wherein carry computer-readable program code.The data-signal of this propagation can take various forms, including --- but It is not limited to --- electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be Any computer-readable medium other than computer readable storage medium, which can send, propagate or Transmission is for by the use of instruction execution system, device or device or program in connection.
The program code for including on computer-readable medium can transmit with any suitable medium, including --- but it is unlimited In --- wireless, electric wire, optical cable, RF etc. or above-mentioned any appropriate combination.
The computer for executing operation of the present invention can be write with one or more programming languages or combinations thereof Program code, described program design language include object oriented program language-such as Java, Smalltalk, C++, Further include conventional procedural programming language-such as " C " language or similar programming language.Program code can be with It fully executes, partly execute on the user computer on the user computer, being executed as an independent software package, portion Divide and partially executes or executed on a remote computer or server completely on the remote computer on the user computer.In Be related in the situation of remote computer, remote computer can pass through the network of any kind --- including local area network (LAN) or Wide area network (WAN)-be connected to subscriber computer, or, it may be connected to outer computer (such as mentioned using Internet service It is connected for quotient by internet).
In several embodiments provided by the present invention, it should be understood that disclosed system, device and method can be with It realizes by another way.For example, the apparatus embodiments described above are merely exemplary, for example, the unit It divides, only a kind of logical function partition, there may be another division manner in actual implementation.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme 's.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit It is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated list Member both can take the form of hardware realization, can also realize in the form of hardware adds SFU software functional unit.
The above-mentioned integrated unit being realized in the form of SFU software functional unit can store and computer-readable deposit at one In storage media.Above-mentioned SFU software functional unit is stored in a storage medium, including some instructions are used so that a computer It is each that equipment (can be personal computer, server or the network equipment etc.) or processor (processor) execute the present invention The part steps of embodiment the method.And storage medium above-mentioned includes: USB flash disk, mobile hard disk, read-only memory (Read- Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic or disk etc. it is various It can store the medium of program code.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all in essence of the invention Within mind and principle, any modification, equivalent substitution, improvement and etc. done be should be included within the scope of the present invention.

Claims (23)

1. a kind of time-delay measuring method of live video, which is characterized in that the described method includes:
According to the PTS of the PTS of the target video frame after coding and the target video frame pre-established and acquire the mesh The incidence relation for marking the acquisition time stamp of video frame obtains the acquisition time stamp of the target video frame；
Acquisition time stamp is encapsulated in code stream；
The code that the data of the target video frame after being packaged with from coding to the transmission of the end CDN and the corresponding acquisition time stab Stream transmits the institute of the target video frame while transmitting the data of the target video frame to broadcasting end for the end CDN Acquisition time stamp is stated, by the broadcasting end when playing the target video frame, measures the time delay of live streaming.
2. the method according to claim 1, wherein acquisition time stamp is encapsulated in code stream, specific packet It includes:
Acquisition time stamp is encapsulated in the SEI of code stream.
3. according to the method described in claim 2, it is characterized in that, the method also includes:
The presence mark of the acquisition time stamp is provided in the SEI of the code stream.
4. method according to claim 1 to 3, which is characterized in that according to the PTS of the target video frame after coding, with And the incidence relation of the PTS of the target video frame pre-established and the acquisition time stamp for acquiring the target video frame, it obtains Before taking the acquisition time of the target video frame to stab, the method also includes:
When acquiring the target video frame, the acquisition time stamp when target video frame is obtained；
Establish the incidence relation that the PTS of the target video frame is stabbed with the corresponding acquisition time.
5. a kind of time-delay measuring method of live video, which is characterized in that the described method includes:
Before the data of the target video frame in the code stream transmitted to plug-flow end carry out transcoding, according to the mesh in the code stream Mark the data of video frame and the acquisition time stamp of the target video frame, establish the PTS of the target video frame with it is corresponding The incidence relation of the acquisition time stamp；
After the data to the target video frame carry out transcoding, according to the incidence relation, the acquisition time is stabbed and is added Into the code stream for the data for being packaged with the target video frame after transcoding；
It is stabbed to the data and the corresponding acquisition time for playing the target video frame after end transmission is packaged with transcoding Code stream measures the time delay of live streaming so that the broadcasting end is when playing the target video frame.
6. according to the method described in claim 5, it is characterized in that, according to the data of the target video frame in the code stream And the acquisition time stamp of the target video frame, the PTS for establishing the target video frame are stabbed with the corresponding acquisition time Incidence relation, specifically include:
The PTS of the target video frame is obtained from the data of the target video frame in the code stream；
The acquisition time stamp of the target video frame is obtained from the SEI in the code stream；
Establish the incidence relation that the PTS of the target video frame is stabbed with the corresponding acquisition time.
7. according to the method described in claim 6, it is characterized in that, obtaining the target video from the SEI in the code stream Before the acquisition time stamp of frame, the method also includes:
According to the presence mark in the SEI in the code stream, determines and carry the acquisition time stamp in the SEI.
8. a kind of time-delay measuring method of live video, which is characterized in that the described method includes:
Before the target video frame in the code stream transmitted to the end CDN plays out, according to the target video in the code stream The acquisition time of the data of frame and target video frame stamp is established the PTS of the target video frame and described is adopted with corresponding Collect the incidence relation of timestamp；
When the target video frame is played, when establishing the broadcasting of PTS and the target video frame of the target video frame Between the corresponding relationship that stabs；
According to the incidence relation and the corresponding relationship, the live streaming time delay for playing the target video frame is obtained.
9. according to the method described in claim 8, it is characterized in that, according to the data of the target video frame in the code stream And the acquisition time stamp of the target video frame, the PTS for establishing the target video frame are stabbed with the corresponding acquisition time Incidence relation, specifically include:
The PTS of the target video frame is obtained from the data of the target video frame in the code stream；
The acquisition time stamp of the target video frame is obtained from the SEI in the code stream；
Establish the incidence relation that the PTS of the target video frame is stabbed with the corresponding acquisition time.
10. according to the method described in claim 9, it is characterized in that, obtaining the target video from the SEI in the code stream Before the acquisition time stamp of frame, the method also includes:
According to the presence mark in the SEI in the code stream, determines and carry the acquisition time stamp in the SEI.
11. a kind of plug-flow end equipment, which is characterized in that the equipment includes:
Module is obtained, for according to the PTS of the target video frame after coding and the target video frame pre-established The incidence relation that PTS and the acquisition time for acquiring the target video frame stab, when obtaining the acquisition of the target video frame Between stab；
Package module, for acquisition time stamp to be encapsulated in code stream；
Transmission module, for the data of the target video frame after being packaged with from coding to the transmission of the end CDN and corresponding described The code stream of acquisition time stamp, while transmitting the data of the target video frame to broadcasting end for the end CDN, described in transmission The acquisition time of target video frame stabs, and by the broadcasting end when playing the target video frame, measures the time delay of live streaming.
12. equipment according to claim 11, which is characterized in that the package module, when being specifically used for the acquisition Between stab and be encapsulated in the SEI of code stream.
13. equipment according to claim 12, which is characterized in that the equipment further include:
Setup module, for being provided with the presence mark of the acquisition time stamp in the SEI of the code stream.
14. any equipment of 1-13 according to claim 1, which is characterized in that the equipment further includes establishing module；
The acquisition module is also used to obtain the acquisition time when target video frame when acquiring the target video frame Stamp；
The incidence relation described to establish module, that the PTS for establishing the target video frame is stabbed with the corresponding acquisition time.
15. a kind of CDN end equipment, which is characterized in that the equipment includes:
Establish module, for the data of the target video frame in the code stream transmitted to plug-flow end carry out transcoding before, according to described The data of the target video frame in code stream and the acquisition time stamp of the target video frame, establish the target video frame PTS and the incidence relation of corresponding acquisition time stamp；
Adding module, for according to the incidence relation, being adopted described after the data to the target video frame carry out transcoding In the code stream for the data that collection timestamp is added to the target video frame after being packaged with transcoding；
Transmission module, for being packaged with the data of the target video frame after transcoding and corresponding described to playing end transmission The code stream of acquisition time stamp measures the time delay of live streaming so that the broadcasting end is when playing the target video frame.
16. equipment according to claim 15, which is characterized in that it is described to establish module, it is specifically used for:
The PTS of the target video frame is obtained from the data of the target video frame in the code stream；
The acquisition time stamp of the target video frame is obtained from the SEI in the code stream；
Establish the incidence relation that the PTS of the target video frame is stabbed with the corresponding acquisition time.
17. equipment according to claim 16, which is characterized in that the equipment further include:
Determining module determines described in carrying in the SEI for being identified according to the presence in the SEI in the code stream Acquisition time stamp.
18. a kind of broadcasting end equipment, which is characterized in that the equipment includes:
Module is established, for before the target video frame in the code stream transmitted to the end CDN plays out, according in the code stream The acquisition time of the data of the target video frame and target video frame stamp, establish the PTS of the target video frame with The incidence relation of the corresponding acquisition time stamp；
It is described to establish module, be also used to when the target video frame is played, establish the PTS of the target video frame with it is described The corresponding relationship of the play time stamp of target video frame；
Module is obtained, for obtaining and playing the straight of the target video frame according to the incidence relation and the corresponding relationship Sowing time prolongs.
19. equipment according to claim 18, which is characterized in that it is described to establish module, it is specifically used for:
The PTS of the target video frame is obtained from the data of the target video frame in the code stream；
The acquisition time stamp of the target video frame is obtained from the SEI in the code stream；
Establish the incidence relation that the PTS of the target video frame is stabbed with the corresponding acquisition time.
20. equipment according to claim 19, which is characterized in that the equipment further include:
Determining module determines described in carrying in the SEI for being identified according to the presence in the SEI in the code stream Acquisition time stamp.
21. a kind of video living transmission system, which is characterized in that including any plug-flow end equipment of claim 11-14 as above, Any CDN end equipment of claim 15-17 as above and any broadcasting end claim 18-20 as above are set It is standby；The CDN end equipment is communicated to connect with the plug-flow end equipment and the broadcasting end equipment respectively.
22. a kind of computer equipment, which is characterized in that the equipment includes:
One or more processors；
Memory, for storing one or more programs；
When one or more of programs are executed by one or more of processors, so that one or more of processors are real The now method as described in any in claim 1-4,5-7 and 8-10.
23. a kind of computer-readable medium, is stored thereon with computer program, which is characterized in that the program is executed by processor Method of the Shi Shixian as described in any in claim 1-4,5-7 and 8-10.
CN201711320836.XA 2017-12-12 2017-12-12 Time-delay measuring method, equipment, system and the computer-readable medium of live video Active CN108234997B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711320836.XA CN108234997B (en) 2017-12-12 2017-12-12 Time-delay measuring method, equipment, system and the computer-readable medium of live video

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711320836.XA CN108234997B (en) 2017-12-12 2017-12-12 Time-delay measuring method, equipment, system and the computer-readable medium of live video

Publications (2)

Publication Number Publication Date
CN108234997A CN108234997A (en) 2018-06-29
CN108234997B true CN108234997B (en) 2019-11-26

Family

ID=62649401

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711320836.XA Active CN108234997B (en) 2017-12-12 2017-12-12 Time-delay measuring method, equipment, system and the computer-readable medium of live video

Country Status (1)

Country Link
CN (1) CN108234997B (en)

Families Citing this family (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108989884A (en) * 2018-08-01 2018-12-11 浙江齐聚科技有限公司 Live video delay performance test method, device, equipment and storage medium
CN109510978B (en) * 2018-11-07 2020-05-26 西安万像电子科技有限公司 Data processing performance detection method and device
CN110225416A (en) * 2019-05-31 2019-09-10 杭州涂鸦信息技术有限公司 A kind of transmission method of video, the network terminal, intelligent terminal and storage device
CN110401850B (en) * 2019-07-30 2021-10-22 网宿科技股份有限公司 Method and device for transparent transmission of custom SEI
CN110708597B (en) * 2019-10-12 2022-01-18 腾讯科技（深圳）有限公司 Live broadcast delay monitoring method and device, electronic equipment and readable storage medium
CN110933449B (en) * 2019-12-20 2021-10-22 北京奇艺世纪科技有限公司 Method, system and device for synchronizing external data and video pictures
CN111654690A (en) * 2020-05-06 2020-09-11 北京百度网讯科技有限公司 Live video delay time determination method and device and electronic equipment
CN112153443B (en) * 2020-09-01 2022-02-22 青岛海信传媒网络技术有限公司 PTS acquisition method and display device
CN112261418B (en) * 2020-09-18 2022-09-30 网宿科技股份有限公司 Method for transmitting live video data and live broadcast acceleration system
CN114257843A (en) * 2020-09-24 2022-03-29 腾讯科技（深圳）有限公司 Multimedia data processing method, device, equipment and readable storage medium
CN112601078B (en) * 2020-12-11 2022-07-26 杭州当虹科技股份有限公司 Automatic encoder delay measuring method based on video
CN113163222A (en) * 2021-03-31 2021-07-23 杭州奥点科技股份有限公司 Video frame synchronization method, system, equipment and readable storage medium
CN113676783A (en) * 2021-08-18 2021-11-19 上海哔哩哔哩科技有限公司 Link node evaluation method and device
CN113923530B (en) * 2021-10-18 2023-12-22 北京字节跳动网络技术有限公司 Interactive information display method and device, electronic equipment and storage medium
CN114554237B (en) * 2022-02-18 2023-12-26 北京有竹居网络技术有限公司 Information processing method, device and equipment
CN115002526A (en) * 2022-05-23 2022-09-02 阿里巴巴（中国）有限公司 Video frame time delay determining method and device
CN115225542A (en) * 2022-07-20 2022-10-21 北京京东乾石科技有限公司 Video information processing method and device, electronic equipment and storage medium
CN116886833A (en) * 2022-12-06 2023-10-13 广州开得联软件技术有限公司 Time synchronization method, device, equipment and storage medium based on heterogeneous system

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3892844B2 (en) * 2003-03-31 2007-03-14 日本放送協会 Delay time measuring method, delay time measuring system, time information setting device, and delay time measuring device
CN101087400B (en) * 2007-06-26 2011-09-21 中兴通讯股份有限公司 Video frame delay detection method and system
CN101212671A (en) * 2007-12-21 2008-07-02 上海广电（集团）有限公司中央研究院 Low-delay real-time transport stream multiplexing method
CN101888412B (en) * 2010-07-05 2012-11-07 优视科技有限公司 Video push processing method and system serving live broadcast of mobile terminal
CN102811373A (en) * 2011-05-30 2012-12-05 上海圣炅商务咨询有限公司 Method for carrying out video broadcast on Internet and mobile Internet by mobile terminal
CN102316363B (en) * 2011-09-09 2014-04-02 华南理工大学 Method and devices for measuring video end-to-end delay by non-intrusive double-ended collection
CN102655585B (en) * 2012-04-16 2015-07-15 中国联合网络通信集团有限公司 Video conference system and time delay testing method, device and system thereof
US20140010517A1 (en) * 2012-07-09 2014-01-09 Sensr.Net, Inc. Reduced Latency Video Streaming
CN104144312B (en) * 2013-05-09 2018-06-05 腾讯科技（深圳）有限公司 A kind of method for processing video frequency and relevant apparatus
CN105791986B (en) * 2014-12-23 2019-06-25 ***通信集团公司 The detection method and device of video time delay
CN106210708B (en) * 2015-04-30 2019-02-01 杭州海康威视数字技术股份有限公司 A kind of detection method and system of picture delay
CN105451096B (en) * 2015-12-02 2018-08-10 北京暴风云科技有限公司 Network direct broadcasting system and live broadcasting method
CN106803967A (en) * 2017-02-13 2017-06-06 北京奇虎科技有限公司 A kind for the treatment of method and apparatus of live data

Also Published As

Publication number Publication date
CN108234997A (en) 2018-06-29

Similar Documents

Publication Publication Date Title
CN108234997B (en) Time-delay measuring method, equipment, system and the computer-readable medium of live video
CN108566558B (en) Video stream processing method and device, computer equipment and storage medium
CN103297452B (en) A kind of in the Internet issue and the method and system of live broadcast stream media
CN109348252B (en) Video playing method, video transmission method, device, equipment and storage medium
CN104918133B (en) A kind of playing method and device for regarding video flowing in networking
CN103841432B (en) The transmission method and equipment of a kind of composite video data
JP5475855B2 (en) Virtual reference decoder for multiview video coding
TW201724086A (en) Selection of coded next generation audio data for transport
CN107872422A (en) A kind of data transmission method, device and electronic equipment
KR101959260B1 (en) Media data transmission apparatus and method, and media data reception apparatus and method in mmt system
CN103650431B (en) Method of transmitting video data and device
WO2022037386A1 (en) Data processing method and apparatus for point cloud media, and device and storage medium
CN105451096B (en) Network direct broadcasting system and live broadcasting method
CN109918306A (en) Audio processing equipment, system, method, apparatus and storage medium
CN110100460A (en) Based on head position and time there is non-head to track stereosonic ambisonics audio
WO2009048235A3 (en) System and method for 3d multimedia contents service using multimedia application file format
CN108847248A (en) Bluetooth equipment audio-frequency processing method, system, readable storage medium storing program for executing and bluetooth equipment
CN108712299A (en) A kind of method, apparatus, equipment and the computer storage media of monitoring live streaming delay
CN103561244A (en) System and method for monitoring model airplane aerial photography data in real time through intelligent mobile phone
US11435977B2 (en) Method for transmitting and receiving audio data related to transition effect and device therefor
US11190724B2 (en) Adaptive bitrate streaming of UHD image data
CN109413371A (en) Video frame rate calculation method and device
CN102625125A (en) Method for transmitting high-resolution and multi-view stereoscopic video in real time
CN107872716A (en) A kind of stream data analytic method, device and the electronic equipment of rearmounted encapsulation
KR20000041813A (en) Method for streaming multimedia through network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant