CN100496133C - Method for testing audio and video frequency out of step of audio and video frequency coding-decoding system - Google Patents
Method for testing audio and video frequency out of step of audio and video frequency coding-decoding system Download PDFInfo
- Publication number
- CN100496133C CN100496133C CNB2004100613204A CN200410061320A CN100496133C CN 100496133 C CN100496133 C CN 100496133C CN B2004100613204 A CNB2004100613204 A CN B2004100613204A CN 200410061320 A CN200410061320 A CN 200410061320A CN 100496133 C CN100496133 C CN 100496133C
- Authority
- CN
- China
- Prior art keywords
- video
- audio
- frame number
- audio signal
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention includes following steps: 1) each frame of image in video sequential file whose video frame rate is given is made frame number identification; 2) the sound signal is generated in audio flow in video sequential file, and the image frame number at the time point generating sound signal is recorded; 3) the audio and video file are inputted into video and audio coding system under test, and are played back; 4) the image frame number at the playing time point of sound signal is recorded; 5) through the formula: delta Tn = delta n/F, the out of alignment time of sound is detected; delta Tn is out of alignment time of sound; delta n is the difference between two frame signals; F is video frame rate of the video sequential file.
Description
Technical field
The present invention relates to the multimedia messages process field, particularly the method for test tone video coding and decoding system audio frequency and video step-out.
Background technology
Studies show that, when the out of alignment time of audio frequency and video surpasses 80 milliseconds, will spectators' subjective sensation be impacted.Therefore, the test of audio frequency and video step-out is very necessary for the performance of estimating a multi-media decoding and encoding system.
The method of existing test tone video coding and decoding system audio frequency and video step-out mainly contains two kinds.A kind of objective examination's method that is based on internal indicator, another kind is the subjective testing method.Objective examination's method based on internal indicator is exactly to utilize the audio-visual synchronization model of formulating in the coding international standard (as system reference clock SCR, Presentation Time Stamp PTS, program clock reference PCR etc.) to calculate the audio frequency and video out of alignment time.For example in MPEG-2, set up system reference clock SCR in decoding end earlier exactly, its initial value is obtained by first effective Presentation Time Stamp PTS.When a video access unit VAU (audio access unit AAU) shows, with SCR and corresponding PTS relatively, if difference beyond certain limit, is then thought the audio/video decoding step-out.Objective examination's method all has introduction in " Shanghai Communications University's journal " the 32nd the 9th phase of volume " audio-visual synchronization and real-time implementation thereof in the MPEG-2 decoding ", " infrared and laser engineering " the 29th volume the 5th phase articles such as " temporal information and audio-visual synchronization in MPEG-2 transport stream ".The subjective testing rule is under the synchronous prerequisite of the audio frequency and video inlet flow that guarantees multimedia system, selects in the audio frequency and video output stream several time point positions to carry out playback at random, utilizes visual and ear to listen the audio frequency and video out of alignment time of comprehensive estimating system.
Though can calculate the audio frequency and video out of alignment time of system based on objective examination's method of internal indicator, it need utilize the audio-visual synchronization model of formulating in the coding international standard, and these need the frame number of the every two field picture of video more accurately.Existing video frame number label method mainly is semantic label method, semantic label method is exactly to utilize semanteme in the video flowing of various coding international standard defineds directly to identify frame number (as the TR in H.263) or identify frame number indirectly (as the time_code among the MPEG-4, modulo_time_base, vop_time_increment), these semantemes are encoded with video data together as the header information of video flowing, decoding these semantemes afterwards will be lost and can't be obtained, therefore from outputting video streams, can't obtain frame number, just can obtain frame number after must obtaining these semantic informations to the encoded video streams decoding of internal system earlier, promptly must be by the internal system code stream is resolved just and can be obtained, therefore require test side to have tested audio/video encoding/decoding internal system code stream is carried out operation permission, that is to say that tested audio/video encoding/decoding system must be a white box system.And this condition all is difficult to satisfy under many circumstances, and for example when the multimedia system product was tested, system under test (SUT) was actually a black-box system, and its internal processes is maintained secrecy.This moment, the objective examination's method based on internal indicator just can not be suitable for.Though the subjective testing method do not require that test side has the internal system program is carried out operation permission, dialogue box system and black-box system all are suitable for, its test data inaccuracy, and randomness is big.
Summary of the invention
Technical problem to be solved by this invention is: the method that a kind of test tone video coding and decoding system audio frequency and video step-out is provided, it is test tone video out of alignment time accurately, and not requiring that test side has carries out operation permission to tested audio/video encoding/decoding internal system program, and dialogue box system and black-box system are all applicable.
The present invention solves the problems of the technologies described above the technical scheme that is adopted:
A kind of method of test tone video coding and decoding system audio frequency and video step-out, it may further comprise the steps:
(1) each two field picture in the video sequence file of one section known video frame per second is carried out the frame number sign;
(2) carrying out that step (1) obtains produce audio signal in the video sequence file corresponding audio stream of frame number sign, and record produces the image frame number on the time point of audio signal;
(3) audio stream of the audio signal with generation that step (2) is obtained is input to audio/video encoding/decoding system to be measured with the video sequence file of having carried out the frame number sign, and plays;
(4) image frame number on the recorde audio signal play time;
(5) pass through formula: Δ Tn=Δ n/F records the audio frequency and video out of alignment time; In the formula: Δ Tn is the audio frequency and video out of alignment time, and Δ n is image frame number on the time point that produces audio signal and the difference between the image frame number on the audio signal play time, and F is the video frame rate of video sequence file.
In the such scheme, produce a plurality of audio signals in audio stream, obtain a plurality of audio frequency and video out of alignment times, pass through formula: Δ T=∑ Δ Tn/N records the average out of alignment time of audio frequency and video; In the formula: Δ T is the average out of alignment time of audio frequency and video, and N is the number that produces audio signal in audio stream.
In the such scheme, the method for the image frame number on the recorde audio signal play time is: utilize the audio frequency monitoring equipment to detect audio signal, and control image frame number on the capture apparatus recorde audio signal play time by the audio frequency monitoring equipment.
In the such scheme, the method for the image frame number on the recorde audio signal play time is: utilize picture pick-up device that image and the sound of playing is carried out record.
In the such scheme, the audio signal that produces on audio stream is a simple signal.
The inventive method is compared with the objective examination's method based on internal indicator, because tested audio/video encoding/decoding system is in video encoding-decoding process, can not change in frame number sign in each two field picture and the audio stream and produce audio signal, therefore, do not need tested audio/video encoding/decoding internal system code stream is resolved, need not utilize the audio-visual synchronization model of formulating in the coding international standard to calculate the audio frequency and video out of alignment time, not requiring that test side has carries out operation permission to tested audio/video encoding/decoding internal system program, and dialogue box system and black-box system are all applicable.
The inventive method is compared with the subjective testing method, and the present invention is the test tone video time accurately, not influenced by subjective factor.
Description of drawings
Fig. 1 is the annexation block diagram between each equipment of embodiment of the present invention
Fig. 2 is the image schematic diagram after the frame number sign
Fig. 3 is image frame number sign software flow pattern
Fig. 4 detects software flow pattern for image frame number
Fig. 5 is input audio/video flow schematic diagram
Fig. 6 is output audio/video flow schematic diagram
Embodiment
Embodiment of the present invention: annexation such as Fig. 1 between each testing equipment and the audio/video encoding/decoding system to be measured, method of testing may further comprise the steps:
(1) each two field picture in one section known video frame per second video sequence file that was 25 frame/seconds is carried out the frame number sign, concrete grammar is:
Utilize zone that computer forms the first row block of pixels of each two field picture in the video sequence (CIF form) as identified areas; In this identified areas, the brightness of interior all pixels of block of pixels and chromatic value all are changed to black, and (brightness value is 255, chromatic value is 0) or white (brightness value is 0, and chromatic value is 0), each block of pixels is combined into a binary number respectively according to each two field picture order in the video sequence.
Image as shown in Figure 2, its resolution are 352 * 288.The zone that 22 block of pixels of first row (each block of pixels is 16 * 16 picture element matrixs herein) on image top are formed is as the two-value identified areas, the brightness of 16 * 16 pixels of each block of pixels and chromatic value all are changed to black, and (brightness value is 255, chromatic value is 0) or white (brightness value is 0, chromatic value is 0), black expression 1 represents 0 in vain.22 block of pixels of such first row according to the left side for high-order the right for the order of low level just can be combined into the binary number of one 22 bit, be illustrated in figure 2 as 0000000000110110010100, decimalize is 3476.If frame number increases progressively from the 0th frame, then this two field picture frame number is exactly the 3476th frame.
The workflow (as Fig. 3) of image frame number sign software.At first import video sequence to be identified, the initial value with frame number F is changed to 0 then, shows that promptly the frame number counting is since 0.Then transfer frame number F to binary number, prepare for carrying out the two-value sign.Next begin to read the data of the 1st two field picture in the video sequence, and the initial value of block of pixels number is changed to 1.Because it is good that the two-value identified areas of each two field picture and block of pixels are predefineds, so can read the data of the 1st block of pixels in the present frame two-value identified areas with that.Get the 1st that has been converted into binary frame number then, if this position is 0, thus just the brightness and the chromatic value of pixel in the current pixel piece is changed to white (brightness value is 0, and chromatic value is 0), otherwise just be changed to black (brightness value is 255, and chromatic value is 0).It is to be noted: the 1st of binary frame number is meant highest order or lowest order, with the 1st block of pixels sign be that highest order or lowest order are consistent.After the 1st block of pixels finished sign, just judge that the current pixel piece is last block of pixels.Get next block of pixels data if not attending school, and block of pixels number is added up 1, read frame number then the 2nd, carry out new judgement and sign.The rest may be inferred, is last block of pixels up to the current pixel piece, also is that the two-value frame number of present frame has identified.Just judge that then present frame is a last frame.If not, attend school the data of taking off a two field picture, frame number adds 1, repeats the sign operation of the 1st two field picture.The rest may be inferred, is last frame up to present frame, also is that the frame number of video sequence has identified, just quit a program, output promptly be the video sequence that each frame has all identified.
Image frame number detects workflow such as Fig. 4 of software.At first import video sequence to be identified, next begin to read the data of the 1st two field picture in the video sequence.Because it is good that the two-value identified areas of each two field picture and block of pixels are predefineds, so can read the data of the 1st block of pixels in the present frame two-value identified areas with that.If the pixel value in the current pixel piece is white (brightness value is 0, and chromatic value is 0), just the 1st with binary system frame number F is designated as 0; Otherwise just the 1st with binary system frame number F is designated as 1.Judge that then the current pixel piece is last block of pixels.Whether, then according to this block of pixels pixel value be white (brightness value be 0, chromatic value be 0), be designated as 0 or 1 with the 2nd of binary system frame number F is corresponding if getting next block of pixels data if not attending school, and block of pixels number is added up 1.The rest may be inferred, is last block of pixels up to the current pixel piece, also is that the two-value frame number sign of present frame has detected.At this moment also just having obtained all value of binary system frame number F, transferred it to decimal number then, is exactly the frame number of present frame.After the frame number of present frame has detected, just judge that present frame is a last frame.If not, attend school the data of getting next frame, repeat the detecting operation of the 1st two field picture.The rest may be inferred, is last frame up to present frame, also is that the frame number of video sequence has detected, just quit a program, output promptly be each frame video frame number.
(2) as Fig. 5, in the audio stream of the video sequence file that step (1) obtains, produce 2 audio signals, audio signal is a simple signal, and writes down the image frame number (the 2nd frame and n frame) on each time point that produces audio signal.
So just set up a kind of relation one to one between simple signal and frame of video, this relation is exactly the foundation of the inventive method test tone video step-out.If coding/decoding system does not cause the audio frequency and video step-out, so this corresponding relation just can not change in the output audio/video flow; Otherwise this corresponding relation will change, and that is to say that the simple signal of output audio no longer is to import the simple signal of audio frequency at pairing that frame of input video at pairing that frame of output video, and both frame numbers can be different.
(3) audio-video document that step (2) is obtained is input to audio/video encoding/decoding system to be measured, and plays;
(4) image frame number on the recorde audio signal play time;
The method of the image frame number on the recorde audio signal play time is: utilize the audio frequency monitoring equipment to detect audio signal, and control image frame number on the capture apparatus recorde audio signal play time by the audio frequency monitoring equipment.
When the audio frequency monitoring equipment monitors a simple signal, send instruction to capture apparatus, take corresponding down frame of video, should guarantee during shooting that two-value video frame number is identified in the picture.Deposit the frame of video of taking in computer then, detect its frame number, compare, calculate the audio frequency and video out of alignment time on this time point with the pairing frame number of this single-frequency audio signal in the input video stream; The rest may be inferred, monitors other single-frequency audio signals, tries to achieve the mean value of the audio frequency and video out of alignment time on all N time point at last.
The simple signal of output audio is pairing video frame number in output video, then with input video in pairing frame number subtract each other that to obtain frame poor.If the frame difference is 0, represent that then corresponding relation does not change, the audio frequency and video out of alignment time is 0; If the frame difference is Δ n, then change has taken place in corresponding relation, step-out occurred, and the time of step-out is exactly the shared time of frame difference Δ n.The frame per second of supposing input video is F frame/second, and the time that each frame occupied is exactly 1/F second so, so out of alignment time Δ Tn=Δ n/F second.For example in the input audio/video flow of Fig. 5 the 1st single-frequency audio signal correspondence be the 2nd frame, the 2nd single-frequency audio signal correspondence be the n frame; And (as Fig. 6) detect the 1st single-frequency audio signal correspondence in the output audio/video flow is the 2nd frame, but the 2nd single-frequency audio signal correspondence is n-1 frame.Just explanation is on the 1st time point for this, and the frame difference is 0, does not have the audio frequency and video step-out; On the 2nd time point, the frame difference is 1.
(5) pass through formula: Δ Tn=Δ n/F records the audio frequency and video out of alignment time; In the formula: Δ Tn is the audio frequency and video out of alignment time, and Δ n is the difference between two image frame number, and F is the video frame rate of video sequence file.
Because frame per second was 25 frame/seconds, calculated: the out of alignment time of audio frequency and video=1 frame/(25 frame/second)=0.04 second.
Pass through formula: Δ T=∑ Δ Tn/N records the average out of alignment time of audio frequency and video; In the formula: Δ T is the average out of alignment time of audio frequency and video, and N is the number that produces audio signal in audio stream.
Calculate: the out of alignment time of the average out of alignment time=audio frequency and video of audio frequency and video/2=0.02 second.
In order to avoid error as far as possible, the number that produces audio signal in the audio stream should be greatly, and distributing also should be even.
The inventive method can also utilize picture pick-up device that image and the sound of playing is carried out record.
Claims (5)
1, a kind of method of test tone video coding and decoding system audio frequency and video step-out, it is characterized in that: it may further comprise the steps:
(1) each two field picture in the video sequence file of one section known video frame per second is carried out the frame number sign;
(2) carrying out that step (1) obtains produce audio signal in the video sequence file corresponding audio stream of frame number sign, and record produces the image frame number on the time point of audio signal;
(3) audio stream of the audio signal with generation that step (2) is obtained is input to audio/video encoding/decoding system to be measured with the video sequence file of having carried out the frame number sign, and plays;
(4) image frame number on the recorde audio signal play time;
(5) pass through formula: △ Tn=△ n/F records the audio frequency and video out of alignment time; In the formula: △ Tn is the audio frequency and video out of alignment time, and △ n is image frame number on the time point that produces audio signal and the difference between the image frame number on the audio signal play time, and F is the video frame rate of video sequence file.
2, the method for claim 1 is characterized in that: produce a plurality of audio signals in audio stream, obtain a plurality of audio frequency and video out of alignment times, pass through formula: △ T=∑ △ Tn/N records the average out of alignment time of audio frequency and video; In the formula: △ T is the average out of alignment time of audio frequency and video, and N is the number that produces audio signal in audio stream.
3, the method for claim 1, it is characterized in that: the method for the image frame number on the recorde audio signal play time is: utilize the audio frequency monitoring equipment to detect audio signal, and control image frame number on the capture apparatus recorde audio signal play time by the audio frequency monitoring equipment.
4, the method for claim 1 is characterized in that: the method for the image frame number on the recorde audio signal play time is: utilize picture pick-up device that image and the sound of playing is carried out record.
5, the method for claim 1 is characterized in that: the audio signal that produces on audio stream is a simple signal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2004100613204A CN100496133C (en) | 2004-12-13 | 2004-12-13 | Method for testing audio and video frequency out of step of audio and video frequency coding-decoding system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2004100613204A CN100496133C (en) | 2004-12-13 | 2004-12-13 | Method for testing audio and video frequency out of step of audio and video frequency coding-decoding system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1645944A CN1645944A (en) | 2005-07-27 |
| CN100496133C true CN100496133C (en) | 2009-06-03 |
Family
ID=34868808
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2004100613204A Expired - Fee Related CN100496133C (en) | 2004-12-13 | 2004-12-13 | Method for testing audio and video frequency out of step of audio and video frequency coding-decoding system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100496133C (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101616331B (en) * | 2009-07-27 | 2011-07-20 | 北京汉邦高科数字技术有限公司 | Method for testing video frequency frame rate and audio-video synchronous performance |
| CN101765027B (en) * | 2009-10-20 | 2013-01-09 | 大连捷成实业发展有限公司 | Method for detecting time-domain weighted digital video static frame |
| CN101902660B (en) * | 2010-08-13 | 2013-01-23 | 深圳创维－Rgb电子有限公司 | Method for testing video output signals of television |
| US10120008B2 (en) * | 2013-08-29 | 2018-11-06 | Keysight Technologies, Inc. | Method and apparatus for estimating the noise introduced by a device |
| CN103561260A (en) * | 2013-10-31 | 2014-02-05 | 北京邮电大学 | Method and device for measuring lost step between network side audio and video streams based on RTSP |
| CN105657412B (en) * | 2015-12-31 | 2018-05-22 | 公安部第三研究所 | A kind of test method of video frame rate |
| CN105657512A (en) * | 2016-01-29 | 2016-06-08 | 四川长虹电器股份有限公司 | Method for monitoring sound-picture asynchronization of digital video |
| CN106375820B (en) * | 2016-08-30 | 2018-07-06 | 京东方科技集团股份有限公司 | The method and apparatus synchronized to audio and video frequency signal |
| CN106358039B (en) * | 2016-09-07 | 2019-02-01 | 深圳Tcl数字技术有限公司 | Sound draws synchronous detecting method and device |
| CN108307191B (en) * | 2018-01-03 | 2020-03-27 | 西安万像电子科技有限公司 | Image data alignment method and device |
| CN113207009B (en) * | 2021-05-11 | 2022-11-29 | 中国工商银行股份有限公司 | Method, device and system for testing audio-video interaction system with out-of-step sound and picture |
| CN114071134A (en) * | 2022-01-13 | 2022-02-18 | 荣耀终端有限公司 | Sound-picture synchronization detection method, sound-picture synchronization generation method, electronic equipment and storage medium |
-
2004
- 2004-12-13 CN CNB2004100613204A patent/CN100496133C/en not_active Expired - Fee Related
Non-Patent Citations (2)
| Title |
|---|
| 多媒体监控***中音视频实时复用的一种策略. 袁登涛等.武汉大学学报（理学版），第48卷第3期. 2002 |
| 多媒体监控***中音视频实时复用的一种策略. 袁登涛等.武汉大学学报(理学版)，第48卷第3期. 2002 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1645944A (en) | 2005-07-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8379735B2 (en) | Automatic video glitch detection and audio-video synchronization assessment | |
| CN100496133C (en) | Method for testing audio and video frequency out of step of audio and video frequency coding-decoding system | |
| US9906782B2 (en) | Source agnostic audio/visual analysis framework | |
| KR100499037B1 (en) | Method and apparatus of dtv lip-sync test | |
| CN103535027B (en) | The method for handling the sequence of encoded video frame | |
| KR100884134B1 (en) | Methods for timecoding media samples | |
| CN102026017B (en) | Efficient test method for video decoding | |
| EP3748972B1 (en) | Video recording method and system | |
| CN101616331B (en) | Method for testing video frequency frame rate and audio-video synchronous performance | |
| CN107018443A (en) | Video recording method, device and electronic equipment | |
| CN108989883B (en) | Live broadcast advertisement method, device, equipment and medium | |
| CN110585702B (en) | Sound and picture synchronous data processing method, device, equipment and medium | |
| CN104092920A (en) | Audio and video synchronizing method | |
| CN106792154A (en) | The frame-skipping synchronization system and its control method of video player | |
| US20060143676A1 (en) | Content reproduce system, reproduce device, and reproduce method | |
| CN101827202A (en) | Image processing equipment, image processing method and program | |
| CN107483916A (en) | The control method of audio frequency and video archival quality detecting system | |
| CN111277894B (en) | Method for automatically detecting fluency of video playing picture | |
| KR20120019872A (en) | A apparatus generating interpolated frames | |
| CN100396108C (en) | A method for testing video frame rate of video coding-decoding system | |
| CN113055711B (en) | Audio and video synchronous detection method and detection system thereof | |
| US6912011B2 (en) | Method and system for measuring audio and video synchronization error of audio/video encoder system and analyzing tool thereof | |
| KR100490154B1 (en) | DTV Test Stream Generator | |
| CN116233411A (en) | Method, device, equipment and computer storage medium for audio and video synchronous test | |
| CN1207922C (en) | Method and system for measuring accuracy of video/audio output synchronization, and analysis means |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20090603 Termination date: 20131213 |