CN112291607A - Video and audio data synchronous output method and system thereof - Google Patents
Video and audio data synchronous output method and system thereof Download PDFInfo
- Publication number
- CN112291607A CN112291607A CN202011178370.6A CN202011178370A CN112291607A CN 112291607 A CN112291607 A CN 112291607A CN 202011178370 A CN202011178370 A CN 202011178370A CN 112291607 A CN112291607 A CN 112291607A
- Authority
- CN
- China
- Prior art keywords
- delay amount
- audio
- value
- unit
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000001360 synchronised effect Effects 0.000 title claims abstract description 22
- 238000000034 method Methods 0.000 title claims abstract description 15
- 239000012146 running buffer Substances 0.000 claims description 65
- 239000000872 buffer Substances 0.000 claims description 14
- 230000007423 decrease Effects 0.000 claims description 8
- 230000001934 delay Effects 0.000 claims description 8
- 230000000694 effects Effects 0.000 description 6
- 230000006978 adaptation Effects 0.000 description 2
- 230000003139 buffering effect Effects 0.000 description 2
- 210000005069 ears Anatomy 0.000 description 2
- 230000008030 elimination Effects 0.000 description 2
- 238000003379 elimination reaction Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 101100016398 Danio rerio hars gene Proteins 0.000 description 1
- 238000005253 cladding Methods 0.000 description 1
- 238000011217 control strategy Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses a video and audio data synchronous output method and a system thereof, comprising the following steps: acquiring a video output delay amount and a first audio output delay amount of a video playing unit, and configuring that the video output delay amount of the video playing unit is the same as the first audio output delay amount, wherein the first audio output delay amount comprises a BSP delay amount and a first ALSA delay amount; acquiring a second audio output delay amount formed by a second ALSA delay amount of the external audio playing unit; obtaining a delay amount difference value between the first audio output delay amount and the second audio output delay amount by using a formula t _ d _ value ═ t _ bsp + t _ spread _ late-t _ usb _ late; when the delay amount difference belongs to the threshold interval, if t _ d _ value > 0, updating the second audio output delay amount by using a formula t _ usb _ late + t _ d _ value, if t _ d _ value < 0, updating the video output delay amount by using a formula t _ hal _ late + | t _ d _ value | and updating the first audio output delay amount by using a formula t _ spread _ late | + t _ space _ d _ value |.
Description
Technical Field
The invention relates to the field of video and audio data processing, in particular to a video and audio data synchronous output method and a system thereof.
Background
Currently, a built-in audio player collocated with a video playing device such as a projection device, a television device, etc. includes: the machine is provided with a loudspeaker, and the problems of insufficient bass and insufficient whole sound effect rendering and the like generally exist. Therefore, a user generally needs to use an external audio playing device to connect with a video playing device, so as to improve the sound experience when the video playing device is used. Under the condition that a user uses the external audio playing device, the internal audio player of the video playing device is matched with the external sound box to make sound at the same time, so that the overall audio-visual experience of the user can be improved at a lower cost.
At present, the common external audio playing device in the market is mainly a sound box using a usb interface. The built-in audio player of the video playing device is matched with the external usb interface sound box, so that the sound effect is improved, and better sound experience is brought. However, compared with the audio output of a built-in audio player of a video playing device, the audio output of the existing USB interface sound box has a delay condition, and when the video playing device is matched with an external USB interface sound box for use, the problem that video and audio cannot be synchronized exists.
Disclosure of Invention
In view of this, the present application provides a video and audio data synchronous output method and system, which solve the problem that video and audio cannot be synchronized when the conventional video playing device is used in cooperation with an external usb interface sound box by improving video and audio output control methods of a video playing unit and an external audio playing unit.
In order to solve the technical problems, the invention provides the following technical scheme: a video and audio data synchronous output method comprises the following steps: s1: acquiring a video output delay amount and a first audio output delay amount of a video playing unit, and configuring that the video output delay amount of the video playing unit is the same as the first audio output delay amount, wherein the first audio output delay amount comprises a BSP delay amount and a first ALSA delay amount; s2: acquiring a second audio output delay quantity formed by a second ALSA delay quantity of an external audio playing unit using a usb interface; s3: obtaining a delay amount difference value between the first audio output delay amount and the second audio output delay amount by using a formula t _ d _ value ═ t _ BSP + t _ spread _ late-t _ usb _ late, wherein t _ d _ value is the delay amount difference value, t _ BSP is the BSP delay amount, t _ spread _ late is the first ALSA delay amount, and t _ usb _ late is the second audio output delay amount; s4: when the delay amount difference belongs to the threshold interval, if t _ d _ value > 0, updating the second audio output delay amount by using a formula t _ usb _ late + t _ d _ value, if t _ d _ value < 0, updating the video output delay amount by using a formula t _ hal _ late + | t _ d _ value | and updating the first audio output delay amount by using a formula t _ spread _ late | + t _ space _ d _ value |.
Further, the S4 further includes: and when the delay difference does not belong to the threshold interval, if t _ d _ value is greater than 0, and audio data are input into a cache unit of the external audio playing unit, transmitting the audio data to a second driving unit of the external audio playing unit after a second ALSA framework unit of the external audio playing unit delays the output of the audio data by t _ d _ value.
Further, the S4 further includes: when the delay amount difference does not belong to the threshold interval, if t _ d _ value is less than 0, updating the video output delay amount by using a formula t _ hal _ late ═ t _ hal _ late + | t _ d _ value | and updating the BSP delay amount by using a formula t _ BSP | -t _ d _ value |.
Further, updating the second audio output delay amount using the formula t _ usb _ late-t _ usb _ late + t _ d _ value includes: gradually increasing, by the second ALSA architecture unit, a second run buffer of the second drive unit, and a numerical amount of the audio data read by the second drive unit from the second run buffer decreases; when the second running buffer increases until t _ usb _ late is t _ usb _ late + t _ d _ value, the second running buffer remains unchanged, and the second driving unit restores the initial value from the numerical amount of the audio data read by the second running buffer.
Further, updating the first audio output delay amount using the formula t _ distributor _ late ═ t _ distributor _ late + | t _ d _ value | includes: gradually increasing a first running buffer of a first driving unit of the video playing unit through a first ALSA architecture unit of the video playing unit, and reducing the numerical quantity of the audio data read by the first running buffer by the first driving unit; when the first running buffer increases until t _ spreader _ late ═ t _ spreader _ late + | t _ d _ value |, the first running buffer remains unchanged, and the first driving unit restores an initial value from the numerical amount of the audio data read by the first running buffer.
Accordingly, the present invention provides a system for synchronously outputting audio and video data, comprising: a video playing unit for outputting a video and a first audio; the external audio playing unit is used for outputting second audio; a management unit of the video playing unit, configured to obtain a video output delay amount and a first audio output delay amount of the video playing unit, configure that the video output delay amount of the video playing unit is the same as the first audio output delay amount, where the first audio output delay amount includes a BSP delay amount and a first ALSA delay amount, and obtain a second audio output delay amount configured by a second ALSA delay amount of the external audio playing unit, and obtain a delay amount difference between the first audio output delay amount and the second audio output delay amount by using a formula t _ d _ value ═ t _ BSP + t _ distributor _ late-t _ usb _ late, where t _ d _ value is the delay amount difference, t _ BSP is the BSP delay amount, t _ distributor _ late is the first ALSA delay amount, and t _ usb _ late is the second audio output delay amount, where the delay amount difference belongs to a threshold interval, if t _ d _ value > 0, updating the second audio output delay amount using a formula t _ usb _ late ═ t _ usb _ late + t _ d _ value, and if t _ d _ value < 0, updating the video output delay amount using a formula t _ hal _ late [ + | t _ d _ value ], and updating the first audio output delay amount using a formula t _ distributor _ late [ + | ] t _ space [ + | t _ d _ value ].
Further, the external audio playing unit includes a second ALSA framework unit, where when the delay amount difference does not belong to the threshold interval, if t _ d _ value is greater than 0, the management unit delays the output of the audio data of the second ALSA framework unit, and after the second ALSA framework unit delays the output of the audio data by t _ d _ value, the audio data is transmitted to a second driving unit of the external audio playing unit.
Further, when the delay amount difference does not belong to the threshold interval, the management unit updates the BSP delay amount using a formula t _ BSP ═ t _ d _ value | if t _ d _ value < 0.
Further, the second driving unit includes a second running buffer, and the management unit updates the second audio output delay amount by dynamically adjusting a size of the second running buffer, wherein the management unit gradually increases the second running buffer by the second ALSA architecture unit, and a numerical amount of the audio data read by the second running buffer by the second driving unit decreases, the management unit controls the second running buffer to remain unchanged when the second running buffer increases until t _ usb _ late is t _ usb _ late + t _ d _ value, and the numerical amount of the audio data read by the second driving unit by the second buffer row restores an initial value.
Further, the video playing unit further includes a first ALSA architecture unit and a first driving unit having a first running buffer, and the management unit updates the first audio output delay amount by dynamically adjusting the size of the first running buffer, wherein the management unit gradually increases the first running buffer of the first driving unit through the first ALSA architecture unit, and the first driving unit decreases the amount of the audio data read by the first running buffer; when the first running buffer increases until t _ spreader _ late ═ t _ spreader _ late + | t _ d _ value |, the first running buffer remains unchanged, and the first driving unit restores an initial value from the numerical amount of the audio data read by the first running buffer.
According to the video and audio data synchronous output method and system, the audio output asynchronous type of the video playing unit and the external audio playing unit is judged, various control strategies are flexibly called, when the audio output asynchronous degree is small, the size of the buffer area is dynamically adjusted to eliminate the delay difference, the problem that video and audio cannot be synchronous when the traditional video playing equipment is matched with an external usb interface sound box for use is solved, and the phenomena of audio playing stagnation and the like caused by the elimination of the delay difference are effectively avoided.
Drawings
FIG. 1 is a flowchart of a method for synchronously outputting audio and video data according to the present invention;
fig. 2 is a block diagram of the video and audio data synchronous output system according to the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood by those skilled in the art, the present invention will be further described in detail with reference to the accompanying drawings and specific embodiments.
As shown in fig. 1, the present embodiment provides a method for synchronously outputting audio and video data, including: s1: acquiring a video output delay amount and a first audio output delay amount of a video playing unit, and configuring that the video output delay amount of the video playing unit is the same as the first audio output delay amount, wherein the first audio output delay amount comprises a BSP delay amount and a first ALSA delay amount; s2: acquiring a second audio output delay quantity formed by a second ALSA delay quantity of an external audio playing unit using a usb interface; s3: obtaining a delay amount difference value between the first audio output delay amount and the second audio output delay amount by using a formula t _ d _ value ═ t _ BSP + t _ spread _ late-t _ usb _ late, wherein t _ d _ value is the delay amount difference value, t _ BSP is the BSP delay amount, t _ spread _ late is the first ALSA delay amount, and t _ usb _ late is the second audio output delay amount; s4: when the delay amount difference belongs to the threshold interval, if t _ d _ value > 0, updating the second audio output delay amount by using a formula t _ usb _ late + t _ d _ value, if t _ d _ value < 0, updating the video output delay amount by using a formula t _ hal _ late + | t _ d _ value | and updating the first audio output delay amount by using a formula t _ spread _ late | + t _ space _ d _ value |. The video output delay amount is controlled by an audio hal layer of the video playing unit, namely a management unit; the BSP delay is the audio output delay of the board-level support cladding, and the scenes influencing the BSP delay comprise: playing videos through a network, playing videos after trapezoidal correction, playing videos after adjusting different image qualities, and the like; the first ALSA delay amount is the buffering delay of a first driving unit of the video playing unit and is controlled by a first ALSA architecture unit; the second ALSA delay amount is a buffering delay of a second driving unit of the external audio playing unit and is controlled by the second ALSA framework unit.
It should be noted that the threshold interval may be (-15ms, +15ms), (-30ms, +30ms), (-50ms, +50ms), and the like, and the present invention does not specifically limit the threshold interval. Specifically, according to the hars effect principle: if two sound sources with the same loudness in the sound field emit the same audio signals, if the time difference of the two sounds transmitted to the human ears is equal to or more than 50ms, the human ears can clearly distinguish the two sounds from different directions; when the time difference of the two sounds transmitted into the human ear is within 50ms, the human ear cannot obviously distinguish the directions of the two sound sources; when the time difference of two sounds transmitted into the human ear is within 17ms, the human ear will feel as if the sounds only come from the undelayed sound source, and will not feel as if the delayed sound source exists. Therefore, the inventor sets the threshold interval to (-50ms, +50ms) at the initial stage of design, and then finds out through a plurality of experiments that when the types/models of the video playing unit and the external audio playing unit are different, the hardware parameters used for constructing the sound field are different, and the software control logic for controlling the audio output strategy is different, and when the threshold interval is not greater than (-50ms, +50ms), the different types of video playing units and external audio playing units have different optimal threshold intervals so as to present the optimal audio playing effect. Therefore, the present invention does not specifically limit the threshold interval.
When the video and audio output is asynchronous, the size of the buffer area is dynamically adjusted to eliminate the delay difference, so that the problem that the video and audio cannot be synchronous when the traditional video playing equipment is matched with an external usb interface sound box is solved, and the phenomena of audio playing jamming and the like caused by the elimination of the delay difference are effectively avoided. Meanwhile, considering that the initial first audio output delay amount is the same as the initial video output delay amount, in order to avoid the problem of audio and video asynchronization caused by updating the first audio output delay amount, the invention updates the video output delay amount before updating the first audio output delay amount so as to ensure the synchronism of the video and audio output.
Further, updating the second audio output delay amount using the formula t _ usb _ late ═ t _ usb _ late + t _ d _ value includes: gradually increasing, by the second ALSA architecture unit, a second run buffer of the second drive unit, and a numerical amount of the audio data read by the second drive unit from the second run buffer decreases; when the second running buffer increases until t _ usb _ late is t _ usb _ late + t _ d _ value, the second running buffer remains unchanged, and the second driving unit restores the initial value from the numerical amount of the audio data read by the second running buffer.
Further, updating the first audio output delay amount using the formula t _ distributor _ late ═ t _ distributor _ late + | t _ d _ value | includes: gradually increasing a first running buffer of a first driving unit of the video playing unit through a first ALSA architecture unit of the video playing unit, and reducing the numerical quantity of the audio data read by the first running buffer by the first driving unit; when the first running buffer increases until t _ spreader _ late ═ t _ spreader _ late + | t _ d _ value |, the first running buffer remains unchanged, and the first driving unit restores an initial value from the numerical amount of the audio data read by the first running buffer.
Specifically, the calculation formula of the run buffer size is as follows: the run buffer size is equal to period _ size and period _ count, where period _ size refers to the minimum amount of data read from the buffer in the audio driver each time, and period _ count refers to how many period _ sizes are in the buffer, that is, the present invention dynamically adjusts the size of the running buffer by dynamically adjusting period _ size and period _ count.
Because the user can hear that two sound sources exist in the sound field when the delay amount difference does not belong to the threshold interval, which causes the user to have a poor listening effect, the S4 further includes: and when the delay difference does not belong to the threshold interval, if t _ d _ value is greater than 0, and audio data are input into a cache unit of the external audio playing unit, transmitting the audio data to a second driving unit of the external audio playing unit after a second ALSA framework unit of the external audio playing unit delays the output of the audio data by t _ d _ value. And if t _ d _ value is less than 0, updating the video output delay amount by using a formula t _ hal _ late ═ t _ hal _ late + | t _ d _ value | and updating the BSP delay amount by using a formula t _ BSP ═ t _ d _ value | when the delay amount difference does not belong to the threshold interval. Therefore, when the listening effect of a user is poor due to the fact that the delay difference is too large, the audio output can be instantly adjusted to eliminate the delay difference, and meanwhile, the problem that sound and picture are not synchronous due to the fact that the first audio output delay is updated is solved by considering that the initial first audio output delay is the same as the initial video output delay.
Accordingly, the present invention provides, as shown in fig. 2, a video and audio data synchronous output system, including: a video playing unit for outputting a video and a first audio; the external audio playing unit is used for outputting second audio; a management unit of the video playing unit, configured to obtain a video output delay amount and a first audio output delay amount of the video playing unit, configure that the video output delay amount of the video playing unit is the same as the first audio output delay amount, where the first audio output delay amount includes a BSP delay amount and a first ALSA delay amount, and obtain a second audio output delay amount configured by a second ALSA delay amount of the external audio playing unit, and obtain a delay amount difference between the first audio output delay amount and the second audio output delay amount by using a formula t _ d _ value ═ t _ BSP + t _ distributor _ late-t _ usb _ late, where t _ d _ value is the delay amount difference, t _ BSP is the BSP delay amount, t _ distributor _ late is the first ALSA delay amount, and t _ usb _ late is the second audio output delay amount, where the delay amount difference belongs to a threshold interval, if t _ d _ value > 0, updating the second audio output delay amount using a formula t _ usb _ late ═ t _ usb _ late + t _ d _ value, and if t _ d _ value < 0, updating the video output delay amount using a formula t _ hal _ late [ + | t _ d _ value ], and updating the first audio output delay amount using a formula t _ distributor _ late [ + | ] t _ space [ + | t _ d _ value ]. The video playing unit can be a video playing device with a built-in audio player, such as a projector and a television; the external audio playing unit can be an external audio playing device using a USB interface; the management unit of the video playing unit is an audio hal layer. Specifically, when the delay amount difference does not belong to the threshold interval, if t _ d _ value is less than 0, the management unit updates the video output delay amount using a formula t _ hal _ late ═ t _ hal _ late + | t _ d _ value | and updates the BSP delay amount using a formula t _ BSP | t _ d _ value |.
Further, the external audio playing unit is provided with a second ALSA framework unit and a second driving unit, when the delay amount difference does not belong to the threshold interval, if t _ d _ value is greater than 0, the management unit delays the output of the audio data of the second ALSA framework unit, and after the second ALSA framework unit delays the output of the audio data by t _ d _ value, the audio data is transmitted to the second driving unit of the external audio playing unit.
Further, the second driving unit includes a second running buffer, and the management unit updates the second audio output delay amount by dynamically adjusting a size of the second running buffer, wherein the management unit gradually increases the second running buffer by the second ALSA framework unit, and a numerical amount of the audio data read by the second running buffer by the second driving unit decreases, the management unit controls the second running buffer to remain unchanged when the second running buffer increases until t _ usb _ late is t _ usb _ late + t _ d _ value, and the numerical amount of the audio data read by the second driving unit by the second buffer row restores an initial value.
Furthermore, the video playing unit further includes a first ALSA framework unit and a first driving unit having a first running buffer, and the management unit updates the first audio output delay amount by dynamically adjusting the size of the first running buffer, wherein the management unit gradually increases the first running buffer of the first driving unit through the first ALSA framework unit, and the first driving unit decreases the amount of the audio data read by the first running buffer; when the first running buffer increases until t _ spreader _ late ═ t _ spreader _ late + | t _ d _ value |, the first running buffer remains unchanged, and the first driving unit restores an initial value from the numerical amount of the audio data read by the first running buffer.
The above is only a preferred embodiment of the present invention, and it should be noted that the above preferred embodiment should not be considered as limiting the present invention, and the protection scope of the present invention should be subject to the scope defined by the claims. It will be apparent to those skilled in the art that various modifications and adaptations can be made without departing from the spirit and scope of the invention, and these modifications and adaptations should be considered within the scope of the invention.
Claims (10)
1. A video and audio data synchronous output method is characterized by comprising the following steps:
s1: acquiring a video output delay amount and a first audio output delay amount of a video playing unit, and configuring that the video output delay amount of the video playing unit is the same as the first audio output delay amount, wherein the first audio output delay amount comprises a BSP delay amount t _ BSP and a first ALSA delay amount t _ spreader _ late;
s2: acquiring a second audio output delay t _ usb _ late formed by a second ALSA delay of the external audio playing unit;
s3: calculating a delay difference t _ d _ value of the first audio output delay compared with the second audio output delay;
s4: and when the delay amount difference value belongs to a threshold interval, if t _ d _ value is greater than 0, dynamically increasing the second audio output delay amount until the second audio output delay amount is the same as the first audio output delay amount, if t _ d _ value is less than 0, updating the video output delay amount to enable the video output delay amount to be the same as the second audio output delay amount, and dynamically increasing the first ALSA delay amount until the first audio output delay amount is the same as the second audio output delay amount.
2. The video/audio data synchronous output method according to claim 1, wherein said S4 further comprises:
and when the delay difference does not belong to the threshold interval, if t _ d _ value is greater than 0, and audio data are input into a cache unit of the external audio playing unit, transmitting the audio data to a second driving unit of the external audio playing unit after a second ALSA framework unit of the external audio playing unit delays the output of the audio data by t _ d _ value.
3. The video/audio data synchronous output method according to claim 2, wherein said S4 further comprises:
when the delay amount difference does not belong to the threshold interval, if t _ d _ value is less than 0, updating the video output delay amount by using a formula t _ hal _ late ═ t _ hal _ late + | t _ d _ value | and updating the BSP delay amount by using a formula t _ BSP | -t _ d _ value |.
4. The video/audio data synchronous output method according to claim 3, wherein dynamically increasing the second audio output delay amount until the second audio output delay amount is the same as the first audio output delay amount comprises:
gradually increasing, by the second ALSA architecture unit, a second run buffer of the second drive unit, and a numerical amount of the audio data read by the second drive unit from the second run buffer decreases;
when the second running buffer increases until t _ usb _ late is t _ usb _ late + t _ d _ value, the second running buffer remains unchanged, and the second driving unit restores the initial value from the numerical amount of the audio data read by the second running buffer.
5. The audio-visual data synchronization output method according to claim 4, wherein dynamically increasing the first ALSA delay amount until the first audio output delay amount is the same as the second audio output delay amount comprises:
gradually increasing a first running buffer of a first driving unit of the video playing unit through a first ALSA architecture unit of the video playing unit, and reducing the numerical quantity of the audio data read by the first running buffer by the first driving unit;
when the first running buffer increases until t _ spreader _ late ═ t _ spreader _ late + | t _ d _ value |, the first running buffer remains unchanged, and the first driving unit restores an initial value from the numerical amount of the audio data read by the first running buffer.
6. A video-audio data synchronous output system, comprising:
a video playing unit for outputting a video and a first audio;
the external audio playing unit is used for outputting second audio;
a management unit of the video playing unit, configured to obtain a video output delay amount and a first audio output delay amount of the video playing unit, configure that the video output delay amount of the video playing unit is the same as the first audio output delay amount, where the first audio output delay amount includes a BSP delay amount and a first ALSA delay amount, and obtain a second audio output delay amount configured by a second ALSA delay amount of the external audio playing unit, and obtain a delay amount difference between the first audio output delay amount and the second audio output delay amount by using a formula t _ d _ value ═ t _ BSP + t _ distributor _ late-t _ usb _ late, where t _ d _ value is the delay amount difference, t _ BSP is the BSP delay amount, t _ distributor _ late is the first ALSA delay amount, and t _ usb _ late is the second audio output delay amount, where the delay amount difference belongs to a threshold interval, if t _ d _ value > 0, updating the second audio output delay amount using a formula t _ usb _ late ═ t _ usb _ late + t _ d _ value, and if t _ d _ value < 0, updating the video output delay amount using a formula t _ hal _ late [ + | t _ d _ value ], and updating the first audio output delay amount using a formula t _ distributor _ late [ + | ] t _ space [ + | t _ d _ value ].
7. The audio-visual data synchronous output system of claim 6, wherein the external audio playing unit comprises a second ALSA architecture unit, wherein,
if t _ d _ value > 0, delaying the output of the audio data of the second ALSA architecture unit by the management unit when the delay amount difference does not belong to the threshold interval,
and after the second ALSA architecture unit delays the output of the audio data by t _ d _ value, transmitting the audio data to a second driving unit of the external audio playing unit.
8. The system of claim 7, wherein the management unit updates the BSP delay amount using the equation t _ BSP ═ t _ d _ value | if t _ d _ value < 0 when the delay amount difference does not belong to the threshold interval.
9. The video/audio data synchronous output system according to claim 8, wherein the second drive unit includes a second running buffer, and the management unit updates the second audio output delay amount by dynamically adjusting a size of the second running buffer, wherein,
the management unit increases the second running buffer step by the second ALSA architecture unit, and the second driving unit decreases the amount of the audio data read by the second running buffer,
when the second running buffer increases until t _ usb _ late is t _ usb _ late + t _ d _ value, the management unit controls the second running buffer to remain unchanged, and the second driving unit restores an initial value from the numerical amount of the audio data read by the second running buffer.
10. The video-audio data synchronous output system according to claim 9, wherein the video playback unit further includes a first ALSA architecture unit and a first driving unit having a first running buffer, the management unit updates the first audio output delay amount by dynamically adjusting a size of the first running buffer, wherein,
the management unit gradually increases a first running buffer of the first driving unit through the first ALSA architecture unit, and the numerical quantity of the audio data read by the first running buffer of the first driving unit is reduced;
when the first running buffer increases until t _ spreader _ late ═ t _ spreader _ late + | t _ d _ value |, the first running buffer remains unchanged, and the first driving unit restores an initial value from the numerical amount of the audio data read by the first running buffer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011178370.6A CN112291607B (en) | 2020-10-29 | 2020-10-29 | Video and audio data synchronous output method and system thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011178370.6A CN112291607B (en) | 2020-10-29 | 2020-10-29 | Video and audio data synchronous output method and system thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112291607A true CN112291607A (en) | 2021-01-29 |
| CN112291607B CN112291607B (en) | 2022-11-29 |
Family
ID=74373850
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011178370.6A Active CN112291607B (en) | 2020-10-29 | 2020-10-29 | Video and audio data synchronous output method and system thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112291607B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102256183A (en) * | 2010-05-21 | 2011-11-23 | 上海捷骐数码科技有限公司 | Mobile-communication-network-based audio and video signal real-time transmission method |
| CN102868908A (en) * | 2011-07-04 | 2013-01-09 | 哈尔滨融智达网络科技有限公司 | High-efficiency streaming media playing method and device |
| CN103078701A (en) * | 2012-10-17 | 2013-05-01 | 大唐移动通信设备有限公司 | Uplink voice data transmission method and device |
| CN103795950A (en) * | 2014-02-21 | 2014-05-14 | 深圳创维－Rgb电子有限公司 | Television audio signal processing device |
| CN103888813A (en) * | 2012-12-21 | 2014-06-25 | 北京计算机技术及应用研究所 | Audio and video synchronization realization method and system |
| US20150243328A1 (en) * | 2014-02-21 | 2015-08-27 | Renesas Electronics Corporation | Content Output Device And Program |
| US20160139784A1 (en) * | 2014-11-14 | 2016-05-19 | Facebook, Inc. | Systems and methods for providing interactive time-lapse selection for captured media content |
| CN106162335A (en) * | 2015-03-23 | 2016-11-23 | 青岛海信电器股份有限公司 | Player methods based on multichannel television audio data, Apparatus and system |
| CN108012177A (en) * | 2017-12-01 | 2018-05-08 | 晨星半导体股份有限公司 | The playback method of audio-video frequency playing system and the voice data being applied thereon |
| US9990937B1 (en) * | 2016-12-07 | 2018-06-05 | Beken Corporation | Device and method for synchronizing speakers |
-
2020
- 2020-10-29 CN CN202011178370.6A patent/CN112291607B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102256183A (en) * | 2010-05-21 | 2011-11-23 | 上海捷骐数码科技有限公司 | Mobile-communication-network-based audio and video signal real-time transmission method |
| CN102868908A (en) * | 2011-07-04 | 2013-01-09 | 哈尔滨融智达网络科技有限公司 | High-efficiency streaming media playing method and device |
| CN103078701A (en) * | 2012-10-17 | 2013-05-01 | 大唐移动通信设备有限公司 | Uplink voice data transmission method and device |
| CN103888813A (en) * | 2012-12-21 | 2014-06-25 | 北京计算机技术及应用研究所 | Audio and video synchronization realization method and system |
| CN103795950A (en) * | 2014-02-21 | 2014-05-14 | 深圳创维－Rgb电子有限公司 | Television audio signal processing device |
| US20150243328A1 (en) * | 2014-02-21 | 2015-08-27 | Renesas Electronics Corporation | Content Output Device And Program |
| US20160139784A1 (en) * | 2014-11-14 | 2016-05-19 | Facebook, Inc. | Systems and methods for providing interactive time-lapse selection for captured media content |
| CN106162335A (en) * | 2015-03-23 | 2016-11-23 | 青岛海信电器股份有限公司 | Player methods based on multichannel television audio data, Apparatus and system |
| US9990937B1 (en) * | 2016-12-07 | 2018-06-05 | Beken Corporation | Device and method for synchronizing speakers |
| CN108012177A (en) * | 2017-12-01 | 2018-05-08 | 晨星半导体股份有限公司 | The playback method of audio-video frequency playing system and the voice data being applied thereon |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112291607B (en) | 2022-11-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10992451B2 (en) | Audio and video playback system and method for playing audio data applied thereto | |
| JP5957760B2 (en) | Video / audio processor | |
| CN104967960A (en) | Voice data processing method, and voice data processing method and system in game live broadcasting | |
| CN109168059B (en) | Lip sound synchronization method for respectively playing audio and video on different devices | |
| CN109275008B (en) | Audio and video synchronization method and device | |
| WO2020038344A1 (en) | Method and device for audio and video synchronization, and display apparatus | |
| AU2014295217B2 (en) | Audio processor for orientation-dependent processing | |
| CN109379613A (en) | Audio-visual synchronization method of adjustment, TV, computer readable storage medium and system | |
| WO2017054377A1 (en) | Audio data processing method, apparatus and system | |
| CN109379619A (en) | Sound draws synchronous method and device | |
| US10747492B2 (en) | Signal processing apparatus, signal processing method, and storage medium | |
| US9886980B2 (en) | Method for synchronizing A/V streams | |
| US12020719B2 (en) | Method, a system, and a program for playing back audio/video signals with automatic adjustment of latency | |
| JP6720566B2 (en) | Audio equipment | |
| US20170019870A1 (en) | Method and apparatus for synchronization in a network | |
| CN112291607B (en) | Video and audio data synchronous output method and system thereof | |
| CN112423074B (en) | Audio and video synchronization processing method and device, electronic equipment and storage medium | |
| CN109495788B (en) | MEMC dynamic control method based on slice source frame rate | |
| CN115767158A (en) | Synchronous playing method, terminal equipment and storage medium | |
| CN101180887B (en) | Equipment for audio-video processing system | |
| KR20010018572A (en) | Apparatus for synchronizing audio/video signal and method thereof | |
| CN115802087A (en) | Sound and picture synchronous processing method and related equipment thereof | |
| CN112770165B (en) | Distributed synchronization method for audio and video streams | |
| CN113301410A (en) | Audio picture synchronization method, device and system and transmission controller | |
| CN111654745B (en) | Multi-channel signal processing method and display device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |