WO2017005066A1 - Method and apparatus for recording audio and video synchronization timestamp - Google Patents

Method and apparatus for recording audio and video synchronization timestamp Download PDF

Info

Publication number
WO2017005066A1
WO2017005066A1 PCT/CN2016/084387 CN2016084387W WO2017005066A1 WO 2017005066 A1 WO2017005066 A1 WO 2017005066A1 CN 2016084387 W CN2016084387 W CN 2016084387W WO 2017005066 A1 WO2017005066 A1 WO 2017005066A1
Authority
WO
WIPO (PCT)
Prior art keywords
audio
data
video
audio data
time stamp
Prior art date
Application number
PCT/CN2016/084387
Other languages
French (fr)
Chinese (zh)
Inventor
焦新旺
王云华
Original Assignee
深圳Tcl数字技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳Tcl数字技术有限公司 filed Critical 深圳Tcl数字技术有限公司
Publication of WO2017005066A1 publication Critical patent/WO2017005066A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4334—Recording operations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4305—Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/8547—Content authoring involving timestamps for synchronizing content

Definitions

  • the present invention relates to the field of multimedia technologies, and in particular, to a method and apparatus for recording an audio and video synchronization time stamp.
  • the main object of the present invention is to provide a method and apparatus for recording audio and video synchronization time stamps, which avoids the problem that recorded audio data and video data are not synchronized during playback.
  • the invention provides a method for recording audio and video synchronization time stamps, comprising the steps of:
  • the length a of the audio data and the frame rate b of the video data are respectively enlarged by n times, and the n is a preset magnification;
  • the video data and audio data that are time-synchronized are obtained by encoding, decoding, and decoding the video data and the audio data according to a new time stamp.
  • the step of encoding, decoding, and synchronizing the video data and the audio data according to the new time stamp further comprises:
  • the step of receiving and storing the audio data collected by the audio collection device further includes:
  • the video capture device When the video capture device is currently connected, the video capture device is controlled to collect the video data.
  • the invention also provides a method for recording an audio and video synchronization time stamp, comprising the steps of:
  • the video data and audio data that are time-synchronized are obtained by encoding, decoding, and decoding the video data and the audio data according to a new time stamp.
  • the length a of the audio data and the frame rate b of the video data are respectively enlarged by n times, and the n is a preset magnification;
  • the step of encoding, decoding, and synchronizing the video data and the audio data according to the new time stamp further comprises:
  • the step of collecting video data and audio data comprises:
  • the step of receiving and storing the audio data collected by the audio collection device further includes:
  • the video capture device When the video capture device is currently connected, the video capture device is controlled to collect the video data.
  • the invention also provides an apparatus for recording an audio and video synchronization time stamp, comprising:
  • An acquisition module for collecting video data and audio data An acquisition module for collecting video data and audio data
  • An acquiring module configured to acquire a length a of the audio data, a frame rate b of the video data, and a preset audio sampling rate h;
  • a codec module configured to obtain the time-synchronized video data and audio data according to a new timestamp encoding, decoding the video data and audio data.
  • the invention recalculates the new timestamp according to the audio length, the audio sampling rate and the video frame rate, and encodes and decodes the audio and video data according to the obtained new timestamp, so that the encoded and decoded audio data and the video data are time synchronized.
  • the smart device plays back the audio and video files, the screen and the sound are played synchronously, which effectively avoids the screen and the sound being out of sync during playback due to recording interference.
  • FIG. 1 is a flowchart of a first embodiment of a method for recording an audio and video synchronization time stamp according to the present invention
  • FIG. 2 is a flowchart of a second embodiment of a method for recording an audio and video synchronization time stamp according to the present invention
  • FIG. 3 is a flowchart of a third embodiment of a method for recording an audio and video synchronization time stamp according to the present invention.
  • FIG. 4 is a flowchart of a fourth embodiment of a method for recording an audio and video synchronization time stamp according to the present invention.
  • FIG. 5 is a flowchart of a fifth embodiment of a method for recording an audio and video synchronization time stamp according to the present invention.
  • FIG. 6 is a schematic block diagram of a first embodiment of an apparatus for recording an audio and video synchronization time stamp according to the present invention
  • FIG. 7 is a schematic block diagram of a second embodiment of an apparatus for recording an audio and video synchronization time stamp according to the present invention.
  • FIG. 8 is a schematic block diagram of a third embodiment of an apparatus for recording an audio and video synchronization time stamp according to the present invention.
  • FIG. 1 is a flowchart of a first embodiment of a method for recording an audio and video synchronization time stamp according to the present invention.
  • the method for recording an audio and video synchronization time stamp mentioned in this embodiment includes:
  • Step S10 collecting video data and audio data
  • the smart device of this embodiment may include a television, a mobile phone, a computer, and the like.
  • the smart device can be connected to a video capture device, such as a camera, to capture video data through the camera.
  • the smart device can also be connected to an audio capture device, such as a microphone, to capture audio data through the microphone.
  • the smart device can also connect other devices with audio capture function, and transmit audio data to the smart device after collecting audio through other devices.
  • the smart device is a television, the other device is a mobile phone, and the mobile phone and the television are connected through a wireless network. After the mobile phone collects audio data through the microphone, the audio data is sent to the television through the wireless network.
  • Step S20 obtaining the length a of the audio data, the frame rate b of the video data, and the preset audio sampling rate h;
  • the smart device can count the frame rate of the current video data according to the resolution of the video capture device, calculate the length of the collected audio data, and obtain a preset audio sample rate from the memory.
  • a new timestamp T a/[b/(a/h)] according to the adaptive timestamp algorithm, where a is the length of the audio data, b is the frame rate of the video data, and h is the preset audio sampling rate.
  • b can be rounded to an integer, eliminating the number after the decimal point.
  • the value after the decimal point of b can also be retained, so that the obtained time stamp is more accurate.
  • Step S40 encoding and decoding the video data and the audio data according to the new time stamp, and obtaining time-synchronized video data and audio data.
  • the obtained new time stamp is input to the codec module, so that the codec module encodes and decodes the video data and the audio data according to the new time stamp, and finally obtains time-synchronized video data and audio data.
  • the smart device plays back the audio and video files, the picture is synchronized with the sound.
  • a new timestamp is recalculated according to the audio length, the audio sampling rate, and the video frame rate, and the audio and video data is encoded and decoded according to the obtained new timestamp, so that the encoded and decoded audio data and the video data are time synchronized.
  • the smart device plays back the audio and video files, the picture and the sound are played synchronously, which effectively avoids the picture and sound being out of sync during playback due to recording interference.
  • FIG. 2 is a flowchart of a second embodiment of a method for recording an audio and video synchronization time stamp according to the present invention.
  • This embodiment includes the steps of the embodiment shown in FIG. 1, wherein step S30 includes:
  • Step S31 respectively, the length a of the audio data and the frame rate b of the video data are enlarged by n times, where n is a preset magnification;
  • Step S32 dividing the length of the amplified audio data by the audio sampling rate h, to obtain an audio length an/h of each Hertz;
  • Step S33 dividing the frame rate of the amplified video data by the audio length an/h of each Hertz, and obtaining the duration bn/(an/h) of each data storage bit;
  • the length a of the audio data is also enlarged by n times, so that the value in the calculation process is further reduced by n times after being enlarged by n times.
  • the length of the amplified audio is divided by the audio sampling rate to obtain the audio length per unit hertz; and the amplified video frame rate is divided by the audio length per hertz to obtain the duration of each data storage bit. That is, the length of each bit in the memory in which the audio data is stored.
  • the audio data is an analog signal at the time of acquisition, and after being sampled by the ADC, the analog waveform becomes a discrete digital signal, and the number of discrete points is corresponding to the audio data.
  • the number of data storage bits, the number of sampling points of the audio data sampled in 1s is two, and the length of each storage bit is 0.5 s; then the audio length is divided by the length of each data storage bit to obtain a new one. Timestamp.
  • the frame rate b is enlarged by n times, so that the obtained timestamp is more accurate, and the audio and video data is encoded and decoded according to the obtained timestamp, so that the encoded and decoded audio data and video data time are obtained. Synchronization effectively prevents the picture and sound from being out of sync during playback due to recording interference.
  • FIG. 3 is a flowchart of a third embodiment of a method for recording an audio and video synchronization time stamp according to the present invention. This embodiment includes the steps of the embodiment shown in FIG. 1. Before step S40, the method further includes:
  • Step S51 it is determined whether the number of data storage bits corresponding to the audio data is greater than the value of the new timestamp; if yes, step S52 is performed; if not, step S53 is performed;
  • Step S52 it is determined that the new timestamp is valid, and step S40 is performed;
  • step S53 it is determined that the new time stamp is invalid.
  • the audio data collected by the audio acquisition device is an analog signal. After sampling by the ADC, the analog waveform becomes a discrete digital signal, and the number of discrete sampling points, that is, the number of data storage bits. Since the number of audio data storage is the number of ADC analog-to-digital conversions, the number of ADC analog-to-digital conversions must be greater than the number of conversions. Therefore, if the number of storages is less than or equal to the new timestamp, Then the timestamp is invalid.
  • the timestamp is invalid, and a prompt window may pop up prompting the user to re-record.
  • a length threshold or a time threshold may be set. After the collected data reaches the length threshold or the time threshold, the time stamp is calculated according to the collected partial data, and the current calculation is determined. Whether the timestamp is valid. If it is invalid, it stops the subsequent data collection and prompts the user to re-record. In this embodiment, the validity of the timestamp is determined, and the problem that the played sound and the picture are still out of synchronization caused by invalid timestamp coding and decoding of the audio and video data is avoided.
  • FIG. 4 is a flowchart of a fourth embodiment of a method for recording an audio and video synchronization time stamp according to the present invention.
  • This embodiment includes the steps of the embodiment shown in FIG. 1, wherein step S10 includes:
  • Step S11 receiving and storing audio data collected by the audio collection device
  • Step S12 it is determined whether the number of storage bits corresponding to the audio data is greater than a threshold; if yes, step S13 is performed; if not, step S14 is performed;
  • Step S13 determining that the audio data is valid, receiving and storing the video data collected by the video capture device, and performing step S20;
  • step S14 it is determined that the audio data is invalid.
  • the audio data when the video data is recorded, the audio data may not be collected due to the interruption of the connection of the audio collection device, so the validity of the audio data needs to be judged.
  • the system acquires the number of stored bits in the memory in which the audio data is stored, and can also be understood as the size of the amount of audio data. When the number of stored bits is greater than the threshold, for example, the threshold is 10, when the number of stored bits is greater than 10, it is determined that enough audio data has been collected, and the audio data is valid. Otherwise, the amount of audio data collected is small. The audio data may not be collected due to the interruption of the connection of the audio collection device.
  • the user may be prompted to check whether the audio collection device is connected and re-recorded.
  • the validity of the audio data is judged to avoid the problem that the audio data cannot be collected due to the interruption of the connection of the audio collection device.
  • FIG. 5 is a flowchart of a fifth embodiment of a method for recording an audio and video synchronization time stamp according to the present invention.
  • This embodiment includes the steps of the embodiment shown in FIG. 4, and before step S11, the method further includes:
  • Step S15 receiving an audio and video recording request
  • Step S16 detecting whether the video capture device is currently connected
  • step S17 when it is detected that the current video capture device is accessed, the video capture device is controlled to collect video data.
  • the user can click on the control of recording audio and video on the smart device to generate an audio and video recording request.
  • the user opens the MTV application, selects the MTV song that he likes, enters the recording interface of the self-singing self-play, and starts singing.
  • the system detects the level of the USB interface. When the hardware level of the USB interface is high, it indicates that the device is connected to the USB interface.
  • the system identifies the device connected to the USB and determines whether it is a video capture device, for example, whether there is a camera connection. USB. If the currently connected USB device is a video capture device, the video capture device is controlled to collect video data.
  • a prompt box is displayed to prompt the user to check whether the video collection device is connected and re-recorded.
  • the video capture device is determined to be accessed, and the problem that the video data cannot be collected due to the video capture device not being accessed is avoided.
  • FIG. 6 is a schematic block diagram of a first embodiment of an apparatus for recording an audio and video synchronization time stamp according to the present invention.
  • the device for recording audio and video synchronization time stamp mentioned in this embodiment includes:
  • the collecting module 10 is configured to collect video data and audio data
  • the obtaining module 20 is configured to obtain a length a of the audio data, a frame rate b of the video data, and a preset audio sampling rate h;
  • the codec module 40 is configured to obtain time-synchronized video data and audio data according to a new time stamp encoding, decoding video data and audio data.
  • the smart device of this embodiment may include a television, a mobile phone, a computer, and the like.
  • the smart device can be connected to a video capture device, such as a camera, to capture video data through the camera.
  • the smart device can also be connected to an audio capture device, such as a microphone, to capture audio data through the microphone.
  • the smart device can also connect other devices with audio capture function, and transmit audio data to the smart device after collecting audio through other devices.
  • the smart device is a television, the other device is a mobile phone, and the mobile phone and the television are connected through a wireless network. After the mobile phone collects audio data through the microphone, the audio data is sent to the television through the wireless network.
  • the smart device can count the frame rate of the current video data according to the resolution of the video capture device, calculate the length of the collected audio data, and obtain a preset audio sample rate from the memory.
  • a is the length of the audio data
  • b is the frame rate of the video data
  • h is the preset audio sampling rate.
  • b can be rounded to an integer, eliminating the number after the decimal point.
  • the value after the decimal point of b can also be retained, so that the obtained time stamp is more accurate.
  • the obtained new time stamp is input to the codec module, so that the codec module encodes and decodes the video data and the audio data according to the new time stamp, and finally obtains time-synchronized video data and audio data.
  • the smart device plays back the audio and video files, the picture is synchronized with the sound.
  • a new timestamp is recalculated according to the audio length, the audio sampling rate, and the video frame rate, and the audio and video data is encoded and decoded according to the obtained new timestamp, so that the encoded and decoded audio data and the video data are time synchronized.
  • the smart device plays back the audio and video files, the picture and the sound are played synchronously, which effectively avoids the picture and sound being out of sync during playback due to recording interference.
  • calculation module 30 is further configured to:
  • the length a of the audio data is also enlarged by n times, so that the value in the calculation process is further reduced by n times after being enlarged by n times.
  • the length of the amplified audio is divided by the audio sampling rate to obtain the audio length per unit hertz; and the amplified video frame rate is divided by the audio length per hertz to obtain the duration of each data storage bit. That is, the length of each bit in the memory in which the audio data is stored.
  • the audio data is an analog signal at the time of acquisition, and after being sampled by the ADC, the analog waveform becomes a discrete digital signal, and the number of discrete points is corresponding to the audio data.
  • the number of data storage bits, the number of sampling points of the audio data sampled in 1s is two, and the length of each storage bit is 0.5 s; then the audio length is divided by the length of each data storage bit to obtain a new one. Timestamp.
  • the frame rate b is enlarged by n times, so that the obtained timestamp is more accurate, and the audio and video data is encoded and decoded according to the obtained timestamp, so that the encoded and decoded audio data and video data time are obtained. Synchronization effectively prevents the picture and sound from being out of sync during playback due to recording interference.
  • FIG. 7 is a schematic block diagram of a second embodiment of an apparatus for recording an audio and video synchronization time stamp according to the present invention.
  • the embodiment includes the module of the embodiment shown in FIG. 6, and further includes a determining module 50, configured to:
  • the audio data collected by the audio acquisition device is an analog signal. After sampling by the ADC, the analog waveform becomes a discrete digital signal, and the number of discrete sampling points, that is, the number of data storage bits. Since the number of audio data storage is the number of ADC analog-to-digital conversions, the number of ADC analog-to-digital conversions must be greater than the number of conversions. Therefore, if the number of storages is less than or equal to the new timestamp, Then the timestamp is invalid.
  • the timestamp is invalid, and a prompt window may pop up prompting the user to re-record.
  • a length threshold or a time threshold may be set. After the collected data reaches the length threshold or the time threshold, the time stamp is calculated according to the collected partial data, and the current calculation is determined. Whether the timestamp is valid. If it is invalid, it stops the subsequent data collection and prompts the user to re-record. In this embodiment, the validity of the timestamp is determined, and the problem that the played sound and the picture are still out of synchronization caused by invalid timestamp coding and decoding of the audio and video data is avoided.
  • the collecting module 10 is further configured to receive and store the audio data collected by the audio collecting device; when the determining module 50 determines that the audio data is valid, receive and store the video data collected by the video collecting device;
  • the determining module 50 is further configured to determine whether the number of storage bits corresponding to the audio data is greater than a threshold; if yes, determine that the audio data is valid; if not, determine that the audio data is invalid.
  • the audio data when the video data is recorded, the audio data may not be collected due to the interruption of the connection of the audio collection device, so the validity of the audio data needs to be judged.
  • the system acquires the number of stored bits in the memory in which the audio data is stored, and can also be understood as the size of the amount of audio data. When the number of stored bits is greater than the threshold, for example, the threshold is 10, when the number of stored bits is greater than 10, it is determined that enough audio data has been collected, and the audio data is valid. Otherwise, the amount of audio data collected is small. The audio data may not be collected due to the interruption of the connection of the audio collection device.
  • the user may be prompted to check whether the audio collection device is connected and re-recorded.
  • the validity of the audio data is judged to avoid the problem that the audio data cannot be collected due to the interruption of the connection of the audio collection device.
  • FIG. 8 is a schematic block diagram of a third embodiment of an apparatus for recording an audio and video synchronization time stamp according to the present invention.
  • the embodiment includes the module of the embodiment shown in FIG. 7, and further includes an instruction receiving module 60 and a detecting module 70;
  • the instruction receiving module 60 is configured to receive an audio and video recording request
  • the detecting module 70 is configured to detect whether a video capturing device is currently connected.
  • the collecting module 10 is further configured to control the video collecting device to collect video data when the detecting module 70 detects that the currently existing video capturing device is accessed.
  • the user can click on the control of recording audio and video on the smart device to generate an audio and video recording request.
  • the user opens the MTV application, selects the MTV song that he likes, enters the recording interface of the self-singing self-play, and starts singing.
  • the system detects the level of the USB interface. When the hardware level of the USB interface is high, it indicates that the device is connected to the USB interface.
  • the system identifies the device connected to the USB and determines whether it is a video capture device, for example, whether there is a camera connection. USB. If the currently connected USB device is a video capture device, the video capture device is controlled to collect video data.
  • a prompt box is displayed to prompt the user to check whether the video collection device is connected and re-recorded.
  • the video capture device is determined to be accessed, and the problem that the video data cannot be collected due to the video capture device not being accessed is avoided.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Television Signal Processing For Recording (AREA)
  • Signal Processing For Digital Recording And Reproducing (AREA)
  • Computer Security & Cryptography (AREA)

Abstract

A method for recording an audio and video synchronization timestamp is disclosed in the present invention. The method comprises: collecting video data and audio data; obtaining the length a of the audio data, the frame rate b of the video data and a preset audio sampling rate h; according to the length a of the audio data, the frame rate b of the video data and the preset audio sampling rate h, calculating a new timestamp T, wherein T=a/[b/(a/h)]; according to the new timestamp, encoding and decoding the video data and audio data to obtain the video data and the audio data which are synchronous in time. An apparatus for recording an audio and video synchronization timestamp is also disclosed in the present invention. According to the present invention, a new timestamp is recalculated according to the audio length, the audio sampling rate and the video frame rate, the audio and video data are encoded and decoded according to the new timestamp, so that the encoded or decoded audio data and the video data are synchronous in time, and the problem that picture and sound are out of synchronization during playing caused by record interference is avoided effectively.

Description

录制音视频同步时间戳的方法和装置  Method and device for recording audio and video synchronization time stamp
技术领域Technical field
本发明涉及多媒体技术领域，特别涉及录制音视频同步时间戳的方法和装置。The present invention relates to the field of multimedia technologies, and in particular, to a method and apparatus for recording an audio and video synchronization time stamp.
背景技术Background technique
目前，智能电视、手机、电脑等智能设备的功能越来越丰富，在智能设备上连接摄像头和录音装置可用于录制用户的音视频文件，并可以在智能设备的显示屏上播放该音视频文件。但是，由于干扰问题的存在，在采集卡采集数据时出现音频与视频不同步。例如原来设置的视频帧率为25帧/秒，但实际采集的视频数据并不是40毫秒的时间间隔，而是50毫秒、60毫秒、甚至100毫秒不等的时间间隔，导致视频数据相对于音频数据滞后，这就给编码后给视频数据和音频数据打时间戳带来很大的困难，出现音视频回放时声音和画面不同步播放的问题。 At present, intelligent devices such as smart TVs, mobile phones, and computers are becoming more and more abundant. Connecting a camera and a recording device on a smart device can be used to record audio and video files of a user, and can play the audio and video files on the display of the smart device. . However, due to the interference problem, the audio and video are out of sync when the data is collected by the capture card. For example, the originally set video frame rate is 25 frames/second, but the actually collected video data is not a time interval of 40 milliseconds, but a time interval of 50 milliseconds, 60 milliseconds, or even 100 milliseconds, resulting in video data relative to audio. The data lags, which makes it difficult to time-stamp the video data and audio data after encoding, and the sound and the picture are not synchronized when the audio and video are played back.
发明内容Summary of the invention
本发明的主要目的为提供一种录制音视频同步时间戳的方法和装置，避免录制的音频数据和视频数据在播放时不同步的问题。The main object of the present invention is to provide a method and apparatus for recording audio and video synchronization time stamps, which avoids the problem that recorded audio data and video data are not synchronized during playback.
本发明提出一种录制音视频同步时间戳的方法，包括步骤：The invention provides a method for recording audio and video synchronization time stamps, comprising the steps of:
接收并存储音频采集设备采集的所述音频数据；Receiving and storing the audio data collected by the audio collection device;
判断所述音频数据对应的存储位的个数是否大于阈值；Determining whether the number of storage bits corresponding to the audio data is greater than a threshold;
如果是，则判定所述音频数据有效，接收并存储视频采集设备采集的所述视频数据；If yes, determining that the audio data is valid, and receiving and storing the video data collected by the video collection device;
如果否，则判定所述音频数据无效；If not, determining that the audio data is invalid;
获取所述音频数据的长度a、所述视频数据的帧率b以及预设的音频采样率h；Obtaining a length a of the audio data, a frame rate b of the video data, and a preset audio sampling rate h;
分别将所述音频数据的长度a和所述视频数据的帧率b放大n倍，所述n为预设放大倍数；The length a of the audio data and the frame rate b of the video data are respectively enlarged by n times, and the n is a preset magnification;
将放大后的所述音频数据的长度除以所述音频采样率h，得到每一赫兹的音频长度an/h；Dividing the length of the amplified audio data by the audio sampling rate h to obtain an audio length an/h per Hertz;
将放大后的所述视频数据的帧率除以每一赫兹的音频长度an/h，得到每一数据存储位的时长bn/(an/h)；Dividing the frame rate of the enlarged video data by the audio length an/h of each Hertz to obtain the duration bn/(an/h) of each data storage bit;
将所述音频数据的长度除以每一数据存储位的时长bn/(an/h)，获得新的时间戳T=a/[bn/(an/h)]；Dividing the length of the audio data by the duration bn/(an/h) of each data storage bit, obtaining a new timestamp T=a/[bn/(an/h)];
根据新的时间戳编码、解码所述视频数据和音频数据，获得时间同步的所述视频数据和音频数据。The video data and audio data that are time-synchronized are obtained by encoding, decoding, and decoding the video data and the audio data according to a new time stamp.
优选地，所述根据新的时间戳编码、解码所述视频数据和音频数据，获得时间同步的所述视频数据和音频数据的步骤之前还包括：Preferably, the step of encoding, decoding, and synchronizing the video data and the audio data according to the new time stamp further comprises:
判断所述音频数据对应的数据存储位的个数是否大于新的时间戳的数值；Determining whether the number of data storage bits corresponding to the audio data is greater than a value of a new timestamp;
如果是，则判定为新的时间戳有效，执行所述根据新的时间戳编码、解码所述视频数据和音频数据，获得时间同步的所述视频数据和音频数据的步骤；If yes, determining that the new timestamp is valid, performing the step of encoding the video data and the audio data according to the new timestamp, obtaining the time-synchronized video data and audio data;
如果否，则判定为新的时间戳无效。If no, it is determined that the new timestamp is invalid.
优选地，所述接收并存储音频采集设备采集的所述音频数据的步骤之前还包括：Preferably, the step of receiving and storing the audio data collected by the audio collection device further includes:
接收音视频录制请求；Receiving an audio and video recording request;
检测当前是否已有所述视频采集设备接入；Detect whether the video capture device is currently connected;
当检测到当前已有所述视频采集设备接入时，控制所述视频采集设备采集所述视频数据。When the video capture device is currently connected, the video capture device is controlled to collect the video data.
本发明还提出一种录制音视频同步时间戳的方法，包括步骤：The invention also provides a method for recording an audio and video synchronization time stamp, comprising the steps of:
采集视频数据和音频数据；Collecting video data and audio data;
获取所述音频数据的长度a、所述视频数据的帧率b以及预设的音频采样率h；Obtaining a length a of the audio data, a frame rate b of the video data, and a preset audio sampling rate h;
根据所述音频数据的长度a、所述视频数据的帧率b以及预设的音频采样率h，计算新的时间戳T=a/[b/(a/h)]；Calculating a new time stamp T=a/[b/(a/h)] according to the length a of the audio data, the frame rate b of the video data, and the preset audio sampling rate h;
根据新的时间戳编码、解码所述视频数据和音频数据，获得时间同步的所述视频数据和音频数据。The video data and audio data that are time-synchronized are obtained by encoding, decoding, and decoding the video data and the audio data according to a new time stamp.
优选地，所述根据所述音频数据的长度a、所述视频数据的帧率b以及预设的音频采样率h，计算新的时间戳T=a/[b/(a/h)]的步骤包括：Preferably, the calculating a new time stamp T=a/[b/(a/h)] according to the length a of the audio data, the frame rate b of the video data, and the preset audio sampling rate h The steps include:
分别将所述音频数据的长度a和所述视频数据的帧率b放大n倍，所述n为预设放大倍数；The length a of the audio data and the frame rate b of the video data are respectively enlarged by n times, and the n is a preset magnification;
将放大后的所述音频数据的长度除以所述音频采样率h，得到每一赫兹的音频长度an/h；Dividing the length of the amplified audio data by the audio sampling rate h to obtain an audio length an/h per Hertz;
将放大后的所述视频数据的帧率除以每一赫兹的音频长度an/h，得到每一数据存储位的时长bn/(an/h)；Dividing the frame rate of the enlarged video data by the audio length an/h of each Hertz to obtain the duration bn/(an/h) of each data storage bit;
将所述音频数据的长度除以每一数据存储位的时长bn/(an/h)，获得新的时间戳T=a/[bn/(an/h)]。The length of the audio data is divided by the duration bn/(an/h) of each data storage bit, and a new time stamp T=a/[bn/(an/h)] is obtained.
优选地，所述根据新的时间戳编码、解码所述视频数据和音频数据，获得时间同步的所述视频数据和音频数据的步骤之前还包括：Preferably, the step of encoding, decoding, and synchronizing the video data and the audio data according to the new time stamp further comprises:
判断所述音频数据对应的数据存储位的个数是否大于新的时间戳的数值；Determining whether the number of data storage bits corresponding to the audio data is greater than a value of a new timestamp;
如果是，则判定为新的时间戳有效，执行所述根据新的时间戳编码、解码所述视频数据和音频数据，获得时间同步的所述视频数据和音频数据的步骤；If yes, determining that the new timestamp is valid, performing the step of encoding the video data and the audio data according to the new timestamp, obtaining the time-synchronized video data and audio data;
如果否，则判定为新的时间戳无效。If no, it is determined that the new timestamp is invalid.
优选地，所述采集视频数据和音频数据的步骤包括：Preferably, the step of collecting video data and audio data comprises:
接收并存储音频采集设备采集的所述音频数据；Receiving and storing the audio data collected by the audio collection device;
判断所述音频数据对应的存储位的个数是否大于阈值；Determining whether the number of storage bits corresponding to the audio data is greater than a threshold;
如果是，则判定所述音频数据有效，接收并存储视频采集设备采集的所述视频数据；If yes, determining that the audio data is valid, and receiving and storing the video data collected by the video collection device;
如果否，则判定所述音频数据无效。If not, it is determined that the audio data is invalid.
优选地，所述接收并存储音频采集设备采集的所述音频数据的步骤之前还包括：Preferably, the step of receiving and storing the audio data collected by the audio collection device further includes:
接收音视频录制请求；Receiving an audio and video recording request;
检测当前是否已有所述视频采集设备接入；Detect whether the video capture device is currently connected;
当检测到当前已有所述视频采集设备接入时，控制所述视频采集设备采集所述视频数据。When the video capture device is currently connected, the video capture device is controlled to collect the video data.
本发明还提出一种录制音视频同步时间戳的装置，包括：The invention also provides an apparatus for recording an audio and video synchronization time stamp, comprising:
采集模块，用于采集视频数据和音频数据；An acquisition module for collecting video data and audio data;
获取模块，用于获取所述音频数据的长度a、所述视频数据的帧率b以及预设的音频采样率h；An acquiring module, configured to acquire a length a of the audio data, a frame rate b of the video data, and a preset audio sampling rate h;
计算模块，用于根据所述音频数据的长度a、所述视频数据的帧率b以及预设的音频采样率h，计算新的时间戳T=a/[b/(a/h)]；a calculation module, configured to calculate a new timestamp T=a/[b/(a/h)] according to a length a of the audio data, a frame rate b of the video data, and a preset audio sampling rate h;
编解码模块，用于根据新的时间戳编码、解码所述视频数据和音频数据，获得时间同步的所述视频数据和音频数据。And a codec module, configured to obtain the time-synchronized video data and audio data according to a new timestamp encoding, decoding the video data and audio data.
发明根据音频长度、音频采样率和视频帧率重新计算新的时间戳，并根据得到的新的时间戳来编码、解码音视频数据，使编码、解码后的音频数据和视频数据时间同步，在智能设备回放该音视频文件时，画面与声音同步播放，有效避免了因录制干扰造成播放时画面与声音不同步。The invention recalculates the new timestamp according to the audio length, the audio sampling rate and the video frame rate, and encodes and decodes the audio and video data according to the obtained new timestamp, so that the encoded and decoded audio data and the video data are time synchronized. When the smart device plays back the audio and video files, the screen and the sound are played synchronously, which effectively avoids the screen and the sound being out of sync during playback due to recording interference.
附图说明DRAWINGS
图1为本发明录制音视频同步时间戳的方法的第一实施例的流程图；1 is a flowchart of a first embodiment of a method for recording an audio and video synchronization time stamp according to the present invention;
图2为本发明录制音视频同步时间戳的方法的第二实施例的流程图；2 is a flowchart of a second embodiment of a method for recording an audio and video synchronization time stamp according to the present invention;
图3为本发明录制音视频同步时间戳的方法的第三实施例的流程图；3 is a flowchart of a third embodiment of a method for recording an audio and video synchronization time stamp according to the present invention;
图4为本发明录制音视频同步时间戳的方法的第四实施例的流程图；4 is a flowchart of a fourth embodiment of a method for recording an audio and video synchronization time stamp according to the present invention;
图5为本发明录制音视频同步时间戳的方法的第五实施例的流程图；5 is a flowchart of a fifth embodiment of a method for recording an audio and video synchronization time stamp according to the present invention;
图6为本发明录制音视频同步时间戳的装置的第一实施例的模块示意图；6 is a schematic block diagram of a first embodiment of an apparatus for recording an audio and video synchronization time stamp according to the present invention;
图7为本发明录制音视频同步时间戳的装置的第二实施例的模块示意图；7 is a schematic block diagram of a second embodiment of an apparatus for recording an audio and video synchronization time stamp according to the present invention;
图8为本发明录制音视频同步时间戳的装置的第三实施例的模块示意图。FIG. 8 is a schematic block diagram of a third embodiment of an apparatus for recording an audio and video synchronization time stamp according to the present invention.
本发明目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。The implementation, functional features, and advantages of the present invention will be further described in conjunction with the embodiments.
具体实施方式detailed description
应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
如图1所示，图1为本发明录制音视频同步时间戳的方法的第一实施例的流程图。本实施例提到的录制音视频同步时间戳的方法，包括：As shown in FIG. 1, FIG. 1 is a flowchart of a first embodiment of a method for recording an audio and video synchronization time stamp according to the present invention. The method for recording an audio and video synchronization time stamp mentioned in this embodiment includes:
步骤S10，采集视频数据和音频数据；Step S10, collecting video data and audio data;
本实施例的智能设备可包括电视机、手机、电脑等设备。智能设备可连接视频采集设备，例如摄像头，通过摄像头采集视频数据。智能设备还可连接音频采集设备，例如麦克风，通过麦克风采集音频数据。此外，智能设备还可以连接其他具有音频采集功能的设备，通过其他设备采集音频后将音频数据传输给智能设备。例如，智能设备为电视机，其他设备为手机，手机与电视机通过无线网络连接，手机通过话筒采集音频数据后，经无线网络将音频数据发送给电视机。The smart device of this embodiment may include a television, a mobile phone, a computer, and the like. The smart device can be connected to a video capture device, such as a camera, to capture video data through the camera. The smart device can also be connected to an audio capture device, such as a microphone, to capture audio data through the microphone. In addition, the smart device can also connect other devices with audio capture function, and transmit audio data to the smart device after collecting audio through other devices. For example, the smart device is a television, the other device is a mobile phone, and the mobile phone and the television are connected through a wireless network. After the mobile phone collects audio data through the microphone, the audio data is sent to the television through the wireless network.
步骤S20，获取音频数据的长度a、视频数据的帧率b以及预设的音频采样率h；Step S20, obtaining the length a of the audio data, the frame rate b of the video data, and the preset audio sampling rate h;
智能设备可根据视频采集设备的分辨率大小统计当前视频数据的帧率，还计算采集的音频数据的长度，并从存储器中获取预设的音频采样率。The smart device can count the frame rate of the current video data according to the resolution of the video capture device, calculate the length of the collected audio data, and obtain a preset audio sample rate from the memory.
步骤S30，根据音频数据的长度a、视频数据的帧率b以及预设的音频采样率h，计算新的时间戳T=a/[b/(a/h)]；Step S30, calculating a new time stamp T=a/[b/(a/h)] according to the length a of the audio data, the frame rate b of the video data, and the preset audio sampling rate h;
根据适配时间戳算法计算新的时间戳T=a/[b/(a/h)]，其中，a为音频数据的长度，b为视频数据的帧率，h预设的音频采样率。为了简化计算，可将b四舍五入为整数，省去小数点之后的数。但为了精确计算结果，也可保留b的小数点后的数值，使获得的时间戳更加准确。Calculate a new timestamp T=a/[b/(a/h)] according to the adaptive timestamp algorithm, where a is the length of the audio data, b is the frame rate of the video data, and h is the preset audio sampling rate. To simplify the calculation, b can be rounded to an integer, eliminating the number after the decimal point. However, in order to accurately calculate the result, the value after the decimal point of b can also be retained, so that the obtained time stamp is more accurate.
步骤S40，根据新的时间戳编码、解码视频数据和音频数据，获得时间同步的视频数据和音频数据。Step S40, encoding and decoding the video data and the audio data according to the new time stamp, and obtaining time-synchronized video data and audio data.
将得到的新的时间戳输入编解码模块，使编解码模块根据新的时间戳编码、解码视频数据和音频数据，最终获得时间同步的视频数据和音频数据。在智能设备回放该音视频文件时，画面与声音同步。The obtained new time stamp is input to the codec module, so that the codec module encodes and decodes the video data and the audio data according to the new time stamp, and finally obtains time-synchronized video data and audio data. When the smart device plays back the audio and video files, the picture is synchronized with the sound.
本实施例根据音频长度、音频采样率和视频帧率重新计算新的时间戳，并根据得到的新的时间戳来编码、解码音视频数据，使编码、解码后的音频数据和视频数据时间同步，在智能设备回放该音视频文件时，画面与声音同步播放，有效避免了因录制干扰造成播放时画面与声音不同步。In this embodiment, a new timestamp is recalculated according to the audio length, the audio sampling rate, and the video frame rate, and the audio and video data is encoded and decoded according to the obtained new timestamp, so that the encoded and decoded audio data and the video data are time synchronized. When the smart device plays back the audio and video files, the picture and the sound are played synchronously, which effectively avoids the picture and sound being out of sync during playback due to recording interference.
如图2所示，图2为本发明录制音视频同步时间戳的方法的第二实施例的流程图。本实施例包括图1所示实施例的步骤，其中的步骤S30包括：As shown in FIG. 2, FIG. 2 is a flowchart of a second embodiment of a method for recording an audio and video synchronization time stamp according to the present invention. This embodiment includes the steps of the embodiment shown in FIG. 1, wherein step S30 includes:
步骤S31，分别将音频数据的长度a和视频数据的帧率b放大n倍，n为预设放大倍数；Step S31, respectively, the length a of the audio data and the frame rate b of the video data are enlarged by n times, where n is a preset magnification;
步骤S32，将放大后的音频数据的长度除以音频采样率h，得到每一赫兹的音频长度an/h；Step S32, dividing the length of the amplified audio data by the audio sampling rate h, to obtain an audio length an/h of each Hertz;
步骤S33，将放大后的视频数据的帧率除以每一赫兹的音频长度an/h，得到每一数据存储位的时长bn/(an/h)；Step S33, dividing the frame rate of the amplified video data by the audio length an/h of each Hertz, and obtaining the duration bn/(an/h) of each data storage bit;
步骤S34，将音频数据的长度除以每一数据存储位的时长bn/(an/h)，获得新的时间戳T=a/[bn/(an/h)]。In step S34, the length of the audio data is divided by the duration bn/(an/h) of each data storage bit, and a new time stamp T=a/[bn/(an/h)] is obtained.
本实施例中，为了让最终获得的时间戳更加精确，在计算时将视频数据的帧率b放大n倍，n的取值可以为10的幂次方，例如n=1000，使帧率b的值精确到了小数点后3位，则获得的时间戳精确到了毫秒级。同时，为了确保最终结果的准确性，还将音频数据的长度a也放大n倍，使得在计算过程中的数值在放大了n倍后又缩小n倍。在计算时，将放大后的音频长度除以音频采样率，得到每单位赫兹的音频长度；再将放大后的视频帧率除以每一赫兹的音频长度，得到每一数据存储位的时长，即存储音频数据的存储器中每一位的时长，比如，音频数据在采集时为模拟信号，将其经过ADC采样后，模拟波形变成离散的数字信号，离散点的个数就是音频数据对应的数据存储位的个数，1s中采样的音频数据的采样点个数为2个，则每一存储位的时长为0.5s；再将音频长度除以每一数据存储位的时长，获得新的时间戳。本实施例在计算时间戳时，将帧率b放大n倍，使获得的时间戳更加精确，根据得到的时间戳来编码、解码音视频数据，使编码、解码后的音频数据和视频数据时间同步，有效避免了因录制干扰造成播放时画面与声音不同步。In this embodiment, in order to make the finally obtained time stamp more accurate, the frame rate b of the video data is enlarged by n times in the calculation, and the value of n may be a power of 10, for example, n=1000, so that the frame rate b The value is accurate to 3 digits after the decimal point, and the obtained timestamp is accurate to the millisecond level. At the same time, in order to ensure the accuracy of the final result, the length a of the audio data is also enlarged by n times, so that the value in the calculation process is further reduced by n times after being enlarged by n times. In the calculation, the length of the amplified audio is divided by the audio sampling rate to obtain the audio length per unit hertz; and the amplified video frame rate is divided by the audio length per hertz to obtain the duration of each data storage bit. That is, the length of each bit in the memory in which the audio data is stored. For example, the audio data is an analog signal at the time of acquisition, and after being sampled by the ADC, the analog waveform becomes a discrete digital signal, and the number of discrete points is corresponding to the audio data. The number of data storage bits, the number of sampling points of the audio data sampled in 1s is two, and the length of each storage bit is 0.5 s; then the audio length is divided by the length of each data storage bit to obtain a new one. Timestamp. In the embodiment, when calculating the timestamp, the frame rate b is enlarged by n times, so that the obtained timestamp is more accurate, and the audio and video data is encoded and decoded according to the obtained timestamp, so that the encoded and decoded audio data and video data time are obtained. Synchronization effectively prevents the picture and sound from being out of sync during playback due to recording interference.
如图3所示，图3为本发明录制音视频同步时间戳的方法的第三实施例的流程图。本实施例包括图1所示实施例的步骤，步骤S40之前还包括：As shown in FIG. 3, FIG. 3 is a flowchart of a third embodiment of a method for recording an audio and video synchronization time stamp according to the present invention. This embodiment includes the steps of the embodiment shown in FIG. 1. Before step S40, the method further includes:
步骤S51，判断音频数据对应的数据存储位的个数是否大于新的时间戳的数值；如果是，则执行步骤S52；如果否，则执行步骤S53；Step S51, it is determined whether the number of data storage bits corresponding to the audio data is greater than the value of the new timestamp; if yes, step S52 is performed; if not, step S53 is performed;
步骤S52，判定为新的时间戳有效，执行步骤S40；Step S52, it is determined that the new timestamp is valid, and step S40 is performed;
步骤S53，判定为新的时间戳无效。In step S53, it is determined that the new time stamp is invalid.
本实施例还对时间戳的有效性进行判断。音频采集设备采集到的音频数据为模拟信号，经过ADC采样后，模拟波形变成离散的数字信号，离散采样点的个数，即数据存储位的个数。由于音频数据存储的个数是通过ADC模数转换后的个数，ADC模数转换后的个数一定会大于转换前的个数，因此，如果存储个数小于或者等于新的时间戳，则该时间戳都是无效的。例如，计算得到的时间戳为4000ms，音频数据对应的存储位的个数为96个，则时间戳无效，可弹出提示窗提示用户重新录制。此外，还可以在采集音频视频数据的过程中，设定一个长度阈值或时间阈值，在采集的数据到达这个长度阈值或时间阈值后，根据已采集的部分数据计算时间戳，并判断当前计算的时间戳是否有效，如果无效，则停止后续的数据采集，提示用户重新录制。本实施例对时间戳的有效性进行判断，避免了采用无效的时间戳编码、解码音视频数据造成播放的声音和画面仍然不同步的问题。This embodiment also judges the validity of the time stamp. The audio data collected by the audio acquisition device is an analog signal. After sampling by the ADC, the analog waveform becomes a discrete digital signal, and the number of discrete sampling points, that is, the number of data storage bits. Since the number of audio data storage is the number of ADC analog-to-digital conversions, the number of ADC analog-to-digital conversions must be greater than the number of conversions. Therefore, if the number of storages is less than or equal to the new timestamp, Then the timestamp is invalid. For example, if the calculated timestamp is 4000ms, and the number of storage bits corresponding to the audio data is 96, the timestamp is invalid, and a prompt window may pop up prompting the user to re-record. In addition, during the process of collecting audio and video data, a length threshold or a time threshold may be set. After the collected data reaches the length threshold or the time threshold, the time stamp is calculated according to the collected partial data, and the current calculation is determined. Whether the timestamp is valid. If it is invalid, it stops the subsequent data collection and prompts the user to re-record. In this embodiment, the validity of the timestamp is determined, and the problem that the played sound and the picture are still out of synchronization caused by invalid timestamp coding and decoding of the audio and video data is avoided.
如图4所示，图4为本发明录制音视频同步时间戳的方法的第四实施例的流程图。本实施例包括图1所示实施例的步骤，其中的步骤S10包括：As shown in FIG. 4, FIG. 4 is a flowchart of a fourth embodiment of a method for recording an audio and video synchronization time stamp according to the present invention. This embodiment includes the steps of the embodiment shown in FIG. 1, wherein step S10 includes:
步骤S11，接收并存储音频采集设备采集的音频数据；Step S11, receiving and storing audio data collected by the audio collection device;
步骤S12，判断音频数据对应的存储位的个数是否大于阈值；如果是，则执行步骤S13；如果否，则执行步骤S14；Step S12, it is determined whether the number of storage bits corresponding to the audio data is greater than a threshold; if yes, step S13 is performed; if not, step S14 is performed;
步骤S13，判定音频数据有效，接收并存储视频采集设备采集的视频数据，并执行步骤S20；Step S13, determining that the audio data is valid, receiving and storing the video data collected by the video capture device, and performing step S20;
步骤S14，判定音频数据无效。In step S14, it is determined that the audio data is invalid.
本实施例在录制视频数据时，可能因音频采集设备连接中断造成音频数据无法采集，因此需对音频数据的有效性进行判断。***在存储音频数据的存储器中获取已存储的位的个数，也可理解为音频数据量的大小。当已存储的位的个数大于阈值时，例如阈值为10，当已存储的位的个数大于10时，则判定为已采集到足够多的音频数据，该音频数据有效。否则，采集的音频数据量较少，可能是音频采集设备连接中断造成音频数据无法采集，此时判定为音频数据无效，可提示用户检查音频采集设备是否连接，并重新录制。本实施例对音频数据的有效性进行判断，避免因音频采集设备连接中断造成音频数据无法采集的问题。In the embodiment, when the video data is recorded, the audio data may not be collected due to the interruption of the connection of the audio collection device, so the validity of the audio data needs to be judged. The system acquires the number of stored bits in the memory in which the audio data is stored, and can also be understood as the size of the amount of audio data. When the number of stored bits is greater than the threshold, for example, the threshold is 10, when the number of stored bits is greater than 10, it is determined that enough audio data has been collected, and the audio data is valid. Otherwise, the amount of audio data collected is small. The audio data may not be collected due to the interruption of the connection of the audio collection device. At this time, it is determined that the audio data is invalid, and the user may be prompted to check whether the audio collection device is connected and re-recorded. In this embodiment, the validity of the audio data is judged to avoid the problem that the audio data cannot be collected due to the interruption of the connection of the audio collection device.
如图5所示，图5为本发明录制音视频同步时间戳的方法的第五实施例的流程图。本实施例包括图4所示实施例的步骤，步骤S11之前还包括：As shown in FIG. 5, FIG. 5 is a flowchart of a fifth embodiment of a method for recording an audio and video synchronization time stamp according to the present invention. This embodiment includes the steps of the embodiment shown in FIG. 4, and before step S11, the method further includes:
步骤S15，接收音视频录制请求；Step S15, receiving an audio and video recording request;
步骤S16，检测当前是否已有视频采集设备接入；Step S16: detecting whether the video capture device is currently connected;
步骤S17，当检测到当前已有视频采集设备接入时，控制视频采集设备采集视频数据。In step S17, when it is detected that the current video capture device is accessed, the video capture device is controlled to collect video data.
本实施例中，用户可在智能设备上点击录制音视频的控件，生成音视频录制请求。例如，用户打开MTV应用，选择自己喜欢的MTV歌曲，进入自唱自演的录制界面，开始演唱。***对USB接口的电平进行检测，当USB接口的硬件电平为高电平时，说明有设备连接USB接口，***对连接USB的设备进行识别，判断是否为视频采集设备，例如是否有摄像头连接USB。如果当前连接的USB设备为视频采集设备，则控制该视频采集设备采集视频数据。否则，判定视频采集设备未接入，可弹出提示框提示用户检查视频采集设备是否连接，并重新录制。本实施例对视频采集设备是否接入进行判断，避免因视频采集设备未接入造成视频数据无法采集的问题。In this embodiment, the user can click on the control of recording audio and video on the smart device to generate an audio and video recording request. For example, the user opens the MTV application, selects the MTV song that he likes, enters the recording interface of the self-singing self-play, and starts singing. The system detects the level of the USB interface. When the hardware level of the USB interface is high, it indicates that the device is connected to the USB interface. The system identifies the device connected to the USB and determines whether it is a video capture device, for example, whether there is a camera connection. USB. If the currently connected USB device is a video capture device, the video capture device is controlled to collect video data. Otherwise, it is determined that the video collection device is not connected, and a prompt box is displayed to prompt the user to check whether the video collection device is connected and re-recorded. In this embodiment, the video capture device is determined to be accessed, and the problem that the video data cannot be collected due to the video capture device not being accessed is avoided.
如图6所示，图6为本发明录制音视频同步时间戳的装置的第一实施例的模块示意图。本实施例提到的录制音视频同步时间戳的装置，包括：As shown in FIG. 6, FIG. 6 is a schematic block diagram of a first embodiment of an apparatus for recording an audio and video synchronization time stamp according to the present invention. The device for recording audio and video synchronization time stamp mentioned in this embodiment includes:
采集模块10，用于采集视频数据和音频数据；The collecting module 10 is configured to collect video data and audio data;
获取模块20，用于获取音频数据的长度a、视频数据的帧率b以及预设的音频采样率h；The obtaining module 20 is configured to obtain a length a of the audio data, a frame rate b of the video data, and a preset audio sampling rate h;
计算模块30，用于根据音频数据的长度a、视频数据的帧率b以及预设的音频采样率h，计算新的时间戳T=a/[b/(a/h)]；The calculating module 30 is configured to calculate a new timestamp T=a/[b/(a/h)] according to the length a of the audio data, the frame rate b of the video data, and the preset audio sampling rate h;
编解码模块40，用于根据新的时间戳编码、解码视频数据和音频数据，获得时间同步的视频数据和音频数据。The codec module 40 is configured to obtain time-synchronized video data and audio data according to a new time stamp encoding, decoding video data and audio data.
本实施例的智能设备可包括电视机、手机、电脑等设备。智能设备可连接视频采集设备，例如摄像头，通过摄像头采集视频数据。智能设备还可连接音频采集设备，例如麦克风，通过麦克风采集音频数据。此外，智能设备还可以连接其他具有音频采集功能的设备，通过其他设备采集音频后将音频数据传输给智能设备。例如，智能设备为电视机，其他设备为手机，手机与电视机通过无线网络连接，手机通过话筒采集音频数据后，经无线网络将音频数据发送给电视机。The smart device of this embodiment may include a television, a mobile phone, a computer, and the like. The smart device can be connected to a video capture device, such as a camera, to capture video data through the camera. The smart device can also be connected to an audio capture device, such as a microphone, to capture audio data through the microphone. In addition, the smart device can also connect other devices with audio capture function, and transmit audio data to the smart device after collecting audio through other devices. For example, the smart device is a television, the other device is a mobile phone, and the mobile phone and the television are connected through a wireless network. After the mobile phone collects audio data through the microphone, the audio data is sent to the television through the wireless network.
智能设备可根据视频采集设备的分辨率大小统计当前视频数据的帧率，还计算采集的音频数据的长度，并从存储器中获取预设的音频采样率。根据适配时间戳算法计算新的时间戳T=a/[b/(a/h)]，使视频数据的时间戳与计算的新的时间戳同步。其中，a为音频数据的长度，b为视频数据的帧率，h预设的音频采样率。为了简化计算，可将b四舍五入为整数，省去小数点之后的数。但为了精确计算结果，也可保留b的小数点后的数值，使获得的时间戳更加准确。The smart device can count the frame rate of the current video data according to the resolution of the video capture device, calculate the length of the collected audio data, and obtain a preset audio sample rate from the memory. A new timestamp T=a/[b/(a/h)] is calculated according to the adapted timestamp algorithm, so that the timestamp of the video data is synchronized with the calculated new timestamp. Where a is the length of the audio data, b is the frame rate of the video data, and h is the preset audio sampling rate. To simplify the calculation, b can be rounded to an integer, eliminating the number after the decimal point. However, in order to accurately calculate the result, the value after the decimal point of b can also be retained, so that the obtained time stamp is more accurate.
将得到的新的时间戳输入编解码模块，使编解码模块根据新的时间戳编码、解码视频数据和音频数据，最终获得时间同步的视频数据和音频数据。在智能设备回放该音视频文件时，画面与声音同步。The obtained new time stamp is input to the codec module, so that the codec module encodes and decodes the video data and the audio data according to the new time stamp, and finally obtains time-synchronized video data and audio data. When the smart device plays back the audio and video files, the picture is synchronized with the sound.
本实施例根据音频长度、音频采样率和视频帧率重新计算新的时间戳，并根据得到的新的时间戳来编码、解码音视频数据，使编码、解码后的音频数据和视频数据时间同步，在智能设备回放该音视频文件时，画面与声音同步播放，有效避免了因录制干扰造成播放时画面与声音不同步。In this embodiment, a new timestamp is recalculated according to the audio length, the audio sampling rate, and the video frame rate, and the audio and video data is encoded and decoded according to the obtained new timestamp, so that the encoded and decoded audio data and the video data are time synchronized. When the smart device plays back the audio and video files, the picture and the sound are played synchronously, which effectively avoids the picture and sound being out of sync during playback due to recording interference.
进一步的，计算模块30还用于：Further, the calculation module 30 is further configured to:
分别将音频数据的长度a和视频数据的帧率b放大n倍，n为预设放大倍数；Enlarging the length a of the audio data and the frame rate b of the video data by n times, where n is a preset magnification;
将放大后的音频数据的长度除以音频采样率h，得到每一赫兹的音频长度an/h；Dividing the length of the amplified audio data by the audio sampling rate h to obtain an audio length an/h per Hertz;
将放大后的视频数据的帧率除以每一赫兹的音频长度an/h，得到每一数据存储位的时长bn/(an/h)；Dividing the frame rate of the amplified video data by the audio length an/h per Hertz, and obtaining the duration bn/(an/h) of each data storage bit;
将音频数据的长度除以每一数据存储位的时长bn/(an/h)，获得新的时间戳T=a/[bn/(an/h)]。The length of the audio data is divided by the duration bn/(an/h) of each data storage bit, and a new time stamp T=a/[bn/(an/h)] is obtained.
本实施例中，为了让最终获得的时间戳更加精确，在计算时将视频数据的帧率b放大n倍，n的取值可以为10的幂次方，例如n=1000，使帧率b的值精确到了小数点后3位，则获得的时间戳精确到了毫秒级。同时，为了确保最终结果的准确性，还将音频数据的长度a也放大n倍，使得在计算过程中的数值在放大了n倍后又缩小n倍。在计算时，将放大后的音频长度除以音频采样率，得到每单位赫兹的音频长度；再将放大后的视频帧率除以每一赫兹的音频长度，得到每一数据存储位的时长，即存储音频数据的存储器中每一位的时长，比如，音频数据在采集时为模拟信号，将其经过ADC采样后，模拟波形变成离散的数字信号，离散点的个数就是音频数据对应的数据存储位的个数，1s中采样的音频数据的采样点个数为2个，则每一存储位的时长为0.5s；再将音频长度除以每一数据存储位的时长，获得新的时间戳。本实施例在计算时间戳时，将帧率b放大n倍，使获得的时间戳更加精确，根据得到的时间戳来编码、解码音视频数据，使编码、解码后的音频数据和视频数据时间同步，有效避免了因录制干扰造成播放时画面与声音不同步。In this embodiment, in order to make the finally obtained time stamp more accurate, the frame rate b of the video data is enlarged by n times in the calculation, and the value of n may be a power of 10, for example, n=1000, so that the frame rate b The value is accurate to 3 digits after the decimal point, and the obtained timestamp is accurate to the millisecond level. At the same time, in order to ensure the accuracy of the final result, the length a of the audio data is also enlarged by n times, so that the value in the calculation process is further reduced by n times after being enlarged by n times. In the calculation, the length of the amplified audio is divided by the audio sampling rate to obtain the audio length per unit hertz; and the amplified video frame rate is divided by the audio length per hertz to obtain the duration of each data storage bit. That is, the length of each bit in the memory in which the audio data is stored. For example, the audio data is an analog signal at the time of acquisition, and after being sampled by the ADC, the analog waveform becomes a discrete digital signal, and the number of discrete points is corresponding to the audio data. The number of data storage bits, the number of sampling points of the audio data sampled in 1s is two, and the length of each storage bit is 0.5 s; then the audio length is divided by the length of each data storage bit to obtain a new one. Timestamp. In the embodiment, when calculating the timestamp, the frame rate b is enlarged by n times, so that the obtained timestamp is more accurate, and the audio and video data is encoded and decoded according to the obtained timestamp, so that the encoded and decoded audio data and video data time are obtained. Synchronization effectively prevents the picture and sound from being out of sync during playback due to recording interference.
如图7所示，图7为本发明录制音视频同步时间戳的装置的第二实施例的模块示意图。本实施例包括图6所示实施例的模块，还包括判断模块50，用于：As shown in FIG. 7, FIG. 7 is a schematic block diagram of a second embodiment of an apparatus for recording an audio and video synchronization time stamp according to the present invention. The embodiment includes the module of the embodiment shown in FIG. 6, and further includes a determining module 50, configured to:
判断音频数据对应的数据存储位的个数是否大于新的时间戳的数值；Determining whether the number of data storage bits corresponding to the audio data is greater than a value of the new timestamp;
如果是，则判定为新的时间戳有效；If yes, it is determined that the new timestamp is valid;
如果否，则判定为新的时间戳无效。If no, it is determined that the new timestamp is invalid.
本实施例还对时间戳的有效性进行判断。音频采集设备采集到的音频数据为模拟信号，经过ADC采样后，模拟波形变成离散的数字信号，离散采样点的个数，即数据存储位的个数。由于音频数据存储的个数是通过ADC模数转换后的个数，ADC模数转换后的个数一定会大于转换前的个数，因此，如果存储个数小于或者等于新的时间戳，则该时间戳都是无效的。例如，计算得到的时间戳为4000ms，音频数据对应的存储位的个数为96个，则时间戳无效，可弹出提示窗提示用户重新录制。此外，还可以在采集音频视频数据的过程中，设定一个长度阈值或时间阈值，在采集的数据到达这个长度阈值或时间阈值后，根据已采集的部分数据计算时间戳，并判断当前计算的时间戳是否有效，如果无效，则停止后续的数据采集，提示用户重新录制。本实施例对时间戳的有效性进行判断，避免了采用无效的时间戳编码、解码音视频数据造成播放的声音和画面仍然不同步的问题。This embodiment also judges the validity of the time stamp. The audio data collected by the audio acquisition device is an analog signal. After sampling by the ADC, the analog waveform becomes a discrete digital signal, and the number of discrete sampling points, that is, the number of data storage bits. Since the number of audio data storage is the number of ADC analog-to-digital conversions, the number of ADC analog-to-digital conversions must be greater than the number of conversions. Therefore, if the number of storages is less than or equal to the new timestamp, Then the timestamp is invalid. For example, if the calculated timestamp is 4000ms, and the number of storage bits corresponding to the audio data is 96, the timestamp is invalid, and a prompt window may pop up prompting the user to re-record. In addition, during the process of collecting audio and video data, a length threshold or a time threshold may be set. After the collected data reaches the length threshold or the time threshold, the time stamp is calculated according to the collected partial data, and the current calculation is determined. Whether the timestamp is valid. If it is invalid, it stops the subsequent data collection and prompts the user to re-record. In this embodiment, the validity of the timestamp is determined, and the problem that the played sound and the picture are still out of synchronization caused by invalid timestamp coding and decoding of the audio and video data is avoided.
进一步的，采集模块10还用于，接收并存储音频采集设备采集的音频数据；当判断模块50判定音频数据有效时，接收并存储视频采集设备采集的视频数据；Further, the collecting module 10 is further configured to receive and store the audio data collected by the audio collecting device; when the determining module 50 determines that the audio data is valid, receive and store the video data collected by the video collecting device;
判断模块50还用于，判断音频数据对应的存储位的个数是否大于阈值；如果是，则判定音频数据有效；如果否，则判定音频数据无效。The determining module 50 is further configured to determine whether the number of storage bits corresponding to the audio data is greater than a threshold; if yes, determine that the audio data is valid; if not, determine that the audio data is invalid.
本实施例在录制视频数据时，可能因音频采集设备连接中断造成音频数据无法采集，因此需对音频数据的有效性进行判断。***在存储音频数据的存储器中获取已存储的位的个数，也可理解为音频数据量的大小。当已存储的位的个数大于阈值时，例如阈值为10，当已存储的位的个数大于10时，则判定为已采集到足够多的音频数据，该音频数据有效。否则，采集的音频数据量较少，可能是音频采集设备连接中断造成音频数据无法采集，此时判定为音频数据无效，可提示用户检查音频采集设备是否连接，并重新录制。本实施例对音频数据的有效性进行判断，避免因音频采集设备连接中断造成音频数据无法采集的问题。In the embodiment, when the video data is recorded, the audio data may not be collected due to the interruption of the connection of the audio collection device, so the validity of the audio data needs to be judged. The system acquires the number of stored bits in the memory in which the audio data is stored, and can also be understood as the size of the amount of audio data. When the number of stored bits is greater than the threshold, for example, the threshold is 10, when the number of stored bits is greater than 10, it is determined that enough audio data has been collected, and the audio data is valid. Otherwise, the amount of audio data collected is small. The audio data may not be collected due to the interruption of the connection of the audio collection device. At this time, it is determined that the audio data is invalid, and the user may be prompted to check whether the audio collection device is connected and re-recorded. In this embodiment, the validity of the audio data is judged to avoid the problem that the audio data cannot be collected due to the interruption of the connection of the audio collection device.
如图8所示，图8为本发明录制音视频同步时间戳的装置的第三实施例的模块示意图。本实施例包括图7所示实施例的模块，还包括指令接收模块60和检测模块70；As shown in FIG. 8, FIG. 8 is a schematic block diagram of a third embodiment of an apparatus for recording an audio and video synchronization time stamp according to the present invention. The embodiment includes the module of the embodiment shown in FIG. 7, and further includes an instruction receiving module 60 and a detecting module 70;
指令接收模块60，用于接收音视频录制请求；The instruction receiving module 60 is configured to receive an audio and video recording request;
检测模块70，用于检测当前是否已有视频采集设备接入；The detecting module 70 is configured to detect whether a video capturing device is currently connected.
采集模块10还用于，当检测模块70检测到当前已有视频采集设备接入时，控制视频采集设备采集视频数据。The collecting module 10 is further configured to control the video collecting device to collect video data when the detecting module 70 detects that the currently existing video capturing device is accessed.
本实施例中，用户可在智能设备上点击录制音视频的控件，生成音视频录制请求。例如，用户打开MTV应用，选择自己喜欢的MTV歌曲，进入自唱自演的录制界面，开始演唱。***对USB接口的电平进行检测，当USB接口的硬件电平为高电平时，说明有设备连接USB接口，***对连接USB的设备进行识别，判断是否为视频采集设备，例如是否有摄像头连接USB。如果当前连接的USB设备为视频采集设备，则控制该视频采集设备采集视频数据。否则，判定视频采集设备未接入，可弹出提示框提示用户检查视频采集设备是否连接，并重新录制。本实施例对视频采集设备是否接入进行判断，避免因视频采集设备未接入造成视频数据无法采集的问题。In this embodiment, the user can click on the control of recording audio and video on the smart device to generate an audio and video recording request. For example, the user opens the MTV application, selects the MTV song that he likes, enters the recording interface of the self-singing self-play, and starts singing. The system detects the level of the USB interface. When the hardware level of the USB interface is high, it indicates that the device is connected to the USB interface. The system identifies the device connected to the USB and determines whether it is a video capture device, for example, whether there is a camera connection. USB. If the currently connected USB device is a video capture device, the video capture device is controlled to collect video data. Otherwise, it is determined that the video collection device is not connected, and a prompt box is displayed to prompt the user to check whether the video collection device is connected and re-recorded. In this embodiment, the video capture device is determined to be accessed, and the problem that the video data cannot be collected due to the video capture device not being accessed is avoided.
以上所述仅为本发明的优选实施例，并非因此限制本发明的专利范围，凡是利用本发明说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本发明的专利保护范围内。The above is only the preferred embodiment of the present invention, and is not intended to limit the scope of the invention, and the equivalent structure or equivalent process transformations made by the description of the invention and the drawings are directly or indirectly applied to other related The technical field is equally included in the scope of patent protection of the present invention.

Claims (17)

  1. 一种录制音视频同步时间戳的方法，其特征在于，包括步骤： A method for recording audio and video synchronization time stamps, comprising the steps of:
    接收并存储音频采集设备采集的所述音频数据；Receiving and storing the audio data collected by the audio collection device;
    判断所述音频数据对应的存储位的个数是否大于阈值；Determining whether the number of storage bits corresponding to the audio data is greater than a threshold;
    如果是，则判定所述音频数据有效，接收并存储视频采集设备采集的所述视频数据；If yes, determining that the audio data is valid, and receiving and storing the video data collected by the video collection device;
    如果否，则判定所述音频数据无效；If not, determining that the audio data is invalid;
    获取所述音频数据的长度a、所述视频数据的帧率b以及预设的音频采样率h；Obtaining a length a of the audio data, a frame rate b of the video data, and a preset audio sampling rate h;
    分别将所述音频数据的长度a和所述视频数据的帧率b放大n倍，所述n为预设放大倍数；The length a of the audio data and the frame rate b of the video data are respectively enlarged by n times, and the n is a preset magnification;
    将放大后的所述音频数据的长度除以所述音频采样率h，得到每一赫兹的音频长度an/h；Dividing the length of the amplified audio data by the audio sampling rate h to obtain an audio length an/h per Hertz;
    将放大后的所述视频数据的帧率除以每一赫兹的音频长度an/h，得到每一数据存储位的时长bn/(an/h)；Dividing the frame rate of the enlarged video data by the audio length an/h of each Hertz to obtain the duration bn/(an/h) of each data storage bit;
    将所述音频数据的长度除以每一数据存储位的时长bn/(an/h)，获得新的时间戳T=a/[bn/(an/h)]；Dividing the length of the audio data by the duration bn/(an/h) of each data storage bit, obtaining a new timestamp T=a/[bn/(an/h)];
    根据新的时间戳编码、解码所述视频数据和音频数据，获得时间同步的所述视频数据和音频数据。The video data and audio data that are time-synchronized are obtained by encoding, decoding, and decoding the video data and the audio data according to a new time stamp.
  2. 如权利要求1所述的录制音视频同步时间戳的方法，其特征在于，所述根据新的时间戳编码、解码所述视频数据和音频数据，获得时间同步的所述视频数据和音频数据的步骤之前还包括： The method for recording an audio/video synchronization time stamp according to claim 1, wherein said encoding the video data and the audio data according to a new time stamp to obtain time-synchronized video data and audio data The steps also include:
    判断所述音频数据对应的数据存储位的个数是否大于新的时间戳的数值；Determining whether the number of data storage bits corresponding to the audio data is greater than a value of a new timestamp;
    如果是，则判定为新的时间戳有效，执行所述根据新的时间戳编码、解码所述视频数据和音频数据，获得时间同步的所述视频数据和音频数据的步骤；If yes, determining that the new timestamp is valid, performing the step of encoding the video data and the audio data according to the new timestamp, obtaining the time-synchronized video data and audio data;
    如果否，则判定为新的时间戳无效。If no, it is determined that the new timestamp is invalid.
  3. 权利要求1所述的录制音视频同步时间戳的方法，其特征在于，所述接收并存储音频采集设备采集的所述音频数据的步骤之前还包括：The method for recording an audio and video synchronization time stamp according to claim 1, wherein the step of receiving and storing the audio data collected by the audio collection device further comprises:
    接收音视频录制请求；Receiving an audio and video recording request;
    检测当前是否已有所述视频采集设备接入；Detect whether the video capture device is currently connected;
    当检测到当前已有所述视频采集设备接入时，控制所述视频采集设备采集所述视频数据。When the video capture device is currently connected, the video capture device is controlled to collect the video data.
  4. 种录制音视频同步时间戳的方法，其特征在于，包括步骤：A method for recording an audio and video synchronization time stamp, comprising the steps of:
    采集视频数据和音频数据；Collecting video data and audio data;
    获取所述音频数据的长度a、所述视频数据的帧率b以及预设的音频采样率h；Obtaining a length a of the audio data, a frame rate b of the video data, and a preset audio sampling rate h;
    根据所述音频数据的长度a、所述视频数据的帧率b以及预设的音频采样率h，计算新的时间戳T=a/[b/(a/h)]；Calculating a new time stamp T=a/[b/(a/h)] according to the length a of the audio data, the frame rate b of the video data, and the preset audio sampling rate h;
    根据新的时间戳编码、解码所述视频数据和音频数据，获得时间同步的所述视频数据和音频数据。The video data and audio data that are time-synchronized are obtained by encoding, decoding, and decoding the video data and the audio data according to a new time stamp.
  5. 权利要求4所述的录制音视频同步时间戳的方法，其特征在于，所述根据所述音频数据的长度a、所述视频数据的帧率b以及预设的音频采样率h，计算新的时间戳T=a/[b/(a/h)]的步骤包括：The method for recording an audio/video synchronization time stamp according to claim 4, wherein the calculating a new one according to a length a of the audio data, a frame rate b of the video data, and a preset audio sampling rate h The steps of the time stamp T=a/[b/(a/h)] include:
    分别将所述音频数据的长度a和所述视频数据的帧率b放大n倍，所述n为预设放大倍数；The length a of the audio data and the frame rate b of the video data are respectively enlarged by n times, and the n is a preset magnification;
    将放大后的所述音频数据的长度除以所述音频采样率h，得到每一赫兹的音频长度an/h；Dividing the length of the amplified audio data by the audio sampling rate h to obtain an audio length an/h per Hertz;
    将放大后的所述视频数据的帧率除以每一赫兹的音频长度an/h，得到每一数据存储位的时长bn/(an/h)；Dividing the frame rate of the enlarged video data by the audio length an/h of each Hertz to obtain the duration bn/(an/h) of each data storage bit;
    将所述音频数据的长度除以每一数据存储位的时长bn/(an/h)，获得新的时间戳T=a/[bn/(an/h)]。The length of the audio data is divided by the duration bn/(an/h) of each data storage bit, and a new time stamp T=a/[bn/(an/h)] is obtained.
  6. 如权利要求5所述的录制音视频同步时间戳的方法，其特征在于，所述根据新的时间戳编码、解码所述视频数据和音频数据，获得时间同步的所述视频数据和音频数据的步骤之前还包括：The method for recording an audio and video synchronization time stamp according to claim 5, wherein said encoding and decoding said video data and audio data according to a new time stamp, obtaining time-synchronized video data and audio data The steps also include:
    判断所述音频数据对应的数据存储位的个数是否大于新的时间戳的数值；Determining whether the number of data storage bits corresponding to the audio data is greater than a value of a new timestamp;
    如果是，则判定为新的时间戳有效，执行所述根据新的时间戳编码、解码所述视频数据和音频数据，获得时间同步的所述视频数据和音频数据的步骤；If yes, determining that the new timestamp is valid, performing the step of encoding the video data and the audio data according to the new timestamp, obtaining the time-synchronized video data and audio data;
    如果否，则判定为新的时间戳无效。If no, it is determined that the new timestamp is invalid.
  7. 权利要求4所述的录制音视频同步时间戳的方法，其特征在于，所述采集视频数据和音频数据的步骤包括：The method for recording an audio/video synchronization time stamp according to claim 4, wherein the step of collecting video data and audio data comprises:
    接收并存储音频采集设备采集的所述音频数据；Receiving and storing the audio data collected by the audio collection device;
    判断所述音频数据对应的存储位的个数是否大于阈值；Determining whether the number of storage bits corresponding to the audio data is greater than a threshold;
    如果是，则判定所述音频数据有效，接收并存储视频采集设备采集的所述视频数据；If yes, determining that the audio data is valid, and receiving and storing the video data collected by the video collection device;
    如果否，则判定所述音频数据无效。If not, it is determined that the audio data is invalid.
  8. 如权利要求5所述的录制音视频同步时间戳的方法，其特征在于，所述采集视频数据和音频数据的步骤包括：The method for recording an audio/video synchronization time stamp according to claim 5, wherein the step of collecting video data and audio data comprises:
    接收并存储音频采集设备采集的所述音频数据；Receiving and storing the audio data collected by the audio collection device;
    判断所述音频数据对应的存储位的个数是否大于阈值；Determining whether the number of storage bits corresponding to the audio data is greater than a threshold;
    如果是，则判定所述音频数据有效，接收并存储视频采集设备采集的所述视频数据；If yes, determining that the audio data is valid, and receiving and storing the video data collected by the video collection device;
    如果否，则判定所述音频数据无效。If not, it is determined that the audio data is invalid.
  9. 权利要求6所述的录制音视频同步时间戳的方法，其特征在于，所述采集视频数据和音频数据的步骤包括：The method for recording an audio/video synchronization time stamp according to claim 6, wherein the step of collecting video data and audio data comprises:
    接收并存储音频采集设备采集的所述音频数据；Receiving and storing the audio data collected by the audio collection device;
    判断所述音频数据对应的存储位的个数是否大于阈值；Determining whether the number of storage bits corresponding to the audio data is greater than a threshold;
    如果是，则判定所述音频数据有效，接收并存储视频采集设备采集的所述视频数据；If yes, determining that the audio data is valid, and receiving and storing the video data collected by the video collection device;
    如果否，则判定所述音频数据无效。If not, it is determined that the audio data is invalid.
  10. 如权利要求7所述的录制音视频同步时间戳的方法，其特征在于，所述接收并存储音频采集设备采集的所述音频数据的步骤之前还包括：The method for recording an audio and video synchronization time stamp according to claim 7, wherein the step of receiving and storing the audio data collected by the audio collection device further comprises:
    接收音视频录制请求；Receiving an audio and video recording request;
    检测当前是否已有所述视频采集设备接入；Detect whether the video capture device is currently connected;
    当检测到当前已有所述视频采集设备接入时，控制所述视频采集设备采集所述视频数据。When the video capture device is currently connected, the video capture device is controlled to collect the video data.
  11. 一种录制音视频同步时间戳的装置，其特征在于，包括：An apparatus for recording an audio and video synchronization time stamp, comprising:
    采集模块，用于采集视频数据和音频数据；An acquisition module for collecting video data and audio data;
    获取模块，用于获取所述音频数据的长度a、所述视频数据的帧率b以及预设的音频采样率h；An acquiring module, configured to acquire a length a of the audio data, a frame rate b of the video data, and a preset audio sampling rate h;
    计算模块，用于根据所述音频数据的长度a、所述视频数据的帧率b以及预设的音频采样率h，计算新的时间戳T=a/[b/(a/h)]；a calculation module, configured to calculate a new timestamp T=a/[b/(a/h)] according to a length a of the audio data, a frame rate b of the video data, and a preset audio sampling rate h;
    编解码模块，用于根据新的时间戳编码、解码所述视频数据和音频数据，获得时间同步的所述视频数据和音频数据。And a codec module, configured to obtain the time-synchronized video data and audio data according to a new timestamp encoding, decoding the video data and audio data.
  12. 如权利要求11所述的录制音视频同步时间戳的装置，其特征在于，所述计算模块还用于：The apparatus for recording an audio and video synchronization time stamp according to claim 11, wherein the calculation module is further configured to:
    分别将所述音频数据的长度a和所述视频数据的帧率b放大n倍，所述n为预设放大倍数；The length a of the audio data and the frame rate b of the video data are respectively enlarged by n times, and the n is a preset magnification;
    将放大后的所述音频数据的长度除以所述音频采样率h，得到每一赫兹的音频长度an/h；Dividing the length of the amplified audio data by the audio sampling rate h to obtain an audio length an/h per Hertz;
    将放大后的所述视频数据的帧率除以每一赫兹的音频长度an/h，得到每一数据存储位的时长bn/(an/h)；Dividing the frame rate of the enlarged video data by the audio length an/h of each Hertz to obtain the duration bn/(an/h) of each data storage bit;
    将所述音频数据的长度除以每一数据存储位的时长bn/(an/h)，获得新的时间戳T=a/[bn/(an/h)]。The length of the audio data is divided by the duration bn/(an/h) of each data storage bit, and a new time stamp T=a/[bn/(an/h)] is obtained.
  13. 权利要求12所述的录制音视频同步时间戳的装置，其特征在于，还包括判断模块，用于：The apparatus for recording an audio and video synchronization time stamp according to claim 12, further comprising: a determining module, configured to:
    判断所述音频数据对应的数据存储位的个数是否大于新的时间戳的数值；Determining whether the number of data storage bits corresponding to the audio data is greater than a value of a new timestamp;
    如果是，则判定为新的时间戳有效；If yes, it is determined that the new timestamp is valid;
    如果否，则判定为新的时间戳无效。If no, it is determined that the new timestamp is invalid.
  14. 如权利要求13所述的录制音视频同步时间戳的装置，其特征在于，所述采集模块还用于，接收并存储音频采集设备采集的所述音频数据；当所述判断模块判定所述音频数据有效时，接收并存储视频采集设备采集的所述视频数据；The apparatus for recording an audio and video synchronization time stamp according to claim 13, wherein the acquisition module is further configured to receive and store the audio data collected by the audio collection device; and when the determining module determines the audio Receiving and storing the video data collected by the video capture device when the data is valid;
    所述判断模块还用于，判断所述音频数据对应的存储位的个数是否大于阈值；如果是，则判定所述音频数据有效；如果否，则判定所述音频数据无效。The determining module is further configured to: determine whether the number of storage bits corresponding to the audio data is greater than a threshold; if yes, determine that the audio data is valid; if not, determine that the audio data is invalid.
  15. 权利要求11所述的录制音视频同步时间戳的装置，其特征在于，所述采集模块还用于，接收并存储音频采集设备采集的所述音频数据；当所述判断模块判定所述音频数据有效时，接收并存储视频采集设备采集的所述视频数据；The apparatus for recording an audio and video synchronization time stamp according to claim 11, wherein the acquisition module is further configured to receive and store the audio data collected by the audio collection device; and when the determining module determines the audio data Receiving and storing the video data collected by the video capture device when valid;
    所述判断模块还用于，判断所述音频数据对应的存储位的个数是否大于阈值；如果是，则判定所述音频数据有效；如果否，则判定所述音频数据无效。The determining module is further configured to: determine whether the number of storage bits corresponding to the audio data is greater than a threshold; if yes, determine that the audio data is valid; if not, determine that the audio data is invalid.
  16. 如权利要求12所述的录制音视频同步时间戳的装置，其特征在于，所述采集模块还用于，接收并存储音频采集设备采集的所述音频数据；当所述判断模块判定所述音频数据有效时，接收并存储视频采集设备采集的所述视频数据；The apparatus for recording an audio and video synchronization time stamp according to claim 12, wherein the acquisition module is further configured to receive and store the audio data collected by the audio collection device; and when the determining module determines the audio Receiving and storing the video data collected by the video capture device when the data is valid;
    所述判断模块还用于，判断所述音频数据对应的存储位的个数是否大于阈值；如果是，则判定所述音频数据有效；如果否，则判定所述音频数据无效。The determining module is further configured to: determine whether the number of storage bits corresponding to the audio data is greater than a threshold; if yes, determine that the audio data is valid; if not, determine that the audio data is invalid.
  17. 如权利要求14所述的录制音视频同步时间戳的装置，其特征在于，还包括指令接收模块和检测模块；The apparatus for recording an audio and video synchronization time stamp according to claim 14, further comprising an instruction receiving module and a detecting module;
    所述指令接收模块，用于接收音视频录制请求；The instruction receiving module is configured to receive an audio and video recording request;
    所述检测模块，用于检测当前是否已有所述视频采集设备接入；The detecting module is configured to detect whether the video capturing device is currently accessed;
    所述采集模块还用于，当所述检测模块检测到当前已有所述视频采集设备接入时，控制所述视频采集设备采集所述视频数据。The collecting module is further configured to: when the detecting module detects that the video capturing device is currently accessed, the video capturing device is controlled to collect the video data.
PCT/CN2016/084387 2015-07-06 2016-06-01 Method and apparatus for recording audio and video synchronization timestamp WO2017005066A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510392437.9A CN105049917B (en) 2015-07-06 2015-07-06 The method and apparatus of recording audio/video synchronized timestamp
CN201510392437.9 2015-07-06

Publications (1)

Publication Number Publication Date
WO2017005066A1 true WO2017005066A1 (en) 2017-01-12

Family

ID=54456040

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/084387 WO2017005066A1 (en) 2015-07-06 2016-06-01 Method and apparatus for recording audio and video synchronization timestamp

Country Status (2)

Country Link
CN (1) CN105049917B (en)
WO (1) WO2017005066A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109635133A (en) * 2018-12-18 2019-04-16 腾讯音乐娱乐科技（深圳）有限公司 Visualize audio frequency playing method, device, electronic equipment and storage medium
CN113596550A (en) * 2021-08-31 2021-11-02 安徽创变信息科技有限公司 Audio and video synchronization control method and device
CN114390314A (en) * 2021-12-30 2022-04-22 咪咕文化科技有限公司 Variable frame rate audio and video processing method, equipment and storage medium

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105049917B (en) * 2015-07-06 2018-12-07 深圳Tcl数字技术有限公司 The method and apparatus of recording audio/video synchronized timestamp
CN106060443A (en) * 2016-07-13 2016-10-26 福州瑞芯微电子股份有限公司 Video recording method and device based on Android system
CN109600563B (en) * 2018-08-01 2020-05-15 北京微播视界科技有限公司 Method and apparatus for determining a timestamp
CN109600665B (en) * 2018-08-01 2020-06-19 北京微播视界科技有限公司 Method and apparatus for processing data
CN109600564B (en) * 2018-08-01 2020-06-02 北京微播视界科技有限公司 Method and apparatus for determining a timestamp
CN109257641B (en) * 2018-09-05 2021-03-16 福建星网智慧科技股份有限公司 Audio and video synchronization method and system in wireless screen transmission
CN109842795B (en) * 2019-02-28 2020-08-11 苏州科达科技股份有限公司 Audio and video synchronization performance testing method and device, electronic equipment and storage medium
CN111726684B (en) * 2019-03-22 2022-11-04 腾讯科技（深圳）有限公司 Audio and video processing method and device and storage medium
JP7208530B2 (en) * 2019-05-31 2023-01-19 日本電信電話株式会社 Synchronization control device, synchronization control method and synchronization control program
CN110933349B (en) * 2019-11-19 2022-03-04 北京奇艺世纪科技有限公司 Audio data generation method, device and system and controller
CN111277896A (en) * 2020-02-13 2020-06-12 上海高重信息科技有限公司 Method and device for splicing network video stream images

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040041946A1 (en) * 2002-08-27 2004-03-04 Gries Patrick J. Method and apparatus for decoding audio and video information
US20100295993A1 (en) * 2009-05-20 2010-11-25 Samsung Electronics Co., Ltd. Apparatus and method for synchronization between video and audio in mobile communication terminal
CN102215429A (en) * 2010-04-01 2011-10-12 安凯(广州)微电子技术有限公司 Recording method for mobile TV
CN102821308A (en) * 2012-06-04 2012-12-12 西安交通大学 Multi-scene streaming media courseware recording and direct-broadcasting method
CN104202648A (en) * 2014-08-25 2014-12-10 四川长虹电器股份有限公司 Television program recording system
CN104618786A (en) * 2014-12-22 2015-05-13 深圳市腾讯计算机***有限公司 Audio/video synchronization method and device
CN105049917A (en) * 2015-07-06 2015-11-11 深圳Tcl数字技术有限公司 Method and device for recording an audio and video synchronization timestamp

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101267572B (en) * 2008-04-30 2010-07-14 中兴通讯股份有限公司 A program stream conversion method and device
US9179118B2 (en) * 2011-05-12 2015-11-03 Intel Corporation Techniques for synchronization of audio and video

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040041946A1 (en) * 2002-08-27 2004-03-04 Gries Patrick J. Method and apparatus for decoding audio and video information
US20100295993A1 (en) * 2009-05-20 2010-11-25 Samsung Electronics Co., Ltd. Apparatus and method for synchronization between video and audio in mobile communication terminal
CN102215429A (en) * 2010-04-01 2011-10-12 安凯(广州)微电子技术有限公司 Recording method for mobile TV
CN102821308A (en) * 2012-06-04 2012-12-12 西安交通大学 Multi-scene streaming media courseware recording and direct-broadcasting method
CN104202648A (en) * 2014-08-25 2014-12-10 四川长虹电器股份有限公司 Television program recording system
CN104618786A (en) * 2014-12-22 2015-05-13 深圳市腾讯计算机***有限公司 Audio/video synchronization method and device
CN105049917A (en) * 2015-07-06 2015-11-11 深圳Tcl数字技术有限公司 Method and device for recording an audio and video synchronization timestamp

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109635133A (en) * 2018-12-18 2019-04-16 腾讯音乐娱乐科技（深圳）有限公司 Visualize audio frequency playing method, device, electronic equipment and storage medium
CN109635133B (en) * 2018-12-18 2022-12-09 腾讯音乐娱乐科技（深圳）有限公司 Visual audio playing method and device, electronic equipment and storage medium
CN113596550A (en) * 2021-08-31 2021-11-02 安徽创变信息科技有限公司 Audio and video synchronization control method and device
CN113596550B (en) * 2021-08-31 2024-05-24 小帧科技(深圳)有限公司 Audio and video synchronous control method and device
CN114390314A (en) * 2021-12-30 2022-04-22 咪咕文化科技有限公司 Variable frame rate audio and video processing method, equipment and storage medium

Also Published As

Publication number Publication date
CN105049917A (en) 2015-11-11
CN105049917B (en) 2018-12-07

Similar Documents

Publication Publication Date Title
WO2017005066A1 (en) Method and apparatus for recording audio and video synchronization timestamp
WO2017177524A1 (en) Audio and video playing synchronization method and device
WO2016091011A1 (en) Subtitle switching method and device
WO2018034462A1 (en) Image display apparatus and method of operating the same
WO2018045682A1 (en) Method and device for testing audio and picture synchronization
WO2017143692A1 (en) Smart television and voice control method therefor
WO2016032021A1 (en) Apparatus and method for recognizing voice commands
WO2017164640A1 (en) System and method for editing video contents automatically technical field
WO2018192253A1 (en) Video signal transmission method and apparatus
WO2017020649A1 (en) Audio/video playback control method and device thereof
WO2015154639A1 (en) Method and apparatus for recording and replaying video of terminal
WO2018166224A1 (en) Target tracking display method and apparatus for panoramic video, and storage medium
WO2018223607A1 (en) Television terminal, method for converting hdr image into sdr image, and computer readable storage medium
WO2018028124A1 (en) Television set and signal source switching method thereof
WO2014187158A1 (en) Method, server, and terminal for controlling cloud sharing of terminal data
WO2018023926A1 (en) Interaction method and system for television and mobile terminal
WO2018032680A1 (en) Method and system for playing audio and video
WO2017054488A1 (en) Television play control method, server and television play control system
WO2017045441A1 (en) Smart television-based audio playback method and apparatus
WO2017113600A1 (en) Video playing method and device
WO2022025423A1 (en) Video quality assessing method and apparatus
WO2016101252A1 (en) Channel information display method and apparatus for intelligent television
WO2016090991A1 (en) Method and apparatus for downloading streaming media data
WO2017201893A1 (en) Video processing method and device
WO2019091128A1 (en) Signal preview method for new access source and television set

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16820717

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 01.06.2018)

122 Ep: pct application non-entry in european phase

Ref document number: 16820717

Country of ref document: EP

Kind code of ref document: A1