Summary of the invention
Embodiments of the invention provide a kind of audio and video synchronization method and electronic equipment, the poor problem of video playing quality seriously having caused due to the video jitter of the asynchronous generation of audio frequency and video while having solved displaying video, can reduce the video jitter that video produces when playing, promote video playing quality.
For achieving the above object, embodiments of the invention adopt following technical scheme:
First aspect, provides a kind of audio and video synchronization method, comprising:
In media content playing process, obtain the timestamp of voice data and the time of system clock that the audio playing module of described media content is play, described media content comprises voice data and video data, and described system clock starts to play moment timing from described media content;
According to the time of the timestamp of described voice data and described system clock, obtain the deviate of the timestamp of described voice data and the time of described system clock;
Described deviate and Preset Time threshold value are made comparisons, obtain comparative result, according to described comparative result, determine whether described system clock to be time-synchronized to the timestamp of described voice data;
If described system clock is time-synchronized to the timestamp of described voice data, according to the time of described system clock after synchronous, obtain the video data corresponding with the time of described system clock, make video data corresponding to time of video playback module from described system clock is synchronous start to continue to play described media content;
If described system clock is not time-synchronized to the timestamp of described voice data, according to the time of described system clock, obtain the video data corresponding with the time of described system clock in real time.
In conjunction with first aspect, in the first mode in the cards, described described deviate and Preset Time threshold value are made comparisons, obtain comparative result, according to described comparative result, the timestamp that is time-synchronized to described voice data of described system clock is comprised:
Described deviate and described Preset Time threshold value are made comparisons;
If described deviate is less than described Preset Time threshold value, determine that described comparative result is that described media content is play normally, and not by the timestamp that is time-synchronized to described voice data of described system clock;
If described deviate is more than or equal to described Preset Time threshold value, determine that described comparative result is that described media content broadcasting is undesired, and by the timestamp that is time-synchronized to described voice data of described system clock;
Wherein, the described timestamp that is time-synchronized to described voice data by described system clock is: the timestamp that is changed to described voice data the time of described system clock.
In conjunction with the first mode in the cards of first aspect, in the second mode in the cards, the described timestamp that if described system clock will be time-synchronized to described voice data, according to the time of described system clock after synchronous, obtain the video data corresponding with the time of described system clock, make video data corresponding to time of video playback module from described system clock is synchronous start to continue to play described media content and comprise:
If by the timestamp that is time-synchronized to described voice data of described system clock, by described video playback module, obtain the time of described system clock after synchronous;
Time after synchronous according to described system clock is obtained video data corresponding to time after synchronous with described system clock, and the current video data that described video playback module is play is updated to video data corresponding to time after synchronous with described system clock;
Video data corresponding to time by described video playback module from described system clock is synchronous starts to continue to play described media content.
In conjunction with the second mode in the cards of first aspect, in the third mode in the cards, the absolute value of the timestamp that described deviate is described voice data and the difference of the time of described system clock.
In conjunction with the third mode in the cards of first aspect, in the 4th kind of mode in the cards, the time of described system clock is linear increment.
Second aspect, provides a kind of electronic equipment, comprising:
Acquiring unit, be used at media content playing process, obtain the timestamp of voice data and the time of system clock that the audio playing module of described media content is play, described media content comprises voice data and video data, and described system clock starts to play moment timing from described media content;
Described acquiring unit, also for obtaining the deviate of the timestamp of described voice data and the time of described system clock according to the time of the timestamp of described voice data and described system clock;
Comparing unit, for described deviate and Preset Time threshold value are made comparisons, obtains comparative result, determines whether described system clock to be time-synchronized to the timestamp of described voice data according to described comparative result;
Processing unit, if for described system clock being time-synchronized to the timestamp of described voice data, according to the time of described system clock after synchronous, obtain the video data corresponding with the time of described system clock, make video data corresponding to time of video playback module from described system clock is synchronous start to continue to play described media content;
Described processing unit, if also for described system clock not being time-synchronized to the timestamp of described voice data, obtain the video data corresponding with the time of described system clock according to the time of described system clock in real time.
In conjunction with second aspect, in the first mode in the cards, described comparing unit specifically for:
Described deviate and described Preset Time threshold value are made comparisons;
If described deviate is less than described Preset Time threshold value, determine that described comparative result is that described media content is play normally, and not by the timestamp that is time-synchronized to described voice data of described system clock;
If described deviate is more than or equal to described Preset Time threshold value, determine that described comparative result is that described media content broadcasting is undesired, and by the timestamp that is time-synchronized to described voice data of described system clock;
Wherein, described comparing unit, when by the timestamp that is time-synchronized to described voice data of described system clock, is that the timestamp by being changed to described voice data the time of described system clock is realized.
In conjunction with the first mode in the cards of second aspect, in the second mode in the cards, described processing unit specifically for:
If by the timestamp that is time-synchronized to described voice data of described system clock, by described video playback module, obtain the time of described system clock after synchronous;
Time after synchronous according to described system clock is obtained video data corresponding to time after synchronous with described system clock, and the current video data that described video playback module is play is updated to video data corresponding to time after synchronous with described system clock;
Video data corresponding to time by described video playback module from described system clock is synchronous starts to continue to play described media content.
In conjunction with the second mode in the cards of second aspect, in the third mode in the cards, the absolute value of the timestamp that described deviate is described voice data and the difference of the time of described system clock.
In conjunction with the second mode in the cards of second aspect, in the third mode in the cards, the time of described system clock is linear increment.
The embodiment of the present invention provides a kind of audio and video synchronization method and electronic equipment, by in media content playing process, obtain the timestamp of voice data and the time of system clock that the audio playing module of media content is play, media content comprises voice data and video data, system clock starts to play moment timing from media content, according to the time of the timestamp of voice data and system clock, obtain the deviate of the timestamp of voice data and the time of system clock, deviate and Preset Time threshold value are made comparisons, obtain comparative result, according to comparative result, determine whether system clock to be time-synchronized to the timestamp of voice data, if system clock is time-synchronized to the timestamp of voice data, according to the time of system clock after synchronous, obtain the video data corresponding with the time of system clock, make video data corresponding to time of video playback module from system clock is synchronous start to continue to play media content, if system clock is not time-synchronized to the timestamp of voice data, according to the time of system clock, obtain the video data corresponding with the time of system clock in real time.
Therefore, no matter whether the time of system clock there is synchronization action with the timestamp of voice data, video playback module is all the time playing video data of reference system clock, like this, the deviate is here less than in the situation of Preset Time threshold value, when namely the timestamp of system clock and voice data is almost consistent, video playback module is with reference to the time of system clock, and because the time of system clock is linear increment, in the time of video playback module reference system clock, obtain video data like this, timestamp in the time of just can keeping the timestamp of video playback module playing video data and audio playing module playing audio-fequency data is consistent, make audio frequency and video reach synchronous, can reduce the video jitter that video produces when playing, promote video playing quality, and in the situation that deviate is greater than or equal to Preset Time threshold value, when the time of system clock is changed to the timestamp of voice data, when video playback module references during the time of system clock, timestamp while just making the timestamp of video playback module playing video data and audio playing module playing audio-fequency data is consistent, make audio frequency and video reach synchronous, solved video playback module in prior art with reference to timestamp during audio playing module playing audio-fequency data may there is notch cuttype matter of time and cause asynchronous the produced video jitter phenomenon of audio frequency and video, promoted video playing quality.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely described, obviously, described embodiment is only the present invention's part embodiment, rather than whole embodiment.Based on the embodiment in the present invention, those of ordinary skills, not making the every other embodiment obtaining under creative work prerequisite, belong to the scope of protection of the invention.
The embodiment of the present invention provides a kind of audio and video synchronization method, as shown in Figure 1, comprising:
101, electronic equipment is in media content playing process, obtain the timestamp of voice data and the time of system clock that the audio playing module of media content is play, media content comprises voice data and video data, and system clock starts to play moment timing from media content.
Wherein, electronic equipment can be the user terminals such as intelligent television, mobile phone, computer.Media content is a video playback file, when electronic equipment is play this video file, by audio playing module and video playback module, play simultaneously, wherein voice data module playing audio-fequency data, video playback module playing video data, user can listen to and watch this played file according to voice data and video data.
Concrete, timestamp during audio playing module playing audio-fequency data can directly obtain from some hardware platforms, system clock is a new clock of introducing in audio playing module, for a reference time in media content playing process, that is to say that video playback module is that obtain with corresponding video data of time system clock according to the time of system clock, wherein, the time of this system clock is linear growth.
102, electronic equipment obtains the deviate of the timestamp of voice data and the time of system clock according to the time of the timestamp of voice data and system clock.
Concrete, electronic equipment is getting the timestamp of voice data and after the time of system clock, the time that the timestamp of voice data can be deducted to system clock is obtained the deviate of the timestamp of voice data and the time of system clock, or deducted to the deviate of the timestamp of voice data the time of system clock, then, get the absolute value of the deviate of acquisition.
103, electronic equipment is made comparisons deviate and Preset Time threshold value, obtains comparative result, determines whether system clock to be time-synchronized to the timestamp of voice data according to comparative result, then enters step 104 or step 105.
Concrete, after the deviate of timestamp when having obtained audio playing module playing audio-fequency data and the time of system clock, deviate and Preset Time threshold value can be made comparisons, if deviate is less than Preset Time threshold value, determine that comparative result is that media content is play normally, not by the timestamp that is time-synchronized to voice data of system clock; If deviate is more than or equal to Preset Time threshold value, determines that comparative result is that media content broadcasting is undesired, and by the timestamp that is time-synchronized to voice data of system clock, be namely updated to the timestamp of voice data the time of system clock.
It should be noted that, when deviate and Preset Time threshold value are compared, be to make comparisons in the situation that electronic equipment has video data to play, to guarantee making audio frequency and video reach synchronous when having video data to play.Wherein, when video data starts to play, can carry out initialization to system clock, when making the time of system clock and now audio playing module starting playing audio-fequency data, first timestamp of voice data keeps synchronizeing, so that the time of video playback module reference system clock is during playing video data, when starting to play, be just consistent with the timestamp of voice data, make audio frequency reach synchronous; Certainly, if when video data starts to play, system clock is not carried out to initialization, and the time of system clock is asynchronous with the timestamp of voice data, once the deviate is here greater than or equal to Preset Time threshold value, thereby by being time-synchronized to of system clock the timestamp of voice data, can not think media content play undesired.That is to say, think media content play undesired be occur in media content play process in the middle of, if make at the beginning by the timestamp that is time-synchronized to voice data of system clock, do not think media content play undesired.
If 104 electronic equipments are time-synchronized to system clock the timestamp of voice data, according to the time of system clock after synchronous, obtain the video data corresponding with the time of system clock, make video data corresponding to time of video playback module from system clock is synchronous start to continue to play media content.
Concrete, if electronic equipment is by the timestamp that is time-synchronized to voice data of system clock, by video playback module, obtain the time of system clock after synchronous, time after synchronous according to system clock is obtained video data corresponding to time after synchronous with system clock, the current video data that video playback module is play is updated to video data corresponding to time of system clock after synchronous, and video data corresponding to time by video playback module from system clock is synchronous starts to continue to play media content.
If 105 are not time-synchronized to system clock the timestamp of voice data, electronic equipment obtains the video data corresponding with the time of system clock according to the time of system clock in real time.
Like this, video playback module is the video data corresponding with system time obtaining according to system time, and system time is again the level and smooth time sequencing that is linear growth, rather than the video data directly obtaining according to the timestamp of voice data.
Comprehensive, no matter whether the time of system clock there is synchronization action with the timestamp of voice data, video playback module is all the time playing video data of reference system clock, like this, just can reduce because audio playing module is at the playback jitter that cannot obtain accurate reproduction time and cause, promote video playing quality.
The embodiment of the present invention provides a kind of audio and video synchronization method, in media content playing process, obtain the timestamp of voice data and the time of system clock that the audio playing module of media content is play, media content comprises voice data and video data, system clock starts to play moment timing from media content, according to the time of the timestamp of voice data and system clock, obtain the deviate of the timestamp of voice data and the time of system clock, deviate and Preset Time threshold value are made comparisons, obtain comparative result, according to comparative result, determine whether system clock to be time-synchronized to the timestamp of voice data, if system clock is time-synchronized to the timestamp of voice data, according to the time of system clock after synchronous, obtain the video data corresponding with the time of system clock, make video data corresponding to time of video playback module from system clock is synchronous start to continue to play media content, can reduce the video jitter that video produces when playing, promote video playing quality.
The embodiment of the present invention provides a kind of audio and video synchronization method, as shown in Figure 2, comprising:
201, electronic equipment is in media content playing process, obtains the timestamp of voice data and the time of system clock that the audio playing module of media content is play.
Wherein, electronic equipment can be the user terminals such as intelligent television, mobile phone and computer.When playing a media content in electronic equipment, namely during a played file, because this media content comprises voice data and video data, voice data is play by audio playing module, video data is play by video playback module, and user just can listen to and watch this media content according to voice data and video data.
Concrete, in the process of playing at media content, timestamp when audio playing module can be obtained playing audio-fequency data from some hardware platforms, system clock is a reference clock of introducing in audio playing module, when audio playing module is obtained voice data according to the timestamp of the voice data obtaining, video playback module can be obtained the time of system clock from system clock.Wherein, the time of system clock is linear growth, can smoothly advance according to time sequencing, and video playback module is according to the time of system clock, to obtain the video data corresponding with the time of system clock and continue to play when playing video data.
202, electronic equipment obtains the deviate of the timestamp of voice data and the time of system clock according to the time of the timestamp of voice data and system clock.
Concrete, obtaining the timestamp of voice data and after the time of system clock, can obtain the timestamp of voice data and the deviate of the time of system clock, concrete this deviate can deduct for the timestamp of the voice data of current broadcasting the difference of the time of system clock, also can deduct for the time of system clock the difference of the timestamp of voice data.Wherein, the absolute value that the deviate of acquisition is difference.
203, electronic equipment compares deviate and Preset Time threshold value, then enters step 204 or step 205.
It should be noted that, when deviate and Preset Time threshold value are compared, be to make comparisons in the situation that electronic equipment has video data to play, to guarantee making audio frequency and video reach synchronous when having video data to play.
If 204 deviates are less than Preset Time threshold value, electronic equipment determines that comparative result is that media content is play normally, and not by the timestamp that is time-synchronized to voice data of system clock.
Concrete, obtaining after deviate, because video playback module is to obtain the video data corresponding with the time of system clock according to the time of system clock when the playing video data, if deviate is less than Preset Time threshold value, timestamp in the time of can thinking time of system clock and audio playing module playing audio-fequency data differs very little, error within the range of permission, the video data that video playback module is play is so almost just that the voice data corresponding with the timestamp of voice data is synchronous, the video jitter that small audio frequency and video asynchrony phenomenon produces can be ignored, user can normally watch this media content, at this moment, just do not need the timestamp that is time-synchronized to voice data of system clock, video playback module is obtained corresponding video data with regard to the time of reference system clock, in the time of system clock, be linear growth guaranteeing, level and smooth playing video data.Wherein, Preset Time threshold value can be 200ms, also can, for other value, not limit here.
If 205 deviates are more than or equal to Preset Time threshold value, electronic equipment determines that comparative result is that media content broadcasting is undesired, and by the timestamp that is time-synchronized to voice data of system clock.
Concrete, if deviate is more than or equal to Preset Time threshold value, can think that the timestamp of voice data and the difference of the time of system clock are larger, for example Preset Time threshold value is 200ms, deviate is more than or equal to 200ms, when the time of video playback module reference system clock is obtained video data, will cause video playback module playing video data and voice data when synchronous, to produce video jitter, electronic equipment determines that comparative result is that media content broadcasting is undesired, at this moment, can be by the timestamp that is time-synchronized to voice data of system clock, so just being conducive to video playback module synchronizeed with the timestamp of voice data during playing video data in the time of reference system clock.
Wherein, when video data starts to play, can carry out initialization to system clock, the timestamp that makes the time of system clock and audio playing module now start first voice data of playing keeps synchronizeing, so that the time of video playback module reference system clock is during playing video data, when starting to play, be just consistent with the timestamp of voice data, make audio frequency reach synchronous; Certainly, if when video data starts to play, system clock is not carried out to initialization, once the deviate is here greater than or equal to Preset Time threshold value, can not think that media content broadcasting is undesired.That is to say, think media content play undesired be occur in media content play process in the middle of, if make at the beginning by the timestamp that is time-synchronized to voice data of system clock, do not think media content play undesired.
If 206 electronic equipments are the timestamp that is time-synchronized to voice data of system clock, by video playback module, obtain the time of system clock after synchronous.
Concrete, at electronic equipment, by after the timestamp that is time-synchronized to voice data of system clock, video playback module just can be obtained the timestamp after the time of system clock and the timestamp of voice data are synchronizeed in the process of the time playing video data of reference system clock in real time.
207, the time of electronic equipment after synchronous according to system clock is obtained video data corresponding to time after synchronous with system clock, and the current video data that video playback module is play is updated to video data corresponding to time after synchronous with system clock.
Concrete, video playback module was obtained after the time after system clock is synchronous, and the time after just can be synchronous according to system clock is obtained video data corresponding to the time after synchronously with system clock, and current in progress video data is upgraded.Here video data is upgraded to the two kinds of situations that can be understood as: a kind of is that the timestamp of current video playing module playing video data is later than the time of system clock after synchronous, can lose the video data of current broadcasting, directly video data corresponding to time from system clock is synchronous starts to play, with holding tone audio video synchronization; Another kind is the time of the timestamp of current video playing module playing video data after synchronous early than system clock, time after can be synchronous according to system clock is obtained the corresponding video data of time after synchronize with system clock, and current in progress video data is updated to video data corresponding to the time after synchronously with system clock.
For instance, a certain media content is being play and is being kept audio-visual synchronization, because audio playing module has been obtained coarse voice data timestamp, audio playing module has been obtained the voice data under inaccuracy timestamp, asynchronous with the video data of video playback module, now, the time of system clock is 1.6s, the timestamp of voice data is 1.3s, the timestamp of video data is 1.5s, the deviate of the time of system clock and the timestamp of voice data is 0.3s, Preset Time threshold value is 200ms, can know that deviate has been greater than Preset Time threshold value, need to be by the timestamp that is time-synchronized to voice data of system clock, namely the time of system clock is changed to 1.3s, due to video playback module references is the video data obtaining time of system clock, like this, video playback module finds that when the time of frame of reference clock the time of system clock upgraded for 1.3s by 1.6s, just directly with reference to this timestamp of 1.3s, obtain the video data that 1.3s is corresponding, like this, just can holding tone audio data and video data synchronously broadcasting under identical timestamp.
208, by video playback module, video data corresponding to time from system clock is synchronous starts to continue to play media content to electronic equipment.
Concrete, when electronic equipment is play in nonsynchronous situation at audio, video data, video playback module is in the time with reference to system clock, because system clock is, linear growth is level and smooth to be advanced, can solve and be notch cuttype when the timestamp of voice data, the video jitter phenomenon causing during the timestamp playing video data of the direct reference audio data of video playback module, in the time of system clock, be linear growth like this, and there is the timestamp of voice data and the time deviation of system clock when larger, make the system clock playing video data of video playback module after can the timestamp that be time-synchronized to voice data of reference system clock, can reduce the video jitter that video produces when playing, promote video playing quality.
In sum, no matter whether the time of system clock there is synchronization action with the timestamp of voice data, and video playback module is all the time playing video data of reference system clock.
The embodiment of the present invention provides a kind of audio and video synchronization method, in media content playing process, obtain the timestamp of voice data and the time of system clock of the voice data that media content plays, media content comprises voice data and video data, system clock starts to play moment timing from media content, according to the time of the timestamp of voice data and system clock, obtain the deviate of the timestamp of voice data and the time of system clock, deviate and Preset Time threshold value are made comparisons, obtain comparative result, according to comparative result, determine whether system clock to be time-synchronized to the timestamp of voice data, if system clock is time-synchronized to the timestamp of voice data, according to the time of system clock after synchronous, obtain the video data corresponding with the time of system clock, make video data corresponding to time of video playback module from system clock is synchronous start to continue to play media content, can reduce the video jitter that video produces when playing, promote video playing quality.
The embodiment of the present invention provides a kind of electronic equipment 01, as shown in Figure 3, comprising:
Acquiring unit 011, be used at media content playing process, obtain the timestamp of voice data and the time of system clock that the audio playing module of media content is play, media content comprises voice data and video data, and system clock starts to play the moment from media content and starts timing.
Acquiring unit 011, also for obtaining the deviate of the timestamp of voice data and the time of system clock according to the time of the timestamp of voice data and system clock.
Comparing unit 012, for deviate and Preset Time threshold value are made comparisons, obtains comparative result, determines whether system clock to be time-synchronized to the timestamp of voice data according to comparative result.
Processing unit 013, if for system clock being time-synchronized to the timestamp of voice data, according to the time of system clock after synchronous, obtain the video data corresponding with the time of system clock, make video data corresponding to time of video playback module from system clock is synchronous start to continue to play media content.
Processing unit 013, if also for system clock not being time-synchronized to the timestamp of voice data, obtain the video data corresponding with the time of system clock according to the time of system clock in real time.
Optionally, comparing unit 012 can be specifically for:
Deviate and Preset Time threshold value are made comparisons;
If deviate is less than Preset Time threshold value, determine that comparative result is that media content is play normally, and not by the timestamp that is time-synchronized to voice data of system clock;
If deviate is more than or equal to Preset Time threshold value, determine that comparative result is that media content broadcasting is undesired, and by the timestamp that is time-synchronized to voice data of system clock.
Wherein, comparing unit 012, when by the timestamp that is time-synchronized to institute's voice data of system clock, is that the timestamp by being changed to voice data the time of system clock is realized.
Optionally, processing unit 013 can be specifically for:
If by the timestamp that is time-synchronized to voice data of system clock, by video playback module, obtain the time of system clock after synchronous;
Time after synchronous according to system clock is obtained video data corresponding to time after synchronous with system clock, and the current video data that video playback module is play is updated to video data corresponding to time after synchronous with system clock;
Video data corresponding to time by video playback module from system clock is synchronous starts to continue to play media content.
The absolute value of the difference of the timestamp that wherein, deviate is voice data and the time of system clock.
The time of system clock is linear increment.
The embodiment of the present invention provides a kind of electronic equipment, in media content playing process, obtain the timestamp of voice data and the time of system clock that the audio playing module of media content is play, media content comprises voice data and video data, system clock starts to play moment timing from media content, according to the time of the timestamp of voice data and system clock, obtain the deviate of the timestamp of data and the time of system clock, deviate and Preset Time threshold value are made comparisons, obtain comparative result, according to comparative result, determine whether system clock to be time-synchronized to the timestamp of voice data, if system clock is time-synchronized to the timestamp of voice data, according to the time of system clock after synchronous, obtain the video data corresponding with the time of system clock, make video data corresponding to time of video playback module from system clock is synchronous start to continue to play media content, can reduce the video jitter that video produces when playing, promote video playing quality.
In the several embodiment that provide in the application, should be understood that disclosed equipment and method can realize by another way.For example, apparatus embodiments described above is only schematic, for example, the division of described unit, be only that a kind of logic function is divided, during actual realization, can have other dividing mode, for example multiple unit or assembly can in conjunction with or can be integrated into another system, or some features can ignore, or do not carry out.Another point, shown or discussed coupling each other or direct-coupling or communication connection can be by some interfaces, indirect coupling or the communication connection of device or unit can be electrically, machinery or other form.
In addition, in equipment and system in each embodiment of the present invention, each functional unit can be integrated in a processing unit, can be also that the independent physics of unit comprises, also can be integrated in a unit two or more unit.And above-mentioned each unit both can adopt the form of hardware to realize, and the form that also can adopt hardware to add SFU software functional unit realizes.
The all or part of step that realizes said method embodiment can complete by the relevant hardware of program command, and aforesaid program can be stored in a computer read/write memory medium, and this program, when carrying out, is carried out the step that comprises said method embodiment; And aforesaid storage medium comprises: USB flash disk, portable hard drive, read-only memory (Read Only Memory, be called for short ROM), the various media that can be program code stored such as random access memory (Random Access Memory, be called for short RAM), magnetic disc or CD.
The above; be only the specific embodiment of the present invention, but protection scope of the present invention is not limited to this, any be familiar with those skilled in the art the present invention disclose technical scope in; can expect easily changing or replacing, within all should being encompassed in protection scope of the present invention.Therefore, protection scope of the present invention should be as the criterion with the protection range of described claim.