CN109635133A - Visualize audio frequency playing method, device, electronic equipment and storage medium - Google Patents
Visualize audio frequency playing method, device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN109635133A CN109635133A CN201811547567.5A CN201811547567A CN109635133A CN 109635133 A CN109635133 A CN 109635133A CN 201811547567 A CN201811547567 A CN 201811547567A CN 109635133 A CN109635133 A CN 109635133A
- Authority
- CN
- China
- Prior art keywords
- audio
- timestamp
- frequency characteristics
- audio frequency
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Television Signal Processing For Recording (AREA)
Abstract
The invention discloses a kind of visualization audio frequency playing method, device, electronic equipment and storage mediums, belong to field of computer technology.This method comprises: obtaining stamp and the second timestamp at the first time；According to first time stamp and the second timestamp, third timestamp is determined；According to third timestamp, from the corresponding timestamp of audio frequency characteristics and audio frequency characteristics of storage, target audio feature is obtained；It generates the image for being used to indicate target audio feature and is rendered；When audio is played at the time of the instruction of third timestamp, display renders obtained image.The present invention is by comprehensively considering the currently playing moment, and the current image that carries out generates the total duration needed with image rendering, to determine third timestamp, and then determine that target audio feature carries out image generation and rendering, when audio is played at the time of the instruction of third timestamp, it is synchronous to achieve the effect that sound is drawn so that the audio for guaranteeing that synchronization plays is consistent with the image of display for the image that display rendering obtains.
Description
Technical field
The present invention relates to field of computer technology, in particular to a kind of visualization audio frequency playing method, device, electronic equipment
And storage medium.
Background technique
With the gradually development of computer technology, people more and more use such as mobile phone in work and life, put down
The electronic equipments such as plate computer.It, can be with the needs of in order to meet user visually when user plays audio using electronic equipment
The image with Audio Matching is shown while playing audio, visualizes the effect for playing audio to realize.
In the related technology, it as shown in Figure 1, being usually first to be decoded to the audio data got, obtains decoded
Audio data.Then decoded audio data is sent to sound card buffer area, after obtaining decoding from sound card buffer area by sound card
Audio data play out.It is currently stored slow to sound card but if the speed stored to sound card buffer area is higher than broadcasting speed
The case where playout-delay may occur in the decoded audio data for rushing area, so as to cause the image that currently shows and currently broadcast
There are error between the audio data put, the unmatched problem of image for currently playing audio occur and currently showing.
Summary of the invention
It, can be with the embodiment of the invention provides a kind of visualization audio frequency playing method, device, electronic equipment and storage medium
Solve the problems in the relevant technologies.The technical solution is as follows:
In a first aspect, providing a kind of visualization audio frequency playing method, which comprises
Obtaining stamp and the second timestamp at the first time, first time stamp is used to indicate the currently playing moment, and described the
Two timestamps, which are used to indicate, carries out the total duration of image generation and rendering at the currently playing moment；
According to first time stamp and second timestamp, determine that third timestamp, the third timestamp are used for
Instruction is from the time of beginning to pass through the total duration at the currently playing moment；
According to the third timestamp, from the corresponding timestamp of audio frequency characteristics and audio frequency characteristics of storage, mesh is obtained
Mark audio frequency characteristics；
It generates the image for being used to indicate the target audio feature and is rendered；
When audio is played at the time of third timestamp instruction, display renders obtained image.
Optionally, the acquisition is at the first time before stamp and the second timestamp, further includes:
Obtain packets of audio data；
The packets of audio data is decoded, decoded audio data is obtained；
Determine and store the audio frequency characteristics of the decoded audio data and the sound of the decoded audio data
The corresponding timestamp of frequency feature.
Optionally, the audio frequency characteristics of the determination decoded audio data and the decoded audio number
According to the corresponding timestamp of audio frequency characteristics, comprising:
FFT (Fast Fourier Transformation, fast Fourier are carried out to the decoded audio data
Transformation), obtain the audio frequency characteristics of the decoded audio data；
It determines between the channel number, the sample rate of the audio and the bit depth three of the audio for playing the audio
Current time and current time are decoded the quotient between the total amount of data and the product of obtained audio data by product before
Value is used as the corresponding timestamp of the decoded audio frequency characteristics.
Optionally, described that the packets of audio data is decoded, after obtaining decoded audio data, further includes:
The decoded audio data is stored to sound card buffer area；
The decoded audio data is obtained from the sound card buffer area by sound card to play out.
Optionally, described according to the third timestamp, from audio frequency characteristics and the audio frequency characteristics corresponding time of storage
In stamp, target audio feature is obtained, comprising:
From the corresponding timestamp of audio frequency characteristics and audio frequency characteristics of storage, corresponding timestamp and the third are searched
The equal audio frequency characteristics of timestamp；
If finding, the audio frequency characteristics found are determined as the target audio feature；
If not finding, from the corresponding timestamp of audio frequency characteristics and audio frequency characteristics of the storage, acquisition and institute
State the corresponding audio frequency characteristics of the smallest timestamp of difference between third timestamp；It is less than the third timestamp alternatively, obtaining
And the corresponding audio frequency characteristics of the smallest timestamp of difference between the third timestamp；It is greater than the third alternatively, obtaining
Timestamp and the corresponding audio frequency characteristics of the smallest timestamp of difference between the third timestamp；The audio frequency characteristics that will acquire
It is determined as the target audio feature.
Optionally, the acquisition is stabbed and the second timestamp at the first time, comprising:
It determines between the channel number, the sample rate of the audio and the bit depth three of the audio for playing the audio
Product, by the quotient between the total amount of data and the product of audio data played before current time and current time
It is stabbed as the first time；
Obtain being averaged for the preset number frame image generated before generating the image for being used to indicate the target audio feature
Generate the average rendering duration of duration and the preset number frame image；
Duration is averagely generated and the sum averagely rendered between duration is determined as the currently playing moment and carries out figure for described
As the total duration for generating and rendering, using the corresponding timestamp of the total duration as second timestamp.
Second aspect, provides a kind of visualization audio playing apparatus, and described device includes:
First obtains module, and for obtaining stamp and the second timestamp at the first time, the first time stamp, which is used to indicate, works as
Preceding playing time, second timestamp, which is used to indicate, carries out the total duration of image generation and rendering at the currently playing moment；
First determining module, for determining third timestamp, institute according to first time stamp and second timestamp
Third timestamp is stated to be used to indicate from the time of beginning to pass through the total duration at the currently playing moment；
Second obtains module, for being corresponded to from the audio frequency characteristics and audio frequency characteristics of storage according to the third timestamp
Timestamp in, obtain target audio feature；
Generation module is used to indicate the image of the target audio feature and is rendered for generating；
Display module, for the figure that when audio is played at the time of third timestamp instruction, display rendering is obtained
Picture.
Optionally, described device further include:
Third obtains module, for obtaining packets of audio data；
Decoder module obtains decoded audio data for being decoded to the packets of audio data；
Second determining module, for determine and store the decoded audio data audio frequency characteristics and the solution
The corresponding timestamp of audio frequency characteristics of audio data after code.
Optionally, first determining module, comprising:
First determines submodule, for carrying out FFT to the decoded audio data, obtains the decoded audio
The audio frequency characteristics of data；
Second determines submodule, for determining the channel number, the sample rate of the audio and the sound that play the audio
Current time and current time are decoded the total data of obtained audio data by the product between the bit depth three of frequency before
Amount and the quotient between the product are as the corresponding timestamp of the decoded audio frequency characteristics.
Optionally, described device further include:
Memory module, for storing the decoded audio data to sound card buffer area；
Playing module is broadcast for obtaining the decoded audio data from the sound card buffer area by sound card
It puts.
Optionally, described second module is obtained, comprising:
Submodule is searched, for searching corresponding from the audio frequency characteristics of storage and the corresponding timestamp of audio frequency characteristics
The timestamp audio frequency characteristics equal with the third timestamp；
Third determines submodule, if it is special that the audio frequency characteristics found are determined as the target audio for finding
Sign；
Third determines submodule, if being also used to not find, from the audio frequency characteristics and audio frequency characteristics pair of the storage
In the timestamp answered, audio frequency characteristics corresponding with the smallest timestamp of difference between the third timestamp are obtained；Alternatively, obtaining
It takes less than the third timestamp and the corresponding audio frequency characteristics of the smallest timestamp of difference between the third timestamp；Or
Person obtains and is greater than the third timestamp and the corresponding audio of the smallest timestamp of difference between the third timestamp is special
Sign；The audio frequency characteristics that will acquire are determined as the target audio feature.
Optionally, described first module is obtained, comprising:
4th determines submodule, for determining the channel number, the sample rate of the audio and the sound that play the audio
Product between the bit depth three of frequency, by the total amount of data of audio data played before current time and current time
Quotient between the product is stabbed as the first time；
Acquisition submodule, for obtaining the present count generated before generating the image for being used to indicate the target audio feature
The average generation duration of mesh frame image and the average rendering duration of the preset number frame image；
5th determines submodule, for the sum between the averagely generation duration and the averagely rendering duration to be determined as
The currently playing moment carries out the total duration of image generation and rendering, when using the corresponding timestamp of the total duration as described second
Between stab.
The third aspect, provides a kind of electronic equipment, and the electronic equipment includes:
Processor and memory；
Wherein, at least one instruction is stored in the memory, at least one instruction is loaded by the processor
And it executes to realize the visualization audio frequency playing method as described in above-mentioned first aspect.
Fourth aspect provides a kind of computer readable storage medium, at least one instruction is stored in the storage medium,
At least one instruction is as processor loads and executes to realize the visualization audio broadcasting side as described in above-mentioned first aspect
Method.
The embodiment of the present invention is stabbed at the first time by acquisition and the second timestamp, comprehensively considers the currently playing moment, and
The total duration that image generates and image rendering needs is carried out at the currently playing moment, determines third timestamp.Later according to third
Timestamp obtains target audio feature from the corresponding timestamp of audio frequency characteristics and audio frequency characteristics of storage, generates for referring to
Show the image of target audio feature and rendered, so that the audio data pair that finally obtained image and third timestamp play
It answers, when audio is played at the time of the instruction of third timestamp, the image that display rendering obtains, to guarantee that synchronization plays
Audio data it is consistent with the image of display, achieve the effect that sound is drawn synchronous, solves sound currently playing in the related technology
Frequency and the unmatched problem of image currently shown.
Detailed description of the invention
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment
Attached drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for
For those of ordinary skill in the art, without creative efforts, it can also be obtained according to these attached drawings other
Attached drawing.
Fig. 1 is the flow chart that the audio of visualization in the prior art provided in an embodiment of the present invention plays；
Fig. 2 is a kind of visualization audio frequency playing method flow chart provided in an embodiment of the present invention；
Fig. 3 is another visualization audio frequency playing method flow chart provided in an embodiment of the present invention；
Fig. 4 is a kind of schematic diagram for showing image provided in an embodiment of the present invention；
Fig. 5 is the schematic diagram of another display image provided in an embodiment of the present invention；
Fig. 6 is the schematic diagram of another display image provided in an embodiment of the present invention；
Fig. 7 is the flow chart that another visualization audio provided in an embodiment of the present invention plays；
Fig. 8 is visualization audio playing apparatus structural schematic diagram provided in an embodiment of the present invention；
Fig. 9 is the structural schematic diagram of a kind of electronic equipment provided in an embodiment of the present invention.
Specific embodiment
To make the object, technical solutions and advantages of the present invention clearer, below in conjunction with attached drawing to embodiment party of the present invention
Formula is described in further detail.
Fig. 2 is a kind of visualization audio frequency playing method flow chart provided in an embodiment of the present invention, referring to fig. 2, this method packet
Include following steps:
Step 201: stamp and the second timestamp at the first time are obtained, stamp is used to indicate the currently playing moment at the first time, the
Two timestamps, which are used to indicate, carries out the total duration of image generation and rendering at the currently playing moment.
Step 202: according to first time stamp and the second timestamp, determining that third timestamp, third timestamp are used to indicate
At the time of beginning to pass through the total duration from the currently playing moment.
Step 203: being obtained from the corresponding timestamp of audio frequency characteristics and audio frequency characteristics of storage according to third timestamp
Take target audio feature.
Step 204: generating the image for being used to indicate target audio feature and rendered.
Step 205: when audio is played at the time of the instruction of third timestamp, display renders obtained image.
In conclusion the embodiment of the present invention is by obtaining stamp and the second timestamp at the first time, stamp is for referring at the first time
Show the currently playing moment, the second timestamp, which is used to indicate, carries out the total duration of image generation and rendering at the currently playing moment；It is comprehensive
It closes and considers the currently playing moment, and carry out the total duration that image generates and image rendering needs at the currently playing moment, determine
Third timestamp.Later according to third timestamp, from the corresponding timestamp of audio frequency characteristics and audio frequency characteristics of storage, obtain
Target audio feature generates the image for being used to indicate target audio feature and is rendered, so that finally obtained image and the
The audio data that three timestamps play is corresponding, and when audio is played at the time of the instruction of third timestamp, display rendering is obtained
Image achievees the effect that sound draws synchronous, solution so that the audio data for guaranteeing that synchronization plays is consistent with the image of display
Currently playing audio and the image unmatched problem currently shown in the related technology.
Optionally, obtain at the first time stamp and the second timestamp before, further includes:
Obtain packets of audio data；
The packets of audio data is decoded, decoded audio data is obtained；
Determine and store the audio frequency characteristics of decoded audio data and the audio frequency characteristics pair of decoded audio data
The timestamp answered.
Optionally it is determined that the audio frequency characteristics of the audio frequency characteristics of decoded audio data and decoded audio data
Corresponding timestamp, comprising:
FFT is carried out to decoded audio data, obtains the audio frequency characteristics of decoded audio data；
Determine the product between the channel number, the sample rate of the audio and the bit depth three of the audio for playing the audio,
Using current time and current time decode before the quotient between the total amount of data and the product of obtained audio data as
The corresponding timestamp of decoded audio frequency characteristics.
Optionally, which is decoded, after obtaining decoded audio data, further includes:
Decoded audio data is stored to sound card buffer area；
Decoded audio data is obtained from sound card buffer area by sound card to play out.
Optionally, it according to third timestamp, from the corresponding timestamp of audio frequency characteristics and audio frequency characteristics of storage, obtains
Target audio feature, comprising:
From the corresponding timestamp of audio frequency characteristics and audio frequency characteristics of storage, corresponding timestamp and third time are searched
Stab equal audio frequency characteristics；
If finding, the audio frequency characteristics found are determined as target audio feature；
If not finding, from the corresponding timestamp of audio frequency characteristics and audio frequency characteristics of storage, when obtaining with third
Between stamp between the corresponding audio frequency characteristics of the smallest timestamp of difference；Alternatively, obtain be less than third timestamp and with the third time
The corresponding audio frequency characteristics of the smallest timestamp of difference between stamp；Alternatively, obtain be greater than third timestamp and with third timestamp
Between the corresponding audio frequency characteristics of the smallest timestamp of difference；The audio frequency characteristics that will acquire are determined as target audio feature.
Optionally, stamp and the second timestamp at the first time are obtained, comprising:
Determine the product between the channel number, the sample rate of the audio and the bit depth three of the audio for playing the audio,
Using the quotient between the total amount of data and the product of audio data played before current time and current time as the
One timestamp；
Obtain the average generation of the preset number frame image generated before generating the image for being used to indicate target audio feature
The average rendering duration of duration and the preset number frame image；
This is averagely generated to duration and the sum averagely rendered between duration is determined as currently playing moment progress image life
At the total duration with rendering, using the corresponding timestamp of the total duration as the second timestamp.
All the above alternatives, can form alternative embodiment of the invention according to any combination, and the present invention is real
It applies example and this is no longer repeated one by one.
Fig. 3 is a kind of visualization audio frequency playing method flow chart provided in an embodiment of the present invention, is applied in electronic equipment.
The embodiment of the present invention will carry out expansion explanation to embodiment shown in FIG. 1 below, and referring to Fig. 3, this method comprises the following steps:
Step 301: obtaining packets of audio data, which is decoded, decoded audio data is obtained.
The embodiment of the present invention can be applied under the scene that electronic equipment plays any audio, and each audio is by multiple sounds
When frequency data packet group obtains packets of audio data at, above-mentioned steps 301, usually successively obtain one by one, correspondingly, to audio
Data packet is decoded, and is often to get a packets of audio data, is decoded, obtains decoded to the packets of audio data
Audio data.
It should be noted that generally realizing audio by the way that the analog audio signal of recording is encoded into digital audio and video signals
Compression, so as to transmission when improve transmission rate, storage when reduce occupy storage resource.So, the present invention is implemented
In example, the data packet got is digital audio and video signals, and while playing audio needs digital audio and video signals converting analogue sound
Frequency signal, is then played out by sound card, therefore, when getting packets of audio data, needs to carry out the packets of audio data
Decoding, obtains decoded audio data, to carry out subsequent broadcasting.Wherein, which is generally PCM
(Pulse Code Modulation, pulse code modulation) data.
Wherein, it is decoded to the packets of audio data, after obtaining decoded audio data, by decoded audio
Data are stored to sound card buffer area；Electronic equipment can obtain decoded audio data from sound card buffer area by sound card later
It plays out.
It should be noted that the sound card buffer area is the buffer area inside sound card, played needed for audio for storing sound card
Decoded audio data.And sound card according to certain broadcasting speed from sound card buffer area obtain decoded audio data into
Row plays.
Step 302: determining and store the audio frequency characteristics of decoded audio data and the sound of decoded audio data
The corresponding timestamp of frequency feature.
Wherein, which includes at least any one of frequency spectrum and loudness, naturally it is also possible to including other, such as save
Clap etc..The corresponding timestamp of the audio frequency characteristics of the decoded audio data refers to: display is used to indicate the decoded audio
The playing time of audio when the image of the audio frequency characteristics of data.
When determining the audio frequency characteristics of decoded audio data, FFT can be carried out to decoded audio data, be solved
The audio frequency characteristics of audio data after code.It, can be true when determining the corresponding timestamp of the audio frequency characteristics of decoded audio data
Surely the product between the bit depth three of the channel number of the audio, the sample rate of the audio and the audio is played, by current time
And current time decodes the quotient between the total amount of data and the product of obtained audio data as decoded sound before
The corresponding timestamp of frequency feature.
It should be noted that the channel number for playing the audio refer to play the audio when sound card in sound equipment number, the sound
The sample rate of frequency refers to the number of the sampled audio within the unit time, and the bit depth of the audio refers to the position of each sampled value
Number, usual bit depth includes 8bits and 16bits.Current time and current time decode the total of obtained audio data before
Quotient between data volume and the product refers to: current time and current time decode the sum of obtained audio data before
The value obtained according to amount divided by the product.
Since decoded audio data is continuous time signal, decoded audio data can be sampled, be obtained
FFT is carried out to the dis-crete sample values of continuous time signal, then to dis-crete sample values, by the dis-crete sample values of continuous time signal
It is transformed into frequency domain from time domain, obtains the audio frequency characteristics of continuous time signal.When audio frequency characteristics are frequency spectrum, by the above process may be used
To obtain the frequency spectrum of continuous time signal.When audio frequency characteristics include loudness, it is also necessary to according to the frequency of continuous time signal come
Calculate the loudness of continuous time signal.Certainly, when audio frequency characteristics further include other feature such as beats, the above method can be passed through
The frequency spectrum of continuous time signal is obtained, the beat of the decoded audio data is extracted from frequency spectrum.
It is corresponding in the audio frequency characteristics of the audio frequency characteristics and decoded audio data that obtain decoded audio data
It, can be corresponding by the audio frequency characteristics of the audio frequency characteristics of decoded audio data and decoded audio data after timestamp
Timestamp stored.For example, can be by the audio frequency characteristics of decoded audio data and decoded audio data
The corresponding timestamp of audio frequency characteristics is stored directly in the ring buffer of electronic equipment.Audio frequency characteristics can certainly first be established
Corresponding relationship between timestamp, by the corresponding timestamp of the audio frequency characteristics of decoded audio data with the shape of corresponding relationship
Formula is stored in ring buffer.
Step 303: stamp and the second timestamp at the first time are obtained, stamp is used to indicate the currently playing moment at the first time, the
Two timestamps, which are used to indicate, carries out the total duration of image generation and rendering at the currently playing moment.
Wherein, stamp is the currently playing moment for referring to audio at the first time, and the second timestamp refers to from the current of audio
Playing time rises, at the time of reached after the total duration that image generates and renders.
When obtaining stamp at the first time, channel number, the bit depth of the sample rate of audio and audio for playing audio can be determined
Product between three, by the total amount of data of audio data played before current time and current time and the product it
Between quotient as at the first time stab.It is available to generate the figure for being used to indicate target audio feature when obtaining the second timestamp
The average generation duration of the preset number frame image generated as before and the average rendering duration of preset number frame image；It will
This averagely generates duration and this averagely renders the sum between duration and be determined as currently playing moment and carries out image generation and rendering
Total duration, using the corresponding timestamp of the total duration as the second timestamp.
It should be noted that the total amount of data of audio data played before current time and current time multiplies with this
Quotient between product refers to: the total amount of data of played audio data is divided by the product before current time and current time
Obtained value.
In embodiments of the present invention, by generating the preset number generated before the image for being used to indicate target audio feature
The average generation duration of frame image and averagely rendering duration determine the second timestamp, are the preset number frame that will be generated before
The average generation duration of image is used to indicate the generation duration of the image of target audio feature as generating, pre- by what is generated before
If rendering duration of the average rendering duration of number frame image as the image for being used to indicate target audio feature, can be compared with subject to
Duration required for true prediction carries out image generation and render to the image for being used to indicate target audio feature, so that determining
The second timestamp it is more accurate, consequently facilitating subsequent accurate determining target audio feature.
In addition, when the image generated before generating the image for being used to indicate target audio feature is less than preset number, it can
To obtain the average generation duration of all images of generation and average wash with watercolours before generating the image for being used to indicate target audio feature
Duration is contaminated, the sum by the average generation duration of all images generated before and averagely between rendering duration was determined as the second time
Stamp.Wherein, when the image for being used to indicate target audio feature is to play the first frame image of the audio, i.e., there is no generate to use
The image generated before the image of instruction target audio feature, then can determine that the second timestamp is zero.
In alternatively possible implementation, the figure of generation before generating the image for being used to indicate target audio feature
It is directly zero by the second time stamp setting when as being less than preset number.
Step 304: according to first time stamp and the second timestamp, determining that third timestamp, third timestamp are used to indicate
At the time of beginning to pass through the total duration from the currently playing moment.
Wherein, according to first time stamp and the second timestamp determine third timestamp when, be determine at the first time stamp and
This is determined as third timestamp with value by between the second timestamp and value.
In the embodiment of the present invention, it is contemplated that every frame image generates and rendering needs certain duration, and electronic equipment is not
It is different with time data volume to be treated, i.e., electronic equipment is possible different in the processing capacity of different time, to lead
Different time is caused to generate different with duration needed for rendering image.And it is played since the speed stored to sound card buffer area is higher than
Speed leads to currently stored the case where being likely to occur playout-delay to the decoded audio data of sound card buffer area.Therefore, originally
Inventive embodiments begun to pass through by the currently playing moment and from the currently playing moment carry out image generation and rendering it is total when
It is long, it determines third timestamp, third timestamp can be determined more accurately, to carry out subsequent processing, reach synchronization
The audio data of broadcasting and the consistent effect of the image of display.
Step 305: being obtained from the corresponding timestamp of audio frequency characteristics and audio frequency characteristics of storage according to third timestamp
Take target audio feature.
According to third timestamp, from the corresponding timestamp of audio frequency characteristics and audio frequency characteristics of storage, target sound is obtained
, can be first determining with the matched timestamp of third timestamp from the corresponding timestamp of the audio frequency characteristics of storage when frequency feature, so
Audio frequency characteristics corresponding with the timestamp are obtained from the audio frequency characteristics of storage afterwards, the audio frequency characteristics that will acquire are determined as target
Audio frequency characteristics.
In one possible implementation, according to third timestamp, from the audio frequency characteristics and audio frequency characteristics pair of storage
It, can be from the corresponding timestamp of audio frequency characteristics and audio frequency characteristics of storage when obtaining target audio feature in the timestamp answered
In, search the corresponding timestamp audio frequency characteristics equal with third timestamp；It is if finding, the audio frequency characteristics found are true
It is set to target audio feature.If not finding, from the corresponding timestamp of audio frequency characteristics and audio frequency characteristics of storage, obtain with
The corresponding audio frequency characteristics of the smallest timestamp of difference between third timestamp；It is less than third timestamp and with the alternatively, obtaining
The corresponding audio frequency characteristics of the smallest timestamp of difference between three timestamps；It is greater than third timestamp and and third alternatively, obtaining
The corresponding audio frequency characteristics of the smallest timestamp of difference between timestamp；The audio frequency characteristics that will acquire are determined as target audio spy
Sign.
It should be noted that the corresponding timestamp of audio frequency characteristics refers to that generation is used to indicate the image and wash with watercolours of the audio frequency characteristics
After dye, at the time of showing the image after rendering.By the way that audio frequency characteristics corresponding with third timestamp equal time stamp are determined as
Image can then be shown that moment and audio playing time match by target audio feature, subsequent in audio so as to guarantee
When playing to any moment, show that the image that the moment should show, the image of the audio data and display of realizing broadcasting are kept
Unanimously.
In addition, can may be not present in the corresponding timestamp of the audio frequency characteristics of storage and third timestamp playing
Equal timestamp.It can so determine corresponding timestamp and the immediate audio frequency characteristics of third timestamp are that target audio is special
Sign, alternatively, determining less than third timestamp and being target audio closest to the corresponding audio frequency characteristics of the timestamp of third timestamp
Feature；Alternatively, being greater than third timestamp and being that target audio is special closest to the corresponding audio frequency characteristics of the timestamp of third timestamp
Sign.In this way, target audio feature is determined by these three modes when the timestamp equal with third timestamp is not present, with
Keep the difference between the determining corresponding timestamp of target audio feature and third timestamp smaller, guarantees the audio finally played
Data are more matched with the image of display.
Step 306: generating the image for being used to indicate target audio feature and rendered.
Wherein, the pattern for being used to indicate the image of target audio feature can be preset, and such as can be rectangle, round,
Wave-like, flame profile, constellation etc. pattern can when generating the image for being used to indicate target audio feature and being rendered
To generate corresponding image according to the variation of target audio feature, and wash with watercolours is carried out according to image of the rendering parameter of setting to generation
Dye increases playability to show the image of different patterns, improves user's viscosity.
For example, it is assumed that target audio feature is frequency spectrum, as shown in figure 4, if the pattern of the image of setting is rectangle, then can
The value in frequency spectrum is represented with the height with rectangle；As shown in figure 5, if the pattern of the image of setting is wave-like, then can be with
The value in frequency spectrum is represented with the fluctuating of wave longitudinal direction；As shown in fig. 6, if the pattern of the image of setting is flare shape, then can
The value in frequency spectrum is represented with the height with flare, to generate the spectrogram for being used to indicate frequency spectrum and be rendered.It certainly can also
To be set as other shapes, such as diamond shape, drop shape.For example, if audio frequency characteristics are loudness, the pattern for the image that can be set
For circle, then the variation of loudness can be indicated with circular radius, to generate image and the rendering for being used to indicate loudness.When
It so may be set to be other patterns, such as rectangle, diamond shape, constellation design.
In addition, when generating image, it can be from timing at the time of starting to carry out image generation, until being generated completely for referring to
Stopping when showing the image of target audio feature obtains the generation duration for generating the image.It correspondingly, can be with when rendering image
From timing at the time of starting to carry out image rendering, the stopping when image rendering until that will be used to indicate target audio feature is completed,
Obtain rendering the rendering duration of the image.
Step 307: when audio is played at the time of the instruction of third timestamp, display renders obtained image.
Wherein, which is the figure for generating and rendering by the target audio feature determined according to third timestamp
Picture, therefore the matched image at the time of image is indicated with third timestamp, then playing in audio to third timestamp
When at the time of instruction, the image is shown, when being at the time of playing time indicates for third timestamp, show mutual with the moment
Matched image, so as to realize that sound draws synchronous effect.
In a kind of mode in the cards, as shown in fig. 7, the visualization audio frequency playing method of the offer of the embodiment of the present invention
Can decoder module by electronic equipment, audio frequency characteristics determining module, feature buffer area, UI (User Interface, user
Interface), image generation module, image rendering module, sound card buffer area and sound card realize.
Electronic equipment obtains packets of audio data, is decoded the packets of audio data by decoder module, then decodes mould
Decoded audio data is sent to audio frequency characteristics computing module harmony card buffer by block.It is obtained by sound card from sound card buffer area
Decoded audio data plays out.Audio frequency characteristics determining module determines decoding when receiving decoded audio data
The corresponding timestamp of the audio frequency characteristics of the audio frequency characteristics of audio data afterwards and decoded audio data, and stored
In feature buffer area.The total amount of data for the audio data that UI played before obtaining current time and current time from sound card,
It is stabbed at the first time with determining, and obtains the second timestamp, then according to first time stamp and the second timestamp, determine the third time
Stamp from the audio frequency characteristics and the corresponding timestamp of audio frequency characteristics that feature buffer area stores, is obtained further according to third timestamp
Target audio feature.Then target audio feature is sent to image generation module by UI.The generation of image generation module is used to indicate
The image of target audio feature, and the image is sent to image rendering module, image rendering module renders the image,
When so that electronic equipment is at the time of audio broadcasting value third timestamp indicating, obtained image is rendered by display screen display.
In conclusion the embodiment of the present invention is by obtaining stamp and the second timestamp at the first time, stamp is for referring at the first time
Show the currently playing moment, the second timestamp, which is used to indicate, carries out the total duration of image generation and rendering at the currently playing moment；It is comprehensive
It closes and considers the currently playing moment, and carry out the total duration that image generates and image rendering needs at the currently playing moment, determine
Third timestamp.Later according to third timestamp, from the corresponding timestamp of audio frequency characteristics and audio frequency characteristics of storage, obtain
Target audio feature generates the image for being used to indicate target audio feature and is rendered, so that finally obtained image and the
The audio data that three timestamps play is corresponding, and when audio is played at the time of the instruction of third timestamp, display rendering is obtained
Image achievees the effect that sound draws synchronous, solution so that the audio data for guaranteeing that synchronization plays is consistent with the image of display
Currently playing audio and the image unmatched problem currently shown in the related technology.
Fig. 8 is a kind of visualization audio playing apparatus structural schematic diagram provided in an embodiment of the present invention.Referring to Fig. 8, the dress
Set includes: the first acquisition module 801, the first determining module 802, second acquisition module 803, generation module 804 and display module
805。
First obtains module 801, and for obtaining stamp and the second timestamp at the first time, stamp is used to indicate current at the first time
Playing time, the second timestamp, which is used to indicate, carries out the total duration of image generation and rendering at the currently playing moment；
First determining module 802, for determining third timestamp according to first time stamp and the second timestamp, when third
Between stab and be used to indicate from the time of beginning to pass through the total duration at the currently playing moment；
Second obtains module 803, is used for according to third timestamp, corresponding from the audio frequency characteristics and audio frequency characteristics of storage
In timestamp, target audio feature is obtained；
Generation module 804 is used to indicate the image of target audio feature and is rendered for generating；
Display module 805, for the figure that when audio is played at the time of the instruction of third timestamp, display rendering is obtained
Picture.
Optionally, the device further include:
Third obtains module, for obtaining packets of audio data；
Decoder module obtains decoded audio data for being decoded to the packets of audio data；
Second determining module, for determining and storing the audio frequency characteristics and decoded sound of decoded audio data
The corresponding timestamp of the audio frequency characteristics of frequency evidence.
Optionally, the first determining module 802, comprising:
First determines submodule, for carrying out FFT to decoded audio data, obtains the sound of decoded audio data
Frequency feature；
Second determines submodule, for determining the position of the channel number for playing the audio, the sample rate of the audio and the audio
Current time and current time are decoded the total amount of data of obtained audio data before and are somebody's turn to do by the product between depth three
Quotient between product is as the corresponding timestamp of decoded audio frequency characteristics.
Optionally, the device further include:
Memory module, for storing decoded audio data to sound card buffer area；
Playing module is played out for obtaining decoded audio data from sound card buffer area by sound card.
Optionally, second module 803 is obtained, comprising:
Submodule is searched, for searching corresponding from the audio frequency characteristics of storage and the corresponding timestamp of audio frequency characteristics
The timestamp audio frequency characteristics equal with third timestamp；
Third determines submodule, if the audio frequency characteristics found are determined as target audio feature for finding；
Third determines submodule, corresponding from the audio frequency characteristics and audio frequency characteristics of storage if being also used to not find
In timestamp, audio frequency characteristics corresponding with the smallest timestamp of difference between third timestamp are obtained；Alternatively, obtaining less than the
Three timestamps and the corresponding audio frequency characteristics of the smallest timestamp of difference between third timestamp；It is greater than third alternatively, obtaining
Timestamp and the corresponding audio frequency characteristics of the smallest timestamp of difference between third timestamp；The audio frequency characteristics that will acquire determine
For target audio feature.
Optionally, first module 801 is obtained, comprising:
4th determines submodule, for determining the position of the channel number for playing the audio, the sample rate of the audio and the audio
Product between depth three multiplies the total amount of data of audio data played before current time and current time with this
Quotient between product is as stamp at the first time；
Acquisition submodule, for obtaining the preset number generated before generating the image for being used to indicate the target audio feature
The average generation duration of frame image and the average rendering duration of the preset number frame image；
5th determines submodule, for this averagely to be generated to duration and this averagely renders the sum between duration and is determined as currently
Playing time carries out the total duration of image generation and rendering, using the corresponding timestamp of the total duration as the second timestamp.
In conclusion the embodiment of the present invention is by obtaining stamp and the second timestamp at the first time, stamp is for referring at the first time
Show the currently playing moment, the second timestamp, which is used to indicate, carries out the total duration of image generation and rendering at the currently playing moment；It is comprehensive
It closes and considers the currently playing moment, and carry out the total duration that image generates and image rendering needs at the currently playing moment, determine
Third timestamp.Later according to third timestamp, from the corresponding timestamp of audio frequency characteristics and audio frequency characteristics of storage, obtain
Target audio feature generates the image for being used to indicate target audio feature and is rendered, so that finally obtained image and the
The audio data that three timestamps play is corresponding, and when audio is played at the time of the instruction of third timestamp, display rendering is obtained
Image achievees the effect that sound draws synchronous, solution so that the audio data for guaranteeing that synchronization plays is consistent with the image of display
Currently playing audio and the image unmatched problem currently shown in the related technology.
It should be understood that visualization audio playing apparatus provided by the above embodiment play visualize audio when, only
The example of the division of the above functional modules, in practical application, can according to need and by above-mentioned function distribution by
Different functional modules is completed, i.e., the internal structure of device is divided into different functional modules, described above complete to complete
Portion or partial function.In addition, visualization audio playing apparatus provided by the above embodiment and visualization audio frequency playing method are real
It applies example and belongs to same design, specific implementation process is detailed in embodiment of the method, and which is not described herein again.
Fig. 9 shows the structural block diagram of the electronic equipment of an illustrative embodiment of the invention offer.The electronic equipment 900
It may is that smart phone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer
III, dynamic image expert's compression standard audio level 3), MP4 (Moving Picture Experts Group Audio
Layer IV, dynamic image expert's compression standard audio level 4) player, laptop or desktop computer.Electronic equipment
900 are also possible to referred to as other titles such as user equipment, portable electronic device, electronic equipment on knee, table type electronic equipment.
In general, electronic equipment 900 includes: processor 901 and memory 902.
Processor 901 may include one or more processing cores, such as 4 core processors, 8 core processors etc..Place
Reason device 901 can use DSP (Digital Signal Processing, Digital Signal Processing), FPGA (Field-
Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array, may be programmed
Logic array) at least one of example, in hardware realize.Processor 901 also may include primary processor and coprocessor, master
Processor is the processor for being handled data in the awake state, also referred to as CPU (Central Processing
Unit, central processing unit)；Coprocessor is the low power processor for being handled data in the standby state.?
In some embodiments, processor 901 can be integrated with GPU (Graphics Processing Unit, image processor),
GPU is used to be responsible for the rendering and drafting of content to be shown needed for display screen.In some embodiments, processor 901 can also be wrapped
AI (Artificial Intelligence, artificial intelligence) processor is included, the AI processor is for handling related machine learning
Calculating operation.
Memory 902 may include one or more computer readable storage mediums, which can
To be non-transient.Memory 902 may also include high-speed random access memory and nonvolatile memory, such as one
Or multiple disk storage equipments, flash memory device.In some embodiments, the non-transient computer in memory 902 can
Storage medium is read for storing at least one instruction, at least one instruction for performed by processor 901 to realize this hair
The visualization audio frequency playing method that bright middle embodiment of the method provides.
In some embodiments, electronic equipment 900 is also optional includes: peripheral device interface 903 and at least one periphery
Equipment.It can be connected by bus or signal wire between processor 901, memory 902 and peripheral device interface 903.It is each outer
Peripheral equipment can be connected by bus, signal wire or circuit board with peripheral device interface 903.Specifically, peripheral equipment includes: to penetrate
At least one in frequency circuit 904, touch display screen 905, camera 906, voicefrequency circuit 907, positioning component 908 and power supply 909
Kind.
Peripheral device interface 903 can be used for I/O (Input/Output, input/output) is relevant outside at least one
Peripheral equipment is connected to processor 901 and memory 902.In some embodiments, processor 901, memory 902 and peripheral equipment
Interface 903 is integrated on same chip or circuit board；In some other embodiments, processor 901, memory 902 and outer
Any one or two in peripheral equipment interface 903 can realize on individual chip or circuit board, the present embodiment to this not
It is limited.
Radio circuit 904 is for receiving and emitting RF (Radio Frequency, radio frequency) signal, also referred to as electromagnetic signal.It penetrates
Frequency circuit 904 is communicated by electromagnetic signal with communication network and other communication equipments.Radio circuit 904 turns electric signal
It is changed to electromagnetic signal to be sent, alternatively, the electromagnetic signal received is converted to electric signal.Optionally, radio circuit 904 wraps
It includes: antenna system, RF transceiver, one or more amplifiers, tuner, oscillator, digital signal processor, codec chip
Group, user identity module card etc..Radio circuit 904 can by least one wireless communication protocol come with other electronic equipments
It is communicated.The wireless communication protocol includes but is not limited to: Metropolitan Area Network (MAN), each third generation mobile communication network (2G, 3G, 4G and 5G), nothing
Line local area network and/or WiFi (Wireless Fidelity, Wireless Fidelity) network.In some embodiments, radio circuit 904
It can also include NFC (Near Field Communication, wireless near field communication) related circuit, the embodiment of the present invention
This is not limited.
Display screen 905 is for showing UI (User Interface, user interface).The UI may include figure, text, figure
Mark, video and its their any combination.When display screen 905 is touch display screen, display screen 905 also there is acquisition to show
The ability of the touch signal on the surface or surface of screen 905.The touch signal can be used as control signal and be input to processor
901 are handled.At this point, display screen 905 can be also used for providing virtual push button and/or dummy keyboard, also referred to as soft button and/or
Soft keyboard.In some embodiments, display screen 905 can be one, and the front panel of electronic equipment 900 is arranged；In other realities
It applies in example, display screen 905 can be at least two, be separately positioned on the different surfaces of electronic equipment 900 or in foldover design；?
In still other embodiments, display screen 905 can be flexible display screen, is arranged on the curved surface of electronic equipment 900 or folds
On face.Even, display screen 905 can also be arranged to non-rectangle irregular figure, namely abnormity screen.Display screen 905 can be adopted
With LCD (Liquid Crystal Display, liquid crystal display), (Organic Light-Emitting Diode, has OLED
Machine light emitting diode) etc. materials preparation.
CCD camera assembly 906 is for acquiring image or video.Optionally, CCD camera assembly 906 include front camera and
Rear camera.In general, the front panel of electronic equipment is arranged in front camera, the back of electronic equipment is arranged in rear camera
Face.In some embodiments, rear camera at least two, be respectively main camera, depth of field camera, wide-angle camera,
Any one in focal length camera, to realize that main camera and the fusion of depth of field camera realize background blurring function, main camera shooting
Head and wide-angle camera fusion realize pan-shot and VR (Virtual Reality, virtual reality) shooting function or its
It merges shooting function.In some embodiments, CCD camera assembly 906 can also include flash lamp.Flash lamp can be monochrome
Warm flash lamp is also possible to double-colored temperature flash lamp.Double-colored temperature flash lamp refers to the combination of warm light flash lamp and cold light flash lamp, can
For the light compensation under different-colour.
Voicefrequency circuit 907 may include microphone and loudspeaker.Microphone is used to acquire the sound wave of user and environment, and will
Sound wave, which is converted to electric signal and is input to processor 901, to be handled, or is input to radio circuit 904 to realize voice communication.
For stereo acquisition or the purpose of noise reduction, microphone can be separately positioned on the different parts of electronic equipment 900 to be multiple.
Microphone can also be array microphone or omnidirectional's acquisition type microphone.Loudspeaker is then used to that processor 901 or radio frequency will to be come from
The electric signal of circuit 904 is converted to sound wave.Loudspeaker can be traditional wafer speaker, be also possible to piezoelectric ceramics loudspeaking
Device.When loudspeaker is piezoelectric ceramic loudspeaker, the audible sound wave of the mankind can be not only converted electrical signals to, can also be incited somebody to action
Electric signal is converted to the sound wave that the mankind do not hear to carry out the purposes such as ranging.In some embodiments, voicefrequency circuit 907 can be with
Including earphone jack.
Positioning component 908 is used for the current geographic position of Positioning Electronic Devices 900, to realize navigation or LBS (Location
Based Service, location based service).Positioning component 908 can be the GPS (Global based on the U.S.
Positioning System, global positioning system), the dipper system of China, Russia Gray receive this system or European Union
The positioning component of Galileo system.
Power supply 909 is used to be powered for the various components in electronic equipment 900.Power supply 909 can be alternating current, direct current
Electricity, disposable battery or rechargeable battery.When power supply 909 includes rechargeable battery, which can support wired
Charging or wireless charging.The rechargeable battery can be also used for supporting fast charge technology.
In some embodiments, electronic equipment 900 further includes having one or more sensors 910.The one or more passes
Sensor 910 includes but is not limited to: acceleration transducer 911, gyro sensor 912, pressure sensor 913, fingerprint sensor
914, optical sensor 915 and proximity sensor 916.
Acceleration transducer 911 can detecte the acceleration in three reference axis of the coordinate system established with electronic equipment 900
Spend size.For example, acceleration transducer 911 can be used for detecting component of the acceleration of gravity in three reference axis.Processor
The 901 acceleration of gravity signals that can be acquired according to acceleration transducer 911, control touch display screen 905 with transverse views or
Longitudinal view carries out the display of user interface.Acceleration transducer 911 can be also used for game or the exercise data of user
Acquisition.
Gyro sensor 912 can detecte body direction and the rotational angle of electronic equipment 900, gyro sensor
912 can cooperate with acquisition user to act the 3D of electronic equipment 900 with acceleration transducer 911.Processor 901 is according to gyroscope
The data that sensor 912 acquires, may be implemented following function: action induction (for example changed according to the tilt operation of user
UI), image stabilization, game control and inertial navigation when shooting.
The lower layer of side frame and/or touch display screen 905 in electronic equipment 900 can be set in pressure sensor 913.When
When the side frame of electronic equipment 900 is arranged in pressure sensor 913, user can detecte to the gripping signal of electronic equipment 900,
Right-hand man's identification or prompt operation are carried out according to the gripping signal that pressure sensor 913 acquires by processor 901.Work as pressure sensing
When the lower layer of touch display screen 905 is arranged in device 913, grasped by processor 901 according to pressure of the user to touch display screen 905
Make, realization controls the operability control on the interface UI.Operability control include button control, scroll bar control,
At least one of icon control, menu control.
Fingerprint sensor 914 is used to acquire the fingerprint of user, collected according to fingerprint sensor 914 by processor 901
The identity of fingerprint recognition user, alternatively, by fingerprint sensor 914 according to the identity of collected fingerprint recognition user.It is identifying
When the identity of user is trusted identity out, the user is authorized to execute relevant sensitive operation, the sensitive operation packet by processor 901
Include solution lock screen, check encryption information, downloading software, payment and change setting etc..Electronics can be set in fingerprint sensor 914
Front, the back side or the side of equipment 900.When being provided with physical button or manufacturer Logo on electronic equipment 900, fingerprint sensor
914 can integrate with physical button or manufacturer Logo.
Optical sensor 915 is for acquiring ambient light intensity.In one embodiment, processor 901 can be according to optics
The ambient light intensity that sensor 915 acquires controls the display brightness of touch display screen 905.Specifically, when ambient light intensity is higher
When, the display brightness of touch display screen 905 is turned up；When ambient light intensity is lower, the display for turning down touch display screen 905 is bright
Degree.In another embodiment, the ambient light intensity that processor 901 can also be acquired according to optical sensor 915, dynamic adjust
The acquisition parameters of CCD camera assembly 906.
Proximity sensor 916, also referred to as range sensor are generally arranged at the front panel of electronic equipment 900.Proximity sensor
916 for acquiring the distance between the front of user Yu electronic equipment 900.In one embodiment, when proximity sensor 916 is examined
When measuring the distance between the front of user and electronic equipment 900 and gradually becoming smaller, touch display screen 905 is controlled by processor 901
Breath screen state is switched to from bright screen state；When proximity sensor 916 detect between user and the front of electronic equipment 900 away from
When from becoming larger, touch display screen 905 being controlled by processor 901 and is switched to bright screen state from breath screen state.
It will be understood by those skilled in the art that structure shown in Fig. 9 does not constitute the restriction to electronic equipment 900, it can
To include perhaps combining certain components than illustrating more or fewer components or being arranged using different components.
A kind of non-transitorycomputer readable storage medium, when the instruction in the storage medium is by the processor of electronic equipment
When execution, so that electronic equipment is able to carry out the visualization audio frequency playing method of above-mentioned Fig. 2 or Fig. 3.
Those of ordinary skill in the art will appreciate that realizing that all or part of the steps of above-described embodiment can pass through hardware
It completes, relevant hardware can also be instructed to complete by program, the program can store in a kind of computer-readable
In storage medium, storage medium mentioned above can be read-only memory, disk or CD etc..
The foregoing is merely presently preferred embodiments of the present invention, is not intended to limit the invention, it is all in spirit of the invention and
Within principle, any modification, equivalent replacement, improvement and so on be should all be included in the protection scope of the present invention.
Claims (14)
1. a kind of visualization audio frequency playing method, which is characterized in that the described method includes:
Obtain stamp and the second timestamp at the first time, first time stamp is used to indicate the currently playing moment, when described second
Between stamp be used to indicate the currently playing moment carry out image generation and rendering total duration；
According to first time stamp and second timestamp, determine that third timestamp, the third timestamp are used to indicate
At the time of beginning to pass through the total duration from the currently playing moment；
According to the third timestamp, from the corresponding timestamp of audio frequency characteristics and audio frequency characteristics of storage, target sound is obtained
Frequency feature；
It generates the image for being used to indicate the target audio feature and is rendered；
When audio is played at the time of third timestamp instruction, display renders obtained image.
2. the method as described in claim 1, which is characterized in that stamp is gone back with before the second timestamp at the first time for the acquisition
Include:
Obtain packets of audio data；
The packets of audio data is decoded, decoded audio data is obtained；
Determine and store the audio frequency characteristics of the decoded audio data and the audio spy of the decoded audio data
Levy corresponding timestamp.
3. method according to claim 2, which is characterized in that the audio of the determination decoded audio data is special
The corresponding timestamp of audio frequency characteristics of sign and the decoded audio data, comprising:
Fast Fourier Transform (FFT) FFT is carried out to the decoded audio data, obtains the sound of the decoded audio data
Frequency feature；
Determine multiplying between the channel number for playing the audio, the sample rate of the audio and the bit depth three of the audio
Current time and current time, are decoded the quotient between the total amount of data and the product of obtained audio data by product before
As the corresponding timestamp of the decoded audio frequency characteristics.
4. method according to claim 2, which is characterized in that it is described that the packets of audio data is decoded, it is decoded
After audio data afterwards, further includes:
The decoded audio data is stored to sound card buffer area；
The decoded audio data is obtained from the sound card buffer area by sound card to play out.
5. the method as described in claim 1, which is characterized in that it is described according to the third timestamp, it is special from the audio of storage
In sign and the corresponding timestamp of audio frequency characteristics, target audio feature is obtained, comprising:
From the corresponding timestamp of audio frequency characteristics and audio frequency characteristics of storage, corresponding timestamp and the third time are searched
Stab equal audio frequency characteristics；
If finding, the audio frequency characteristics found are determined as the target audio feature；
If not finding, from the corresponding timestamp of audio frequency characteristics and audio frequency characteristics of the storage, obtain and described the
The corresponding audio frequency characteristics of the smallest timestamp of difference between three timestamps；Alternatively, obtain be less than the third timestamp and with
The corresponding audio frequency characteristics of the smallest timestamp of difference between the third timestamp；It is greater than the third time alternatively, obtaining
Stamp and the corresponding audio frequency characteristics of the smallest timestamp of difference between the third timestamp；The audio frequency characteristics that will acquire determine
For the target audio feature.
6. the method as described in claim 1, which is characterized in that the acquisition is stabbed at the first time and the second timestamp, comprising:
Determine multiplying between the channel number for playing the audio, the sample rate of the audio and the bit depth three of the audio
Product makees the quotient between the total amount of data and the product of audio data played before current time and current time
For first time stamp；
Obtain the average generation of the preset number frame image generated before generating the image for being used to indicate the target audio feature
The average rendering duration of duration and the preset number frame image；
It is raw that the sum averagely generated between duration and the averagely rendering duration is determined as currently playing moment progress image
At the total duration with rendering, using the corresponding timestamp of the total duration as second timestamp.
7. a kind of visualization audio playing apparatus, which is characterized in that described device includes:
First obtains module, and for obtaining stamp and the second timestamp at the first time, the first time stamp, which is used to indicate, currently to be broadcast
It puts the moment, second timestamp, which is used to indicate, carries out the total duration of image generation and rendering at the currently playing moment；
First determining module, for determining third timestamp according to first time stamp and second timestamp, described the
Three timestamps are used to indicate from the time of beginning to pass through the total duration at the currently playing moment；
Second obtains module, for according to the third timestamp, when from the audio frequency characteristics and corresponding audio frequency characteristics of storage
Between stamp in, obtain target audio feature；
Generation module is used to indicate the image of the target audio feature and is rendered for generating；
Display module, for the image that when audio is played at the time of third timestamp instruction, display rendering is obtained.
8. device as claimed in claim 7, which is characterized in that described device further include:
Third obtains module, for obtaining packets of audio data；
Decoder module obtains decoded audio data for being decoded to the packets of audio data；
Second determining module, after audio frequency characteristics and the decoding for determining and storing the decoded audio data
Audio data the corresponding timestamp of audio frequency characteristics.
9. device as claimed in claim 8, which is characterized in that first determining module, comprising:
First determines submodule, for carrying out Fast Fourier Transform (FFT) FFT to the decoded audio data, obtains the solution
The audio frequency characteristics of audio data after code；
Second determines submodule, for determining the channel number for playing the audio, the sample rate of the audio and the audio
Product between bit depth three, by current time and current time decode before the total amount of data of obtained audio data with
Quotient between the product is as the corresponding timestamp of the decoded audio frequency characteristics.
10. device as claimed in claim 8, which is characterized in that described device further include:
Memory module, for storing the decoded audio data to sound card buffer area；
Playing module is played out for obtaining the decoded audio data from the sound card buffer area by sound card.
11. device as claimed in claim 7, which is characterized in that described second obtains module, comprising:
Submodule is searched, for searching the corresponding time from the audio frequency characteristics of storage and the corresponding timestamp of audio frequency characteristics
Stab the audio frequency characteristics equal with the third timestamp；
Third determines submodule, if the audio frequency characteristics found are determined as the target audio feature for finding；
Third determines submodule, corresponding from the audio frequency characteristics and audio frequency characteristics of the storage if being also used to not find
In timestamp, audio frequency characteristics corresponding with the smallest timestamp of difference between the third timestamp are obtained；Alternatively, obtaining small
In the third timestamp and the corresponding audio frequency characteristics of the smallest timestamp of difference between the third timestamp；Alternatively,
It obtains and is greater than the third timestamp and the corresponding audio frequency characteristics of the smallest timestamp of difference between the third timestamp；
The audio frequency characteristics that will acquire are determined as the target audio feature.
12. device as claimed in claim 7, which is characterized in that described first obtains module, comprising:
4th determines submodule, for determining the channel number for playing the audio, the sample rate of the audio and the audio
Product between bit depth three, by the total amount of data of audio data played before current time and current time and institute
The quotient stated between product is stabbed as the first time；
Acquisition submodule, for obtaining the preset number frame generated before generating the image for being used to indicate the target audio feature
The average generation duration of image and the average rendering duration of the preset number frame image；
5th determines submodule, for the sum between the averagely generation duration and the averagely rendering duration to be determined as currently
Playing time carries out the total duration of image generation and rendering, using the corresponding timestamp of the total duration as second time
Stamp.
13. a kind of electronic equipment, which is characterized in that the electronic equipment includes:
Processor and memory；
Wherein, at least one instruction is stored in the memory, at least one instruction is loaded and held by the processor
Row is to realize the visualization audio frequency playing method as described in any of claim 1 to 6 claim.
14. a kind of computer readable storage medium, which is characterized in that be stored at least one instruction, institute in the storage medium
State at least one instruction as processor loads and execute with realize as described in any of claim 1 to 6 claim can
Depending on changing audio frequency playing method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811547567.5A CN109635133B (en) | 2018-12-18 | 2018-12-18 | Visual audio playing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811547567.5A CN109635133B (en) | 2018-12-18 | 2018-12-18 | Visual audio playing method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109635133A true CN109635133A (en) | 2019-04-16 |
| CN109635133B CN109635133B (en) | 2022-12-09 |
Family
ID=66074971
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811547567.5A Active CN109635133B (en) | 2018-12-18 | 2018-12-18 | Visual audio playing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109635133B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111131917A (en) * | 2019-12-26 | 2020-05-08 | 国微集团(深圳)有限公司 | Real-time audio frequency spectrum synchronization method and playing device |
| CN111813970A (en) * | 2020-07-14 | 2020-10-23 | 广州酷狗计算机科技有限公司 | Multimedia content display method, device, terminal and storage medium |
| CN112423120A (en) * | 2020-01-22 | 2021-02-26 | 上海哔哩哔哩科技有限公司 | Audio time delay detection method and system |
| CN112464019A (en) * | 2020-12-03 | 2021-03-09 | 广州酷狗计算机科技有限公司 | Audio playing method, device, terminal and storage medium |
| CN113422997A (en) * | 2021-06-21 | 2021-09-21 | 北京小米移动软件有限公司 | Method and device for playing audio data and readable storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017005066A1 (en) * | 2015-07-06 | 2017-01-12 | 深圳Tcl数字技术有限公司 | Method and apparatus for recording audio and video synchronization timestamp |
| CN107566890A (en) * | 2017-09-15 | 2018-01-09 | 深圳国微技术有限公司 | Handle audio stream broadcasting abnormal method, apparatus, computer installation and computer-readable recording medium |
| CN107948735A (en) * | 2017-12-06 | 2018-04-20 | 北京金山安全软件有限公司 | Video playing method and device and electronic equipment |
-
2018
- 2018-12-18 CN CN201811547567.5A patent/CN109635133B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017005066A1 (en) * | 2015-07-06 | 2017-01-12 | 深圳Tcl数字技术有限公司 | Method and apparatus for recording audio and video synchronization timestamp |
| CN107566890A (en) * | 2017-09-15 | 2018-01-09 | 深圳国微技术有限公司 | Handle audio stream broadcasting abnormal method, apparatus, computer installation and computer-readable recording medium |
| CN107948735A (en) * | 2017-12-06 | 2018-04-20 | 北京金山安全软件有限公司 | Video playing method and device and electronic equipment |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111131917A (en) * | 2019-12-26 | 2020-05-08 | 国微集团(深圳)有限公司 | Real-time audio frequency spectrum synchronization method and playing device |
| CN111131917B (en) * | 2019-12-26 | 2021-12-28 | 国微集团(深圳)有限公司 | Real-time audio frequency spectrum synchronization method and playing device |
| CN112423120A (en) * | 2020-01-22 | 2021-02-26 | 上海哔哩哔哩科技有限公司 | Audio time delay detection method and system |
| CN111813970A (en) * | 2020-07-14 | 2020-10-23 | 广州酷狗计算机科技有限公司 | Multimedia content display method, device, terminal and storage medium |
| CN112464019A (en) * | 2020-12-03 | 2021-03-09 | 广州酷狗计算机科技有限公司 | Audio playing method, device, terminal and storage medium |
| CN113422997A (en) * | 2021-06-21 | 2021-09-21 | 北京小米移动软件有限公司 | Method and device for playing audio data and readable storage medium |
| CN113422997B (en) * | 2021-06-21 | 2023-02-28 | 北京小米移动软件有限公司 | Method and device for playing audio data and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109635133B (en) | 2022-12-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109635133A (en) | Visualize audio frequency playing method, device, electronic equipment and storage medium | |
| CN108401124A (en) | The method and apparatus of video record | |
| CN109348247A (en) | Determine the method, apparatus and storage medium of audio and video playing timestamp | |
| CN109448761B (en) | Method and device for playing songs | |
| CN110290421A (en) | Frame per second method of adjustment, device, computer equipment and storage medium | |
| CN109147757A (en) | Song synthetic method and device | |
| CN108922506A (en) | Song audio generation method, device and computer readable storage medium | |
| EP3618055B1 (en) | Audio mixing method and terminal, and storage medium | |
| CN109327608A (en) | Method, terminal, server and the system that song is shared | |
| CN108491748B (en) | Graphic code identification and generation method and device and computer readable storage medium | |
| CN108848394A (en) | Net cast method, apparatus, terminal and storage medium | |
| CN108965757A (en) | video recording method, device, terminal and storage medium | |
| CN108965922A (en) | Video cover generation method, device and storage medium | |
| CN110324689A (en) | Method, apparatus, terminal and the storage medium that audio-visual synchronization plays | |
| CN109346111A (en) | Data processing method, device, terminal and storage medium | |
| CN110121094A (en) | Video is in step with display methods, device, equipment and the storage medium of template | |
| CN109065068B (en) | Audio processing method, device and storage medium | |
| CN108900925A (en) | The method and apparatus of live streaming template are set | |
| CN109003621A (en) | A kind of audio-frequency processing method, device and storage medium | |
| CN109192218A (en) | The method and apparatus of audio processing | |
| CN109192223A (en) | The method and apparatus of audio alignment | |
| CN109982129A (en) | Control method for playing back, device and the storage medium of short-sighted frequency | |
| CN109218751A (en) | The method, apparatus and system of recommendation of audio | |
| CN109102811A (en) | Generation method, device and the storage medium of audio-frequency fingerprint | |
| CN108845777A (en) | The method and apparatus for playing frame animation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |