CN1321964A - Sound and beat image display method and equipment - Google Patents
Sound and beat image display method and equipment Download PDFInfo
- Publication number
- CN1321964A CN1321964A CN 00108270 CN00108270A CN1321964A CN 1321964 A CN1321964 A CN 1321964A CN 00108270 CN00108270 CN 00108270 CN 00108270 A CN00108270 A CN 00108270A CN 1321964 A CN1321964 A CN 1321964A
- Authority
- CN
- China
- Prior art keywords
- color
- sound
- frequency
- coordinate
- waveform
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 230000008859 change Effects 0.000 claims abstract description 26
- 238000005311 autocorrelation function Methods 0.000 claims abstract description 10
- 230000033764 rhythmic process Effects 0.000 claims abstract description 10
- 230000000694 effects Effects 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 14
- 230000006870 function Effects 0.000 claims description 10
- 238000010586 diagram Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 5
- 238000005070 sampling Methods 0.000 claims description 5
- 230000015572 biosynthetic process Effects 0.000 claims description 4
- 239000004744 fabric Substances 0.000 claims description 4
- 239000000203 mixture Substances 0.000 claims description 4
- 230000000737 periodic effect Effects 0.000 claims description 4
- 230000009467 reduction Effects 0.000 claims description 4
- 230000007704 transition Effects 0.000 claims description 4
- 230000001052 transient effect Effects 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 claims description 3
- 238000004422 calculation algorithm Methods 0.000 claims description 2
- 230000000750 progressive effect Effects 0.000 claims description 2
- 238000004064 recycling Methods 0.000 claims description 2
- 230000008034 disappearance Effects 0.000 claims 2
- OAICVXFJPJFONN-UHFFFAOYSA-N Phosphorus Chemical compound [P] OAICVXFJPJFONN-UHFFFAOYSA-N 0.000 claims 1
- 230000008602 contraction Effects 0.000 claims 1
- 238000009792 diffusion process Methods 0.000 claims 1
- 230000005484 gravity Effects 0.000 claims 1
- 230000002688 persistence Effects 0.000 claims 1
- 230000001105 regulatory effect Effects 0.000 claims 1
- 101150029607 SS18L1 gene Proteins 0.000 description 22
- 238000006243 chemical reaction Methods 0.000 description 9
- 238000004364 calculation method Methods 0.000 description 8
- 239000003086 colorant Substances 0.000 description 7
- 238000002156 mixing Methods 0.000 description 3
- 210000000214 mouth Anatomy 0.000 description 3
- 238000001228 spectrum Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 210000001260 vocal cord Anatomy 0.000 description 3
- NNBFNNNWANBMTI-UHFFFAOYSA-M brilliant green Chemical compound OS([O-])(=O)=O.C1=CC(N(CC)CC)=CC=C1C(C=1C=CC=CC=1)=C1C=CC(=[N+](CC)CC)C=C1 NNBFNNNWANBMTI-UHFFFAOYSA-M 0.000 description 2
- 230000003467 diminishing effect Effects 0.000 description 2
- 238000005755 formation reaction Methods 0.000 description 2
- 210000001331 nose Anatomy 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 235000005976 Citrus sinensis Nutrition 0.000 description 1
- 240000002319 Citrus sinensis Species 0.000 description 1
- 241000196324 Embryophyta Species 0.000 description 1
- 230000000739 chaotic effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000035876 healing Effects 0.000 description 1
- 239000004615 ingredient Substances 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 210000003928 nasal cavity Anatomy 0.000 description 1
- 230000010355 oscillation Effects 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000005622 photoelectricity Effects 0.000 description 1
- 230000035807 sensation Effects 0.000 description 1
- 125000006850 spacer group Chemical group 0.000 description 1
- 238000010183 spectrum analysis Methods 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 230000036962 time dependent Effects 0.000 description 1
- 230000032258 transport Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/06—Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/27—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Auxiliary Devices For Music (AREA)
- Electrophonic Musical Instruments (AREA)
Abstract
The present invention relates to a method for displaying sound essence by using graph and its equipment capable of displaying waveform of sound, main frequency characteristics, beat and change produced with time. Said invention adopts colour, form position and movement, and can display several graphs simultaneously in the mode of concentric circles or placing them at adjacent position, can overlap or no overlap them to represent essence of sound in the different field. The characteristic frequency of sound can be calculated by using rapid Fourier transform. Said invention contains an algorith based on "zero point space" and with low calculating quantity and matches it with autocorrelation function to emphasize main frequency characteristics. It also contains a helical coordinate, and can provide higher accuracy to easily identify the characteristics of rhythm, chord and tone change, etc.
Description
Put it briefly, the present invention shows the essence of sound with a new form.It can be finished by write formula on multi-purpose computer, or cooperates software to finish with special electronic circuit, can also add machinery and optical devices and produce image.Specifically, the present invention relates to apparatus and method, can show its waveform, main frequency feature, beat and the variation of making in time with graphic presentation sound essence.What adopt is color, shape, position and mobile form.A plurality of figures can concentrically ringed modes or are placed on two adjacent positions and show simultaneously, can be overlapping or not overlapping, and with the essence of the different aspect that highlights sound.
Music performance and language pronouncing normally carry out under the guidance that the experience tutor is arranged.Though the tutor can discover the problem place of student's pronunciation, and tells how the student proofreaies and correct, and the degree of makeing mistakes then can not be put into words out quantitatively.In addition, also there are some equipment that sound frequency and other parameter are provided on some degree.These equipment comprise:
(1) oscillograph
Oscillograph is used in waveform and has shown for a long time, and the frequency of sound also can be dealt with very well-to-doly.But except some very stable and pure sound, oscillograph also fails to show stable waveform, and has only mixed and disorderly picture, Li Shayu (ISSAJOUS) figure is passed to two relevant signals respectively X and Y-axis and has avoided unsettled scanning to trigger, someone transports to stereosonic left and right acoustic channels the X-Y axle and obtains some stable figure, but still fails to produce the significant figure of people.
(2) metronome
The beat of musical performance is to make the greatest factor of playing the tool timing.In the performance of being made up of many musical instruments or group's performance, timing is to be produced by cooperatively interacting of many musical instruments.Just can detect the imbalance of different players on rhythm although depend merely on people's ear, but will accurately judge it is that whose imbalance is just more difficult.Whether in addition, find the relative duration of forming the beat each several part meets the requirements and also has any problem.
There are some instruments can assist to grasp rhythm.A kind of metronome commonly used is the pendulum of a upside-down mounting, can transfer to such an extent that make it by fixing periodic oscillations, ticktocks in each cycle, and this also has other scheme in Machine Design.In these instruments, rhythm is set up in advance, and the performance personnel play according to the rhythm that mixes up.Rhythm is closelyed follow the quality of degree by subjective judgement.
Have many tone measuring instruments can supply the musical instrument tuning, that is to be undertaken by the frequency that shows the tone that produces.Yet these measuring instruments are just lack scope for their abilities in the general vertiginous occasion of playing procedure medium pitch.Even measuring instrument can play a role, the numeral that the player also can't fast understanding shows.
(3) spectrogram
Usually spectrogram is mainly used in transformation and the resolution " sound class " of identifying tone (has reference frequency clearly, so frequency spectrum is clear) and " non-sound class " (reference frequency clearly not, so frequency spectrum disperses), the four tones of standard Chinese pronunciation that can be used to learn Chinese (or nine or other family of languages of Guangdong language).
Select as another kind, spectrogram can show that sound embodies the fourier transform of tone and formant frequency transient change, can also demonstrate each ingredient of sound all in the multiple performance or human speech.Suitably mark frequency axis and ruling and just can express each tone of sound.But intensity of variation is difficult to the naked eye see accurately, and the variation of same tone is also different in low-and high-frequency, so spectrogram is just very limited as the effect of a teaching tool.
Spectrogram also has such shortcoming: the scale of distinguishing the tone height is intensive, and the minor deviations that can't with the naked eye find out at spectrogram acoustically may have great difference, and the overtone of tone also makes chart seem chaotic.Therefore,, seldom be applied in the musical performance, even in voice application, also seldom be used for training in real time and feedback though frequency spectrum is widely used for analyzing speech.
An object of the present invention is to provide the image display method of a kind of sound and beat.
Another object of the present invention provides the image display unit of a kind of sound and beat.
Utilization of the present invention reflects the different essential of sound in every way with pattern and the color that sound changes, waveform and melody that reference period that utilization calculates from waveform or preset period drawing waveforms figure and spectrogram counterweight cover can obtain more stable image, to the sound in changing then by the corresponding track variation of graphic presentation.
The characteristic frequency of sound can be calculated with fast Fourier transformation (FAST FOURIERTRANSFORM).The present invention comprises one based on the low calculated amount algorithm of " zero point at interval " (TIMEBETWEEN ZERO-CROSSING) and cooperate autocorrelation function (AUTO CORRELATION) to go to emphasize the main frequency feature.Another spiral coordinate provides than the higher precision of rectangular coordinate and polar coordinates and can make feature easy identification such as rhythm, chord harmony modulationization, by the mode of imagery show the waveform, frequecy characteristic, mutual relationship of sound and over time, pronunciation or player just can improve their performance.
Spiral coordinate of the present invention provides the resolution higher than rectangular coordinate, has avoided the overlapping phenomenon of per 360 degree of angle axle in the polar coordinates again.And the relative position of each overtone is the angle of fixing.
Luminous point disappearing in time provides the sensation to the time in the traditional instrument.In this invention, the various eigenwerts of sound be converted into color characteristic (luminance saturation and color), position and position over time or their mixing change.
In addition, also can make the feature of color for example represent 7 tones to cooperate luminosity and concentration to represent accuracy in pitch with the rainbow color corresponding to the feature of sound.This helps the accuracy in pitch training of some musical instrument such as violin etc.
Fig. 1 is the block scheme of the image display apparatus of sound of the present invention and beat,
Fig. 2 is signal processing flow figure,
Fig. 3 is the window function that uses in the Fourier conversion,
Fig. 4 utilizes fourier transform to find out characteristic frequency,
Fig. 5 is measuring at interval at zero point,
Fig. 6 is based on and draws different reduction oscillograms zero point by spacing,
Fig. 7 is the flaabellum shape pattern that tone m and t cause,
Fig. 8 is spiral coordinate tone figure,
Fig. 9 is the Line Chart of spiral coordinate on peak value display frequency,
Figure 10 is polar coordinates beat figure,
Figure 11 is the trajectory diagram that shows on the spiral coordinate that joint sound degree point inwardly shrinks with beat,
Figure 12 utilizes form and aspect, saturation degree and light secretly to reflect the frequency plot of sound accuracy,
Figure 13 is the result of sound after the Fourier conversion
Figure 14 is the real face oscillogram of sound frequency and Strength Changes on the spiral coordinate,
Figure 15 is the crest outside drawing of sound frequency and Strength Changes on the spiral coordinate,
Figure 16 utilizes the waveform weighted mean to reduce the schematic diagram of scan speed.
With reference to the accompanying drawings, describe most preferred embodiment of the present invention in detail.
At first consult Fig. 1, there is shown block scheme, label 1 expression sound importation according to the image display apparatus of sound of the present invention and beat, 2 expression signal Processing and control sections, 3 expression user interfaces, 4 expression display driving parts, 4 expression display entities.
Sound at first becomes signal Processing and control section 2 acceptable forms by sound importation 1, and the simulation process circuit can be accepted time dependent voltage or electric current as input.Computing machine or digital circuit can be accepted a frequency and the enough numeral inputs of waveform resolution.Recording that has stored or music score can convert available analog or digital form earlier to and originate as sound.The user chooses the pattern of demonstration and related parameter is arranged by user interface 3.Signal Processing and control section carry out computing according to voice signal of receiving and user's selection then.Operation result is transferred into display driving part 4 acceptable forms.5 outputs and acoustic phase graph of a correspondence in the display part at last.Wherein signal processing 2 can be fixedly electronic circuit or computing machine, the combination of software and hardware accelerator.Display driving part 4 can be the combination of simple numerical analog converter or photoelectricity and mechanical hook-up.Display part 5 can be general computer monitor or recording camera, can also be the reflective thing of projecting beam and front.
The present invention also provides effective implementation method to reach to required dynamic effect to above dot matrix and mechanical means graphing.
The present invention is with the essence of a new form demonstration sound, and it can be finished by write formula on multi-purpose computer, or cooperates software to finish with special electronic circuit, can also add machinery and optical devices and produce image.
The present invention adopts following method that the sound input is converted to figure output.The flow process of signal Processing sees also Fig. 1.
The details of each part of signal processing flow figure among Fig. 2 is as follows.
(1) extracts frequecy characteristic
(1) fourier transform
The frequency content of calculating sound at present all is to adopt fast Fourier transformation (FFT).Its frequency accuracy is average.For example precision is 5 hertz (cycles per seonds), it is 5/1000=0.5% 100 hertz precision, 100 hertz precision then is 5/100=5%, in 12 average scales, the gap of scale only has an appointment 5%, for the precision that makes visual low frequency part improves, in lower sampling frequency but than the FFT processing that adds under the long time.The frequency that does not accord with requirement in a certain precision begins to calculate additional FFT with lower sampling frequency and on the time than long window function.In the processing of lowest frequency, will increase the delay of demonstration time because of strengthening window function, so window function is designed to more flat to replace general cusp configuration:
Fig. 3 represents the window function of use in the Fourier conversion
When actual computation, what the width of window function was determined to carry out is broadband or arrowband analysis, has the branch of broadband and arrowband the same with spectrum analysis figure.
Fig. 4 represents to utilize fourier transform to find out characteristic frequency
In Fig. 4, find out each local peaking earlier.If peak value is too low, just this small peak to be cast out, too wealthy crest belongs to noise and also can not want.Remaining local peaking measures their width separately, Yi Bian half makes starting point till the another side from being equal to peak value exactly.This width multiply by the area that peak value approximates crest, can be considered the intensity of local peaking, or the intensity of this frequency content.Thereby draw one or above quaternary (time t, crest sequence number n, frequency f, intensity ordinal number a).
After fourier transform, delete tiny and wealthy excessively local peaking, if sound contains reference frequency, frequency peak can be not deleted clearly for some.The f of these local peakings is exactly a characteristic frequency.
(2) zero point, interval and waveform parameter extracted
Fig. 5 represents measuring of interval at zero point
Waveform from negative by forward to zero point on the occasion of time be exactly zero point at interval, as the t among Fig. 5
1And t
3Similarly, from the occasion of the interval that forwards negative value to, as t
2, also be zero point at interval, adjacent addition at interval at zero point more than is as t
1+ t
3, also zero point is at interval at last.For stable and simple waveform, zero point is also very stable at interval, also equals the sound wave time in one week.The for example sine wave in one 500 week, then t
1=t
2=t
3The reference period of=1/500 second=0.002 second this frequecy characteristic that is sound wave.Its inverse (negative first power) is exactly a reference frequency.For more multidata, t are provided
4, t
5, t
6, t
7As the layout at zero point, their inverse is exactly zero point by frequency Deng also, the actual sound mixing of a plurality of frequency contents often, or even do not have stable frequency as grating etc.This characteristic feature of an invention be each all be at interval at zero point orderly five dimension values (time, every number, interbody spacer, autocorrelation value, wave amplitude) or note work (t, n, x, r, q), time is the mean value of this section on time coordinate, one of them five dimensions point as the waveform on Fig. 4 is (t, c, x, r, a)=(t
4-6, 3, t
4, t
5, t
6, r
4-6, a
4-6), r wherein
4-6Be exactly with t
4-6For the center time interval is t
4+ t
5+ t
6Autocorrelation function.
Autocorrelation function except by its mathematical definition directly from the waveform sampling value is calculated, characteristic that also can be by the hypothesis waveform is also measured (or to calculate in the sampling) some character numerical value and is simplified.
Fig. 6 by the difference setting mainly based on draw different reduction waveforms zero point by spacing.Wherein:
(a) equivalent hypothesis: promptly waveform only may be one fixing on the occasion of or the rectangular wave of negative value.
(b) square hypothesis: the height of rectangular wave crest is proportional to and passes through interlude zero point.
(c) trapezoidal hypothesis: but waveform is the trapezoidal of a certain selected shape.
(d) para-curve mean value revises hypothesis: waveform is a para-curve and mean value is identical with original waveform.
Fig. 6 represents based on draw different reduction oscillograms zero point by spacing
More than four kinds of hypothesis have only (d) need measure or calculate the additional parameter wave-average filtering value of compartment (zero point) of original waveform beyond at interval at zero point.
Based on different hypothesis and different window functions, can simplify the calculating of autocorrelation function to some extent.Cooperate a square window function as the equivalence hypothesis, then computational short cut is a plus and minus calculation.
From the above zero point that calculates by at interval and the calculating of autocorrelation function can reflected waveform frequecy characteristic.Autocorrelation value is bigger, more is important characteristic frequency.And minimum characteristic frequency is exactly a reference frequency, and its inverse is called reference period.Have only one can use following calculation to choose data as known main audible frequency.
If at (t
i, n
i, x
i, r
i, a
i) in, r
IVery approaching, as＞0.9, x then
IBe about reference period or its integral multiple.As (t in addition
j, nk, x
j, r
j, a
j) in, r
jAlso very near 1, as＞0.9, x
i≈ mx
j, t
j≈ t
j, wherein m is a positive integer that numerical value is very little, as 2 or 3, and (t then
i, n
i, x
i, r
i, a
i) middle x
IThe m that is about reference period doubly, and reference period is x, because of finding x
jSo, do not need display dot (t
i, n
i, x
i, r
i, a
i)
In the mankind's sounding, the vibrations of vocal cords are exactly that the common per second of reference frequency was lower than for 400 weeks.But the resonant tank effect meeting of oral cavity and nasal cavity causes the resonant frequency of higher-frequency to 1,000 to 3,000 week of per second.They can find by the interval from zero point.When processed voice is imported, can do low frequency and High frequency filter to sound respectively earlier, and find out the reference frequency of vocal cords vibrations and the resonance frequency in mouth and nose chamber respectively at the low frequency and the high-frequency range calculating autocorrelation function of expection.
More than based on zero point method at interval be fit to handle simple musical sound.When enough arithmetic capabilities are arranged, can directly calculate the Fourier conversion of waveform, find out reference frequency then.
(2) coordinate Calculation
Emphasize above-mentioned frequecy characteristic data and the reference frequency that calculates, can utilize following coordinate Calculation method.
(1) waveform coordinate Calculation
Utilize the reference period or the predetermined period drawing waveforms figure of waveform.The instantaneous reference frequency of utilizing (one) to find out cooperates following oscillogram impasto, and the shape invariance of the waveform that draws under different frequency uses preset value then to utilize figure to highlight the difference of different tones.As musically, predeterminable frequency is 1/256 or the fixed value selected based on the works of playing and sing.In the following discussion, with " T " expression, t then is a time variable, and s (t) is the voice signal value of time " t ".
Use (a) Cartesian coordinate and (b) polar coordinates respectively, (being angle, length)
(a) X-Y track
X: horizontal ordinate
Y: ordinate
Cartesian coordinate (x, y)=(S (t), S (t-kT))
Wherein k is a phase parameter, elects the value between 0 to 1 usually as.
Wherein T is a cycle parameter, elects that find or default reference period usually as.
The frequency of transferring with central C is an example, and T equals 1/256 second; Central authorities C heighten octave then T like 1/ (512) second.When k=1/4, the cycle is that the sine wave of T can be drawn out a circle.Other tone then can draw the ellipse of different degree of tilt.And impure sinusoidal wave musical instrument can cause irregular edge.
(b) θ-r track
θ: angle coordinate
R: length coordinate
(rθ)=(s(t)+K
r,K
θ·2π·t/T)
Polar coordinates
K
r: the orbital radius parameter
K
θ: the track scanning speed parameter
As the value of s (t) between-1 and 1
K
r=0.5
K
θ=0.5
T=1/262 second
Nature melody medium pitch d, r, m, f, s, l, the cycle of t and K
2The ratio of/T is respectively 2,9/4,5/2,8/3,3,10/3,15/4,4.The K that tone m and t draw
2/ T is 5/2 and 15/4, draws the flaabellum shape simulation pattern of figure below respectively.
Fig. 7 represents the flaabellum shape pattern that tone m and t cause.
Figure is that the frequency between the note of music is simple proportionate relationship, or very near simple ratio, if the frequency of choosing a fundamental note as T, and K
2Be simple fraction, then the track of Hui Zhiing can form and stable heavily cover track (accurately simple ratio), or does rotation (deviation being arranged with simple ratio) at a slow speed.
If being waveform, T derives out, then can time to time change, and can use the reference period moving average that finds in short (as 0.1 second) at interval as T.Simultaneously can be with the period T of an above characteristic frequency
1, T
2Show simultaneously Deng calculating respectively and drawing a plurality of patterns.
(2) tone coordinate Calculation
Adopt following each coordinate system to emphasize the frequecy characteristic coordinate conversion.
(a) Cartesian coordinate (X, Y), i.e. rectangular coordinate
(b) polar coordinates (θ, r), i.e. (angle, length)
(c) the spiral coordinate (θ, Y), promptly (angle, highly)
Wherein flute card and polar coordinates all have a detailed description in the mathematics textbook.
The spiral coordinate that the present invention uses is both mixing.It can be considered as a rolled-up Cartesian coordinate of X-axis or the angle axle does not have overlapping polar coordinates.(θ, (X, Y) X-axis is accurate and be more convenient for highlighting periodic relation for θ axial ratio Y).
Above coordinate in order to mark the characteristic frequency data (t, n, x, r, a) or (f when two axles a) are drawn, generally uses frequency and non-periodic for t, n, thus can with (t, n, f, r, a)=(t, n, l/x, r is a) as the data of drawing usefulness.
(a) based on the characteristic that will emphasize, can t, f, r, a wherein two as coordinate (x, y), (θ, r), (θ is on diaxon y).N is then in order to distinguish different data sets or to ignore.
(b) especially when the spiral coordinate (θ, when θ axle y) is frequency, adopt a circle (i.e. 360 degree) be equivalent to one musically octave or with the corresponding ratio of linguistic tone variation, then the distance of each note or tone is all corresponding to the differential seat angle distance on the coordinate.As 12 tones of 12 average scales as corresponding to 12 hours on the clock, as shown in Figure 8.When Y is time coordinate, suppose two very zero seconds and 1 second of time coordinate, inner terminal is the peak value of present (zero second is preceding), outermost end then is the peak value before 1 second.Peak point is healed bright representative voice more by force: a little less than more then representative voice is healed.In example just now, it is dark that the sound peaks of crescendo can become a inner, the Line Chart that the outer end is bright.When sound frequency is unstable (as the tremolo effect occurring), just can show the curve of a swing.After sound disappeared, the crest frequency line then can shorten from inside to outside gradually, complete obiteration after a second.
Fig. 8 represents spiral coordinate tone figure.
Fig. 9 represents the Line Chart of spiral coordinate on peak value display frequency.
(c) (t, n, f, r, a) or (t, n, f, a) in time close and n different characteristic Frequency point pairing, as (x, the y) diaxon of coordinate is as two data (t
i, n
i, f
i, r
i, a
i) and (t
j, n
j, f
j, r
j, a
j) t
I≈ t
jAnd n
i≠ n
jThen can draw point (x, y)=(f
i, f
j), the wide and height of the frequecy characteristic point of its track reflection sound is corresponding to a
iAnd a
jTo reflect their relative intensity.
(3) beat coordinate Calculation
The coordinate Calculation of beat can have following (a) to reach (b) two kinds of methods:
(a) polar coordinates show beat (θ, r)=(time, frequency)
The present invention shows the fourier transform and the predominant frequency of sound with polar coordinates.In fact, this is the polar coordinates type spectrogram that has added predominant frequency.When equilibrium separated tone, frequency axis (radial axle) adopted logarithmic coordinate.But with regard to the rhythm pattern, also can adopt log-log coordinate, linear coordinate or other coordinate and draw similar effects.In addition, near the center shown be that all it doesn't matter for high frequency or low-frequency sound.The camber line that music tone marks always is directly proportional with its duration.
Be accurately to distinguish the height of each acoustic tones, further through handling, overlap the centre frequency higher or that show with outstanding color of brightness on the frequency display graphics to detect mass tone through the signal of fourier transform.Angle time coordinate axle indicates each minute musical note of the rhythm pattern that requires.When making general service, it is the needs that 15 degree fan-shaped will be satisfied most of general beat type that circle is divided into 24 central angles, for example 2/2,4/4,4/8,3/3 etc.Each note has just shown as the part of beat with from the duration that the deviation that correct meter is clapped produces like this.
In the spectrogram of standard, curve is not that volume (SCROLL) goes out to provide circulation exactly to write full.Circulation is write and completely is applicable to polar curve.Yet, the difference of beat after total hope can be compared.Thereby preferably can use the overlapping of translucent effect.
For demonstrating the contrast between higher pitch sounds and the low pitch sound, the unwanted frequency scope should as far as possible little covering in the frequency coordinate.This is actively to dwindle and enlarge frequency coordinate according to the frequency range of previous several beats to carry out.
Figure 10 represents polar coordinates beat figure
(b) utilize spiral sit to show beat (θ, Y)=(time, frequency)
Above introduced the variation that the method for utilizing concentrically ringed angle to spend coordinate as beat coordinate (being θ=time) and radius thereof in a low voice as height shows beat.But equate or when similar, the sound degree track of new beat just can cover the old sound degree track rear that does not disappear as yet, the phenomenon that formation overlaps each other with old when sending new sound degree.
Make the trajectory diagram more than the circle show simultaneously, avoid occurring the phenomenon that new and old track overlaps each other simultaneously, just can utilize a spiral coordinate that constantly inwardly shrinks to show beat.As shown in figure below, the sound degree track on the spiral coordinate can be along with new syllable beat, along the stretching, extension of constantly circling round of spirality coordinate.As long as with the spiral coordinate and above the figure that drawn is whole inwardly shrinks simultaneously, to keep whole size constant.This method not only trajectory diagram of equal sound degree can be revealed on the coordinate of different beats, and has avoided the ad infinitum abducent problem of sound degree track.Simultaneously the speed that rolls out new data that directly moves at the X-Y coordinate of the velocity ratio of figure radial shrinkage will be slowly many, make things convenient for eye-observation.
Figure 11 represents to show on the spiral coordinate that joint sound degree point is with the inside trajectory diagram that shrinks of beat
(3) color calculates
During generating writing pattern, can show the data that do not have use in two axial coordinates with change in color.Color can be expressed as three-dimensional value: H (form and aspect, the i.e. variation of the yellowish green grade of blood orange), S (saturation degree, promptly distinct or light white), V (brightness).For example (t, n, f, r, a) → (X, Y, H, S, V):
X=F
1(t)
Y=F
2(f)
H=F
3(r)
S=is from the value of establishing
V=F
4(a)
Then form and aspect and brightness correspond respectively to autocorrelation value and loudness of a sound degree and do not reflect n.
(1) emphasizes the frequency accuracy of sounding with nonlinear relation
The present invention utilizes different form and aspect to show various tones, and the saturation degree of color shows whether tone accurate, bright with show slinkingly the power of showing tone.As in figure below horizontal coordinate, the red c sound of representing, green is represented the e sound, so analogizes.The height of vertical coordinate is represented the height of color saturation.When sound is c timing very accurately, saturate redness, promptly cerise just can be revealed.If sound is to begin to depart from c to transfer, and trends towards the d timing, shown redness can become more and more light and be partial to orange gradually.The nonlinear relationship of form and aspect and frequency makes small detonieren produce the obvious color transformation, the light of adding color is dark, can make change with the power of difference tone accent itself, thereby draw out the accuracy of demonstration tone, tone and the strong and weak triadic relation's of tone frequency figure.
Figure 12 represents to utilize the frequency plot of form and aspect, saturation degree and the accuracy of luminance brightness reflection sound
(4) graphing.
At Fourier conversion back frequency data, the intensity of sound also can strengthen, weakens or keep to stablize in fixing frequency in time gradually, makes waveform change rapidly and can not stably show.The present invention then utilizes the color of gradual change, comes the relation of express time and intensity of sound, utilizes the mode of filling up whole real face or describing the crest profile, to show the conversion of intensity of sound in a period of time than stable status.Explain that for convenient Figure 13 is the result of sound after the Fourier conversion.
Figure 13 represents the result of sound after the Fourier conversion
(1) fills up real face
Utilize color to show that time coordinate can have following hypothesis: the expression color of (before zero second) now is an emerald green, becomes dim in time gradually, and orange representative is before half second, before black is represented one second.Hypothesis has three kinds of sound frequencies now again, and three crests are represented crescendo, stable and three kinds of intensities of sound of diminuendo respectively.The sound that strengthens gradually in time, its waveform can become greatly gradually, is not having under the situation of transparent effect, and more near now, the huge bud green crest of healing can constantly occur and cover become before dimness and more tiny crest.The result forms one and becomes big bud green crest gradually.Certain is waterborne when the intensity of sound continues to be fixed on, and the size of crest also can not become big gradually or dwindles because of the time.And more near present bud green crest, the crest that has become dim before can just covering, the result forms a stable and bud green crest.When the intensity of sound constantly weakens, more near now, the crest that color heals bright-coloured can dwindle gradually because intensity constantly weakens, and area is bigger before the order, but that the crest that color has dimmed out is covered fully.The result forms a periphery and becomes dim gradually and disappear, and presents emerald green inside gradually, diminishing from level to level figure.With the horizontal coordinate twist up, just form the display result of figure below again.
Figure 14 represents the real face oscillogram of sound frequency and Strength Changes on the spiral coordinate.
(2) describe the crest profile
Real face with display frequency figure changes the profile of display frequency figure into now.Utilize (a) same hypothesis, the waveform of crescendo can be shown as an inside because having avoided the mutual situation about hiding of real face and become dim gradually, and the periphery presents crest bud green and that outwards enlarge gradually.Stable sound frequency can be shown as one and depict the crest profile, and inside does not have the bud green crest of color.And the sound frequency of diminuendo then is opposite phenomenon with the situation of sound frequency crescendo.Promptly be a peripheral dimness, inside presents bud green and diminishing crest.On the spiral coordinate, just form the display result of figure below.
Figure 15 represents the crest outside drawing of sound frequency and Strength Changes on the spiral coordinate.
(5) image output
The above-mentioned figure of drawing, can be with following (1) and (2) two kinds of way of outputs:
(1) for the display device of dot matrix, then can utilize software in order to following method, hardware or combination thereof form dynamic color.
(a) upgrade pixel (Pixel) color.
In with the continuous graphing of new voice data, the color of every bit can be upgraded to reach color changeable effect at regular intervals on the image.General mode is R
N+1, G
N+1, B
N+1,=F (R
n, G
n, B
n).Show that in the RGB mode different colours is an example, 0 representative is the most black, and 1 represents light, when color dot is painted out, though its numerical value be 0,1 or boundary 0 to 1 between any numerical value, can its numerical value be deducted 0.02 with per 1/50 second speed.If when beginning, the numerical value of pixel was 1, just it can be in one second is become black and disappeared by light; The numerical value of pixel was 0.5 when the false beginning began, and it just can disappear in half second, so analogized.Can reach fade effect by other transformation rule from a kind of color transition to other different colours.For example adopting HSV is that (form and aspect, concentration, brightness) color coordinates can utilize the change form and aspect and cause the rainbow color changeable effect.
(b) upgrade palette (Palette).
Utilize the change of time and upgrade color numerical value on the palette, also can reach dynamic tonal variation.Mode Show Color with RGB is an example, supposes by in 256 kinds of colors of 0 to 255, sets 100 kinds of dynamic colors that change usefulness of colors conduct of 0 to 99, recycles.Back several 50 colors with the numbering of crossing of the color code of current use are formed required progressive formations, as from bud green orange, the Huang and red of carrying out the transition to.The color of 50 remaining numberings is background colour entirely, is the independent background color code and get numbering 255 arbitrarily in addition, and remaining numerical value is then made other purposes.When graphing, the color code of drawing usefulness added 1 until 99 in per 1/50 second, later on then again since 0.The color of palette is also upgraded simultaneously, forms required gradual transition with the back several 50 kinds of colors of the color that keeps current use.At each pixel of checking within each second on the picture,, promptly be rewritten as background color numbering 255 entirely in addition as for temporary transient 50 kinds of color codes not using as the composition gradual change.This step there is no the change graphic color, but is that the color code Recycling is necessary.
(2) to the display device that comprises machinery with than the low-velocity scanning track
With respect to the conspicuous sound waveform of hundreds of, the variation that the speed that machinery scans equipment often fails to follow waveform shows synchronously.So machinery scans the waveform synthesis that equipment can before fail in time to show, and the weighted mean value that calculates these waveforms so scans out again.As shown in figure below, suppose that sound frequency is than fast ten times of the scan speed that can finish one-period with respect to plant equipment.Finish with the speed of sound frequency 1/10 and scan a sound wave during cycle when machinery scans equipment, ignored the data in 9 audio frequency cycles.The data in these nine cycles in time be reflected, mechanical scanner have to utilize their weighted mean value, scans the track data in cycle as the next one.
Figure 16 represents to utilize the waveform weighted mean to reduce the schematic diagram of scan speed.
Based on above-mentioned calculating and display mode, following variation can be arranged:
(1) above introduced and utilized various calculating and display mode, the image of various waveforms and the frequency of on Cartesian coordinate, polar coordinates and spiral coordinate, painting out.These are because of the indivedual images that calculate and the display mode difference is depicted as, can or adjacent to each otherly provide multiple visual feedback with concentric form in fact, for example dynamically change their position, ratio and color selection etc., it was both regular to become a width of cloth width of cloth, and the sound image that changes all the time.Effect is better than current stage or laser light.Except image, the cycle of detected sound, beat and other parameter also can show together with the form of numeral.
(2) music score or the music that records in advance can be utilized the calculating of the present invention to music, and the music with being about to play shows in advance with figure.As the Karaoke of current trend, the chanteur can be before each sound begins, and sees through graphic change and knows in advance and assorted petty music will occur, gets ready to when reaching how correctly to go into to sing.
(3) when sound when source that more than one occur, can other sound or pre-record and default music score and musical instrument expect that the sound that sends makes independent processing, and the general after treatment indivedual graphics overlays and merge and become a comprehensive figure.This figure that combines a plurality of sound sources can be with the difference of indivedual sound sources of making comparisons, as when carrying out the relevant training of band's exercise or music, can know immediately that deviation appears in the there.
(4) same sound source is high, in and the part of low frequency separate separate processes then with wave filter, plot other figure.This way can independent displaying sound composition or band in the character of various sound.The vibration frequency of vocal cords was less than about 400 hertz greatly when for example, the people spoke; The resonant frequency that mouth and nose cause then is higher than 1000 hertz.With low-and high-frequency part separate processes, can obtain several simple and be easy to distinguish images earlier, after again other figure being merged, the convenient multiple characteristic of analyzing in the same sound.
Claims (26)
1. the method that sound is become figure comprises:
(1) the fourier transform and the zero point of waveform extracting the sound frequency feature at interval, and when sound contained reference frequency, week was to find out one or above reference frequency of sound.
(2) with the waveform values (SIGNALVALUE) of frequecy characteristic data or original waveform calculating drawing usefulness, emphasize waveform respectively, tone and beat.
(3) with the color of frequecy characteristic data computation drawing usefulness.
(4) with (2) and (3) resulting Data Dynamic ground graphing.
(5) the dot matrix display device is made dynamic change color with hardware, software or software and hardware cooperation.
(6) mechanical display device utilization is calculated the weighted mean value of sound waveform to scan the track of sound waveform than low velocity.
2. the method for claim 1 is characterized in that fourier transform begins to calculate additional fourier transform with lower sampling frequency and on the time than long window function in the frequency that a certain precision does not accord with requirement.In the processing of lowest frequency, window function is designed to more flat to replace general cusp configuration.After the fourier transform, delete tiny and wide local peaking, remaining local peaking is exactly a characteristic frequency.
3. the method for claim 1, the zero point that it is characterized in that calculating waveform at interval and corresponding autocorrelation function as the frequecy characteristic data.The inverse at interval at zero point that one or above corresponding autocorrelation function numerical value are big is exactly a reference frequency.
4. method as claimed in claim 3, it is characterized in that with zero point of waveform at interval and simple parameter comprise that zero point, peak value and centre of gravity place lowered the calculated amount of autocorrelation function with the waveform of mathematical approach algorithm reduction through simplifying at interval.
5. the method for claim 1 is characterized in that utilizing the reference period that found or default reference period to draw the video figure, and the figure that order is drawn out is more stable.
6. the method for claim 1, it is characterized in that drawing (x, y) the track Cartesian coordinate (x, y)=(S (t), S (t-kT), wherein
S (t): waveform is in the value of time t,
K: phase parameter, elect the value between 0 to 1 usually as,
T: cycle parameter, elect that find or default reference period usually as,
X: horizontal ordinate,
Y: ordinate.
7. the method for claim 1 is characterized in that (angle is a time shaft for r, θ) track, and radius is the waveform axle in drafting.(r,θ)=(S(t)+K
r,K
2·2π·t/T)
Wherein
S (t): waveform is in the value of time t,
K
r: radius parameter, elect the value between 0 to 1 usually as,
K
2: simple fraction,
T: cycle parameter, elect that find or default reference period usually as,
θ: angle coordinate,
R: length coordinate.
8. the method for claim 1 is characterized in that the coordinate with a diaxon, and the polar coordinates and the spiral coordinate that comprise rectangular cartesian coordinate, represent with length and angle are to reflect periodic relation.
9. the method for claim 1, it is characterized in that when the spiral coordinate (θ, when θ axle y) is frequency, adopt a circle (i.e. 360 degree) be equivalent to one musically octave or with the corresponding ratio of linguistic tone variation.
10. the method for claim 1 is characterized in that (θ, Y-axis y) is as time coordinate the spiral coordinate.
11. the method for claim 1 is characterized in that polar coordinates show that (θ is the polar coordinates type spectrogram that has added predominant frequency r)=(time, frequency) to beat, and the angle of the camber line that music tone marks always is directly proportional with its duration.For accurately distinguishing the height of each acoustic tones, further process processing of signal through fourier transform, overlap the centre frequency higher or that show with outstanding color of brightness on the frequency display graphics to detect mass tone, angle time coordinate axle indicates each minute musical note of the rhythm pattern that requires, when making general service, it is the fan-shaped of 15 degree that circle is divided into 24 central angles.
12. as the method for claim 11, it is characterized in that the brightness of the image retouched out simulates the attenuation of phosphor persistence effect, thereby make the overlapping folder that demonstrates of current and previous beat that the performance personnel see and will gain enlightenment.
13. method as claim 11, it is characterized in that, the color time to time change of the image of retouching out, the image of newly retouching uses translucent effect and overlaps on several images that scan, thereby make the imbalance on the beat cause the edge diffusion phenomena, the performance personnel have seen the enlightenment that just obtains the beat imbalance from the color aspect.
14. the method for claim 1, it is characterized in that utilizing the spiral coordinate show beat (θ, Y)=(time, frequency),
Utilize a spiral coordinate that constantly inwardly shrinks to show beat.Sound degree track on the spiral coordinate can be along with new music beat, along the stretching, extension of constantly circling round of spirality coordinate.As long as with the whole inwardly contraction simultaneously of spiral coordinate, to keep whole size constant, this method not only trajectory diagram of equal sound degree can be revealed on the coordinate of different beats, and has avoided the ad infinitum abducent problem of sound degree track.
15. the method for claim 1, when it is characterized in that generating writing pattern, change in color shows the data that do not have use in two axial coordinates.
16. method as claimed in claim 15 is characterized in that emphasizing with nonlinear relation the frequency accuracy of sounding.Form and aspect show various tones, the saturation degree of color shows whether tone is accurate, light shows slinkingly the power of showing tone, when sound is very accurately during tone, saturate bright-colored just can be revealed, if sound begins to depart from, the form and aspect of shown color can change, color saturation can become more and more light simultaneously, the light of adding color is dark, can make change along with the power of tone itself, thereby draw out the accuracy of demonstration tone, tone and the strong and weak triadic relation's of tone frequency figure.
17. the method for claim 1 is characterized in that utilizing the color of gradual change, comes the relation of express time and intensity of sound, utilizes the mode of filling up whole real face or describing the crest profile, to show the transformation of intensity of sound in a period of time than stable status.
18. the method for claim 1, it is characterized in that display device for dot matrix, utilize software, hardware or combination thereof to cause dynamic color, the color of every bit can be upgraded to reach color changeable effect at regular intervals on the image, color with the RGB coding, at regular intervals, the numerical value of redgreenblue is deducted a free numerical value, constantly heavily cover until last disappearance.
19. the method for claim 1, it is characterized in that display device for dot matrix, utilize software, hardware or combination thereof to form dynamic color, the color of every bit can be upgraded at regular intervals to reach color changeable effect on the image, at regular intervals, the numerical value of luminosity is deducted a free numerical value, simultaneously form and aspect or concentration are changed, constantly heavy coated with reaching rainbow color changeable effect and last the disappearance.
20. the method for claim 1, it is characterized in that utilizing the change of time and upgrade the color of regulating on the colour disk, in all available color codes, the color of choosing a certain quantity is as the color that dynamically changes usefulness, recycle, the color code of current use is back counted the color of a certain quantity and is formed required progressive formation, the color that remaining conduct dynamically changes usefulness is background colour entirely, and get a certain color code arbitrarily in addition is the independent background color code, remaining numerical value is then as other purposes, when graphing, each fixes time the color code of drawing usefulness and goes to next available color code, recycle, the color of palette is also upgraded simultaneously, back count the color code of used a certain quantity with the color that keeps current use and form required gradual transition, in addition before all dynamic color codes use up, check each picture point on the picture, as the temporary transient color code of not using as the composition gradual change, then be rewritten as the background color numbering entirely, this step there is no the change graphic color, but is that the color code Recycling is necessary.
21. the method for claim 1, it is characterized in that the display device that comprises machinery with than low-velocity scanning, the speed in sound frequency cycle, the scan speed that can finish one-period than plant equipment is for fast, when machinery scanned equipment and scans with the speed that is lower than a certain ratio of sound frequency, the weighted mean value of the voice data that mechanical scanner utilization is not scanned scanned the track data in cycle as the next one, they are scanned out, become machinery and scan the equipment next one and scan the cycle.
22. the method for claim 1, it is characterized in that will be because of the individual image of calculating and the display mode difference is depicted as, with concentric form or adjacent to each otherly provide multiple visual feedback, dynamically change their position, ratio and color selection etc., it was both regular to become a width of cloth width of cloth, the audiovideo that changes all the time again, except image, the cycle of detected sound, beat and other parameter also can show together with the form of numeral.
23. the method for claim 1 is characterized in that music score or the music that records in advance, can utilize the calculating of the present invention to music, the music with being about to play shows in advance with figure.
24. the method for claim 1, when more than one sound source occurring, other sound can be made independent processing, and indivedual graphics overlays that will be after treatment and merge and become a comprehensive figure, this figure that combines a plurality of sound sources can be used for the deviation of more indivedual sound sources.
25. the method for claim 1, it is characterized in that with the height in same sound source, in and the part of low frequency separate separate processes then with wave filter, plot other figure, this way can independent displaying sound composition or band in various sound character and obtain several simple and be easy to distinguish images, the convenient multiple characteristics of analyzing in the same sound.
26. an enforcement is characterized in that comprising: acoustic input dephonoprojectoscope, signal Processing and control device, user interface, display drive apparatus and display device according to the equipment of the method for claim 1.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB001082701A CN1151485C (en) | 2000-05-02 | 2000-05-02 | Sound and beat image display method and equipment |
| PCT/CN2001/000620 WO2001088905A1 (en) | 2000-05-02 | 2001-04-27 | Method and apparatus for displaying sound graph |
| AU2001275634A AU2001275634A1 (en) | 2000-05-02 | 2001-04-27 | Method and apparatus for displaying sound graph |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB001082701A CN1151485C (en) | 2000-05-02 | 2000-05-02 | Sound and beat image display method and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1321964A true CN1321964A (en) | 2001-11-14 |
| CN1151485C CN1151485C (en) | 2004-05-26 |
Family
ID=4579150
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB001082701A Expired - Fee Related CN1151485C (en) | 2000-05-02 | 2000-05-02 | Sound and beat image display method and equipment |
Country Status (3)
| Country | Link |
|---|---|
| CN (1) | CN1151485C (en) |
| AU (1) | AU2001275634A1 (en) |
| WO (1) | WO2001088905A1 (en) |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100365545C (en) * | 2001-11-16 | 2008-01-30 | 皇家飞利浦电子股份有限公司 | Dynamically configurable virtual window manager |
| CN100465964C (en) * | 2005-12-20 | 2009-03-04 | 索尼株式会社 | Content reproducing apparatus, list correcting apparatus, content reproducing method, and list correcting method |
| CN101634588B (en) * | 2008-07-25 | 2011-02-09 | 北京大学 | Method and device for drawing audio waveforms |
| CN101577114B (en) * | 2009-06-18 | 2012-01-25 | 无锡中星微电子有限公司 | Method and device for implementing audio visualization |
| CN101645268B (en) * | 2009-08-19 | 2012-03-14 | 李宋 | Computer real-time analysis system for singing and playing |
| CN101702316B (en) * | 2009-11-20 | 2014-04-09 | 北京中星微电子有限公司 | Method for converting MIDI music into color information and system thereof |
| CN104574453A (en) * | 2013-10-17 | 2015-04-29 | 付晓宇 | Software for expressing music with images |
| CN104820218A (en) * | 2015-05-26 | 2015-08-05 | 中国科学院声学研究所东海研究站 | Shallow sea seabed single parameter inversion method based on frequency domain autocorrelation |
| CN106128470A (en) * | 2016-06-29 | 2016-11-16 | 努比亚技术有限公司 | The device and method of display audible spectrum |
| CN106328164A (en) * | 2016-08-30 | 2017-01-11 | 上海大学 | Ring-shaped visualized system and method for music spectra |
| CN106373579A (en) * | 2016-08-31 | 2017-02-01 | 天脉聚源（北京）科技有限公司 | Method and device for displaying audio information |
| CN106468997A (en) * | 2016-09-13 | 2017-03-01 | 华为机器有限公司 | The display packing of information and terminal |
| CN107172483A (en) * | 2017-05-05 | 2017-09-15 | 广州华多网络科技有限公司 | A kind of tonequality under live scene knows method for distinguishing, device and terminal device |
| CN107329980A (en) * | 2017-05-31 | 2017-11-07 | 福建星网视易信息***有限公司 | A kind of real-time linkage display methods and storage device based on audio |
| CN107393517A (en) * | 2017-07-03 | 2017-11-24 | 腾讯音乐娱乐（深圳）有限公司 | A kind of frequency spectrum display methods and device |
| CN107901694A (en) * | 2017-11-14 | 2018-04-13 | 上海电机学院 | A kind of interactive drawing method and device based on sound frequency value |
| CN109120983A (en) * | 2018-09-28 | 2019-01-01 | 腾讯音乐娱乐科技（深圳）有限公司 | A kind of audio-frequency processing method and device |
| CN111341355A (en) * | 2019-12-17 | 2020-06-26 | 中原工学院 | Method for generating image, picture and pattern based on sound |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| HUP0600540A2 (en) * | 2006-06-27 | 2008-03-28 | Ave Fon Kft | System for and method of visualizing audio signals |
| JP2022051974A (en) * | 2019-02-12 | 2022-04-04 | ソニーグループ株式会社 | Information processing device, method, and program |
| US11532293B2 (en) * | 2020-02-06 | 2022-12-20 | James K. Beasley | System and method for generating harmonious color sets from musical interval data |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4040112A (en) * | 1972-12-29 | 1977-08-02 | Research Corporation | Direct electronic fourier transforms of optical images |
| JPH05265481A (en) * | 1992-03-16 | 1993-10-15 | Csk Corp | Speech editing device for natural moving picture |
| FI110220B (en) * | 1993-07-13 | 2002-12-13 | Nokia Corp | Compression and reconstruction of speech signal |
| JP3110215B2 (en) * | 1993-08-10 | 2000-11-20 | 沖電気工業株式会社 | Pronunciation training device |
| JPH07160676A (en) * | 1993-12-06 | 1995-06-23 | Fujitsu Ltd | Fast fourier transformation device and fast fourier transformation method |
| KR100273436B1 (en) * | 1998-07-31 | 2001-04-02 | 구자홍 | Sound generating apparatus and method |
-
2000
- 2000-05-02 CN CNB001082701A patent/CN1151485C/en not_active Expired - Fee Related
-
2001
- 2001-04-27 AU AU2001275634A patent/AU2001275634A1/en not_active Abandoned
- 2001-04-27 WO PCT/CN2001/000620 patent/WO2001088905A1/en active Application Filing
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100365545C (en) * | 2001-11-16 | 2008-01-30 | 皇家飞利浦电子股份有限公司 | Dynamically configurable virtual window manager |
| CN100465964C (en) * | 2005-12-20 | 2009-03-04 | 索尼株式会社 | Content reproducing apparatus, list correcting apparatus, content reproducing method, and list correcting method |
| CN101634588B (en) * | 2008-07-25 | 2011-02-09 | 北京大学 | Method and device for drawing audio waveforms |
| CN101577114B (en) * | 2009-06-18 | 2012-01-25 | 无锡中星微电子有限公司 | Method and device for implementing audio visualization |
| CN101645268B (en) * | 2009-08-19 | 2012-03-14 | 李宋 | Computer real-time analysis system for singing and playing |
| CN101702316B (en) * | 2009-11-20 | 2014-04-09 | 北京中星微电子有限公司 | Method for converting MIDI music into color information and system thereof |
| CN104574453A (en) * | 2013-10-17 | 2015-04-29 | 付晓宇 | Software for expressing music with images |
| CN104820218A (en) * | 2015-05-26 | 2015-08-05 | 中国科学院声学研究所东海研究站 | Shallow sea seabed single parameter inversion method based on frequency domain autocorrelation |
| CN106128470A (en) * | 2016-06-29 | 2016-11-16 | 努比亚技术有限公司 | The device and method of display audible spectrum |
| CN106328164A (en) * | 2016-08-30 | 2017-01-11 | 上海大学 | Ring-shaped visualized system and method for music spectra |
| CN106373579A (en) * | 2016-08-31 | 2017-02-01 | 天脉聚源（北京）科技有限公司 | Method and device for displaying audio information |
| US10694020B2 (en) | 2016-09-13 | 2020-06-23 | Huawei Technologies Co., Ltd. | Information displaying method and terminal |
| CN106468997B (en) * | 2016-09-13 | 2020-02-21 | 华为机器有限公司 | Information display method and terminal |
| CN106468997A (en) * | 2016-09-13 | 2017-03-01 | 华为机器有限公司 | The display packing of information and terminal |
| US10778832B2 (en) | 2016-09-13 | 2020-09-15 | Huawei Technologies Co., Ltd. | Information displaying method and terminal |
| US11025768B2 (en) | 2016-09-13 | 2021-06-01 | Huawei Technologies Co., Ltd. | Information displaying method and terminal |
| CN107172483A (en) * | 2017-05-05 | 2017-09-15 | 广州华多网络科技有限公司 | A kind of tonequality under live scene knows method for distinguishing, device and terminal device |
| CN107329980A (en) * | 2017-05-31 | 2017-11-07 | 福建星网视易信息***有限公司 | A kind of real-time linkage display methods and storage device based on audio |
| CN107329980B (en) * | 2017-05-31 | 2022-04-12 | 福建星网视易信息***有限公司 | Real-time linkage display method based on audio and storage device |
| CN107393517A (en) * | 2017-07-03 | 2017-11-24 | 腾讯音乐娱乐（深圳）有限公司 | A kind of frequency spectrum display methods and device |
| CN107393517B (en) * | 2017-07-03 | 2020-10-09 | 腾讯音乐娱乐（深圳）有限公司 | Frequency spectrum display method and device |
| CN107901694A (en) * | 2017-11-14 | 2018-04-13 | 上海电机学院 | A kind of interactive drawing method and device based on sound frequency value |
| CN109120983A (en) * | 2018-09-28 | 2019-01-01 | 腾讯音乐娱乐科技（深圳）有限公司 | A kind of audio-frequency processing method and device |
| CN111341355A (en) * | 2019-12-17 | 2020-06-26 | 中原工学院 | Method for generating image, picture and pattern based on sound |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2001088905A1 (en) | 2001-11-22 |
| CN1151485C (en) | 2004-05-26 |
| AU2001275634A1 (en) | 2001-11-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1151485C (en) | Sound and beat image display method and equipment | |
| US10789921B2 (en) | Audio extraction apparatus, machine learning apparatus and audio reproduction apparatus | |
| US10354625B2 (en) | Digital sight-singing piano with a fixed-solfège keyboard, continuous keys and adjustable tones by kneading piano keys | |
| CN1172291C (en) | Formant conversion device for correcting singing sound for imitating standard sound | |
| US6791568B2 (en) | Electronic color display instrument and method | |
| US11830466B2 (en) | System and method for generating harmonious color sets from musical interval data | |
| US11568857B2 (en) | Machine learning method, audio source separation apparatus, and electronic instrument | |
| Barrass | A perceptual framework for the auditory display of scientific data | |
| WO2007105927A1 (en) | Method and apparatus for converting image to sound | |
| Polo et al. | Musical Vision: an interactive bio-inspired sonification tool to convert images into music | |
| Horton et al. | Harmony through melody: The interaction of melody, counterpoint, and harmony in Western music | |
| Horner | Nested modulator and feedback FM matching of instrument tones | |
| CN110853457B (en) | Interactive music teaching guidance method | |
| JP2020021098A (en) | Information processing equipment, electronic apparatus, and program | |
| Lesbros | From images to sounds, a dual representation | |
| JP4665123B2 (en) | Musical scale creation method, musical scale creation program, computer-readable recording medium recording the program, musical scale creation apparatus, performance apparatus, and illumination apparatus | |
| Wu et al. | A study of image-based music composition | |
| JP2022521357A (en) | Systems and methods for generating harmonious color sets from pitch data | |
| US20160247494A1 (en) | Method of displaying musical score | |
| Fineberg | Sculpting sound. An introduction to the spectral movement: Its ideas, techniques and music | |
| Joo | Sonifyd: A graphical approach for sound synthesis and synesthetic visual expression | |
| JP2590725B2 (en) | Music data input device | |
| Pantalony | Seeing a voice: Rudolph Koenig's instruments for studying vowel sounds | |
| US20050268773A1 (en) | Apparatus and method for pitch training | |
| Andersen et al. | Grisey’s' Talea': Musical Representation As An Interactive 3D Map |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C06 | Publication | ||
| PB01 | Publication | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C19 | Lapse of patent right due to non-payment of the annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |