CN106531184A - Intelligent hardware piano music finishing training guidance method based on melody characteristic matching - Google Patents
Intelligent hardware piano music finishing training guidance method based on melody characteristic matching Download PDFInfo
- Publication number
- CN106531184A CN106531184A CN201610847637.3A CN201610847637A CN106531184A CN 106531184 A CN106531184 A CN 106531184A CN 201610847637 A CN201610847637 A CN 201610847637A CN 106531184 A CN106531184 A CN 106531184A
- Authority
- CN
- China
- Prior art keywords
- music
- intelligent hardware
- piano
- piano music
- melody
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 15
- 238000012549 training Methods 0.000 title abstract description 5
- 238000012937 correction Methods 0.000 abstract description 3
- 230000015572 biosynthetic process Effects 0.000 abstract 1
- 238000011161 development Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 229910000831 Steel Inorganic materials 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 230000033764 rhythmic process Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
- 239000010959 steel Substances 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B15/00—Teaching music
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Acoustics & Sound (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Health & Medical Sciences (AREA)
- Signal Processing (AREA)
- Computational Linguistics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Educational Technology (AREA)
- Business, Economics & Management (AREA)
- Educational Administration (AREA)
- Quality & Reliability (AREA)
- Electrically Operated Instructional Devices (AREA)
- Auxiliary Devices For Music (AREA)
Abstract
The invention discloses an intelligent hardware piano music finishing training guidance method based on melody characteristic matching. The intelligent hardware piano music finishing training guidance method based on melody characteristic matching includes the steps: utilizing portable intelligent hardware equipment to perform real-time recording of the performed music, wave velocity formation on a microphone array, and removal of background noise, and then perform fundamental frequency estimation on the recorded music; acquiring the melody feature of the piano music by solving the frequency of all the harmonic wave components; acquiring the corresponding notes in the piano music to obtain a time sequence of the notes; and by means of matching and comparison of a dynamic time planning, a standard note sequence and a playing note sequence, acquiring the similarity value and regularizing the matching value and scoring. The whole system is based on an intelligent hardware platform, thus being convenient to carry and use, and having low requirement for the platform requirement. Aiming at the playing feature of the piano music, based on the melody feature matching, identification and comparison of notes can be carried out quickly and efficiently. The intelligent hardware piano music finishing training guidance method based on melody characteristic matching can help a piano learner quickly realizing self error correction and self improvement.
Description
Technical field
The present invention relates to a kind of Intelligent hardware piano music matched based on melody characteristicses practices guidance method again, belong to computer
Music recognition field.
Background technology
With the change and the raising of living condition of living environment, more and more children is with adult for piano study is produced
Raw interest.The current mode of study piano, main the carrying out by teacher's teach-by-doing such as attend a training class are instructed, and otherwise student does not have
Method tells the correctness of oneself played piano music music rhythm.In recent years, along with the quick of the Internet and Intelligent hardware
The music recognition software of development, some web-based teachings and characteristic has also obtained development quickly, particularly with Wearable device
Deng the development of Intelligent hardware, for people study and entertainment life generate great impact, but student still can not be
In the case of without counselor, self-correction, ego trip is realized.
Nearly twenties years, the related algorithm of music recognition and theory obtained more in-depth study, the spy based on melody
Levying matching algorithm also has a lot, such as one kind that Chinese publication CN103247286A is proposed is using GPU to counterpoint
The parallelization method of melody extraction is carried out, although the algorithm significantly reduces the hardware resource required for extracting, while improve calculation
The development progress of method, but GPU costs and algorithm complex are higher, and while being unfavorable for the application of the little hardware of intelligence, also China is public
Patent CN103853836A and CN101916250A are opened it is also proposed that some algorithms have been used in many music based on melody
It is in searching system but not notable for piano music feature identification application effect.
The Music Recognition Algorithm matched based on melody characteristicses is combined by the present invention with Intelligent hardware, it is proposed that a kind of piano
Song gets guidance method again, is scored by the piano playing of dynamic realtime, just can quickly understand the deficiency grasped in piano playing.
The content of the invention
The present invention is proposed a kind of Intelligent hardware piano music matched based on melody characteristicses and practices guidance method again, can be used to refer to
Leading the music such as child student carries out the review performance of piano music.
The present invention is adopted the following technical scheme that for solving its technical problem：
A kind of Intelligent hardware piano music matched based on melody characteristicses practices guidance method again, comprises the following steps：
Step (1) is recorded by the enforcement performed music by portable intelligent hardware device, is entered using microphone array
Traveling wave speed is formed, and removes background noise to improve music signal recording quality.
Step (1-1) is recorded by intelligent hardware devices, the short-time energy, wavelet transformation sub-band coefficients according to sound
Amplitude, carries out music starting and the breaking point detection for terminating.
Step (1-2) carries out velocity of wave by microphone array and is formed, and directional reception music signal, to from different azimuth
Interference noise is suppressed.
In the music signal of the recording described in step (2) step (1), fundamental frequency estimation is carried out, ask for all harmonic components
Frequency, as the melody characteristicses of piano music.
Step (2-1) asks for the corresponding frequency values of auto-correlation maximum, as harmonic component by auto-correlation computation
Fundamental frequency,
Step (2-2) deducts the harmonic component fundamental frequency described in step (2-1) from primary signal, repeats auto-correlation
Computing, extracts remaining frequencies of harmonic components, used as the melody characteristicses of piano music.
Step (3) is converted to corresponding note in piano music according to harmonic component fundamental frequency described in step (2), obtains sound
The time serieses of symbol.
Step (3-1) is played in recording from piano music standard, and the fundamental frequency described in extraction step (2) is converted to standard pronunciation
Symbol sequence.
In the piano music recording that step (3-1) is played from student, the fundamental frequency described in extraction step (2) is converted to performance
Sequence of notes.
Step (4) carries out the standard note sequence and musical notes sequence described in step (3) by dynamic time programming
Matching, obtain Similarity value, carry out Regularization to matching numerical value, obtain score, for instructing student correctly to play.
The beneficial effects of the present invention is：The present invention by dynamic realtime recording, feature extraction, matching and can score
Journey, is scored to the piano music that student plays, and so as to student can carry out self confusing correction according to fraction, lifts the steel of oneself
Qin plays level, additionally, being realized based on Intelligent hardware, scoring can be allowed with interest, so can repeated to student
In experience fast happy progress, strengthen the confidence that student perfects piano.
Description of the drawings
Fig. 1 is that white silk instructs system flow chart to piano music according to the present invention again；
Fig. 2 is that microphone array according to the present invention suppresses noise；
Fig. 3 is the Dynamic Matching marking of sequence of notes according to the present invention.
Specific embodiment
The present invention is further elaborated below in conjunction with the accompanying drawings：
As shown in Figure 1, it is necessary first to gather the recording that the standard of piano music is played, typically can be entered by teacher
OK, during teaching demonstration, recorded by portable intelligent hardware devices.Exercise of being in is carried out in student or child
During, intelligent hardware devices to be opened, is started music and is practiced guiding function again, the melody to playing is recorded, and gathers audio frequency
Signal.
Needed for obtaining after audio signal, by frequency estimation algorithm, recording is converted to into sequence of notes, 88 of piano
Key difference respective frequencies scope is A2 (27.5Hz) to c5 (4186Hz).After sequence of notes is obtained, entered by dynamic programming
Row Optimized Matching, changes to the similarity that matching is obtained, and obtains final score.
As shown in Figure 2, as microphone position is different, the signal component that mike is received is also different, using blind source point
From technology, noise component(s) and musical components are separated.
Receive signal X to be expressed as
X=AS (1)
Wherein,
S=[S (n), N (n)]T (2)
S is the bivector that music source and noise source are produced.X=[X1, X2]TFor two-dimensional observation data.2 × N matrix A is
Transmission matrix, in the case where A and S are unknown, determines transfer matrix W according to observation vector X, makes the output Y after conversion
For the estimation of original signal S：
Y=WX (3)
For two-microphone array, it is impossible to separate the sound source signal more than two, but for long-range noise, multiple noises
After the superposition of source, the signal on two mikes is basically identical, it is believed that be a source of sound.By to music and background noise
Separation after, we further can carry out the suppression of noise by filtering.
As shown in Figure 3, solid line represents the frequency transformation curve of a sequence of notes, and dotted line represents another sequence of notes
Frequency transformation curve.By finding most like frequency values, sequence of notes is aligned, after being aligned, is calculated frequency
Difference distance between rate, adds up total distance difference, obtains similarity.
During the alignment of similar frequencies note is carried out, need to travel through all of possible sound around current note
Symbol, by dynamic time warping algorithm, completes this optimization process, and the calculating process of dynamic time warping is as follows：
Sequence of notes is stored in S1, in S2,
Read sequence length：
Int n=s1.length ()；
Int m=s2.length ()；
Application calculator memory is used for storage apart from matched data：
ArrayList<Float>Dtw=new ArrayList<Float>()；
The range difference of all possible matching way is initialized away from amount：
Setting search range parameter：
Int degree=2；
dtw.set(0,(float)0.0)；
Int w=(n+m)/2；//DTW parameter local constrain
W=w+degree；
Dynamic programming travels through optimizing：
Sequence of calculation distance：
Cost=dist (s1, s2)；
Float minimum=0；
The minimum matching approach of determination frequency gap：
Obtain the Similarity value of two sequences：
Dist=dtw.get (n*m-1)；
As shown in table 1,88 keyboards of piano have corresponding relation with frequency, therefore by the estimation to frequency, can
Reflect melody characteristicses information, and then evaluate to playing correctness, instruct user or child to practice again.
Table 1
Embodiment described above, simply preferred embodiments of the invention, not limiting the practical range of the present invention, thus it is all according to
Equivalence changes or modification that construction, feature and principle described in scope of the present invention patent is done, all should be included in the present invention
In patent claim.
Claims (1)
1. a kind of Intelligent hardware piano music matched based on melody characteristicses practices guidance method again, it is characterised in that including following step
Suddenly：
Step (1) is recorded by the enforcement performed music by portable intelligent hardware device, enters traveling wave using microphone array
Speed is formed, and removes background noise to improve music signal recording quality.
In music signal of the step (2) to the recording described in step (1), fundamental frequency estimation is carried out, all harmonic components are asked for
Frequency, used as the melody characteristicses of piano music.
Step (3) is converted to corresponding note in piano music according to harmonic component fundamental frequency described in step (2), obtains note
Time serieses.
Step (4) by dynamic time programming, carry out standard note sequence and musical notes sequence described in step (3)
Match somebody with somebody, obtain Similarity value, Regularization is carried out to matching numerical value, obtain score, for instructing student correctly to play.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610847637.3A CN106531184A (en) | 2016-09-23 | 2016-09-23 | Intelligent hardware piano music finishing training guidance method based on melody characteristic matching |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610847637.3A CN106531184A (en) | 2016-09-23 | 2016-09-23 | Intelligent hardware piano music finishing training guidance method based on melody characteristic matching |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106531184A true CN106531184A (en) | 2017-03-22 |
Family
ID=58344056
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610847637.3A Pending CN106531184A (en) | 2016-09-23 | 2016-09-23 | Intelligent hardware piano music finishing training guidance method based on melody characteristic matching |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106531184A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107564540A (en) * | 2017-07-05 | 2018-01-09 | 珠海市维想科技有限公司 | A kind of Multifunctional piano identifies teaching auxiliary system |
| CN110120213A (en) * | 2019-05-12 | 2019-08-13 | 陕西学前师范学院 | A kind of piano performance points-scoring system and its method |
| CN113327482A (en) * | 2021-06-17 | 2021-08-31 | 上海松鼠课堂人工智能科技有限公司 | String instrument playing teaching method and system based on video monitoring |
| CN113657184A (en) * | 2021-07-26 | 2021-11-16 | 广东科学技术职业学院 | Evaluation method and device for piano playing fingering |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103354092A (en) * | 2013-06-27 | 2013-10-16 | 天津大学 | Audio music-score comparison method with error detection function |
| CN103531189A (en) * | 2013-09-25 | 2014-01-22 | 熊世林 | Performance evaluator for intelligent electric piano |
| CN104978962A (en) * | 2014-04-14 | 2015-10-14 | 安徽科大讯飞信息科技股份有限公司 | Query by humming method and system |
| CN205541906U (en) * | 2016-04-14 | 2016-08-31 | 安徽克洛斯威智能乐器科技有限公司 | Intelligence piano system with tuning function of grading and record |
-
2016
- 2016-09-23 CN CN201610847637.3A patent/CN106531184A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103354092A (en) * | 2013-06-27 | 2013-10-16 | 天津大学 | Audio music-score comparison method with error detection function |
| CN103531189A (en) * | 2013-09-25 | 2014-01-22 | 熊世林 | Performance evaluator for intelligent electric piano |
| CN104978962A (en) * | 2014-04-14 | 2015-10-14 | 安徽科大讯飞信息科技股份有限公司 | Query by humming method and system |
| CN205541906U (en) * | 2016-04-14 | 2016-08-31 | 安徽克洛斯威智能乐器科技有限公司 | Intelligence piano system with tuning function of grading and record |
Non-Patent Citations (2)
| Title |
|---|
| 刘乐: ""钢琴演奏评价***研究"", 《中国优秀硕士学位论文全文数据库 工程科技I辑》 * |
| 王佳迪: ""鲁棒的音乐评分方法研究"", 《中国优秀硕士学位论文全文数据库 哲学与人文科学辑》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107564540A (en) * | 2017-07-05 | 2018-01-09 | 珠海市维想科技有限公司 | A kind of Multifunctional piano identifies teaching auxiliary system |
| CN110120213A (en) * | 2019-05-12 | 2019-08-13 | 陕西学前师范学院 | A kind of piano performance points-scoring system and its method |
| CN113327482A (en) * | 2021-06-17 | 2021-08-31 | 上海松鼠课堂人工智能科技有限公司 | String instrument playing teaching method and system based on video monitoring |
| CN113657184A (en) * | 2021-07-26 | 2021-11-16 | 广东科学技术职业学院 | Evaluation method and device for piano playing fingering |
| CN113657184B (en) * | 2021-07-26 | 2023-11-07 | 广东科学技术职业学院 | Piano playing fingering evaluation method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Cao et al. | Convolutional neural network with second-order pooling for underwater target classification | |
| CN105023573B (en) | It is detected using speech syllable/vowel/phone boundary of auditory attention clue | |
| CN102664016B (en) | Singing evaluation method and system | |
| CN110246488B (en) | Voice conversion method and device of semi-optimized cycleGAN model | |
| CN106531184A (en) | Intelligent hardware piano music finishing training guidance method based on melody characteristic matching | |
| CN104217729A (en) | Audio processing method, audio processing device and training method | |
| Liu et al. | Simultaneous utilization of spectral magnitude and phase information to extract supervectors for speaker verification anti-spoofing | |
| CN108109613A (en) | For the audio training of Intelligent dialogue voice platform and recognition methods and electronic equipment | |
| US20110123965A1 (en) | Speech Processing and Learning | |
| CN110222841A (en) | Neural network training method and device based on spacing loss function | |
| Fan et al. | Singing voice separation and pitch extraction from monaural polyphonic audio music via DNN and adaptive pitch tracking | |
| CN103714806A (en) | Chord recognition method combining SVM with enhanced PCP | |
| Dong | Application of artificial intelligence software based on semantic web technology in english learning and teaching | |
| CN107767850A (en) | A kind of singing marking method and system | |
| CN111191787B (en) | Training method and device of neural network for extracting speaker embedded features | |
| Kang et al. | Fast DCTTS: efficient deep convolutional text-to-speech | |
| Li et al. | Oriental language recognition (OLR) 2020: Summary and analysis | |
| Zhang | [Retracted] Practice and Exploration of Music Solfeggio Teaching Based on Data Mining Technology | |
| Zhang et al. | Meta-generalization for domain-invariant speaker verification | |
| Kim et al. | Light-Weight Speaker Verification with Global Context Information. | |
| Dat et al. | Robust speaker verification using low-rank recovery under total variability space | |
| CN111368129A (en) | Humming retrieval method based on deep neural network | |
| Küçükbay et al. | Audio event detection using adaptive feature extraction scheme | |
| Zhang | Mobile music recognition based on deep neural network | |
| Zhang et al. | Recognition of score words in freestyle kayaking using improved DTW matching |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20190529 Address after: 518000 Tian'an Science and Technology Venture Park A Block 501, Tairan Five Road, Futian District, Shenzhen City, Guangdong Province Applicant after: Shenzhen bean Mdt InfoTech Ltd Address before: 210000 Guohua Building 23D, No. 45 Pailou Lane, Gulou District, Nanjing, Jiangsu Province Applicant before: Nanjing bean Mdt InfoTech Ltd |
|
| TA01 | Transfer of patent application right | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170322 |
|
| RJ01 | Rejection of invention patent application after publication |