US20090255396A1 - Self-adjusting music scrolling system - Google Patents
Self-adjusting music scrolling system Download PDFInfo
- Publication number
- US20090255396A1 US20090255396A1 US12/266,327 US26632708A US2009255396A1 US 20090255396 A1 US20090255396 A1 US 20090255396A1 US 26632708 A US26632708 A US 26632708A US 2009255396 A1 US2009255396 A1 US 2009255396A1
- Authority
- US
- United States
- Prior art keywords
- musician
- music score
- midi file
- music
- midi
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 8
- 238000012544 monitoring process Methods 0.000 claims 2
- 238000000034 method Methods 0.000 description 28
- 238000006243 chemical reaction Methods 0.000 description 4
- 230000005236 sound signal Effects 0.000 description 4
- 230000001360 synchronised effect Effects 0.000 description 4
- ZYXYTGQFPZEUFX-UHFFFAOYSA-N benzpyrimoxan Chemical compound O1C(OCCC1)C=1C(=NC=NC=1)OCC1=CC=C(C=C1)C(F)(F)F ZYXYTGQFPZEUFX-UHFFFAOYSA-N 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 238000013179 statistical model Methods 0.000 description 1
- 230000017105 transposition Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H1/00—Details of electrophonic musical instruments
- G10H1/0008—Associated control or indicating means
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/26—Speech to text systems
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
- G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
- G10H2210/066—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for pitch analysis as part of wider processing for musical purposes, e.g. transcription, musical performance evaluation; Pitch recognition, e.g. in polyphonic sounds; Estimation or use of missing fundamental
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
- G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
- G10H2210/086—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for transcription of raw audio or music data to a displayed or printed staff representation or to displayable MIDI-like note-oriented data, e.g. in pianoroll format
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
- G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
- G10H2210/091—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for performance evaluation, i.e. judging, grading or scoring the musical qualities or faithfulness of a performance, e.g. with respect to pitch, tempo or other timings of a reference performance
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2220/00—Input/output interfacing specifically adapted for electrophonic musical tools or instruments
- G10H2220/005—Non-interactive screen display of musical or status data
- G10H2220/015—Musical staff, tablature or score displays, e.g. for score reading during a performance
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2240/00—Data organisation or data communication aspects, specifically adapted for electrophonic musical tools or instruments
- G10H2240/011—Files or data streams containing coded musical information, e.g. for transmission
- G10H2240/046—File format, i.e. specific or non-standard musical file format used in or adapted for electrophonic musical instruments, e.g. in wavetables
- G10H2240/056—MIDI or other note-oriented file format
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2240/00—Data organisation or data communication aspects, specifically adapted for electrophonic musical tools or instruments
- G10H2240/011—Files or data streams containing coded musical information, e.g. for transmission
- G10H2240/046—File format, i.e. specific or non-standard musical file format used in or adapted for electrophonic musical instruments, e.g. in wavetables
- G10H2240/071—Wave, i.e. Waveform Audio File Format, coding, e.g. uncompressed PCM audio according to the RIFF bitstream format method
Definitions
- the present invention relates in general to the display of words or music notes, and more particularly, to an automatic scrolling system and method of adjusting a music score shown on a display based upon recognition of the music notes or words already played or sung by a musician.
- existing solutions are generally not automatic and/or are not synchronized with the execution of music notes played by the musician. These solutions exempt the musician from undertaking manual actions in only very limited circumstances.
- a cursor is displayed on a screen containing sheet music in order to indicate the current beat set by the director.
- the musician is then forced to determine for himself what the correct position of the cursor should be.
- the musician may activate the scrolling of music on a display by using his or her foot.
- this type of system may not be used in conjunction with instruments that are played, at least in part, with the feet (e.g. a piano or an organ).
- automatic time-based scrolling is used to scroll the music notes on a display based upon a predetermined time interval.
- the automatic scrolling solution utilizing a customizable time interval is often times inaccurate, thereby scrolling either ahead of or behind time and disrupting the musician's ability to play the proper notes at the proper tempo.
- the present invention solves the foregoing problems by providing a system for automatically adjusting a music score comprising a display screen and a self-adjusting music scrolling system having a music score database, a memory, and a digital device for recording musical notes, the music scrolling system configured to perform the steps of selecting a music score to be played by a musician, wherein the music score is cataloged and stored in the music score database as a first MIDI file, displaying a first portion of the selected music score on the display screen, recording musical notes with the digital device, storing the recorded musical notes in the memory as a WAV file, converting the WAV file into a second MIDI file, comparing the first MIDI file and the second MIDI file with a MIDI comparison algorithm, determining if the first MIDI file substantially matches the second MIDI file, automatically adjusting the music score on the display screen to show a second portion of the selected music score upon determining that the first MIDI file substantially matches the second MIDI file, and displaying one or more mistakes
- FIG. 1 is a flowchart illustrating one embodiment of an automatic music scrolling method in accordance with the present invention.
- FIG. 2 is a flowchart illustrating an alternative embodiment of an automatic scrolling system and method in accordance with the present invention that utilizes speech recognition.
- the present invention includes an automatic music scrolling system and method capable of adjusting a music score or notes shown on a display screen based upon recognition of the music notes already played or sung by a musician.
- one exemplary embodiment of the present invention is designed to both recognize the instrument timbre as well as the music played.
- the invention generally combines music recognition with an electronic music display that visually depicts the real-time position in the music score.
- the automatic scrolling system and method recognizes the notes already played, tracks the performance on the displayed music score, and advances the music score on the screen accordingly and without the need for intervention by the musician.
- the musician may be able to select the type of scrolling suitable for him, such as by beats, lines or pages.
- page scrolling the musician may be able to select when the system scrolls to the next page. For example, the musician may select the number of beats remaining on a page before the system scrolls to the next page.
- line and beat scrolling the musician may choose the fixed position of the current line or beat. The musician may also select that a preview of the next beats/lines/pages be shown on a portion of the screen to help the musician visualize the notes that will be coming up shortly. It is important to note that it is not necessary for the automatic scrolling system of the present invention to always scroll sequentially. For example, the system may be configured such that it may automatically jump pages, in either direction, as required by the music score.
- FIG. 1 is a flowchart illustrating one exemplary embodiment of a method 10 for providing automatic scrolling of a previously selected music score.
- the method begins at step 12 where the musical notes played by a musician are recorded and stored in memory by a digital device in WAV format.
- the recording of the musical notes played by the musician may be accomplished using any one of a variety of commercially available WAV recording devices, as will be appreciated by those skilled in the art.
- step 14 the WAV file recorded and stored in step 12 is converted into a MIDI file.
- the conversion between a WAV and a MIDI file in step 14 may be performed by any one of a variety of WAV to MIDI programs as will be appreciated by those skilled in the art.
- one type of solution that may be used for the WAV to MIDI conversion of step 14 is a software program that uses an algorithm configured to take a sound signal as it is input and attempts to capture the loudness, pitch, dynamics, and performance style present in the sound signal. All of these features captured by the program are then written into a MIDI file. When replayed, this MIDI file will sound very similar to the original sound signals input into the converter.
- Another solution that may be used for the WAV to MIDI conversion of step 14 includes a real-time WAV to MIDI converter that is capable of analyzing a live or recorded solo performance (such as a singing voice or a musical instrument) and converting the corresponding sound signals into a standard MIDI file.
- This type of software program may not only be capable of reading standard WAV audio files, but also may accept live microphone input.
- the method continues at step 16 where the MIDI file created in step 14 is compared to a MIDI file corresponding to the music score previously selected by the musician, which may be cataloged in and accessed from a music repository database.
- the music repository database preferably stores the music scores in MIDI format, although those skilled in the art will appreciate that the music scores may be stored in another format, such as in WAV format, and later converted into a MIDI file such as with the WAV to MIDI programs previously described.
- the comparison between the MIDI file corresponding to the music notes recorded in step 12 and the MIDI file corresponding to the predetermined music score stored in the music repository database may be based on the events constituting the MIDI format without the need to focus on all aspects of the event. For example, the comparison may be performed based on pitch (either relative or absolute) and/or the duration of the event. In one embodiment, the comparison algorithm may place a greater weighting on the duration of the music notes rather than on their pitch.
- Step 18 a determination is made whether or not the MIDI file corresponding to the music notes recorded in step 12 substantially match the MIDI file corresponding to the pre-selected music score.
- Step 18 preferably incorporates an “error allowance” that may be selected by the musician, thus providing the musician with the power to select the degree of a “mistake” that is considered acceptable.
- a mistake may be one or more consecutive incorrect notes played by the musician.
- the automatic scrolling will continue tracking the performance of the musician ignoring the incorrect notes.
- a mechanism of matching the next played beats with the music will take place and will re-synchronize the musician's performance to the music score.
- the musician may have the opportunity to select the search range of the re-synchronization mechanism, for example in the same piece, a selected range of pieces, the same page, etc.
- an error tolerance is needed because, after a mistake, the musician could stop and restart the performance from another point, either prior to or after the beat of the mistake.
- the possibility of modulating the allowance level will permit the musician to adjust the synchronization mechanism to the ability of the musician in performing the piece.
- the mechanism of matching played beats with music may also exempt the musician from having to select the piece before starting to play. For example, the musician could simply start to play the first beats, which may thereafter be automatically synchronized to the right piece at the right beat.
- step 18 If it is determined in step 18 that the MIDI file corresponding to the music notes recorded in step 12 substantially match the MIDI file corresponding to the pre-selected music score, then the method continues at step 20 where the music score shown on the display is automatically scrolled so that the musician may continue playing without any interruption.
- step 18 if it is determined in step 18 that the MIDI file corresponding to the musical notes recorded in step 12 do not substantially match the MIDI file corresponding to the pre-selected musical score, then the method continues at step 22 where the user is provided with feedback regarding the mistakes detected in the comparison step.
- the synchronization in accordance with the present invention may also happen as a result of musical notes being played by other musicians.
- the music scrolling systems of the group may be connected and always synchronized, delegating the scrolling task to only one instrument or by necessity, one scrolling system may be temporarily synchronized to another.
- the tolerance of the music scrolling system in accordance with the present invention may largely depend on the tolerance of all the involved algorithms.
- the errors produced by the music recording and MIDI comparison steps are usually negligible when compared to the errors that may be produced by the WAV to MIDI conversion step.
- the main source of error in the WAV to MIDI step is due to the fact that several harmonics played by an instrument may be associated with the same note.
- This type of error may greatly affect the transposition of played music into music notation.
- the pitch of notes is not the only parameter that may be relied upon.
- another parameter that may be used is duration.
- the scrolling system and method of the present invention may be set up based only on the pitch recognition, thereby introducing a larger source of error.
- the scrolling system and method of the present invention may be set up based on the duration of the notes, thereby minimizing the potential errors.
- FIG. 2 is a flowchart illustrating one exemplary embodiment of a method 100 for providing automatic scrolling based on speech recognition.
- the method operates similar to method 10 previously described, and thus, will be discussed only briefly.
- the method begins at step 102 where the voice signals from words spoken or sung by an individual are recorded and stored in memory.
- the method continues at step 104 where the voice signals recorded and stored in step 102 are converted into a text file with speech recognition software.
- step 106 the text file created in step 104 is compared to a text file corresponding to a script or music lyrics previously selected by the individual.
- step 108 a determination is made whether or not the text file corresponding to the words recorded in step 102 substantially matches the text file corresponding to the pre-selected script or music lyrics. If it is determined in step 108 that the text file corresponding to the words recorded in step 102 substantially matches the text file corresponding to the pre-selected script or music lyrics, then the method continues at step 110 where the text shown on the display is automatically scrolled accordingly. Alternatively, if it is determined in step 108 that the text file corresponding to the words recorded in step 102 do not substantially match the text file corresponding to the pre-selected script or music lyrics, then the method continues at step 112 where the user is provided with feedback regarding the mistakes detected in the comparison step. As will be appreciated by those skilled in the art, method 100 may be adapted for use with numerous other speech-type applications without departing from the intended scope of the present invention.
- HMM Hidden Markov Model
- the HMM speech recognition method is a statistical model which treats the voice as a signal.
- HMM receives a voice signal as an input and produces an output that typically includes a representation (in plain text) of the recognized word or set of words.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Auxiliary Devices For Music (AREA)
Abstract
A system for automatically adjusting a music score comprises a display screen and a music scrolling system configured to perform the steps of selecting a music score, the music score being cataloged and stored in a music score database as a first MIDI file, displaying a first portion of the selected music score on the display screen, recording musical notes with a digital device, storing the recorded musical notes in memory as a WAV file, converting the WAV file into a second MIDI file, comparing the first MIDI file and the second MIDI file with a MIDI comparison algorithm, determining if the first MIDI file substantially matches the second MIDI file, automatically adjusting the music score on the display screen upon determining that the first MIDI file substantially matches the second MIDI file, and displaying one or more mistakes upon determining that the first MIDI file does not substantially match the second MIDI file.
Description
- The present application is a continuation application of application Ser. No. 12/100,125 filed Apr. 9, 2008, entitled “Self-Adjusting Music Scrolling System,” which is hereby incorporated by reference in its entirety.
- The present invention relates in general to the display of words or music notes, and more particularly, to an automatic scrolling system and method of adjusting a music score shown on a display based upon recognition of the music notes or words already played or sung by a musician.
- Previous systems and methods have attempted to provide electronic music displays as an alternative to requiring musicians to manually turn pages. However, such solutions have not conquered the many obstacles that exist in providing electronic music displays because, for example, they require manual intervention, either directly to activate the scrolling or as a consequence of mistakes made by the musician (i.e., playing the wrong note).
- More particularly, existing solutions are generally not automatic and/or are not synchronized with the execution of music notes played by the musician. These solutions exempt the musician from undertaking manual actions in only very limited circumstances. For example, in one existing system designed for use by an orchestra, a cursor is displayed on a screen containing sheet music in order to indicate the current beat set by the director. However, if the director makes a mistake, the musician is then forced to determine for himself what the correct position of the cursor should be. In another existing system, the musician may activate the scrolling of music on a display by using his or her foot. However, for obvious reasons, this type of system may not be used in conjunction with instruments that are played, at least in part, with the feet (e.g. a piano or an organ). In yet another existing system, automatic time-based scrolling is used to scroll the music notes on a display based upon a predetermined time interval. However, the automatic scrolling solution utilizing a customizable time interval is often times inaccurate, thereby scrolling either ahead of or behind time and disrupting the musician's ability to play the proper notes at the proper tempo.
- Furthermore, existing solutions do not adequately take into account the fact that reading “sheet” music is not always sequential like reading a book. For example, when reading sheet music, sets of beats may be repeated and may terminate differently at any repetition. Furthermore, under certain circumstances, there may be “jumps” from one page to another page.
- Based on the foregoing, there exists a need for an improved system and method for automatically scrolling music on a display based upon recognition of the music notes previously played or sung by a musician.
- The present invention solves the foregoing problems by providing a system for automatically adjusting a music score comprising a display screen and a self-adjusting music scrolling system having a music score database, a memory, and a digital device for recording musical notes, the music scrolling system configured to perform the steps of selecting a music score to be played by a musician, wherein the music score is cataloged and stored in the music score database as a first MIDI file, displaying a first portion of the selected music score on the display screen, recording musical notes with the digital device, storing the recorded musical notes in the memory as a WAV file, converting the WAV file into a second MIDI file, comparing the first MIDI file and the second MIDI file with a MIDI comparison algorithm, determining if the first MIDI file substantially matches the second MIDI file, automatically adjusting the music score on the display screen to show a second portion of the selected music score upon determining that the first MIDI file substantially matches the second MIDI file, and displaying one or more mistakes detected on the display screen upon determining that the first MIDI file does not substantially match the second MIDI file. The musical notes played by the musician may be played on any type of instrument, including both acoustic and electronic instruments. The comparison algorithm compares both pitch and duration parameters of musical notes in order to determine if the first MIDI file substantially matches the second MIDI file.
-
FIG. 1 is a flowchart illustrating one embodiment of an automatic music scrolling method in accordance with the present invention. -
FIG. 2 is a flowchart illustrating an alternative embodiment of an automatic scrolling system and method in accordance with the present invention that utilizes speech recognition. - The present invention includes an automatic music scrolling system and method capable of adjusting a music score or notes shown on a display screen based upon recognition of the music notes already played or sung by a musician. In particular, one exemplary embodiment of the present invention is designed to both recognize the instrument timbre as well as the music played. The invention generally combines music recognition with an electronic music display that visually depicts the real-time position in the music score. Thus, the automatic scrolling system and method recognizes the notes already played, tracks the performance on the displayed music score, and advances the music score on the screen accordingly and without the need for intervention by the musician.
- In accordance with the present invention, the musician may be able to select the type of scrolling suitable for him, such as by beats, lines or pages. Additionally, with regard to page scrolling the musician may be able to select when the system scrolls to the next page. For example, the musician may select the number of beats remaining on a page before the system scrolls to the next page. With regard to line and beat scrolling, the musician may choose the fixed position of the current line or beat. The musician may also select that a preview of the next beats/lines/pages be shown on a portion of the screen to help the musician visualize the notes that will be coming up shortly. It is important to note that it is not necessary for the automatic scrolling system of the present invention to always scroll sequentially. For example, the system may be configured such that it may automatically jump pages, in either direction, as required by the music score.
-
FIG. 1 is a flowchart illustrating one exemplary embodiment of amethod 10 for providing automatic scrolling of a previously selected music score. The method begins atstep 12 where the musical notes played by a musician are recorded and stored in memory by a digital device in WAV format. The recording of the musical notes played by the musician may be accomplished using any one of a variety of commercially available WAV recording devices, as will be appreciated by those skilled in the art. - The method continues at
step 14 where the WAV file recorded and stored instep 12 is converted into a MIDI file. The conversion between a WAV and a MIDI file instep 14 may be performed by any one of a variety of WAV to MIDI programs as will be appreciated by those skilled in the art. For example, one type of solution that may be used for the WAV to MIDI conversion ofstep 14 is a software program that uses an algorithm configured to take a sound signal as it is input and attempts to capture the loudness, pitch, dynamics, and performance style present in the sound signal. All of these features captured by the program are then written into a MIDI file. When replayed, this MIDI file will sound very similar to the original sound signals input into the converter. Another solution that may be used for the WAV to MIDI conversion ofstep 14 includes a real-time WAV to MIDI converter that is capable of analyzing a live or recorded solo performance (such as a singing voice or a musical instrument) and converting the corresponding sound signals into a standard MIDI file. This type of software program may not only be capable of reading standard WAV audio files, but also may accept live microphone input. - The method continues at
step 16 where the MIDI file created instep 14 is compared to a MIDI file corresponding to the music score previously selected by the musician, which may be cataloged in and accessed from a music repository database. The music repository database preferably stores the music scores in MIDI format, although those skilled in the art will appreciate that the music scores may be stored in another format, such as in WAV format, and later converted into a MIDI file such as with the WAV to MIDI programs previously described. The comparison between the MIDI file corresponding to the music notes recorded instep 12 and the MIDI file corresponding to the predetermined music score stored in the music repository database may be based on the events constituting the MIDI format without the need to focus on all aspects of the event. For example, the comparison may be performed based on pitch (either relative or absolute) and/or the duration of the event. In one embodiment, the comparison algorithm may place a greater weighting on the duration of the music notes rather than on their pitch. - Next, in
step 18, a determination is made whether or not the MIDI file corresponding to the music notes recorded instep 12 substantially match the MIDI file corresponding to the pre-selected music score.Step 18 preferably incorporates an “error allowance” that may be selected by the musician, thus providing the musician with the power to select the degree of a “mistake” that is considered acceptable. In general, a mistake may be one or more consecutive incorrect notes played by the musician. In case of a mistake within the allowance limit set by the musician, the automatic scrolling will continue tracking the performance of the musician ignoring the incorrect notes. However, when the mistake is above the allowance limit a mechanism of matching the next played beats with the music will take place and will re-synchronize the musician's performance to the music score. The musician may have the opportunity to select the search range of the re-synchronization mechanism, for example in the same piece, a selected range of pieces, the same page, etc. - The definition of an error tolerance is needed because, after a mistake, the musician could stop and restart the performance from another point, either prior to or after the beat of the mistake. The possibility of modulating the allowance level will permit the musician to adjust the synchronization mechanism to the ability of the musician in performing the piece. The mechanism of matching played beats with music may also exempt the musician from having to select the piece before starting to play. For example, the musician could simply start to play the first beats, which may thereafter be automatically synchronized to the right piece at the right beat.
- If it is determined in
step 18 that the MIDI file corresponding to the music notes recorded instep 12 substantially match the MIDI file corresponding to the pre-selected music score, then the method continues atstep 20 where the music score shown on the display is automatically scrolled so that the musician may continue playing without any interruption. - Alternatively, if it is determined in
step 18 that the MIDI file corresponding to the musical notes recorded instep 12 do not substantially match the MIDI file corresponding to the pre-selected musical score, then the method continues atstep 22 where the user is provided with feedback regarding the mistakes detected in the comparison step. - In a musical group, the synchronization in accordance with the present invention may also happen as a result of musical notes being played by other musicians. In particular, the music scrolling systems of the group may be connected and always synchronized, delegating the scrolling task to only one instrument or by necessity, one scrolling system may be temporarily synchronized to another.
- The tolerance of the music scrolling system in accordance with the present invention may largely depend on the tolerance of all the involved algorithms. For example, in the system and method previously described, the errors produced by the music recording and MIDI comparison steps are usually negligible when compared to the errors that may be produced by the WAV to MIDI conversion step.
- More particularly, the main source of error in the WAV to MIDI step is due to the fact that several harmonics played by an instrument may be associated with the same note. This type of error may greatly affect the transposition of played music into music notation. However, it will not generally impact an automatic scrolling system in the same manner because in such a system, the pitch of notes is not the only parameter that may be relied upon. For example, another parameter that may be used is duration. Thus, the scrolling system and method of the present invention may be set up based only on the pitch recognition, thereby introducing a larger source of error. Alternatively, the scrolling system and method of the present invention may be set up based on the duration of the notes, thereby minimizing the potential errors. However, it may be desirable to use a combination of both of the methods in order to reduce the potential errors by a considerable amount.
- The system and method of the present invention has been described above with reference to musical notes generated by a musician playing an instrument. However, the present invention may also be extended to words spoken or sung by an individual, which may then be compared to a script, music lyrics, or the like.
-
FIG. 2 is a flowchart illustrating one exemplary embodiment of amethod 100 for providing automatic scrolling based on speech recognition. The method operates similar tomethod 10 previously described, and thus, will be discussed only briefly. The method begins atstep 102 where the voice signals from words spoken or sung by an individual are recorded and stored in memory. The method continues atstep 104 where the voice signals recorded and stored instep 102 are converted into a text file with speech recognition software. Then, instep 106, the text file created instep 104 is compared to a text file corresponding to a script or music lyrics previously selected by the individual. Next, instep 108, a determination is made whether or not the text file corresponding to the words recorded instep 102 substantially matches the text file corresponding to the pre-selected script or music lyrics. If it is determined instep 108 that the text file corresponding to the words recorded instep 102 substantially matches the text file corresponding to the pre-selected script or music lyrics, then the method continues atstep 110 where the text shown on the display is automatically scrolled accordingly. Alternatively, if it is determined instep 108 that the text file corresponding to the words recorded instep 102 do not substantially match the text file corresponding to the pre-selected script or music lyrics, then the method continues atstep 112 where the user is provided with feedback regarding the mistakes detected in the comparison step. As will be appreciated by those skilled in the art,method 100 may be adapted for use with numerous other speech-type applications without departing from the intended scope of the present invention. - One exemplary speech recognition method that may be used in conjunction with the present invention is the Hidden Markov Model (HMM). The HMM speech recognition method is a statistical model which treats the voice as a signal. Generally speaking, HMM receives a voice signal as an input and produces an output that typically includes a representation (in plain text) of the recognized word or set of words.
- Although the present invention has been described with reference to preferred embodiments, workers skilled in the art will recognize that changes may be made in form and detail without departing from the spirit and scope of the invention.
Claims (1)
1. A system for automatically adjusting a music score shown on a display comprising:
a display screen; and
a self-adjusting music scrolling system having a music score database, a memory, and a digital device for recording musical notes, the music scrolling system configured to perform the steps of:
selecting a music score to be played by a musician, wherein the music score is cataloged and stored in the music score database as a first MIDI file;
defining an acceptable musician error, wherein the acceptable musician error is a configurable parameter selected by the musician, and wherein the acceptable musician error may be defined as the occurrence of one or more consecutive incorrect musical notes played by the musician;
displaying a first portion of the selected music score on the display screen;
recording musical notes played by the musician with the digital device;
storing the recorded musical notes in the memory as a WAV file;
converting the WAV file into a second MIDI file;
comparing the first MIDI file and the second MIDI file with a MIDI comparison algorithm, wherein the comparison algorithm compares both pitch and duration parameters of musical notes;
determining if the first MIDI file substantially matches the second MIDI file;
executing an error monitoring sequence to determine if a mistake in the musical notes played by the musician exceeds the acceptable musician error selected by the musician;
automatically adjusting the music score on the display screen to show a second portion of the selected music score upon determining that the first MIDI file substantially matches the second MIDI file; and
re-synchronizing a position of the musician within the selected music score upon detecting a mistake that exceeds the acceptable musician error defined by the musician, wherein re-synchronizing includes monitoring a string of correct musical notes played after the mistake is detected and matching the string of correct musical notes with a corresponding string of musical notes in the selected music score.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/266,327 US20090255396A1 (en) | 2008-04-09 | 2008-11-06 | Self-adjusting music scrolling system |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/100,125 US7482529B1 (en) | 2008-04-09 | 2008-04-09 | Self-adjusting music scrolling system |
US12/266,327 US20090255396A1 (en) | 2008-04-09 | 2008-11-06 | Self-adjusting music scrolling system |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/100,125 Continuation US7482529B1 (en) | 2008-04-09 | 2008-04-09 | Self-adjusting music scrolling system |
Publications (1)
Publication Number | Publication Date |
---|---|
US20090255396A1 true US20090255396A1 (en) | 2009-10-15 |
Family
ID=40275416
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/100,125 Expired - Fee Related US7482529B1 (en) | 2008-04-09 | 2008-04-09 | Self-adjusting music scrolling system |
US12/266,327 Abandoned US20090255396A1 (en) | 2008-04-09 | 2008-11-06 | Self-adjusting music scrolling system |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/100,125 Expired - Fee Related US7482529B1 (en) | 2008-04-09 | 2008-04-09 | Self-adjusting music scrolling system |
Country Status (1)
Country | Link |
---|---|
US (2) | US7482529B1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100095828A1 (en) * | 2006-12-13 | 2010-04-22 | Web Ed. Development Pty., Ltd. | Electronic System, Methods and Apparatus for Teaching and Examining Music |
US20110132176A1 (en) * | 2009-12-04 | 2011-06-09 | Stephen Maebius | System for displaying and scrolling musical notes |
Families Citing this family (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7579541B2 (en) * | 2006-12-28 | 2009-08-25 | Texas Instruments Incorporated | Automatic page sequencing and other feedback action based on analysis of audio performance data |
US7674970B2 (en) * | 2007-05-17 | 2010-03-09 | Brian Siu-Fung Ma | Multifunctional digital music display device |
US20090173213A1 (en) * | 2008-01-09 | 2009-07-09 | Ming Jiang | Music Score Recognizer and Its Applications |
US8629342B2 (en) * | 2009-07-02 | 2014-01-14 | The Way Of H, Inc. | Music instruction system |
US8357848B2 (en) * | 2009-12-22 | 2013-01-22 | Keith Michael Andrews | System and method for policy based automatic scoring of vocal performances |
US8445766B2 (en) * | 2010-02-25 | 2013-05-21 | Qualcomm Incorporated | Electronic display of sheet music |
FR2966635A1 (en) * | 2010-10-20 | 2012-04-27 | France Telecom | Method for displaying e.g. song lyrics of audio content under form of text on e.g. smartphone, involves recognizing voice data of audio content, and displaying recognized voice data in form of text on device |
US20140129235A1 (en) * | 2011-06-17 | 2014-05-08 | Nokia Corporation | Audio tracker apparatus |
US8884148B2 (en) * | 2011-06-28 | 2014-11-11 | Randy Gurule | Systems and methods for transforming character strings and musical input |
ES2609444T3 (en) * | 2014-03-12 | 2017-04-20 | Newmusicnow, S.L. | Method, device and software to move a musical score |
JP6467887B2 (en) * | 2014-11-21 | 2019-02-13 | ヤマハ株式会社 | Information providing apparatus and information providing method |
US9672799B1 (en) * | 2015-12-30 | 2017-06-06 | International Business Machines Corporation | Music practice feedback system, method, and recording medium |
JP6801225B2 (en) | 2016-05-18 | 2020-12-16 | ヤマハ株式会社 | Automatic performance system and automatic performance method |
US11030983B2 (en) | 2017-06-26 | 2021-06-08 | Adio, Llc | Enhanced system, method, and devices for communicating inaudible tones associated with audio files |
US10460709B2 (en) | 2017-06-26 | 2019-10-29 | The Intellectual Property Network, Inc. | Enhanced system, method, and devices for utilizing inaudible tones with music |
US11017751B2 (en) * | 2019-10-15 | 2021-05-25 | Avid Technology, Inc. | Synchronizing playback of a digital musical score with an audio recording |
US11972693B2 (en) | 2020-12-02 | 2024-04-30 | Joytunes Ltd. | Method, device, system and apparatus for creating and/or selecting exercises for learning playing a music instrument |
US11893898B2 (en) * | 2020-12-02 | 2024-02-06 | Joytunes Ltd. | Method and apparatus for an adaptive and interactive teaching of playing a musical instrument |
US11670188B2 (en) | 2020-12-02 | 2023-06-06 | Joytunes Ltd. | Method and apparatus for an adaptive and interactive teaching of playing a musical instrument |
US11900825B2 (en) | 2020-12-02 | 2024-02-13 | Joytunes Ltd. | Method and apparatus for an adaptive and interactive teaching of playing a musical instrument |
Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4779510A (en) * | 1987-11-20 | 1988-10-25 | Den Abbeel Paul Van | Electronic apparatus for displaying music |
US5760323A (en) * | 1996-06-20 | 1998-06-02 | Music Net Incorporated | Networked electronic music display stands |
US20010037719A1 (en) * | 2000-05-03 | 2001-11-08 | Gardner Gary L. | Digital sheet music display system |
US20020011143A1 (en) * | 2000-06-21 | 2002-01-31 | Haruki Uehara | Musical score display for musical performance apparatus |
US6365819B2 (en) * | 1999-12-24 | 2002-04-02 | Roland Corporation | Electronic musical instrument performance position retrieval system |
US7064261B2 (en) * | 2003-10-15 | 2006-06-20 | Sunplus Technology Co., Ltd. | Electronic musical score device |
US20060288842A1 (en) * | 1996-07-10 | 2006-12-28 | Sitrick David H | System and methodology for image and overlaid annotation display, management and communicaiton |
US7189912B2 (en) * | 2001-05-21 | 2007-03-13 | Amusetec Co., Ltd. | Method and apparatus for tracking musical score |
US20070113728A1 (en) * | 2005-09-02 | 2007-05-24 | Qrs Music Technologies, Inc. | Method and apparatus for playing in synchronism with a digital audio file an automated musical instrument |
US20070119292A1 (en) * | 2005-09-26 | 2007-05-31 | Yamaha Corporation | Apparatus for automatically starting add-on progression to run with inputted music, and computer program therefor |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH01277893A (en) | 1988-04-30 | 1989-11-08 | Shimadzu Corp | Musical instrument |
-
2008
- 2008-04-09 US US12/100,125 patent/US7482529B1/en not_active Expired - Fee Related
- 2008-11-06 US US12/266,327 patent/US20090255396A1/en not_active Abandoned
Patent Citations (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4779510A (en) * | 1987-11-20 | 1988-10-25 | Den Abbeel Paul Van | Electronic apparatus for displaying music |
US5760323A (en) * | 1996-06-20 | 1998-06-02 | Music Net Incorporated | Networked electronic music display stands |
US20060288842A1 (en) * | 1996-07-10 | 2006-12-28 | Sitrick David H | System and methodology for image and overlaid annotation display, management and communicaiton |
US6365819B2 (en) * | 1999-12-24 | 2002-04-02 | Roland Corporation | Electronic musical instrument performance position retrieval system |
US20010037719A1 (en) * | 2000-05-03 | 2001-11-08 | Gardner Gary L. | Digital sheet music display system |
US20020011143A1 (en) * | 2000-06-21 | 2002-01-31 | Haruki Uehara | Musical score display for musical performance apparatus |
US6392132B2 (en) * | 2000-06-21 | 2002-05-21 | Yamaha Corporation | Musical score display for musical performance apparatus |
US7189912B2 (en) * | 2001-05-21 | 2007-03-13 | Amusetec Co., Ltd. | Method and apparatus for tracking musical score |
US7064261B2 (en) * | 2003-10-15 | 2006-06-20 | Sunplus Technology Co., Ltd. | Electronic musical score device |
US20070113728A1 (en) * | 2005-09-02 | 2007-05-24 | Qrs Music Technologies, Inc. | Method and apparatus for playing in synchronism with a digital audio file an automated musical instrument |
US20070119292A1 (en) * | 2005-09-26 | 2007-05-31 | Yamaha Corporation | Apparatus for automatically starting add-on progression to run with inputted music, and computer program therefor |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100095828A1 (en) * | 2006-12-13 | 2010-04-22 | Web Ed. Development Pty., Ltd. | Electronic System, Methods and Apparatus for Teaching and Examining Music |
US20110132176A1 (en) * | 2009-12-04 | 2011-06-09 | Stephen Maebius | System for displaying and scrolling musical notes |
US8530735B2 (en) * | 2009-12-04 | 2013-09-10 | Stephen Maebius | System for displaying and scrolling musical notes |
Also Published As
Publication number | Publication date |
---|---|
US7482529B1 (en) | 2009-01-27 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7482529B1 (en) | Self-adjusting music scrolling system | |
US7579541B2 (en) | Automatic page sequencing and other feedback action based on analysis of audio performance data | |
US8027631B2 (en) | Song practice support device | |
US6166314A (en) | Method and apparatus for real-time correlation of a performance to a musical score | |
US7189912B2 (en) | Method and apparatus for tracking musical score | |
US9595256B2 (en) | System and method for singing synthesis | |
US20080115656A1 (en) | Tempo detection apparatus, chord-name detection apparatus, and programs therefor | |
US9355634B2 (en) | Voice synthesis device, voice synthesis method, and recording medium having a voice synthesis program stored thereon | |
US10504498B2 (en) | Real-time jamming assistance for groups of musicians | |
US20060112812A1 (en) | Method and apparatus for adapting original musical tracks for karaoke use | |
US8859872B2 (en) | Method for giving feedback on a musical performance | |
Arzt et al. | Artificial intelligence in the concertgebouw | |
JP4212446B2 (en) | Karaoke equipment | |
JP2010025972A (en) | Code name-detecting device and code name-detecting program | |
JP2008040259A (en) | Musical piece practice assisting device, dynamic time warping module, and program | |
JP2007334364A (en) | Karaoke machine | |
JP4204941B2 (en) | Karaoke equipment | |
JP2009169103A (en) | Practice support device | |
JP4222919B2 (en) | Karaoke equipment | |
JP2009003225A (en) | Chord name detector and program for chord name detection | |
JP4048249B2 (en) | Karaoke equipment | |
KR101546331B1 (en) | Method for reproducing music using musical scales and apparatus using the method | |
JP2005107332A (en) | Karaoke machine | |
JP4159961B2 (en) | Karaoke equipment | |
JP5983670B2 (en) | Program, information processing apparatus, and data generation method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO PAY ISSUE FEE |