CN112581976B - Singing scoring method and system based on streaming media - Google Patents
Singing scoring method and system based on streaming media Download PDFInfo
- Publication number
- CN112581976B CN112581976B CN201910933048.0A CN201910933048A CN112581976B CN 112581976 B CN112581976 B CN 112581976B CN 201910933048 A CN201910933048 A CN 201910933048A CN 112581976 B CN112581976 B CN 112581976B
- Authority
- CN
- China
- Prior art keywords
- singing
- electronic device
- time difference
- time
- song
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013077 scoring method Methods 0.000 title claims abstract description 40
- 230000003111 delayed effect Effects 0.000 claims description 18
- 238000001228 spectrum Methods 0.000 claims 2
- 238000013461 design Methods 0.000 description 9
- 238000000034 method Methods 0.000 description 6
- 238000009825 accumulation Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000003139 buffering effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000001934 delay Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H1/00—Details of electrophonic musical instruments
- G10H1/0008—Associated control or indicating means
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H1/00—Details of electrophonic musical instruments
- G10H1/36—Accompaniment arrangements
- G10H1/361—Recording/reproducing of accompaniment for use with an external source, e.g. karaoke systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Computational Linguistics (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Reverberation, Karaoke And Other Acoustics (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses a singing scoring method and a singing scoring system based on streaming media, which not only calculate the first time difference that a streaming video player starts to play the song and an electronic device starts a recording program and a scoring engine, but also continuously calculate the play time difference of the streaming video player in every fixed time interval of the system time of the electronic device, and transmit the play time difference to the scoring engine to accumulate to form a second time difference, and then the scoring engine can adjust the singing time of each note in a singing score according to the first time difference and the second time difference, so that the singing scoring method and the singing scoring system are not limited by the technical problems derived from streaming.
Description
Technical Field
The present invention relates to a singing scoring method and system, and more particularly, to a singing scoring method and system based on Streaming Media.
Background
Existing singing scoring methods typically store song files (e.g., music movies) in advance in local electronic devices (e.g., karaoke machines or smart phones). Therefore, the existing singing scoring method is limited by the storage capacity of the local electronic device, and it is difficult to expand the song library. In addition, since the music movie with high definition image is usually large in file, the waiting time before singing is relatively long if the song file is downloaded from the cloud server to the local electronic device. However, in the case of copyright usage, if the copyright owner does not agree with the authorized form of download but only allows streaming, the prior art cannot be implemented.
On the other hand, even if the streaming is a more general authorized form for copyright owners, the prior art cannot solve the technical problems derived from the streaming, such as: network latency, buffering time of streaming, and time precision of playing streaming media by the video player are inferior to that of playing local files. These technical problems all cause that the existing singing scoring method cannot accurately compare the singing voice data of the user with the singing music, so that the scoring is incorrect and the user experience is poor. Therefore, how to design a singing scoring method and a singing scoring system based on streaming media becomes an important issue in the art.
Disclosure of Invention
In view of the above, an embodiment of the present invention provides a singing scoring method based on streaming media, which is executed in an electronic device, the electronic device is provided with an application program (Application Program), and after the electronic device starts the application program, the electronic device generates an operation interface, and the singing scoring method includes the following steps. When a user selects a song to be scored through the operation interface, the electronic device downloads the singing music of the song from the music server through the application program, and starts the streaming video player to start streaming the song from the streaming server. Secondly, the electronic device detects whether the streaming video player starts playing the song through the application program, when the streaming video player starts playing the song, the electronic device immediately records the system time of the electronic device through the application program, starts a recording program in the application program to record from a microphone device of the electronic device, and simultaneously starts a scoring engine in the application program to compare singing music with sound data recorded by the recording program. Then, the electronic device calculates a first time difference between the start of playing the song by the streaming video player and the start of the recording program and the scoring engine by the electronic device through the application program, and transmits the first time difference to the scoring engine. Then, the electronic device continuously calculates the play time difference of the streaming video player in every fixed time of the system time through the application program, and transmits the play time difference to the scoring engine for accumulation to form a second time difference, and then the scoring engine adjusts the singing music according to the first time difference and the second time difference, and performs singing scoring by comparing the adjusted singing music with sound data recorded by the recording program.
In addition, the embodiment of the invention further provides a singing scoring system. The singing scoring system comprises the singing score server, the streaming server and the electronic device of the foregoing embodiment, and when the electronic device starts an application program to generate an operation interface, the application program is used for instructing the electronic device to execute the singing scoring method of the foregoing embodiment.
For a further understanding of the nature and the technical aspects of the present invention, reference should be made to the following detailed description of the invention and the accompanying drawings, which are provided for purposes of reference only and are not intended to limit the invention.
Drawings
Fig. 1 is a block diagram of a singing scoring system provided by an embodiment of the present invention.
Fig. 2 is a flowchart illustrating steps of a singing scoring method according to an embodiment of the present invention.
Detailed Description
The following embodiments of the present invention are described in terms of specific examples, and those skilled in the art will appreciate the advantages and effects of the present invention from the disclosure provided herein. The invention is capable of other and different embodiments and its several details are capable of modifications and various other uses and applications, all of which are obvious from the description, without departing from the spirit of the invention. The drawings of the present invention are merely schematic illustrations, and are not intended to be drawn to actual dimensions. The following embodiments will further illustrate the related art content of the present invention in detail, but the content provided is not intended to limit the scope of the present invention.
It will be understood that, although the terms "first," "second," "third," etc. may be used herein to describe various components or signals, these components or signals should not be limited by these terms. These terms are used primarily to distinguish one element from another element or signal from another signal. In addition, the term "or" as used herein shall include any one or combination of more of the associated listed items as the case may be.
First, referring to fig. 1 and fig. 2, fig. 1 is a block diagram of a singing scoring system according to an embodiment of the present invention, and fig. 2 is a flowchart illustrating steps of a singing scoring method according to an embodiment of the present invention. It should be noted that, the singing scoring method of fig. 2 may be executed in the electronic device 10 of fig. 1, but the present invention is not limited to the singing scoring method of fig. 2 being executed only in the electronic device 10 of fig. 1, and as shown in fig. 1, the electronic device 10, the score server 20 and the stream server 30 further constitute a singing scoring system 1. In the present embodiment, the electronic device 10 will be connectable to the music server 20 and the streaming server 30 via the internet 40. Thus, the electronic device 10 may be implemented as a desktop computer, a notebook computer, a smart phone, a tablet computer, or any electronic device with networking function, for example, but the invention is not limited thereto. In summary, those skilled in the art will appreciate that the electronic device 10 is comprised of suitable circuitry and hardware, such as a central processing unit and a memory. In addition, the electronic device 10 may include an Operating System (OS) 110 and a microphone device 140, and the Operating System 110 is installed with an application 120.
The application 120 may be implemented by a plurality of program codes and instructions for instructing the electronic device 10 to perform the singing scoring method of fig. 2. That is, after (the operating system 110 of) the electronic device 10 installs the application 120, the electronic device 10 may selectively launch the application 120. It should be noted that the present invention is not limited to the specific implementation of the electronic device 10 when the application program 120 is installed and started, and those skilled in the art should be able to design according to the actual requirements or applications. In addition, after the electronic device 10 starts the application 120, the electronic device 10 may generate an operation interface through the application 120. In this embodiment, in addition to the operating system 110 having the application 120, the operating system 110 also has a streaming video player 130. The streaming audio player 130 and the application 120 may be released by different software developers, or the application 120 may actively include the streaming audio player 130, but the electronic device 10 may be capable of simultaneously presenting the streaming audio player 130 with the above-mentioned operation interface, which is not limited to the specific implementation of the streaming audio player 130.
After the electronic device 10 starts the application program 120, as shown in fig. 2, when the user selects a song to be scored for singing via the operation interface of the application program 120 in step S201, the electronic device 10 downloads the singing score of the song from the score server 20 via the application program 120, and starts the streaming audio/video player 130 to stream the song from the streaming server 30 via the application program 120 in step S203. It should be noted that the singing score is not limited to a general song score, but refers to any numerical data representing Pitch (Pitch) of a song, singer singing skills, or characteristic values obtained by analyzing a song, etc., which can be used to facilitate singing scoring. Next, in step S205, the electronic device 10 detects whether the streaming audio/video player 130 starts playing the song through the application 120; alternatively, the electronic device 10 may begin playing by the application 120 detecting whether the streaming video player 130 receives the song file streamed from the streaming server 30 and completes the buffering. If not (play is started), the electronic device 10 executes step S207; if yes, the electronic device 10 executes step S209. In step S207, the electronic device 10 checks whether a first allowable time, e.g. 5 seconds, has elapsed since the electronic device 10 started the streaming audio player 130 (from the streaming server 30) to stream the song through the application 120. If not, the electronic device 10 returns to step S205; if yes, the electronic device 10 returns to step S203.
For example, when the electronic device 10 in step S203 starts the streaming audio player 130 to start streaming the song from the streaming server 30, the system time of the electronic device 10 is (gmt+8) 2019, 8, 5, 09:58:00:000. next, because the electronic device 10 of step S205 detects that the streaming audio/video player 130 has not started playing the song, the electronic device 10 of step S207 checks whether the system time is from (gmt+8) 2019, 8, 5, 09:58:00:000 and 5 seconds. If the system time at this time is from (gmt+8) 2019, 8, 5, 09:58:00:000 for only 0.2 seconds, i.e., (gmt+8) 2019, 8, 5, 09:58:00:200, the electronic device 10 returns to step S205, and so on, until the electronic device 10 of step S207 checks that the system time is from (gmt+8) 2019, 8, 5, 09:58:00:000 for 5 seconds, i.e., (gmt+8) 2019, 8, 5, 09:58:05:000, which means that the streaming audio player 130 has not started playing the song 5 seconds after the electronic device 10 starts streaming the song, so the electronic device 10 can determine that the streaming audio player 130 is unable to stream normally, and the electronic device 10 returns to step S203 to restart the streaming audio player 130 to start streaming the song from the streaming server 30. It should be noted that, for convenience of description of the present embodiment, the system time of the electronic device 10 is the time of year, month, day, and year: the method comprises the following steps: second: millisecond form, which is not intended to limit the invention.
It should be understood that the above-mentioned first allowable time of 5 seconds is only an example, and the present invention is not limited to the specific value of the first allowable time, and those skilled in the art can design according to actual needs or applications. In other embodiments, the electronic device 10 may omit the step S207, which does not affect the implementation of the present invention. That is, in the embodiment without step S207, when the electronic device 10 has started the streaming audio player 130 to start streaming the song, but the streaming audio player 130 has not started playing the song, the electronic device 10 continues to perform step S205 until detecting that the streaming audio player 130 starts playing the song. Incidentally, the electronic device 10 may read the playing time of the streaming audio player 130 at intervals by the application 120 to detect whether the streaming audio player 130 starts playing the song, which is not limited to the implementation of step S205. In addition, in step S209, the electronic device 10 immediately records the system time of the electronic device 10 through the application 120, and starts a recording program (not shown) in the application 120 to start recording from the microphone device 140, and also starts a scoring engine (not shown) in the application 120 to start comparing the singing score with the sound data recorded by the recording program.
It should be noted that, limited to the memory and the cpu of the electronic device 10, when the streaming audio/video player 130 is detected to start playing the song, the electronic device 10 cannot start the recording program and the scoring engine immediately, but starts the recording program and the scoring engine after a certain period of time, so that the user can sing ahead according to the playing of the song, and the recording program also records the sound data later, so that the scoring engine cannot accurately compare the singing sound data with the singing music of the user. Therefore, in step S211 after step S209, the electronic device 10 calculates a first time difference between the start of playing the song by the streaming audio/video player 130 and the start of the recording and scoring engine by the electronic device 10 through the application 120. For example, if the electronic device 10 detects that the system time recorded when the streaming audio video player 130 starts playing the song is (gmt+8) 2019, 8, 5, 10:00:00:000, the system time when the electronic device 10 starts the recording program and scoring engine is (gmt+8) 2019, 8, 5, 10:00:03:000, which means that the electronic device 10 can calculate the first time difference to be 3 seconds by the application 120. In other words, when the electronic device 10 detects that the streaming audio/video player 130 starts playing the song, the electronic device 10 delays for 3 seconds to start the recording program and the scoring engine, so that the user can sing 3 seconds earlier and the recording program records 3 seconds later of sound data according to the playing of the song.
The scoring engine is not affected by the user's early singing and the late recording of the recording program, but the effective scoring is continued, and will be described hereinafter by other contents, so details thereof will not be repeated here. It should be understood that, since the positive number is used to indicate how long the electronic device 10 starts the recording and scoring engine after a delay, the value of the first time difference may be greater than or equal to 0, and if the value is 0, this indicates that the electronic device 10 also starts the recording and scoring engine immediately when the electronic device 10 detects that the streaming audiovisual player 130 starts playing the song. In contrast, the electronic device 10 can also calculate the first time difference by the application 120 comparing the playing time of the streaming audio/video player 130. For example, if the playback time for the streaming audio/video player 130 to start playing the song is 0:00:00:000, but the playback time of the streaming audio/video player 130 when the electronic device 10 starts the recording procedure and the scoring engine is 0:00:03:000, which means that the electronic device 10 can also calculate the first time difference by the application 120 for 3 seconds, and in summary, the present invention is not limited to the specific implementation of calculating the first time difference by the application 120, and those skilled in the art should be able to design according to the actual needs or applications. Similarly, for convenience of description of the present embodiment, the playback time of the streaming audio/video player 130 is as follows: the method comprises the following steps: second: in milliseconds, but is not meant to limit the invention.
In addition, in other embodiments, the user may select to score singing from the sub-song of the song through the operation interface of the application 120, so that the playing time of the streaming audio player 130 to begin playing the song is not 0:00:00:000, this in reverse does not affect the implementation of the present invention. Next, in step S213, the electronic device 10 checks whether the first time difference is greater than a second allowable time, such as 5 seconds, through the application 120. If yes, the electronic device 10 determines that the scoring engine at this time is seriously affected by the user singing too early and the recording program recording too late and cannot score effectively, so the electronic device 10 returns to execute step S203 to restart the streaming video player 130 to start streaming the song from the streaming server 30; if not, the electronic device 10 executes step S215 to transmit the first time difference to the scoring engine. Similarly, the second allowable time is 5 seconds only for example, and the present invention is not limited to the specific value of the second allowable time, and those skilled in the art can design according to the actual requirement or application. In other embodiments, the electronic device 10 may omit step S213, which does not affect the implementation of the present invention.
Finally, in step S217, the electronic device 10 continuously calculates a play time difference of the streaming audio/video player 130 at regular intervals (e.g. 0.1 second) in the system time through the application program 120, and transmits the play time difference to the scoring engine for accumulation to form a second time difference, and then the scoring engine performs singing scoring by comparing the adjusted singing score with the sound data recorded by the recording program according to the first time difference and the second time difference. For example, if the system time of the electronic device 10 is 10 from (gmt+8) 2019, 8, 5, 10:00:00:000 to (gmt+8) 2019, 8, 5, 10:00:00:100 for 0.1 seconds, then the playback time of the streaming audio/video player 130 should also be from 0:00:00:000 to 0:00:00:100 for 0.1 seconds. However, due to the unstable network propagation, the streaming may be delayed or advanced. For example, assume that the electronic device 10 detects this time by the application 120, but the play time is from 0:00:00:000 to 0:00:00:090 only takes 0.09 seconds, which means that the stream is delayed by 0.01 seconds, so the electronic device 10 can calculate the play time difference by +0.01 seconds, i.e. 0.1 seconds minus 0.09 seconds, through the application 120. The present embodiment may simply refer to the play time difference at this time as D1, and transmit the play time difference D1 to the scoring engine, and since the scoring engine only receives the play time difference D1, the second time difference at this time is +0.01 seconds.
The scoring engine adjusts the singing score according to the first time difference and the second time difference, and performs singing scoring by comparing the adjusted singing score with the sound data recorded by the recording program, which will be described hereinafter by other contents, so details thereof will not be repeated herein. It should be understood that the above-mentioned fixed time of 0.1 seconds is only an example, and the present invention is not limited to the specific value of the fixed time, and those skilled in the art can design according to actual needs or applications. Next, if the system time of the electronic device 10 is from (gmt+8) 2019, 8 months 5 days 10:00:00:100 to (gmt+8) 2019, 8, 5, 10:00:00:200 further passes 0.1 seconds, the playback time of the streaming video player 130 should also be from 0:00:00:090 to 0:00:00:190 for 0.1 seconds, but the play time at this time is from 0:00:00:090 to 0:00:00:170 for 0.08 seconds, which means that the streaming is delayed by 0.02 seconds, the electronic device 10 can calculate the play time difference by the application 120 to be +0.02 seconds, i.e. 0.1 seconds minus 0.08 seconds. The embodiment can simply refer to the play time difference at this time as D2, and transmit the play time difference D2 to the scoring engine, and since the scoring engine has received the play time difference D1 immediately before, the second time difference at this time is +0.03 seconds, i.e. the sum of the play time differences D1-D2.
Similarly, if the system time of the electronic device 10 is from (gmt+8) 2019, 8, 5, 10:00:00:200 to (gmt+8) 2019, 8, 5, 10:00:00:300 for another 0.1 seconds, but the playing time at this time is from 0:00:00:170 to 0:00:00:280 last 0.11 seconds, which means that the current stream is advanced by 0.01 seconds, so that the electronic device 10 can calculate the play time difference D by the application 120 3 Is-0.01 seconds and the second time difference at this time is +0.02 seconds. In other words, when the system time of the electronic device 10 is 0.1 seconds, the electronic device 10 can detect whether the playing time of the streaming audio/video player 130 is synchronized for 0.1 seconds through the application 120, and calculate the difference of the playing time as the playing time difference D i . Since the following details are already described in the foregoing, no further description is given here. In summary, since the present embodiment uses positive and negative numbers to represent how long the stream is delayed and how long the stream is advanced, respectively, the playback time difference D i And the value of the second time difference can be zero, positive value or negative value, and the accumulated formula of the second time difference can be expressed as
Where i is a positive integer from 1 to (T/dt) and dt is the fixed time as described above. In addition, in this embodiment, T may be assumed to be the total duration of the song, for example 180 seconds, so i is a positive integer from 1 to 1800.
In order to reduce the computational burden of the electronic device 10, the application 120 may also design T to be a third allowable time, for example, 10 seconds, so i is a positive integer from 1 to 100. That is, the electronic device 10 continuously calculates the playing time difference of the streaming audio/video player 130 at every 0.1 second by the application 120 within the 10 seconds, and transmits the playing time difference to the scoring engine for accumulation to form the second time difference. One of the benefits of this is that if the electronic device 10 detects that the value of the second time difference is greater than or equal to a positive threshold (e.g., +3) or less than or equal to a negative threshold (e.g., -3) within the 10 seconds, the electronic device 10 can determine that the current streaming is too late or early for the scoring engine to be able to score effectively, so the electronic device 10 can also return to step S203 to restart the streaming video player 130 to start streaming the song from the streaming server 30.
It should be noted that, in this embodiment, the positive number and the negative number are used to represent how long the stream is delayed and how long the stream is advanced, respectively, so when the value of the second time difference is greater than or equal to the positive threshold, the electronic device 10 determines that the stream is delayed too severely at this time; conversely, when the value of the second time difference is less than or equal to the negative threshold, the electronic device 10 recognizes that the streaming is advanced too much. However, if the negative number and the positive number are used instead to indicate how long the stream is delayed and how long the stream is advanced, respectively, the above-mentioned determination results are just opposite, so that the description thereof will not be repeated here. In addition, the present invention does not limit that the absolute values of the positive and negative thresholds are the same or different, but the electronic device 10 may also be simplified to detect whether the absolute value of the second time difference is greater than or equal to a threshold (e.g. 3) by the application 120 if the absolute values of the positive and negative thresholds are the same. Since the following details are already described in the foregoing, further description is omitted here. Similarly, the third allowable time is 10 seconds only for example, and the present invention is not limited to the specific value of the third allowable time, and those skilled in the art can design according to the actual requirement or application.
On the other hand, when the electronic device 10 starts the recording program and the scoring engine, the electronic device 10 may start counting a singing time. Assuming that the song has N notes, each note is in N k Represented where k is a positive integer from 1 to n, and the original for each noteSinging time is first performed by T k The representation, therefore, of the present embodiment can use the singing score of the song as (N) k ，T k ) To represent. That is, in the prior art, the scoring engine compares the user's singing time T one by one k Whether the musical note sung down is identical to the musical note N in the singing score k According to the above, the difference between the two values is calculated to be used as the scoring value at the moment, and finally all scoring values are counted to form the scoring result of the song sung by the user.
However, taking the foregoing example as a continuation, since the electronic device 10 starts the recording process and the scoring engine after 3 seconds is delayed when the streaming audio player 130 starts playing the song, so that the user can sing 3 seconds earlier according to the playing of the song, or the scoring engine starts to use a singing time 3 seconds later than the playing time of the streaming audio player 130, and the streaming may also have a delay or advance phenomenon since the streaming audio player 130 starts playing the song, when the scoring engine obtains the first time difference and the second time difference, the first time difference and the second time difference are respectively referred to as Δt1 and Δt2, the scoring engine can immediately adjust the singing score to be (N) k ，T k - Δt1+Δt2) and by comparing the user's singing time (T k - Δt1+Δt2) whether the musical note sung is identical to the musical note N in the singing score k Matching to calculate the scoring value at this time.
For example, assume that the original singing time for one note "Do" is 4.05 seconds, and that the electronic device 10 is from its system time of (gmt+8) 2019, 8 months, 5 days 10:00:03:900 to (gmt+8) 2019, 8, 5, 10:00:04:000, this means that the scoring engine can immediately adjust the singing time of the note "Do" to 1.1 seconds, i.e., 4.05-3+0.05=1.1, and then calculate the score value at this time by comparing whether the note that the user sings at 1.1 seconds coincides with the note "Do". Similarly, assume that the original singing time for the further note "Re" is at 4.37 seconds, and that the electronic device 10 is from its system time of (gmt+8) 2019, 8, 5, 10:00:04:200 to (gmt+8) 2019, 8, 5, 10:00:04: if the second time difference accumulated at 300 is 0 seconds, this means that the scoring engine can immediately adjust the singing time of the note "Re" to 1.37 seconds, i.e., 4.37-3+0 =1.37, and then calculate the score value at this time by comparing whether the note sung by the user at the singing time of 1.37 seconds coincides with the note "Re".
Since the following details are as described above, the details are not repeated here, and in summary, each note N k The first time difference is subtracted from the original singing time of each note Nk to cancel out the effect that the singing time starts later than the playing time of the streaming audio-visual player 130, and the second time difference is added up by the system time nearest to the original singing time of each note Nk to cancel out the delay or advance effect of the streaming in this period. It should be understood that if other embodiments instead use negative numbers to indicate how long the electronic device 10 is delayed to start the recording process and the scoring engine, and use negative numbers and positive numbers to indicate how long the stream is delayed and how long the stream is advanced, respectively, the scoring engine is instead adapted to adjust the singing score to (N) k ，T k +Δt1- Δt2) and further by comparing the user's singing time (T k +Δt1- Δt2) whether the musical note sung is identical to the musical note N in the singing score k The scoring value at this time is calculated in agreement, but this does not affect the implementation of the present invention. In addition, the fixed time dt must be smaller than the singing time difference between any two consecutive notes, for example, the singing time difference between the note "Do" and the note "Re" is 0.32 seconds, i.e. 4.37-4.05=0.32, so the application 120 does not need to design the fixed time dt to be equal to or greater than 0.32 seconds.
In summary, the embodiments of the present invention provide a method and a system for scoring songs based on streaming media, which can calculate the start of playing songs by a streaming audio/video player, start a recording program by an electronic device, and score an engineIn addition to the first time difference, the singing scoring method and the singing scoring system also continuously calculate the playing time difference of the streaming video player in every other fixed time in the system time of the electronic device, and transmit the playing time difference to the scoring engine for accumulation to form a second time difference, and then the scoring engine calculates each note N in the adjustable singing music according to the first time difference and the second time difference k For example, the adjusted singing time is (T k - Δt1+Δt2), and the scoring engine compares the user's singing time (T k - Δt1+Δt2) whether the note sung at Δt1+Δt2) matches the note N k The scoring value at this time is calculated in correspondence, and thus the singing scoring method and the singing scoring system will not be limited by technical difficulties derived from streaming.
The above disclosure is only a preferred embodiment of the present invention and is not intended to limit the claims of the present invention, so that all equivalent technical changes made by the application of the specification and the drawings of the present invention are included in the claims of the present invention.
Claims (20)
1. The singing scoring method based on streaming media is executed in an electronic device, the electronic device is provided with an application program, and after the electronic device starts the application program, the electronic device generates an operation interface, and the singing scoring method comprises the following steps:
when a user selects a song to be scored through the operation interface, the electronic device downloads a singing music of the song from a music server through the application program, and starts a streaming video player to start streaming the song from a streaming server;
the electronic device detects whether the streaming video player starts playing the song through the application program, when the streaming video player starts playing the song, the electronic device immediately records a system time of the electronic device through the application program, starts a recording program in the application program to start recording from a microphone device of the electronic device, and also starts a scoring engine in the application program to start comparing the singing music spectrum with sound data recorded by the recording program;
the electronic device calculates a first time difference between the start of playing the song by the streaming video player and the start of the recording program and the scoring engine by the electronic device through the application program, and transmits the first time difference to the scoring engine; and
the electronic device continuously calculates the play time difference of the streaming video player in every fixed time of the system time through the application program, transmits the play time difference to the scoring engine to be accumulated to form a second time difference, and then adjusts the singing score according to the first time difference and the second time difference, and performs singing scoring by comparing the adjusted singing score with the sound data recorded by the recording program.
2. The singing scoring method of claim 1, wherein the electronic device reads the playing time of the streaming video player at intervals by the application program to detect whether the streaming video player starts playing the song.
3. The singing scoring method as recited in claim 1, further comprising:
when detecting that the streaming video player has not started playing the song, the electronic device checks whether the system time of the electronic device passes a first allowable time since the electronic device starts the streaming video player to start streaming the song through the application program;
if not, the electronic device returns to execute the step of detecting whether the streaming video player starts playing the song through the application program; and
if yes, the electronic device returns to execute the step of starting the streaming video player to start streaming the song from the streaming server.
4. The singing scoring method as recited in claim 3 wherein the singing scoring method further comprises:
the electronic device checks whether the first time difference is larger than a second allowable time or not through the application program;
if yes, the electronic device returns to execute the step of starting the streaming video player to start streaming the song from the streaming server; and
if not, the electronic device transmits the first time difference to the scoring engine.
5. The singing scoring method of claim 4 wherein the application uses positive numbers to indicate how long the electronic device is delayed to start the recording and scoring engine, and uses positive numbers and negative numbers to indicate how long the stream is delayed and how long the stream is advanced, respectively.
6. The singing scoring method as recited in claim 5 wherein the song has a plurality of notes, each of the notes corresponding to a singing time to form the singing score, and the scoring engine adjusts the singing time of each of the notes in the singing score according to the first time difference and the second time difference, wherein the adjusted singing time is (T k -Δt1+Δt2)，T k For the singing time before the adjustment, Δt1 and Δt2 are the first time difference and the second time difference, respectively.
7. The singing scoring method of claim 4 wherein the application uses a negative number to indicate how long the electronic device is delayed to start the recording and scoring engine, and uses a negative number and a positive number to indicate how long the stream is delayed and how long the stream is advanced, respectively.
8. The singing scoring method of claim 7 wherein the song has a plurality of notes, each of the notes corresponding to a singing time to form the singing score, and according to the first time difference and the first time differenceTwo time differences, the scoring engine adjusts the singing time of each of the notes in the singing score, wherein the adjusted singing time is (T) k +Δt1-Δt2)，T k For the singing time before the adjustment, Δt1 and Δt2 are the first time difference and the second time difference, respectively.
9. The singing scoring method of claim 4, wherein the formula accumulating the second time difference is expressed as
D i For the playing time difference, T is a third allowable time, dt is the fixed time, and the singing scoring method further includes:
the electronic device detects whether the value of the second time difference is greater than or equal to a positive threshold value or less than or equal to a negative threshold value through the application program;
if yes, the electronic device returns to execute the step of starting the streaming video player to start streaming the song from the streaming server; and
if not, the scoring engine adjusts the singing score according to the first time difference and the second time difference, and performs singing scoring by comparing the adjusted singing score with the sound data recorded by the recording program.
10. The singing scoring method of claim 4, wherein the formula accumulating the second time difference is expressed as
D i For the playing time difference, T is a third allowable time, dt is the fixed time, and the singing scoring method further includes:
the electronic device detects whether the absolute value of the second time difference is greater than or equal to a threshold value through the application program;
if yes, the electronic device returns to execute the step of starting the streaming video player to start streaming the song from the streaming server; and
if not, the scoring engine adjusts the singing score according to the first time difference and the second time difference, and performs singing scoring by comparing the adjusted singing score with the sound data recorded by the recording program.
11. A singing scoring system based on streaming media, the singing scoring system comprising:
a singing music server;
a streaming server; and
an electronic device is provided with an application program, and after the electronic device starts the application program to generate an operation interface, the application program is used for instructing the electronic device to execute a singing scoring method, and the singing scoring method comprises the following steps:
when a user selects a song to be scored through the operation interface, the electronic device downloads a singing music of the song from the music server through the application program, and starts a streaming video player to start streaming the song from the streaming server;
the electronic device detects whether the streaming video player starts playing the song through the application program, when the streaming video player starts playing the song, the electronic device immediately records a system time of the electronic device through the application program, starts a recording program in the application program to start recording from a microphone device of the electronic device, and also starts a scoring engine in the application program to start comparing the singing music spectrum with sound data recorded by the recording program;
the electronic device calculates a first time difference between the start of playing the song by the streaming video player and the start of the recording program and the scoring engine by the electronic device through the application program, and transmits the first time difference to the scoring engine; and
the electronic device continuously calculates the play time difference of the streaming video player in every fixed time of the system time through the application program, transmits the play time difference to the scoring engine to be accumulated to form a second time difference, and then adjusts the singing score according to the first time difference and the second time difference, and performs singing scoring by comparing the adjusted singing score with the sound data recorded by the recording program.
12. The singing scoring system of claim 11, wherein the electronic device reads the playback time of the streaming video player at intervals by the application to detect whether the streaming video player begins playing the song.
13. The singing scoring system of claim 11, wherein the singing scoring method further comprises:
when detecting that the streaming video player has not started playing the song, the electronic device checks whether the system time of the electronic device passes a first allowable time since the electronic device starts the streaming video player to start streaming the song through the application program;
if not, the electronic device returns to execute the step of detecting whether the streaming video player starts playing the song through the application program; and
if yes, the electronic device returns to execute the step of starting the streaming video player to start streaming the song from the streaming server.
14. The singing scoring system of claim 13, wherein the singing scoring method further comprises:
the electronic device checks whether the first time difference is larger than a second allowable time or not through the application program;
if yes, the electronic device returns to execute the step of starting the streaming video player to start streaming the song from the streaming server; and
if not, the electronic device transmits the first time difference to the scoring engine.
15. The singing scoring system of claim 14 wherein the application uses positive numbers to indicate how long the electronic device is delayed to start the recording and scoring engine and positive numbers and negative numbers to indicate how long the stream is delayed and how long the stream is advanced, respectively.
16. The singing scoring system of claim 15 wherein the song has a plurality of notes, each of the notes corresponding to a singing time to form the singing score, and the scoring engine adjusts the singing time of each of the notes in the singing score based on the first time difference and the second time difference, wherein the adjusted singing time is (T k -Δt1+Δt2)，T k For the singing time before the adjustment, Δt1 and Δt2 are the first time difference and the second time difference, respectively.
17. The singing scoring system of claim 14 wherein the application uses a negative number to indicate how long the electronic device is delayed to start the recording and scoring engine, and uses a negative number and a positive number to indicate how long the stream is delayed and how long the stream is advanced, respectively.
18. The singing scoring system of claim 17 wherein the song has a plurality of notes, each of the notes corresponding to a singing time to form the singing score, and the scoring engine adjusts the singing time of each of the notes in the singing score based on the first time difference and the second time difference, wherein the adjusted singing time is (T k +Δt1-Δt2)，T k For the singing time before the adjustment, Δt1 and Δt2 are the first time difference and the second time difference, respectively.
19. The singing scoring system of claim 14, wherein the formula accumulating the second time difference is expressed as
D i For the playing time difference, T is a third allowable time, dt is the fixed time, and the singing scoring method further includes:
the electronic device detects whether the value of the second time difference is greater than or equal to a positive threshold value or less than or equal to a negative threshold value through the application program;
if yes, the electronic device returns to execute the step of starting the streaming video player to start streaming the song from the streaming server; and
if not, the scoring engine adjusts the singing score according to the first time difference and the second time difference, and performs singing scoring by comparing the adjusted singing score with the sound data recorded by the recording program.
20. The singing scoring system of claim 14, wherein the formula accumulating the second time difference is expressed as
D i For the playing time difference, T is a third allowable time, dt is the fixed time, and the singing scoring method further includes:
the electronic device detects whether the absolute value of the second time difference is greater than or equal to a threshold value through the application program;
if yes, the electronic device returns to execute the step of starting the streaming video player to start streaming the song from the streaming server; and
if not, the scoring engine adjusts the singing score according to the first time difference and the second time difference, and performs singing scoring by comparing the adjusted singing score with the sound data recorded by the recording program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910933048.0A CN112581976B (en) | 2019-09-29 | 2019-09-29 | Singing scoring method and system based on streaming media |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910933048.0A CN112581976B (en) | 2019-09-29 | 2019-09-29 | Singing scoring method and system based on streaming media |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112581976A CN112581976A (en) | 2021-03-30 |
| CN112581976B true CN112581976B (en) | 2023-06-27 |
Family
ID=75111366
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910933048.0A Active CN112581976B (en) | 2019-09-29 | 2019-09-29 | Singing scoring method and system based on streaming media |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112581976B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102761523A (en) * | 2011-04-27 | 2012-10-31 | 宏碁股份有限公司 | Network streaming content playing method and system |
| CN103886854A (en) * | 2012-12-24 | 2014-06-25 | 詹江村 | Online singing system and singing method thereof |
| CN105578202A (en) * | 2014-09-29 | 2016-05-11 | 纬创资通股份有限公司 | Video and audio sharing method and system |
| CN106157976A (en) * | 2015-04-10 | 2016-11-23 | 科大讯飞股份有限公司 | A kind of singing evaluating method and system |
| CN107452361A (en) * | 2017-08-08 | 2017-12-08 | 腾讯音乐娱乐（深圳）有限公司 | Song subordinate sentence method and device |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8438131B2 (en) * | 2009-11-06 | 2013-05-07 | Altus365, Inc. | Synchronization of media resources in a media archive |
| US9774911B1 (en) * | 2016-07-29 | 2017-09-26 | Rovi Guides, Inc. | Methods and systems for automatically evaluating an audio description track of a media asset |
-
2019
- 2019-09-29 CN CN201910933048.0A patent/CN112581976B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102761523A (en) * | 2011-04-27 | 2012-10-31 | 宏碁股份有限公司 | Network streaming content playing method and system |
| CN103886854A (en) * | 2012-12-24 | 2014-06-25 | 詹江村 | Online singing system and singing method thereof |
| CN105578202A (en) * | 2014-09-29 | 2016-05-11 | 纬创资通股份有限公司 | Video and audio sharing method and system |
| CN106157976A (en) * | 2015-04-10 | 2016-11-23 | 科大讯飞股份有限公司 | A kind of singing evaluating method and system |
| CN107452361A (en) * | 2017-08-08 | 2017-12-08 | 腾讯音乐娱乐（深圳）有限公司 | Song subordinate sentence method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112581976A (en) | 2021-03-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101582436B1 (en) | Methods and systems for syschronizing media | |
| US20140074783A1 (en) | Synchronizing metadata across devices | |
| US11755280B2 (en) | Media content system for enhancing rest | |
| US20140074663A1 (en) | Integrating purchase history and metadata across devices | |
| KR20150095957A (en) | Methods and systems for processing a sample of media stream | |
| JP2005321668A (en) | Device and method for processing information, and program | |
| US20140280272A1 (en) | Media content substitution | |
| EP3839938B1 (en) | Karaoke query processing system | |
| JP2006293246A (en) | Information processing device, method, and program | |
| CN110089120B (en) | System and method for synchronized playback of media items on multiple remote devices | |
| US20110231426A1 (en) | Song transition metadata | |
| US9502017B1 (en) | Automatic audio remixing with repetition avoidance | |
| TWI727432B (en) | Singing scoring method and singing scoring system based on streaming media | |
| CN112581976B (en) | Singing scoring method and system based on streaming media | |
| CN110970059A (en) | Multimedia information playing method and device and readable storage medium | |
| JP2006300970A (en) | Information processor, method, and program | |
| Tabone et al. | Automated page turner for musicians | |
| KR101002732B1 (en) | Online digital contents management system | |
| WO2017107309A1 (en) | Control method, control device, terminal, and synchronous audio playback system | |
| US10368114B2 (en) | Media channel creation based on free-form media input seeds | |
| JP7261276B2 (en) | Electronic device and method of operation for synchronizing multimedia content and audio sources | |
| US11606606B1 (en) | Systems and methods for detecting and analyzing audio in a media presentation environment to determine whether to replay a portion of the media | |
| AU2019101257A4 (en) | Method for integrating different types of media | |
| KR102171479B1 (en) | Method and system for digital audio co-play service | |
| US10817562B2 (en) | Disregarding audio content |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |