CN112992106A - Music creation method, device, equipment and medium based on hand-drawn graph - Google Patents
Music creation method, device, equipment and medium based on hand-drawn graph Download PDFInfo
- Publication number
- CN112992106A CN112992106A CN202110307587.0A CN202110307587A CN112992106A CN 112992106 A CN112992106 A CN 112992106A CN 202110307587 A CN202110307587 A CN 202110307587A CN 112992106 A CN112992106 A CN 112992106A
- Authority
- CN
- China
- Prior art keywords
- note
- pitch
- string
- nodes
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 230000001419 dependent effect Effects 0.000 claims abstract description 8
- 230000006870 function Effects 0.000 claims description 71
- 239000000203 mixture Substances 0.000 claims description 24
- 238000004590 computer program Methods 0.000 claims description 14
- 238000003860 storage Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 7
- 230000015572 biosynthetic process Effects 0.000 claims description 5
- 239000006185 dispersion Substances 0.000 claims description 5
- 238000003786 synthesis reaction Methods 0.000 claims description 5
- 238000010586 diagram Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000010801 machine learning Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008450 motivation Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 230000001755 vocal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H1/00—Details of electrophonic musical instruments
- G10H1/0008—Associated control or indicating means
- G10H1/0025—Automatic or semi-automatic music composition, e.g. producing random music, applying rules from music theory or modifying a musical piece
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
- G10H2210/101—Music Composition or musical creation; Tools or processes therefor
- G10H2210/111—Automatic composing, i.e. using predefined musical rules
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Electrophonic Musical Instruments (AREA)
Abstract
The invention discloses a music creation method based on hand-drawn graphics, which comprises the following steps: obtaining a line drawn by a user; modeling the line to obtain a pitch variation function corresponding to the line, wherein the independent variable of the pitch variation function is time, and the dependent variable is initial pitch height; generating note nodes according to the initial pitch heights at the adjacent moments in the pitch change function; combining the note nodes according to a time sequence to obtain a note string corresponding to the line; generating a pitch height of the note string from an initial pitch height of each note node in the note string; performing tone matching processing on the note strings according to the tone heights of the note strings to obtain tone information corresponding to the note strings; and generating an audio file according to the note string, the pitch height of the note string and the tone information corresponding to the note string. The invention solves the problems of complex operation and poor user experience when music creation software in the prior art is used for creating music.
Description
Technical Field
The invention belongs to the technical field of information, and particularly relates to a music creation method, device, equipment and medium based on hand-drawn graphics.
Background
The existing music making software is mainly provided for professional musicians to use, strange patterns and relatively professional English abbreviations are paved on complex pages, and the existing music making software is not friendly to beginners or users without musician knowledge although the existing music making software is powerful in function. The motivation for many users to use music production software may be simply to experience the music production process, or it may be curiosity, which creates a situation where tools are chosen instead of people. Moreover, in the aspects of performance and storage, the existing music making software is too large in size, occupies a large amount of system disk space, and adopts a complex algorithm and poor performance. Sometimes even tens of thousands of notes need to be processed, the requirement on hardware equipment used by a user is high, and the use scenes of music production software are further limited.
Disclosure of Invention
The embodiment of the invention provides a music creation method, a device, equipment and a medium based on hand-drawn graphics, which aim to solve the problems of complex operation and poor performance in music creation in the prior art.
A music composition method based on hand-drawn graphics comprises the following steps:
obtaining a line drawn by a user;
modeling the line to obtain a pitch variation function corresponding to the line, wherein the independent variable of the pitch variation function is time, and the dependent variable is initial pitch height;
generating a note string corresponding to the line according to the initial pitch height of the pitch variation function, wherein the note string is composed of a plurality of note nodes;
generating a pitch height of the note string from an initial pitch height of each note node in the note string;
and generating an audio file according to the note strings and the tone heights of the note strings.
Optionally, the generating of the note string corresponding to the line according to the initial pitch height of the pitch change function includes:
generating note nodes according to the initial pitch heights at the adjacent moments in the pitch change function;
and combining the note nodes according to the time sequence to obtain a note string corresponding to the line.
Optionally, the generating note nodes according to the initial pitch heights at adjacent times in the pitch change function includes:
acquiring initial pitch heights of adjacent moments in the pitch change function, and calculating the difference between the initial pitch heights of the adjacent moments;
and comparing the difference of the initial pitch heights with a variation threshold of the pitch variation function, and determining whether to generate a note node according to a comparison result.
Optionally, the comparing the difference between the initial pitch heights with a variation threshold of the pitch variation function, and the determining whether to generate a note node according to the comparison result includes:
when the difference of the initial pitch heights of the adjacent moments is larger than the variation threshold, if the note node is generated, the note node is kept, and if the note node is not generated, the note node is generated;
and when the difference of the initial pitch heights at the adjacent moments is smaller than the variation threshold, closing the note node if the note node is generated, and not generating the note node if the note node is not generated.
Optionally, the generating the pitch height of the note string according to the initial pitch height of each note node in the note string comprises:
determining a pitch value for each note node in the string of notes according to MIDI protocols;
calculating association probability among the note nodes according to the pitch values of the note nodes, wherein the association probability comprises the probability of two same note nodes appearing continuously, the probability of two same note nodes appearing continuously in a preset interval and the probability of corresponding sum of the note nodes appearing in corresponding bar positions;
and calculating the pitch height of the note string according to the contribution degree of the association probability between the note nodes.
Optionally, the calculating the pitch height of the note string according to the contribution of the association probability between the note nodes comprises:
calculating the contribution degree of the association probability, and calculating the pitch height of the note string according to the contribution degree of the association probability, wherein the pitch height calculation formula of the note string is as follows:
F(N)＝w0sim(N)+w1seq1(N)+w2seq2(N)+w3harm(N)+w4ent(N)
in the above formula, sim (N) represents the sum of the similarity, seq, of the pitch value of the note node in the note string and the pitch change function1(N) contribution, seq, representing the probability of two identical note nodes appearing in succession2(N) contribution degree representing probability of two same note nodes appearing continuously in a preset interval, harm (N) contribution degree representing probability of corresponding note node and corresponding rotation appearing in corresponding bar position, ent (N) representing discrete degree, w0A weight, w, representing the sum of the pitch values of the nodes of notes in the note string and the similarity of the pitch variation function1Weight, w, representing the degree of contribution of the probability of two identical note nodes occurring in succession2Weight, w, representing the degree of contribution of the probability that two identical note nodes occur consecutively within a certain interval3Weights, w, representing the degree of contribution of a note node to and the probability of its rotation appearing at the corresponding bar position4A weight representing a degree of dispersion.
Optionally, the generating an audio file according to the note string and the pitch height of the note string comprises:
performing tone matching processing on the note strings according to the tone heights of the note strings to obtain tone information corresponding to the note strings;
and generating an audio file according to the note string, the pitch height of the note string and the tone information corresponding to the note string.
A hand-drawn graphics based music composition apparatus comprising:
the input module is used for acquiring lines drawn by a user;
the graph modeling module is used for modeling the line to obtain a pitch variation function corresponding to the line, wherein the independent variable of the pitch variation function is time, and the dependent variable is initial pitch height;
the note string generating module is used for generating a note string corresponding to the line according to the initial pitch height of the pitch change function, and the note string is composed of a plurality of note nodes;
the pitch height calculation module is used for generating the pitch height of the note string according to the initial pitch height of each note node in the note string;
and the audio synthesis module is used for generating an audio file according to the note string and the pitch height of the note string.
A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the hand-drawn graph-based music composition method when executing the computer program.
A computer-readable storage medium, in which a computer program is stored, and the computer program is executed by a processor to implement the method for creating music based on hand-drawn graphics.
According to the embodiment of the invention, the lines drawn by the user are obtained, the lines are converted into the note strings, the tone heights corresponding to the note strings are generated, and the audio files are generated according to the note strings and the tone heights corresponding to the note strings, so that the problem of complex operation in music creation in the prior art is solved, the performance of music creation software is optimized, the difficulty in music creation is reduced, and the user experience is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
FIG. 1 is a flowchart of a music composition method based on hand-drawn graphics according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating step S3 of the method for creating music based on hand-drawn graphics according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating step S31 of the method for creating music based on hand-drawn graphics according to an embodiment of the present invention;
FIG. 4 is a flowchart illustrating step S4 of the method for creating music based on hand-drawn graphics according to an embodiment of the present invention;
FIG. 5 is a flowchart illustrating step S5 of the method for creating music based on hand-drawn graphics according to an embodiment of the present invention;
FIG. 6 is a schematic block diagram of a music creation apparatus based on hand-drawn graphics according to an embodiment of the present invention;
FIG. 7 is a schematic block diagram of a note string generating module of the hand-drawn graph-based music composition apparatus according to an embodiment of the present invention;
FIG. 8 is a schematic block diagram of a pitch height calculation module of the hand-drawn music composition apparatus according to an embodiment of the present invention;
FIG. 9 is a schematic block diagram of an audio synthesis module of the hand-drawn graphics based music composition apparatus according to an embodiment of the present invention;
FIG. 10 is a diagram of a computer device according to an embodiment of the invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment provides a music creation method based on hand-drawn graphics. The following will describe in detail the music composition method based on hand-drawn graphics provided in this embodiment, as shown in fig. 1, the music composition method based on hand-drawn graphics includes:
in step S1, a user-drawn line is acquired.
Embodiments of the present invention herein provide for obtaining user drawn lines from a conventional input device, a touch input device, or other type of input device. For example, a user may draw a line through a conventional input device such as a mouse or a keyboard, may draw a line through a touch screen of a smart phone or a tablet computer, and may draw a line on a supporting input panel using an electronic pen. The user can draw at will when drawing the lines, and does not need to have professional vocal music knowledge.
In step S2, the line is modeled to obtain a pitch variation function corresponding to the line, where the independent variable of the pitch variation function is time, and the dependent variable is initial pitch height.
In this embodiment, after obtaining a line drawn by a user, analyzing the line, modeling the line to obtain a pitch change function { y (t)) } of the line drawn by the user, specifying that an x axis in the pitch change function { y (t)) } represents a time axis, and a y axis represents a pitch height, so that a value of y (t)) represents an initial pitch height corresponding to a point at a time t, and the function { y (t)) } represents a curve function in which the initial pitch height changes with time.
After the line drawn by the user is modeled to obtain the pitch change function { y (t) }, the attribute of music needs to be given to the function { y (t)) }, and in this embodiment, the function { y (t)) } needs to be converted into a note string composed of note nodes one by one.
In step S3, a note string corresponding to the line is generated according to the initial pitch height of the pitch variation function, and the note string is composed of several note nodes.
In the step, after obtaining the lines drawn by the user, the lines drawn by the user are firstly converted into note nodes one by one, and then the pitch height of each note node is determined, and the step aims to endow the irregular lines drawn by the user with basic attributes of music.
Optionally, as a preferred example of the present invention, as shown in fig. 2, the step S3 of generating the note string corresponding to the line according to the initial pitch height of the pitch variation function includes:
in step S31, a note node is generated from the initial pitch heights at adjacent times in the pitch change function.
In this embodiment, the fluctuation condition of the pitch change function { y (t) } is obtained by analyzing the function value at the adjacent time of the pitch change function { y (t) }, and each note node in the pitch change function { y (t) } is determined by using a note switching algorithm according to the fluctuation condition of { y (t) }.
Optionally, as a preferred example of the present invention, as shown in fig. 3, step S31 further includes:
in step S311, initial pitch heights at adjacent time points in the pitch change function are obtained, and a difference between the initial pitch heights at the adjacent time points is calculated.
The present embodiment obtains the fluctuation condition of the pitch variation function by calculating the difference between the function values at the adjacent time instants, where the function value of the pitch variation function is the initial pitch height.
In step S312, the difference between the initial pitch heights is compared with a variation threshold of the pitch variation function, and whether to generate a note node is determined according to the comparison result.
In this embodiment, a set R 'is used to record the comparison result between the function value difference of adjacent moments of the pitch variation function and the preset variation threshold, where the set R' is obtained by the following formula:
r ' is a set containing a plurality of elements, delta is a variation threshold of the pitch variation function, and the values of the elements in R ' are only two cases, namely 1 or 0, as can be seen from the expression of the set R '. In this embodiment, we stipulate that when the value of the element in R' is 1, it represents the start of the note node, and when the value is 0, it represents the end of the note node.
Further, in this step, when the difference between the initial pitch heights is greater than the variation threshold, the note node is maintained if the note node is generated, and the note node is generated if the note node is not generated.
In this embodiment, when the difference between the function values is greater than the variation threshold, and at this time, when the element value corresponding to R 'is 1, it indicates that the note node is turned on, and if the element value in the adjacent R' before and this time is 1, it indicates that the note node has been turned on before, and at this time, it only needs to keep the note node in the on state. If the element value in the adjacent R' before and this time is 0, it indicates that no note node is turned on before, and at this time, a note node needs to be generated and taken as the node where the note node is turned on.
Further, in this step, when the difference between the initial pitch heights is smaller than the variation threshold, the note node is turned off if the note node is generated, and the note node is not generated if the note node is not generated.
In this embodiment, when the difference between the function values is smaller than the variation threshold, and at this time, when the element value corresponding to R 'is 0, it represents that the note node is ended, and if the element value in the adjacent R' before and this time is 1, it indicates that the note node has been turned on before, and at this time, it is necessary to end the note node. If the element value in the adjacent R' before and this time is 0, the note node is not turned on before, and the state that no note is turned on only needs to be kept at the moment.
In step S32, the note nodes are combined in time sequence to obtain a note string corresponding to the line.
In this embodiment, after traversing the pitch variation function, a group of note strings consisting of several note nodes is obtained, where M ═ M (M ═ M)0...mL-1) To represent a string of musical notes, where miThe ith note node in the note string is represented, and L represents the number of note nodes in the note string M.
Further, after the corresponding note string is generated according to the pitch change function, the corresponding pitch height needs to be determined for the note string.
In step S4, the pitch altitudes of the note string are generated according to the initial pitch altitude of each note node in the note string.
In this embodiment, the pitch heights of the final note strings are obtained by matching the normalized pitch heights to the note string M and correcting the pitch heights. The standardized pitch height refers to a pitch height value conforming to a Musical Instrument Digital Interface (MIDI) protocol. For example, the MIDI protocols specify standard notes C, C #, D #, E, F #, G #, a #, and B with pitch values of 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, and 71, respectively.
Optionally, as a preferred example of the present invention, as shown in fig. 4, step S4 further includes:
in step S41, the pitch value of each note node in the note string is determined according to the MIDI protocol.
In this embodiment, the pitch value of the standard note closest to the initial pitch height of each note node is taken as the pitch value of the note node.
The specific method comprises the following steps: provided that N is (N)0...nL-1) Is the string of notes M ═ M0...mL-1) Corresponding set of pitch values, niIs the ith note node M in the note string MiCorresponding pitch value, niHas a value of and the function value y (t)i) Pitch value, y (t), corresponding to the closest standard notei) Is shown at note node miMoment of opening tiA function value of the pitch change function of (1). For example, note node m1Corresponding to the turn-on time of (f) of1) When the value of (3) is 66.8, t is set1Note node m that is turned on at any moment1Corresponding number n1Determine the pitch value 67 of the standard note G closest to 66.8 if note node m2Corresponding to the turn-on time of (f) of2) When the value of (A) is just 66.5, then the random slave ANDSelecting one of the pitch values of the standard notes F # and G closest to 66.5 as the t2Note node m that is turned on at any moment2Corresponding pitch value n2. By analogy, the note string M ═ M (M) can be determined finally0...mL-1) Corresponding set of pitch values N ═ N (N)0...nL-1)。
In step S42, the association probability between the note nodes is calculated according to the pitch values of the note nodes, wherein the association probability includes the probability of two identical note nodes appearing consecutively, the probability of two identical note nodes appearing consecutively within a preset interval, and the probability of the sum of the note nodes appearing at the corresponding bar position.
According to a large amount of previous statistical results, the fact that a certain correlation exists between different note nodes of a note string generated according to a curve drawn by a user is similar to the correlation between characters in a language, and therefore, the correlation relationship between the note nodes can be analyzed to provide assistance for determining the final pitch height of the note string.
In this embodiment, a classic language model in machine learning is used to train and simulate the association relationship between the note nodes, and machine learning can obtain: probability of successive occurrence of two nodes of the same note, P (n)i|ni-1) To represent; probability of two same note nodes appearing continuously in a preset interval, P (n)i-ni-1|ni-1-ni-2) To represent; probability that the sum corresponding to the note node appears at the corresponding bar position, P (n)i|ci,bi) Is shown in (c) isiIs shown at tiChord name of time, biIs shown at tiThe bar position of the moment.
In step S43, the pitch height of the note string is calculated from the contribution of the association probability between the note nodes.
In this embodiment, the pitch height of the note string is calculated by the following formula:
F(N)＝w0sim(N)+w1seq1(N)+w2seq2(N)+w3harm(N)+w4ent(N)
in the above-mentioned formula, the compound of formula,
sim (N) represents the sum of the similarity of the pitch value of the note node in the note string and the pitch change function,
seq1(N) a contribution degree representing a probability of two same note nodes occurring consecutively,
seq2(N) a contribution degree representing a probability that two same note nodes consecutively appear within a preset interval,
harm (n) represents the degree of contribution of the tone node to and the probability of the rotation occurring at the corresponding bar position,
ent (N) represents the degree of dispersion,
ent(N)＝-(H(N)-Hmean-ε)2
wherein, w0A weight, w, representing the sum of the pitch values of the nodes of notes in the note string and the similarity of the pitch variation function1Weight, w, representing the degree of contribution of the probability of two identical note nodes occurring in succession2Weight, w, representing the degree of contribution of the probability that two identical note nodes occur consecutively within a certain interval3Weights, w, representing the degree of contribution of a note node to and the probability of its rotation appearing at the corresponding bar position4Weights representing degrees of dispersionHeavy, can be adjusted by adjusting w0,w1,w2,w3,w4The emphasis points of F (N) are adjusted to realize the correction of the calculation result. H (N) is an entropy value based on N, HmeanThe average entropy value of the melody is represented, and epsilon is an adjusting parameter, and the dispersion degree can be adjusted by adjusting epsilon.
The embodiment converts lines into note strings through graphic modeling, obtains the association relation of note nodes through a language model based on machine learning, and calculates the pitch height of the note strings according to the algorithm of the embodiment, so that the lines are endowed with basic music attributes. In the process, the user only needs to draw lines without any other operation, so that the user threshold is reduced, and the user experience is improved.
In step S5, an audio file is generated from the note string and the pitch height of the note string.
The present embodiment also needs to solve the problem of how to convert a string of notes into audio that can be used for playing.
Optionally, as a preferred example of the present invention, as shown in fig. 5, step S5 further includes:
in step S51, a tone matching process is performed on the note string according to the pitch height of the note string, and tone information corresponding to the note string is obtained.
The tone is the sound heard by people, and different sound producing bodies produce different sounds due to different materials and structures. Note nodes of the same pitch height match different timbres to emit different sounds, for example, the sounds emitted by pianos, violins and people are different, and the sounds emitted by everyone are different. Thus, tone color can be understood as a characteristic of sound. In this embodiment, the SoundFont is used to match the corresponding tone to the note string according to the pitch height of the note string. The sound library of SoundFont stores various timbres conforming to the MIDI standard, and the pitch information of each note node in the note string can be matched to the corresponding timbres in the sound library of SoundFont.
In order to further improve the user experience, the embodiment may further provide a plurality of tone libraries for the user to select.
In step S52, an audio file is generated based on the note string, the pitch height of the note string, and the tone color information corresponding to the note string.
In this embodiment, the note strings with matched timbres are arranged in time sequence, and a final audio file is formed through simple format conversion, so that the user can listen to the audio in real time.
The music creation method based on the hand-drawn graph in the embodiment abandons complex, redundant and fussy operation processes, and enables a user to create at will. The method can simulate corresponding notes and tones according to the lines drawn by the user and match the tones with the notes, and the time consumption of the process is less than 0.5s, so that the real-time drawing and real-time playing smooth experience can be brought to the user. The algorithm model provides powerful guarantee for the success of graphic analysis and audio conversion. The method can be operated on different platforms, and can optimize the performance of music creation software through a webpage, a client, a small program and the like. In the embodiment, the music creation method based on the hand-drawn graph enables more users who do not know music to join in the line of creating music, and breaks through the situation of selecting people by tools.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
In one embodiment, a music composition apparatus based on hand-drawn graphics is provided, and the music composition apparatus corresponds to the music composition method based on hand-drawn graphics in the above embodiments one to one. As shown in fig. 6, the music composition apparatus includes an input module 10, a graphic modeling module 20, a note string generating module 30, a pitch height calculating module 40, and an audio synthesizing module 50. The functional modules are explained in detail as follows:
the input module 10 is used for acquiring lines drawn by a user;
the graph modeling module 20 is configured to model the line to obtain a pitch variation function corresponding to the line, where an independent variable of the pitch variation function is time, and a dependent variable is an initial pitch height;
a note string generating module 30, configured to generate a note string corresponding to the line according to the initial pitch height of the pitch variation function, where the note string is composed of a plurality of note nodes;
a pitch height calculation module 40 for generating a pitch height of the note string according to an initial pitch height of each note node in the note string;
and an audio synthesis module 50 for generating an audio file according to the note string and the pitch height of the note string.
Optionally, as shown in fig. 7, the note string generating module 30 includes:
a note node generating unit 301, configured to generate a note node according to the initial pitch heights at adjacent times in the pitch change function;
a note string generating unit 302, configured to combine the note nodes in a time sequence to obtain a note string corresponding to the line.
Optionally, the note node generating unit 301 further includes:
a function value comparison subunit, configured to obtain initial pitch heights of adjacent times in the pitch change function, calculate a difference between the initial pitch heights of the adjacent times, and then compare the difference between the initial pitch heights with a variation threshold of the pitch change function;
a note node switching subunit configured to, when the difference in the initial pitch heights is greater than the variation threshold, maintain the note node if the note node is generated, and generate the note node if the note node is not generated; and when the difference of the initial pitch heights is smaller than the variation threshold value, closing the note node if the note node is generated, and not generating the note node if the note node is not generated.
Alternatively, as shown in fig. 8, the pitch height calculating module 40 includes:
a pitch value determining unit 401 for determining a pitch value of each note node in the note string according to the MIDI protocol;
a note node relation calculating unit 402, configured to calculate association probabilities between note nodes according to the pitch values of the note nodes, where the association probabilities include probabilities that two same note nodes appear continuously, and probabilities that two same note nodes and corresponding note nodes appear at corresponding bar positions continuously in a preset interval;
a pitch height calculating unit 403, configured to calculate a pitch height of the note string according to a contribution of the association probability between the note nodes, where the pitch height calculating formula of the note string is:
F(N)＝w0sim(N)+w1seq1(N)+w2seq2(N)+w3harm(N)+w4ent(N)
in the above formula, sim (N) represents the sum of the similarity, seq, of the pitch value of the note node in the note string and the pitch change function1(N) contribution, seq, representing the probability of two identical note nodes appearing in succession2(N) contribution degree representing probability of two same note nodes appearing continuously in a preset interval, harm (N) contribution degree representing probability of corresponding note node and corresponding rotation appearing in corresponding bar position, ent (N) representing discrete degree, w0A weight, w, representing the sum of the pitch values of the nodes of notes in the note string and the similarity of the pitch variation function1Weight, w, representing the degree of contribution of the probability of two identical note nodes occurring in succession2Weight, w, representing the degree of contribution of the probability that two identical note nodes occur consecutively within a certain interval3Weights, w, representing the degree of contribution of a note node to and the probability of its rotation appearing at the corresponding bar position4A weight representing a degree of dispersion.
Alternatively, as shown in fig. 9, the audio synthesis module 50 includes:
the tone matching unit is used for performing tone matching processing on the note strings according to the tone heights of the note strings to obtain tone information corresponding to the note strings;
and the audio generation unit is used for generating an audio file according to the note string, the pitch height of the note string and the tone information corresponding to the note string.
In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 10. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of music composition based on hand-drawn graphics.
In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program:
obtaining a line drawn by a user;
modeling the line to obtain a pitch variation function corresponding to the line, wherein the independent variable of the pitch variation function is time, and the dependent variable is initial pitch height;
generating a note string corresponding to the line according to the initial pitch height of the pitch variation function, wherein the note string is composed of a plurality of note nodes;
generating a pitch height of the note string from an initial pitch height of each note node in the note string;
and generating an audio file according to the note strings and the tone heights of the note strings.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, databases, or other media used in embodiments provided herein may include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.
Claims (10)
1. A music composition method based on hand-drawn graphics is characterized by comprising the following steps:
obtaining a line drawn by a user;
modeling the line to obtain a pitch variation function corresponding to the line, wherein the independent variable of the pitch variation function is time, and the dependent variable is initial pitch height;
generating a note string corresponding to the line according to the initial pitch height of the pitch variation function, wherein the note string is composed of a plurality of note nodes;
generating a pitch height of the note string from an initial pitch height of each note node in the note string;
and generating an audio file according to the note strings and the tone heights of the note strings.
2. The hand-drawn graphic-based music composition method of claim 1, wherein the generating the string of notes corresponding to the line according to the initial pitch height of the pitch change function comprises:
generating note nodes according to the initial pitch heights at the adjacent moments in the pitch change function;
and combining the note nodes according to the time sequence to obtain a note string corresponding to the line.
3. The hand-drawn graphic-based music composition method of claim 2, wherein the generating note nodes according to the initial pitch heights at adjacent times in the pitch change function comprises:
acquiring initial pitch heights of adjacent moments in the pitch change function, and calculating the difference between the initial pitch heights of the adjacent moments;
and comparing the difference of the initial pitch heights with a variation threshold of the pitch variation function, and determining whether to generate a note node according to a comparison result.
4. The hand-drawn graphic-based music composition method of claim 3, wherein the comparing the difference between the initial pitch heights with the variation threshold of the pitch variation function, and the determining whether to generate a note node according to the comparison result comprises:
when the difference of the initial pitch heights of the adjacent moments is larger than the variation threshold, if the note node is generated, the note node is kept, and if the note node is not generated, the note node is generated;
and when the difference of the initial pitch heights at the adjacent moments is smaller than the variation threshold, closing the note node if the note node is generated, and not generating the note node if the note node is not generated.
5. The hand-drawn graphic-based music composition method of claim 1, wherein said generating the pitch altitudes of the note strings according to the initial pitch altitudes of each of the note nodes in the note strings comprises:
determining a pitch value for each note node in the string of notes according to MIDI protocols;
calculating association probability among the note nodes according to the pitch values of the note nodes, wherein the association probability comprises the probability of two same note nodes appearing continuously, the probability of two same note nodes appearing continuously in a preset interval and the probability of corresponding sum of the note nodes appearing in corresponding bar positions;
and calculating the pitch height of the note string according to the contribution degree of the association probability between the note nodes.
6. The hand-drawn graphic-based music composition method of claim 5, wherein said calculating the pitch height of the note string according to the contribution of the association probability between the note nodes comprises:
calculating the contribution degree of the association probability, and calculating the pitch height of the note string according to the contribution degree of the association probability, wherein the pitch height calculation formula of the note string is as follows:
F(N)＝w0sim(N)+w1seq1(N)+w2seq2(N)+w3harm(N)+w4ent(N)
in the above formula, sim (N) represents the sum of the similarity, seq, of the pitch value of the note node in the note string and the pitch change function1(N) contribution, seq, representing the probability of two identical note nodes appearing in succession2(N) contribution degree representing probability of two same note nodes appearing continuously in a preset interval, harm (N) contribution degree representing probability of corresponding note node and corresponding rotation appearing in corresponding bar position, ent (N) representing discrete degree, w0A weight, w, representing the sum of the pitch values of the nodes of notes in the note string and the similarity of the pitch variation function1Weight, w, representing the degree of contribution of the probability of two identical note nodes occurring in succession2Weight, w, representing the degree of contribution of the probability that two identical note nodes occur consecutively within a certain interval3Weights, w, representing the degree of contribution of a note node to and the probability of its rotation appearing at the corresponding bar position4A weight representing a degree of dispersion.
7. The hand-drawn graphic-based music composition method of claim 1, wherein the generating an audio file according to the string of musical notes and the pitch heights of the string of musical notes comprises:
performing tone matching processing on the note strings according to the tone heights of the note strings to obtain tone information corresponding to the note strings;
and generating an audio file according to the note string, the pitch height of the note string and the tone information corresponding to the note string.
8. A hand-drawn graphic-based music composition apparatus, the apparatus comprising:
the input module is used for acquiring lines drawn by a user;
the graph modeling module is used for modeling the line to obtain a pitch variation function corresponding to the line, wherein the independent variable of the pitch variation function is time, and the dependent variable is initial pitch height;
the note string generating module is used for generating a note string corresponding to the line according to the initial pitch height of the pitch change function, and the note string is composed of a plurality of note nodes;
the pitch height calculation module is used for generating the pitch height of the note string according to the initial pitch height of each note node in the note string;
and the audio synthesis module is used for generating an audio file according to the note string and the pitch height of the note string.
9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor when executing the computer program implements a method of hand-drawn graphics based music composition according to any one of claims 1 to 6.
10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the hand-drawn graphic-based music composition method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110307587.0A CN112992106B (en) | 2021-03-23 | 2021-03-23 | Music creation method, device, equipment and medium based on hand-drawn graph |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110307587.0A CN112992106B (en) | 2021-03-23 | 2021-03-23 | Music creation method, device, equipment and medium based on hand-drawn graph |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112992106A true CN112992106A (en) | 2021-06-18 |
| CN112992106B CN112992106B (en) | 2024-06-25 |
Family
ID=76333056
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110307587.0A Active CN112992106B (en) | 2021-03-23 | 2021-03-23 | Music creation method, device, equipment and medium based on hand-drawn graph |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112992106B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002311951A (en) * | 2001-04-12 | 2002-10-25 | Yamaha Corp | Device and program for automatic music composition |
| US20110004476A1 (en) * | 2009-07-02 | 2011-01-06 | Yamaha Corporation | Apparatus and Method for Creating Singing Synthesizing Database, and Pitch Curve Generation Apparatus and Method |
| JP2011158516A (en) * | 2010-01-29 | 2011-08-18 | Ryukoku Univ | Automatic music arrangement system and automatic music arrangement method |
| CN102289778A (en) * | 2011-05-10 | 2011-12-21 | 南京大学 | Method for converting image into music |
| CN106098058A (en) * | 2016-06-23 | 2016-11-09 | 腾讯科技（深圳）有限公司 | Tone line generates method and apparatus |
| CN110162659A (en) * | 2019-05-27 | 2019-08-23 | 湖南工程学院 | It is a kind of that rhythm newly bent generation method is determined based on pitch network |
-
2021
- 2021-03-23 CN CN202110307587.0A patent/CN112992106B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002311951A (en) * | 2001-04-12 | 2002-10-25 | Yamaha Corp | Device and program for automatic music composition |
| US20110004476A1 (en) * | 2009-07-02 | 2011-01-06 | Yamaha Corporation | Apparatus and Method for Creating Singing Synthesizing Database, and Pitch Curve Generation Apparatus and Method |
| JP2011158516A (en) * | 2010-01-29 | 2011-08-18 | Ryukoku Univ | Automatic music arrangement system and automatic music arrangement method |
| CN102289778A (en) * | 2011-05-10 | 2011-12-21 | 南京大学 | Method for converting image into music |
| CN106098058A (en) * | 2016-06-23 | 2016-11-09 | 腾讯科技（深圳）有限公司 | Tone line generates method and apparatus |
| CN110162659A (en) * | 2019-05-27 | 2019-08-23 | 湖南工程学院 | It is a kind of that rhythm newly bent generation method is determined based on pitch network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112992106B (en) | 2024-06-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11562722B2 (en) | Cognitive music engine using unsupervised learning | |
| US5617486A (en) | Continuous reference adaptation in a pattern recognition system | |
| WO2020015153A1 (en) | Method and device for generating music for lyrics text, and computer-readable storage medium | |
| JP2017107228A (en) | Singing voice synthesis device and singing voice synthesis method | |
| JP2021056489A (en) | Voice synthesis model training method, device, electronics apparatus and storage media | |
| CN111613199B (en) | MIDI sequence generating device based on music theory and statistical rule | |
| Nakamura et al. | Statistical piano reduction controlling performance difficulty | |
| Manzelli et al. | An end to end model for automatic music generation: Combining deep raw and symbolic audio networks | |
| JP2020003535A (en) | Program, information processing method, electronic apparatus and learnt model | |
| US20220301528A1 (en) | Systems, Devices, and Methods for Harmonic Structure in Digital Representations of Music | |
| CN112992106B (en) | Music creation method, device, equipment and medium based on hand-drawn graph | |
| JP4945465B2 (en) | Voice information processing apparatus and method | |
| WO2019176954A1 (en) | Machine learning method, electronic apparatus, electronic musical instrument, model generator for part selection, and method of part determination | |
| CN115810341A (en) | Audio synthesis method, apparatus, device and medium | |
| WO2022244818A1 (en) | Sound generation method and sound generation device using machine-learning model | |
| Dhara et al. | Automatic note transcription system for Hindustani classical music | |
| CN113488007B (en) | Information processing method, information processing device, electronic equipment and storage medium | |
| US20230419932A1 (en) | Information processing device and control method thereof | |
| TWI836255B (en) | Method and apparatus in designing a personalized virtual singer using singing voice conversion | |
| WO2022244403A1 (en) | Musical score writing device, training device, musical score writing method and training method | |
| Arunnehru et al. | Chord Craft: Exploring Musical Frontiers with Machine Learning | |
| CN117672258A (en) | Fundamental frequency generation method, computer device, and computer-readable storage medium | |
| CN116486765A (en) | Singing voice generating method, computer device and storage medium | |
| Jiang | Generating Musical Compositions through a Data-Driven Approach along with Static Implementations of Theoretical Principles | |
| CN117746832A (en) | Singing voice synthesizing method, singing voice synthesizing device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |