CN112037738A - Music data processing method and device and computer storage medium - Google Patents
Music data processing method and device and computer storage medium Download PDFInfo
- Publication number
- CN112037738A CN112037738A CN202010906667.3A CN202010906667A CN112037738A CN 112037738 A CN112037738 A CN 112037738A CN 202010906667 A CN202010906667 A CN 202010906667A CN 112037738 A CN112037738 A CN 112037738A
- Authority
- CN
- China
- Prior art keywords
- track data
- music
- sound source
- target
- sound
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 14
- 238000012545 processing Methods 0.000 claims abstract description 104
- 230000000694 effects Effects 0.000 claims abstract description 71
- 238000000034 method Methods 0.000 claims abstract description 59
- 230000005236 sound signal Effects 0.000 claims abstract description 54
- 238000012986 modification Methods 0.000 claims description 31
- 230000004048 modification Effects 0.000 claims description 31
- 230000006870 function Effects 0.000 claims description 24
- 238000004590 computer program Methods 0.000 claims description 10
- 230000002194 synthesizing effect Effects 0.000 claims description 9
- 230000008859 change Effects 0.000 claims description 8
- 230000006978 adaptation Effects 0.000 claims description 4
- 230000017105 transposition Effects 0.000 description 29
- 230000008569 process Effects 0.000 description 25
- 238000010586 diagram Methods 0.000 description 8
- 238000004458 analytical method Methods 0.000 description 4
- 230000003321 amplification Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000003199 nucleic acid amplification method Methods 0.000 description 3
- 241001342895 Chorus Species 0.000 description 2
- 210000004556 brain Anatomy 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000004040 coloring Methods 0.000 description 2
- HAORKNGNJCEJBX-UHFFFAOYSA-N cyprodinil Chemical compound N=1C(C)=CC(C2CC2)=NC=1NC1=CC=CC=C1 HAORKNGNJCEJBX-UHFFFAOYSA-N 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 210000004958 brain cell Anatomy 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 230000036039 immunity Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007830 nerve conduction Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000033764 rhythmic process Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 210000003454 tympanic membrane Anatomy 0.000 description 1
- 230000001755 vocal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H1/00—Details of electrophonic musical instruments
- G10H1/0008—Associated control or indicating means
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H1/00—Details of electrophonic musical instruments
- G10H1/0091—Means for obtaining special acoustic effects
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H1/00—Details of electrophonic musical instruments
- G10H1/02—Means for controlling the tone frequencies, e.g. attack or decay; Means for producing special musical effects, e.g. vibratos or glissandos
- G10H1/04—Means for controlling the tone frequencies, e.g. attack or decay; Means for producing special musical effects, e.g. vibratos or glissandos by additional modulation
- G10H1/043—Continuous modulation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/305—Electronic adaptation of stereophonic audio signals to reverberation of the listening space
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/307—Frequency adjustment, e.g. tone control
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
- G10H2210/155—Musical effects
- G10H2210/265—Acoustic effect simulation, i.e. volume, spatial, resonance or reverberation effects added to a musical sound, usually by appropriate filtering or delays
- G10H2210/281—Reverberation or echo
- G10H2210/291—Reverberator using both direct, i.e. dry, and indirect, i.e. wet, signals or waveforms, indirect signals having sustained one or more virtual reflections
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
- G10H2210/155—Musical effects
- G10H2210/265—Acoustic effect simulation, i.e. volume, spatial, resonance or reverberation effects added to a musical sound, usually by appropriate filtering or delays
- G10H2210/295—Spatial effects, musical uses of multiple audio channels, e.g. stereo
- G10H2210/305—Source positioning in a soundscape, e.g. instrument positioning on a virtual soundstage, stereo panning or related delay or reverberation changes; Changing the stereo width of a musical source
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
- H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Electrophonic Musical Instruments (AREA)
Abstract
The embodiment of the invention discloses a music data processing method, a device and a computer storage medium, wherein the method comprises the following steps: acquiring a sound signal of music to be processed, and performing track splitting processing on the sound signal to obtain N pieces of audio track data, wherein different audio track data are used for representing different sound source objects; setting a sound field position of a sound source object in a virtual space based on the sound source object; and performing first orientation modulation on the audio track data corresponding to the sound source object according to the sound field position of the sound source object in the virtual space to obtain target music with the performance effect of a band in the real space. The invention obtains a plurality of music track data based on the track splitting processing of the sound signals of the music, and obtains the target music with the playing effect of the band in the real space by modulating the plurality of music track data, thereby realizing the intelligent management of the music data.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for processing music data, and a computer storage medium.
Background
In the fast-paced information age, music becomes an important tool for public sentiment. The music can relax the body and relieve the stress; the music can knock off the closed soul and relieve the mind of depression and stuffiness; music can stimulate the brain and activate brain cells; the music can improve creativity, penguing power and stimulate the right brain; the music can help sleep, improve immunity, increase nerve conduction rate, and enhance memory and attention; the melody of the music can make the baby breathe calmly and the heartbeat slow down, so that the baby does not cry and cry any more. Music is a tool that is currently used more frequently. Therefore, some processing techniques for music have been developed.
At present, the processing mode of a piece of music is usually to separate background music or human voice, or to modulate the angle of stereo music to achieve the surround effect. Therefore, how to realize more effective processing of music becomes an urgent problem to be solved.
Disclosure of Invention
The embodiment of the invention provides a music data processing method, a device and a computer storage medium, which are used for obtaining a plurality of audio track data based on the split-track processing of the sound signals of music and obtaining target music with the playing effect of a band in a real space by modulating the audio track data, thereby realizing the intelligent management of the music data.
In a first aspect, the present invention discloses a music data processing method, including:
acquiring a sound signal of music to be processed;
performing track splitting processing on the sound signal to obtain N pieces of track data, wherein different track data are used for representing different sound source objects, and N is a positive integer greater than or equal to 2;
determining a sound field position of the sound source object in a virtual space;
and performing first orientation modulation on the track data corresponding to the sound source object according to the sound field position of the sound source object in the virtual space to obtain target music, wherein the target music has the music effect that a plurality of sound source objects perform at different space positions.
In a second aspect, an embodiment of the present invention discloses a device for processing music data, including:
the acquisition module is used for acquiring a sound signal of music to be processed;
the processing module is used for performing track splitting processing on the sound signal to obtain N pieces of audio track data, wherein different audio track data are used for representing different sound source objects, and N is a positive integer greater than or equal to 2;
the processing module is used for setting the sound field position of the sound source object in the virtual space;
the processing module is further configured to perform first orientation modulation on the track data corresponding to the sound source object according to the sound field position of the sound source object in the virtual space, so as to obtain target music, where the target music has a music effect that a plurality of sound source objects perform at different spatial positions.
In a third aspect, an embodiment of the present invention provides a terminal, including a processor, an input device, an output device, and a memory, where the processor, the input device, the output device, and the memory are connected to each other, where the memory is used to store a computer program, and the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method according to the first aspect.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored, the computer program comprising program instructions that, when executed by a processor, cause the processor to perform the method of the first aspect.
In the embodiment of the invention, a terminal acquires a sound signal of music to be processed, performs track splitting processing on the sound signal to obtain N pieces of music track data, wherein different music track data are used for representing different sound source objects, sets the sound field position of the sound source object in a virtual space based on the sound source object, and performs first azimuth modulation on the music track data corresponding to the sound source object according to the sound field position of the sound source object in the virtual space to obtain target music with a playing effect of a band in a real space.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flowchart illustrating a music data processing method according to an embodiment of the present invention;
fig. 2 is a schematic diagram illustrating an orientation effect of music data according to an embodiment of the present invention;
fig. 3 is a flowchart illustrating another music data processing method according to an embodiment of the present invention;
fig. 4 is a flowchart illustrating a music data processing method according to another embodiment of the present invention;
FIG. 5 is a diagram illustrating an apparatus for processing music data according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a terminal according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The music data processing method provided by the embodiment of the invention is realized on a terminal, the terminal comprises electronic equipment such as a smart phone, a tablet personal computer, a digital audio and video player, an electronic reader, a handheld game machine, a smart watch and the like, and the terminal can support interface display and can receive various operations of a user side, such as click operation, input operation or sliding operation and the like.
The music data processing method provided by the embodiment of the invention mainly comprises the following steps: the method comprises the steps that terminal equipment obtains sound signals of music to be processed, and performs track splitting processing on the sound signals of the music to be processed by means of machine learning to obtain N pieces of audio track data, wherein different audio track data correspond to different sound source objects, for example, sound sources corresponding to human voice, piano voice and the like are different; further, after N pieces of audio track data are obtained, the sound field positions of the sound source objects corresponding to different pieces of audio track data in the virtual space are determined, and then the audio track data corresponding to the sound source objects are subjected to first azimuth modulation according to the sound field positions of the sound source objects in the virtual space, so that target music with the playing effect of a band in the real space is obtained. The playing effect is embodied in that different music track data of target music are played at different spatial positions, for example, a singer stands in the middle of a stage, and a guitar player and a piano player are respectively positioned at two sides of the stage. Wherein N is an integer of 2 or more.
In the embodiment of the present invention, a method for processing music data is further provided, where the method mainly aims at the music track data of a target type in N pieces of music track data, that is, a terminal acquires music to be processed, acquires a sound signal of the music to be processed, performs track splitting processing on the sound signal to obtain N pieces of music track data, where different music track data correspond to different sound source objects, analyzes the N pieces of music track data, if it is determined according to an analysis result that the music track data of the target type exists in the N pieces of music track data, invokes a transposition tool to perform transposition processing on the music track data of the target type to obtain N pieces of transposed music track data, obtains processed music according to the N pieces of transposed music track data and other music track data of the N pieces of music track data except the music track data of the target type, and can only process the music track data of the target type, to obtain different effects of the target music. The target type of track data may refer to one or more of a human voice, a piano voice, or a guitar voice, among others. If the sound is the human voice, only the human voice is subjected to the tone modification processing, and other audio track data are not adjusted.
The specific flow of the music data processing method provided by the embodiment of the present invention can be specifically described by the following flowchart.
Referring to fig. 1, fig. 1 is a schematic flowchart of a music data processing method according to an embodiment of the present invention, which includes the following steps:
step S101, the terminal acquires a sound signal of music to be processed.
Specifically, the terminal obtains music to be processed, the music to be processed also refers to music files to be processed, the music files include music files which are uploaded to the server and authorized, and the user side can perform playing operation, downloading operation and the like on the music files through the terminal interface. After the terminal acquires the music file to be processed, the sound signal of the music file to be processed is acquired, and for example, the music file to be processed may be processed by a tool such as Python, MATLAB, or the like to obtain the sound signal.
Here, the sound signal refers to a sound wave signal whose frequency and amplitude are regularly changed. The three elements of sound are pitch, intensity and timbre. There are three important parameters of a sound wave or sine wave: frequency ω 0, amplitude An and phase ψ n, which determine the characteristics of the sound signal.
Step S102, the terminal performs track splitting processing on the sound signal to obtain N pieces of track data, wherein different track data are used for representing different sound source objects, and N is an integer greater than or equal to 2.
Wherein different track data correspond to different sound source objects, for example: the track 1 corresponds to track data of a human voice, and the sound source object of the track data is the human voice; the track 2 corresponds to the track data of piano sound, and the sound source object is a piano, and may include the track data of drumbeats, the track data of bass, the track data of other sounds, and the like. The number of tracks is customizable, and is not limited, but is at least 2. Note that, although one separation result is to separate the track data of the human voice and the track data of the background music, such track processing is relatively simple, and in order to achieve a more complicated music processing effect, the embodiment of the present application may separate more various track data.
One representation of the soundtrack data is in the form of parallel "tracks" of one of the sequencer software. Each track has its own attributes such as timbre, timbre library, number of channels, input/output ports, volume, etc.
In one possible implementation, the audio signal is subjected to a track splitting process to obtain N pieces of audio track data, specifically: inputting the sound signal into a trained sound signal processing machine model for separation to obtain N pieces of audio track data, wherein the setting of the number of audio tracks can be input by a user based on a terminal interface, the input mode comprises a key-in mode or a voice input mode, namely, the terminal interface comprises an icon control for setting the number of audio tracks, and the setting of the number of audio tracks is realized by performing corresponding key-in operation or voice input operation on the icon control.
Step S103, the terminal sets the sound field position of the sound source object in the virtual space.
In a possible implementation manner, the terminal sets a sound field position of the sound source object in the virtual space, specifically: and the terminal displays a target interface, wherein the target interface comprises an icon element used for representing a sound source object corresponding to the audio track data, and then the sound field position input by the user for the sound source object corresponding to the icon element through the target interface is set as the sound field position of the sound source object corresponding to the icon element in the virtual space.
In a possible implementation manner, the terminal determines the sound field position of the sound source object in the virtual space, specifically: and the terminal responds to an operation instruction of dragging the icon element of the sound source object from the first position to the second position of the target interface by the user, and determines the sound field position corresponding to the second position as the sound field position of the sound source object in the virtual space. For any sound source object, the user side can perform a dragging operation on the icon element corresponding to the sound source object to determine the sound field position, so as to achieve the directional effect diagram shown in fig. 2.
In another possible implementation manner, the icon elements of the sound source object corresponding to each piece of sound track data are displayed on the target interface, and when the icon element of one sound source object is clicked, angle information may be input for the sound track data corresponding to the sound source object, or the icon elements of all the sound source objects may be clicked respectively and then corresponding angles are input. The terminal responds to angle input operation of a user on the icon elements of the sound source object, the sound field position of the sound source object in the virtual space is determined according to the angle corresponding to the angle input operation, the icon elements of the sound source object corresponding to all the audio track data are displayed on the target display interface, and the icon elements can be clicked at the angle corresponding to all the audio track data for viewing. For example, assuming that there are five pieces of track data including human track data, piano track data, bass track data, drum track data, and other track data, setting angles of icons corresponding to sound source objects of the five pieces of track data in the target interface, for example, setting the human track data to be 5 degrees on the left, the piano track data to be 10 degrees on the left, the bass track data to be 30 degrees on the left, the drum track data to be 30 degrees on the right, and the other track data to be 10 degrees on the right, after receiving the settings, the terminal determines sound field positions of icon elements corresponding to the respective track data in the virtual space according to the angle information, and obtains an azimuth effect diagram of a music file as shown in fig. 2.
In the upper half of the azimuth effect diagram shown in fig. 2, each element corresponds to a sound source object, and the sound source objects are set according to the number of the track data, that is, the number of the track data is several, and there are several sound source objects.
And step S104, the terminal carries out first azimuth modulation on the music track data corresponding to the sound source object according to the sound field position of the sound source object in the virtual space so as to obtain target music, and the target music has the performance effect of a band in the real space.
Among them, the azimuth modulation of the sound source object is mainly realized by a Head-Related transfer function (HRTF). HRTF is used to describe the frequency response system where any point sound source in space arrives at the binaural tympanic membrane. The system is affected by factors such as pinna, head shape, shoulders, etc. The human ear mainly judges the sound source position according to the binaural cue provided by the system.
The specific application of the function is as follows: assuming that any one of the audio track data x (N) of the N audio track data is obtained by the track splitting processing, the HRTF functions of the target angle are respectively: HLeft (n) and HRight (n). HRTF functions of the American CICICICIIC database can be adopted, or other data of an open source database can be adopted, wherein the transfer functions of non-measurement points are generated by adopting the fitting of the distance between adjacent four points as weight. The method of orientation modulation is to convolute the audio track data x (n) with the HRTF function respectively, namely:
left channel signal:
right channel signal:
wherein, the symbol
Representing a convolution operation.
In a possible implementation manner, the terminal performs first orientation modulation on the audio track data corresponding to the sound source object according to the sound field position of the sound source object in the virtual space, and the combining with the angle function may specifically include: the terminal takes the angle information of the sound field position of the sound source object in the virtual space as the input of the angle function of the sound track data, and obtains the output of the angle function after the angle function processing, wherein the angle function corresponds to HLeft (n) and HRight (n). After the output of the angle function is obtained, convolution operation is carried out on the output of the angle function and track data (any track data in N track data, corresponding to x (N)), so that modulation data of first azimuth modulation of the track data corresponding to a sound source object is obtained.
Taking voice track data as an example, receiving an angle setting instruction for the voice track data on a target interface, determining the sound field position of the voice track data to obtain corresponding angle information, and performing angle convolution on the voice track data at each moment by using an HRTF function to realize the effect of voice rotation in the whole music file playing process, so that the voice is equivalent to a singer constantly changing positions in terms of a concert. Wherein, the surround rotation effect of the audio track data in the playing process can be realized by setting the angular speed.
The obtained target music with the performance effect of the band in the real space is different from the music to be processed, wherein the space sound field position of the audio track data is changed, and a plurality of audio track data form a plurality of left channel data and right channel data after being subjected to angle modulation of the HRTF function. All left channel data may be shuffled together, and similarly all right channel data may be shuffled together. The difference between the processed music signal and the original music signal is that the spatial sound field position of each instrument is changed according to the intention of the user.
In the embodiment of the invention, a terminal acquires a sound signal of music to be processed, performs track splitting processing on the sound signal to obtain N pieces of music track data, wherein different music track data are used for representing different sound source objects, sets the sound field position of the sound source object in a virtual space based on the sound source object, and performs first azimuth modulation on the music track data corresponding to the sound source object according to the sound field position of the sound source object in the virtual space to obtain target music with a band playing effect in a real space.
Referring to fig. 3, fig. 3 is a schematic flowchart of another music file processing method according to an embodiment of the present invention, where the method includes:
step S301, the terminal acquires the sound signal of the music to be processed.
The step is as described in step S101, and is not described in detail here.
Step S302, if the music to be processed is stereo music, the terminal converts the sound signal of the music to be processed into a monaural sound signal.
In one possible implementation manner, after acquiring the sound signal of the music to be processed, the terminal performs music type detection on the music to be processed to determine the type of the music to be processed, where the music type includes: stereo (two-channel) music types, mono music types. And if the music type is detected to be the stereo music type, converting the sound signal of the music to be processed into a sound signal of a single sound channel music type. A binaural musical composition includes a left channel (denoted as L) and a right channel (denoted as R). Methods of converting a sound signal of a stereo music type into a sound signal of a mono music type include various methods. For example, a mono music file can be obtained by a channel signal averaging method, i.e., (L + R)/2, or, for example, a stereo music type sound signal can be input into an FFmpeg (short for Fast Forward Mpeg) program, and the program combines the sound signals of the left and right channels into one channel from the perspective of the frequency domain. Where FFmpeg is a set of open source computer programs that can be used to process digital audio and video.
Step S303, the terminal performs a track splitting process on the audio signal to obtain N pieces of track data, where different track data are used to represent different sound source objects, and N is an integer greater than or equal to 2.
The step is as described in step S102, and is not described in detail here.
Step S304, the terminal sets the sound field position of the sound source object in the virtual space.
The step is as described in step S103, and is not described in detail here.
Step S305 performs first orientation modulation on the track data corresponding to the sound source object according to the sound field position of the sound source object in the virtual space.
The step is as described in step S104, and is not described in detail here.
And S306, if the N pieces of track data comprise the target type of track data, the terminal performs tonal modification processing on the target type of track data to obtain tonal modification track data, and synthesizes the tonal modification track data and the track data modulated in the first direction to obtain target music which is played by a band in a real space and has different tone ensemble effects.
In a possible implementation manner, after acquiring the N audio track data, the terminal analyzes the N audio track data, and if it is determined that the target type audio track data exists in the N audio track data according to the analysis result, a transposition tool is called to perform transposition processing on the target type audio track data to obtain transposition audio track data. The transposition track data is obtained by transposing the target type of track data, and the track data before transposition still exists. Wherein, the pitch-changing tool means that the pitch is not changed, namely, the fundamental frequency of the sound is changed, and simultaneously the speech speed and the semantics are kept unchanged. If the target type of track data includes track data of a human voice, the following processes are performed on the track data of the human voice: a process may adjust a sound signal in a first frequency range to a sound signal in a second frequency range, such as the effect of modulating male soundtrack data to female soundtrack data or modulating female soundtrack data to male soundtrack data; a process can subject a target type of soundtrack data to a plurality of different amplitude transposition processes to obtain soundtrack data in a plurality of different frequency ranges.
For example, several specific forms of the transposition audio track data are illustrated, such as calling a transposition tool to perform an up-conversion process on a male audio track data to obtain a corresponding female audio track data, or calling a transposition tool to perform an down-conversion process on a female audio track data to obtain a male audio track data. And synthesizing the tonal modification music track data and the music track data modulated in the first direction to obtain target music with a antiphonal singing effect for men and women, or obtain target music with a double-person chorusing effect, or obtain target music with a multi-person chorusing effect.
After the target type of track data is subjected to transposition processing to obtain transposition track data, the terminal synthesizes the transposition track data and the track data modulated in the first direction to obtain target music which is played by a band in a real space and has different tone ensemble effects.
In a possible implementation manner, the terminal determines target playing time periods corresponding to the target types of audio track data and the tonal modification audio track data according to the playing time of each audio track data, and by controlling the playing time, a stage music effect of sing by men and women (for example, "one sentence for boys and one sentence for girls") can be achieved, and a stage music effect of chorusing by multiple persons can also be achieved according to the control of the playing time periods.
The terminal can also perform second azimuth modulation on the target type audio track data and the tonal modification audio track data in the target playing time period, wherein the second azimuth modulation mainly aims at the target audio track data and the tonal modification audio track data, and music with different tone ensemble and changing effect of different tone relative positions can be realized after the second azimuth modulation is performed.
If the terminal synthesizes the audio track data obtained after the second azimuth modulation and the first azimuth modulation, the target music with the effects that a plurality of sound source objects perform at different spatial positions, different tones are integrated, and the relative positions of different tones change can be obtained. For example, the track data obtained by the first azimuth modulation may include other track data than the target type of track data, such as piano sound, guitar sound, and so on, in which case, the combination of the second azimuth modulation and the track data obtained by the first azimuth modulation may result in a performance effect of the band in real space: the musical instruments are positioned at different positions of the stage, and if the vocal track data is of a target type, the singing effect that different singers play in concert and the singers change at the positions of the stage can be realized, for example, the singing effect that male and female singers sing on the stage and exchange positions can be realized.
It should be noted that if the obtained tonal modification audio track data is greater than two, an effect of changing the positions of multiple sounds on a stage can be realized by the second azimuth modulation process, or a stage music effect of multiple persons singing together can be realized by controlling the playing time period. Note that a plurality of pieces of modified track data are mixed after being delayed for a certain time from one another on the time axis, thereby providing an effect of a chorus of a plurality of persons. Since it is difficult for the human ear to distinguish similar sounds within 10ms, the duration of the delay may be 10 ms.
Step S307, acquiring a pulse signal corresponding to the target scene from the database, and performing convolution processing on the pulse signal corresponding to the target scene and the pulse signal of the target music to obtain the target music with the performance effect in the target scene.
In one possible implementation, the target music may be obtained by first-position-modulating the track data corresponding to the sound source object, or may be obtained by synthesizing the transposed track data and the first-position-modulated track data. In both cases, the terminal may obtain the pulse signal corresponding to the target scene from the database. The database stores pulse signals of different scenes, the specific scenes include but are not limited to a stadium, a theater, a concert hall, a gymnasium, a library and the like, and a user can select the scenes through a terminal interface. And carrying out convolution processing on the pulse signal corresponding to the target scene and the pulse signal of the updated music file target music to obtain the target music with the target scene playing effect. For example, when a music file with a special effect in a concert hall scene is to be realized, the terminal directly obtains a pulse signal corresponding to the concert hall scene from the database, and then convolves the pulse signal with a pulse signal of target music to obtain the target music of the concert hall scene. According to the method, special effect music of different scenes can be simulated to meet different requirements of users. Optionally, the simulation of special effect music of different scenes can also be realized by using an artificial reverberator (the artificial reverberator is one of the most important devices for beautifying sound), which is mainly set by a user according to a terminal interface.
In a possible implementation manner, on a display interface of the terminal, a mute switch corresponding to each piece of audio track data is included, specifically, each piece of audio track data has a corresponding audio track data identifier, each audio track data identifier corresponds to one mute switch, and if the terminal receives an on instruction of a mute switch through the target interface, the on instruction of the mute switch of some piece of audio track data is performed, and the on instruction indicates the corresponding audio track data to be subjected to mute processing. And other audio track data which do not receive the opening instruction of the mute switch are normally played, so that the user can autonomously control the music file. The user can clearly know the role played by each piece of track data in a music file and the music effect obtained by the presence or absence of the piece of track data. For example, if a music file including a plurality of pieces of track data is muted for piano track data and other track data than the human voice track data, the musical effect of piano singing can be achieved. In addition, different styles of musical arrangement effects can be achieved through other combinations.
In a possible implementation manner, the terminal determines the respective energy adjustment multiple of each music track data, and adjusts the energy of each music track data of the target music according to the respective energy adjustment multiple of each music track data to obtain the target music with the adaptation style. Specifically, the target interface of the terminal may include energy proportion setting controls corresponding to all audio track data, and the user may set the energy proportion through the target interface, so as to achieve a music effect with a strong or weak rhythm. The muting effect can also be achieved by means of energy setting, i.e. setting the energy of the audio track data to be muted to 0%. The target music may be obtained by performing first-position modulation on track data corresponding to the sound source object, or may be obtained by synthesizing transposed track data and the track data after the first-position modulation, or may be obtained by performing convolution processing on an impulse signal corresponding to the target scene and an impulse signal of the target music.
In a possible implementation manner, the terminal displays a playing interface of the terminal, where the playing interface includes icon elements corresponding to each music track data of the target music, and during playing of the recomposed target music, the terminal may detect real-time energy of each music track data of the recomposed target music and determine the real-time energy. If the real-time energy of the audio track data is greater than the first energy value, performing prompt processing on an icon element of the audio track data corresponding to the real-time energy, wherein the prompt processing may include performing various processing such as coloring processing and amplification effect processing on the icon element, the color of the coloring processing may be set by a user, and the rendered color may indicate that the audio track data is in a playing or singing state; if the real-time energy of the audio track data is less than the second energy value, the icon element of the audio track data corresponding to the real-time energy is subjected to non-cue processing, the non-cue processing may include performing gray processing on the icon element and performing size-invariant or reduction processing on the icon element, and the icon element corresponding to the audio track data remains still, indicating that the audio track data is in an intermittent state at the moment. As shown in fig. 2, each audio track data corresponds to an icon element, and each audio track data has corresponding energy fluctuation during the playing process of the target music, and if there is energy fluctuation, the terminal interface displays the amplification effect of the icon element, and the amplification effect is more obvious when the energy is larger. Generally, in the process of playing target music, whether the instrument or the human voice is in a playing or singing state, so that the target music can be checked through real-time energy. By the method, the target music can achieve the effect of coordinating sound and pictures in the playing process, and a user can visually see which musical instruments are played in which specific time periods in the music playing process.
In the embodiment of the present invention, the terminal, in addition to performing the process shown in fig. 1 on the music file to be processed, may perform a tone-changing process on the target music to realize alternate playing of different tones or simultaneous playing of different tones, obtain special effect music in the target scene through a scene mixing process, and realize picture rendering during music playing through a picture rendering process. Through the embodiment, intelligent processing of various kinds of music can be realized to a great extent, so that diversified requirements of users for hearing music with different sound effects are met.
Referring to fig. 4, fig. 4 is a schematic flowchart illustrating a method for processing music data according to another embodiment of the present invention, the method includes the following steps:
step S401, the terminal obtains the sound signal of the music to be processed.
The step is as described in step S101, and is not described in detail here.
Step S402, the terminal performs track splitting processing on the sound signal to obtain N pieces of track data, wherein different track data are used for representing different sound source objects, and N is an integer greater than or equal to 2.
The step is as described in step S102, and is not described in detail here.
Step S403, if the N pieces of audio track data include audio track data of the target type, performing transposition processing on the audio track data of the target type to obtain transposed audio track data.
Specifically, the terminal analyzes the N audio track data, and if it is determined that the target type of audio track data exists in the N audio track data according to the analysis result, a transposition tool is called to perform transposition processing on the target type of audio track data to obtain transposition audio track data. The number of the transposition audio track data is n, and n is an integer greater than or equal to 2.
In a possible implementation manner, the terminal analyzes N pieces of track data, detects whether there is track data of a target type, and if there is track data of a corresponding target type, invokes a transposition tool to perform transposition processing on the track data of the target type to obtain N pieces of transposition track data, where N is an integer greater than or equal to 2, and if there is one track data, the transposition processing is meaningless. The n pieces of transposition track data may include female track data obtained by transposition processing of male track data, male track data obtained by transposition processing of female track data, or a plurality of track data obtained by transposition processing of arbitrary human track data. The target track data may be a human voice type track data or other voice type track data.
And S404, the terminal obtains target music according to the tonal modification music track data and other music track data except the music track data of the target type in the N music track data.
In one possible implementation, the tone of only the target type of track data is changed while keeping the sound characteristics of other track data than the target type of track data unchanged, thereby generating target music of a chorus of a male or a female or a group of persons. Specifically, the characteristics of other track data of the original music, including the positions of other track data, are not changed, and only the target type of track data is changed, such as the tone of one person to the sound of two persons or more persons.
In one possible implementation manner, the azimuth modulation processing or the playing time control processing may be performed on the N transposed track data, and the terminal generates the processed target music according to the N transposed track data obtained by the azimuth modulation processing or the playing time control processing and the track data other than the target type of track data in the N track data. Because the tonal modification music track data is modulated, the sound signal corresponding to the processed music file has music effects played alternately or at different angles in the playing process, and the experience presented to the client is the antiphonal singing effect of the human voice in the music or the surround effect of the human voice.
Note that, in this method, the angle modulation may be performed on the audio track data other than the target type of audio track data among the N pieces of audio track data, or the modulation may not be performed.
In the implementation of the invention, the terminal acquires the sound signal of the music to be processed, performs the split-track processing on the sound signal to obtain N pieces of audio track data, different audio track data are used for representing different sound source objects, analyzing the N pieces of audio track data, if the audio track data of the target type exists in the N pieces of audio track data according to the analysis result, then a tonal modification tool is called to perform tonal modification processing on the target type of audio track data to obtain n pieces of tonal modification audio track data, the processed target music is obtained according to the N pieces of tonal modification music track data and other music track data except the music track data of the target type in the N pieces of music track data, the invention mainly obtains a plurality of music track data through the track splitting processing of the music, and the tone modification processing is carried out on the target type of the audio track data to obtain target music with various auditory effects, thereby realizing the intelligent management of various music.
Referring to fig. 5, fig. 5 is a schematic diagram of a music data processing apparatus according to an embodiment of the present invention, where the apparatus 50 includes: an acquisition module 501 and a processing module 502. Wherein:
an obtaining module 501, configured to obtain a sound signal of music to be processed;
a processing module 502, configured to perform a track splitting process on the sound signal to obtain N pieces of track data, where different track data are used to represent different sound source objects, and N is an integer greater than or equal to 2;
the processing module 502 is configured to set a sound field position of the sound source object in a virtual space;
the processing module 502 is further configured to perform first azimuth modulation on the track data corresponding to the sound source object according to the sound field position of the sound source object in the virtual space, so as to obtain target music, where the target music has an effect of a band playing in a real space.
In a possible implementation manner, the processing module 502 is configured to perform a first orientation modulation on track data corresponding to the sound source object according to a sound field position of the sound source object in a virtual space, and includes:
taking the angle information of the sound field position of the sound source object in the virtual space as the input of the angle function of the sound track data corresponding to the sound source object to obtain the output result of the angle function;
and performing convolution operation on the output result of the angle function and the audio track data to realize first azimuth modulation on the audio track data corresponding to the sound source object.
In a possible implementation manner, the processing module 502 is configured to set a sound field position of the sound source object in a virtual space, and includes:
displaying a target interface including an icon element for representing a sound source object corresponding to the track data;
and setting the sound field position input by the user based on the icon element as the sound field position of the sound source object corresponding to the icon element in the virtual space.
In a possible implementation manner, the processing module 502 is configured to set a sound field position input by a user based on the icon element as a sound field position of a sound source object corresponding to the icon element in a virtual space, and includes:
responding to an operation that a user drags the icon element of the sound source object from the first position to the second position of the target interface, and determining the sound field position corresponding to the second position as the sound field position of the sound source object in the virtual space.
In a possible implementation manner, after performing the first orientation modulation on the track data corresponding to the sound source object according to the sound field position of the sound source object, the processing module 502 is further configured to:
if the N pieces of audio track data comprise audio track data of a target type, performing tonal modification processing on the audio track data of the target type to obtain tonal modification audio track data;
synthesizing the tonal modification music track data and the music track data modulated in the first direction to obtain target music with different tone ensemble effects and played by a band in a real space
In a possible implementation manner, the processing module 502 is configured to combine the transposition audio track data with the first azimuth-modulated audio track data, and specifically includes:
determining the target type of the audio track data and the target playing time interval corresponding to the tonal modification audio track data according to the playing time of the audio track data;
performing second azimuth modulation of relative change of sound field positions on the target type audio track data and the tonal modification audio track data in the target playing time period;
and synthesizing the second azimuth modulation and the track data after the first azimuth modulation to obtain target music with the effects of the band playing in real space, different tone ensemble and different tone relative position change.
In a possible implementation manner, the obtaining module 501 is further configured to obtain a pulse signal corresponding to a target scene from a database, where the database stores pulse signals corresponding to different scenes;
the processing module 502 is further configured to perform convolution processing on the pulse signal corresponding to the target scene and the pulse signal of the target music to obtain the target music with the performance effect in the target scene.
In a possible implementation manner, the processing module 502 is further configured to:
displaying the target interface, wherein the target interface comprises a mute switch corresponding to the audio track data;
and if an opening instruction of the mute switch is received through the target interface, carrying out mute processing on the audio track data corresponding to the opening instruction.
In one possible implementation, as shown in fig. 5, the apparatus may further include a determining module 503 and a detecting module 504.
A determining module 503, configured to determine an energy adjustment multiple of each of the audio track data;
the processing module 502 is further configured to adjust the energy of each of the music track data of the target music according to the respective energy adjustment multiple of each of the music track data, so as to obtain the target music with an adaptation style.
In a possible implementation manner, the processing module 502 is configured to display a playing interface, where the playing interface includes an icon element corresponding to each piece of music track data of the target music;
the detecting module 504 is configured to detect real-time energies of respective track data of the recomposition-style target music during the playing of the recomposition-style target music;
the processing module 502 is configured to:
if the real-time energy of the audio track data is larger than the first energy value, prompting the icon elements of the audio track data corresponding to the real-time energy;
and if the real-time energy of the audio track data is less than the second energy value, carrying out non-prompt processing on the icon element of the audio track data corresponding to the real-time energy.
In the embodiment of the invention, a terminal acquires a sound signal of music to be processed, performs track splitting processing on the sound signal to obtain N pieces of music track data, wherein different music track data are used for representing different sound source objects, sets the sound field position of the sound source object in a virtual space based on the sound source object, and performs first azimuth modulation on the music track data corresponding to the sound source object according to the sound field position of the sound source object in the virtual space to obtain target music with a playing effect of a band in a real space.
Referring to fig. 6, fig. 6 is a schematic structural diagram of a terminal according to an embodiment of the present invention, and as shown in fig. 6, the terminal device includes: at least one processor 601, input devices 603, output devices 604, memory 605, at least one communication bus 602. Wherein a communication bus 602 is used to enable the connection communication between these components. The input device 603 may be a control panel, a microphone, or the like, and the output device 604 may be a display screen, or the like. The memory 605 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 605 may optionally be at least one storage device located remotely from the processor 601. Wherein the processor 601 may be combined with the apparatus described in fig. 5, the memory 605 stores a set of program codes, and the processor 601, the input device 603, and the output device 604 call the program codes stored in the memory 605 to:
acquiring a sound signal of music to be processed;
performing track splitting processing on the sound signal to obtain N pieces of track data, wherein different track data are used for representing different sound source objects, and N is an integer greater than or equal to 2;
setting the sound field position of the sound source object in a virtual space;
and performing first azimuth modulation on the audio track data corresponding to the sound source object according to the sound field position of the sound source object in the virtual space to obtain target music, wherein the target music has the performance effect of a band in the real space.
In one possible implementation manner, the processor 601 is configured to perform a first orientation modulation on track data corresponding to the sound source object according to a sound field position of the sound source object in a virtual space, and includes:
taking the angle information of the sound field position of the sound source object in the virtual space as the input of the angle function of the sound track data corresponding to the sound source object to obtain the output result of the angle function;
and performing convolution operation on the output result of the angle function and the audio track data to realize first azimuth modulation on the audio track data corresponding to the sound source object.
In one possible implementation manner, the processor 601 is configured to set a sound field position of the sound source object in a virtual space, and includes (for):
displaying a target interface including an icon element for representing a sound source object corresponding to the track data;
and setting the sound field position input by the user based on the icon element as the sound field position of the sound source object corresponding to the icon element in the virtual space.
In one possible implementation manner, the processor 601 is configured to determine to set the sound field position input by the user based on the icon element as the sound field position of the sound source object corresponding to the icon element in the virtual space, and includes (for):
responding to an operation that a user drags an icon element of the sound source object from a first position to a second position of the target interface, and determining a sound field position corresponding to the second position as a sound field position of the sound source object in a virtual space; alternatively, the first and second electrodes may be,
and responding to the angle input operation of the user on the icon element of the sound source object, and determining the sound field position corresponding to the angle input operation as the sound field position of the sound source object in the virtual space.
In a possible implementation manner, after performing the first orientation modulation on the track data corresponding to the sound source object according to the sound field position of the sound source object, the processor 601 is further configured to:
if the N pieces of audio track data comprise audio track data of a target type, performing tonal modification processing on the audio track data of the target type to obtain tonal modification audio track data;
and synthesizing the tonal modification music track data and the music track data modulated in the first direction to obtain target music which is played by a band in a real space and has different tone ensemble effects.
In one possible implementation, the processor 601 is configured to combine the transposed audio track data with the first azimuth-modulated audio track data, and includes:
determining the target type of the audio track data and the target playing time interval corresponding to the tonal modification audio track data according to the playing time of the audio track data;
performing second azimuth modulation of relative change of sound field positions on the target type audio track data and the tonal modification audio track data in the target playing time period;
and synthesizing the second azimuth modulation and the track data after the first azimuth modulation to obtain target music with the effects of the band playing in real space, different tone ensemble and different tone relative position change.
In a possible implementation manner, the processor 601 is further configured to:
acquiring pulse signals corresponding to a target scene from a database, wherein the database stores the pulse signals corresponding to different scenes;
and carrying out convolution processing on the pulse signal corresponding to the target scene and the pulse signal of the target music to obtain the target music with the playing effect in the target scene.
In a possible implementation manner, the processor 601 is further configured to:
displaying the target interface, wherein the target interface comprises a mute switch corresponding to the audio track data;
and if an opening instruction of the mute switch is received through the target interface, carrying out mute processing on the audio track data corresponding to the opening instruction.
In a possible implementation manner, the processor 601 is further configured to:
determining a respective energy adjustment multiple for each of the audio track data;
and adjusting the energy of each music track data of the target music according to the respective energy adjustment multiple of each music track data to obtain the target music with the adaptation style.
In a possible embodiment, the processor 601 is further configured to display a playing interface, where the playing interface includes icon elements corresponding to respective audio track data of the target music;
detecting real-time energy of each track data of the recomposition-style target music during the playing of the recomposition-style target music;
if the real-time energy of the audio track data is larger than the first energy value, the icon elements of the audio track data corresponding to the real-time energy are subjected to lifting processing;
and if the real-time energy of the audio track data is less than the second energy value, carrying out non-prompt processing on the icon element of the audio track data corresponding to the real-time energy.
In the embodiment of the present invention, the processor 601 of the terminal obtains a plurality of pieces of music track data based on the split-track processing of the sound signal of the music to be processed, and performs modulation processing on the plurality of pieces of music track data to obtain updated diversified target music, so as to implement intelligent management on various types of music. And the method also comprises the transposition processing of the target type of the audio track data so as to realize more diversified auditory effects and meet the diversified requirements of users.
Further, a computer-readable storage medium is provided in the embodiments of the present invention, and the computer-readable storage medium stores a computer program, where the computer program includes program instructions, and the program instructions, when executed by a processor, may perform the steps performed in the embodiments of fig. 1, fig. 3, and fig. 4.
It should be understood that in the embodiment of the present invention, the Processor 601 may be a Central Processing Unit (CPU), and the Processor may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware associated with computer program instructions, and the programs can be stored in a computer readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
Reference herein to "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
While the invention has been described with reference to a number of embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (13)
1. A method of processing music data, the method comprising:
acquiring a sound signal of music to be processed;
performing track splitting processing on the sound signal to obtain N pieces of track data, wherein different track data are used for representing different sound source objects, and N is an integer greater than or equal to 2;
setting the sound field position of the sound source object in a virtual space;
and performing first orientation modulation on the track data corresponding to the sound source object according to the sound field position of the sound source object in the virtual space to obtain target music, wherein the target music has the music effect that a plurality of sound source objects perform at different space positions.
2. The method according to claim 1, wherein the performing the first orientation modulation on the track data corresponding to the sound source object according to the sound field position of the sound source object in the virtual space comprises:
taking the angle information of the sound field position of the sound source object in the virtual space as the input of the angle function of the sound track data corresponding to the sound source object to obtain the output result of the angle function;
and performing convolution operation on the output result of the angle function and the audio track data to realize first azimuth modulation on the audio track data corresponding to the sound source object.
3. The method according to claim 1, wherein the setting of the sound field position of the sound source object in the virtual space comprises:
displaying a target interface including an icon element for representing a sound source object corresponding to the track data;
and setting the sound field position input by the user based on the icon element as the sound field position of the sound source object corresponding to the icon element in the virtual space.
4. The method according to claim 3, wherein the setting of the sound field position input by the user based on the icon element as the sound field position of the sound source object corresponding to the icon element in the virtual space comprises:
responding to an operation that a user drags the icon element of the sound source object from the first position to the second position of the target interface, and determining the sound field position corresponding to the second position as the sound field position of the sound source object in the virtual space.
5. The method according to claim 1, wherein after the first orientation modulation is performed on the track data corresponding to the sound source object according to the sound field position of the sound source object, the method further comprises:
if the N pieces of audio track data comprise audio track data of a target type, performing tonal modification processing on the audio track data of the target type to obtain tonal modification audio track data;
and synthesizing the tonal modification music track data and the music track data modulated in the first direction to obtain target music which is played by a band in a real space and has different tone ensemble effects.
6. The method of claim 5, wherein said combining the transposed soundtrack data with the first azimuth modulated soundtrack data comprises:
determining the target type of the audio track data and the target playing time interval corresponding to the tonal modification audio track data according to the playing time of the audio track data;
performing second azimuth modulation of relative change of sound field positions on the target type audio track data and the tonal modification audio track data in the target playing time period;
and synthesizing the second azimuth modulation and the track data after the first azimuth modulation to obtain target music with the effects that a plurality of sound source objects perform at different spatial positions, different tones are ensemble, and the relative positions of different tones change.
7. The method according to any one of claims 1-6, further comprising:
acquiring pulse signals corresponding to a target scene from a database, wherein the database stores the pulse signals corresponding to different scenes;
and carrying out convolution processing on the pulse signal corresponding to the target scene and the pulse signal of the target music to obtain the target music with the playing effect in the target scene.
8. The method according to any one of claims 1-6, further comprising:
displaying the target interface, wherein the target interface comprises a mute switch corresponding to the audio track data;
and if an opening instruction of the mute switch is received through the target interface, carrying out mute processing on the audio track data corresponding to the opening instruction.
9. The method according to any one of claims 1-6, further comprising:
determining a respective energy adjustment multiple for each of the audio track data;
and adjusting the energy of each music track data of the target music according to the respective energy adjustment multiple of each music track data to obtain the target music with the adaptation style.
10. The method of claim 9, further comprising:
displaying a playing interface, wherein the playing interface comprises icon elements corresponding to all the audio track data of the target music;
detecting real-time energy of each track data of the recomposition-style target music during the playing of the recomposition-style target music;
if the real-time energy of the audio track data is larger than the first energy value, prompting the icon elements of the audio track data corresponding to the real-time energy;
and if the real-time energy of the audio track data is less than the second energy value, carrying out non-prompt processing on the icon element of the audio track data corresponding to the real-time energy.
11. An apparatus for processing music data, the apparatus comprising:
the acquisition module is used for acquiring a sound signal of music to be processed;
the processing module is used for performing track splitting processing on the sound signal to obtain N pieces of audio track data, wherein different audio track data are used for representing different sound source objects, and N is a positive integer greater than or equal to 2;
the processing module is used for setting the sound field position of the sound source object in the virtual space;
the processing module is further configured to perform first orientation modulation on the track data corresponding to the sound source object according to the sound field position of the sound source object in the virtual space, so as to obtain target music, where the target music has a music effect that a plurality of sound source objects perform at different spatial positions.
12. A terminal, characterized in that it comprises a processor, an input device, an output device and a memory, said processor, input device, output device and memory being interconnected, wherein said memory is used to store a computer program comprising program instructions, said processor being configured to invoke said program instructions to perform the method of processing music data according to any of claims 1-10.
13. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program comprising program instructions that, when executed by a processor, cause the processor to execute the processing method of music data according to any one of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010906667.3A CN112037738B (en) | 2020-08-31 | 2020-08-31 | Music data processing method and device and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010906667.3A CN112037738B (en) | 2020-08-31 | 2020-08-31 | Music data processing method and device and computer storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112037738A true CN112037738A (en) | 2020-12-04 |
| CN112037738B CN112037738B (en) | 2024-05-28 |
Family
ID=73592219
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010906667.3A Active CN112037738B (en) | 2020-08-31 | 2020-08-31 | Music data processing method and device and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112037738B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113365191A (en) * | 2021-06-04 | 2021-09-07 | 广州酷狗计算机科技有限公司 | Music playing method, device, equipment and storage medium |
| CN113707165A (en) * | 2021-09-07 | 2021-11-26 | 联想(北京)有限公司 | Audio processing method and device, electronic equipment and storage medium |
| CN113747337A (en) * | 2021-09-03 | 2021-12-03 | 杭州网易云音乐科技有限公司 | Audio processing method, medium, device and computing equipment |
| CN113986191A (en) * | 2021-12-27 | 2022-01-28 | 广州酷狗计算机科技有限公司 | Audio playing method and device, terminal equipment and storage medium |
| CN114286275A (en) * | 2021-12-20 | 2022-04-05 | Oppo广东移动通信有限公司 | Audio processing method and device and storage medium |
| CN114598985A (en) * | 2022-03-07 | 2022-06-07 | 安克创新科技股份有限公司 | Audio processing method and device |
| CN115174959A (en) * | 2022-06-21 | 2022-10-11 | 咪咕文化科技有限公司 | Video 3D sound effect setting method and device |
| CN115278350A (en) * | 2021-04-29 | 2022-11-01 | 华为技术有限公司 | Rendering method and related equipment |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1797538A (en) * | 2004-12-01 | 2006-07-05 | 创新科技有限公司 | System and method for forming and rendering 3d midi messages |
| CN102118670A (en) * | 2011-03-17 | 2011-07-06 | 杭州赛利科技有限公司 | Earphone capable of generating three-dimensional stereophonic sound effect |
| KR20110119339A (en) * | 2010-04-27 | 2011-11-02 | (주)아이지솔루션즈 | Music synthesis technique for synchroning with rhythm and it's service method |
| CN106952637A (en) * | 2017-03-15 | 2017-07-14 | 北京时代拓灵科技有限公司 | The creative method and experience apparatus of a kind of interactive music |
| CN108919953A (en) * | 2018-06-29 | 2018-11-30 | 咪咕文化科技有限公司 | A kind of music method of adjustment, device and storage medium |
| WO2019002909A1 (en) * | 2017-06-26 | 2019-01-03 | Latypov Ray | A method of providing to user an interactive music composition |
| WO2019057530A1 (en) * | 2017-09-20 | 2019-03-28 | Nokia Technologies Oy | An apparatus and associated methods for audio presented as spatial audio |
| CN110111761A (en) * | 2019-03-28 | 2019-08-09 | 深圳芒果未来教育科技有限公司 | The method followed in real time and Related product are played to musical sound |
| CN110170170A (en) * | 2019-05-30 | 2019-08-27 | 维沃移动通信有限公司 | A kind of information display method and terminal device |
| CN110992970A (en) * | 2019-12-13 | 2020-04-10 | 腾讯音乐娱乐科技（深圳）有限公司 | Audio synthesis method and related device |
-
2020
- 2020-08-31 CN CN202010906667.3A patent/CN112037738B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1797538A (en) * | 2004-12-01 | 2006-07-05 | 创新科技有限公司 | System and method for forming and rendering 3d midi messages |
| KR20110119339A (en) * | 2010-04-27 | 2011-11-02 | (주)아이지솔루션즈 | Music synthesis technique for synchroning with rhythm and it's service method |
| CN102118670A (en) * | 2011-03-17 | 2011-07-06 | 杭州赛利科技有限公司 | Earphone capable of generating three-dimensional stereophonic sound effect |
| CN106952637A (en) * | 2017-03-15 | 2017-07-14 | 北京时代拓灵科技有限公司 | The creative method and experience apparatus of a kind of interactive music |
| WO2019002909A1 (en) * | 2017-06-26 | 2019-01-03 | Latypov Ray | A method of providing to user an interactive music composition |
| CN110915240A (en) * | 2017-06-26 | 2020-03-24 | 雷.拉蒂波夫 | Method for providing interactive music composition to user |
| WO2019057530A1 (en) * | 2017-09-20 | 2019-03-28 | Nokia Technologies Oy | An apparatus and associated methods for audio presented as spatial audio |
| CN108919953A (en) * | 2018-06-29 | 2018-11-30 | 咪咕文化科技有限公司 | A kind of music method of adjustment, device and storage medium |
| CN110111761A (en) * | 2019-03-28 | 2019-08-09 | 深圳芒果未来教育科技有限公司 | The method followed in real time and Related product are played to musical sound |
| CN110170170A (en) * | 2019-05-30 | 2019-08-27 | 维沃移动通信有限公司 | A kind of information display method and terminal device |
| CN110992970A (en) * | 2019-12-13 | 2020-04-10 | 腾讯音乐娱乐科技（深圳）有限公司 | Audio synthesis method and related device |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115278350A (en) * | 2021-04-29 | 2022-11-01 | 华为技术有限公司 | Rendering method and related equipment |
| WO2022228174A1 (en) * | 2021-04-29 | 2022-11-03 | 华为技术有限公司 | Rendering method and related device |
| CN113365191A (en) * | 2021-06-04 | 2021-09-07 | 广州酷狗计算机科技有限公司 | Music playing method, device, equipment and storage medium |
| CN113747337A (en) * | 2021-09-03 | 2021-12-03 | 杭州网易云音乐科技有限公司 | Audio processing method, medium, device and computing equipment |
| CN113747337B (en) * | 2021-09-03 | 2024-05-10 | 杭州网易云音乐科技有限公司 | Audio processing method, medium, device and computing equipment |
| CN113707165A (en) * | 2021-09-07 | 2021-11-26 | 联想(北京)有限公司 | Audio processing method and device, electronic equipment and storage medium |
| CN114286275A (en) * | 2021-12-20 | 2022-04-05 | Oppo广东移动通信有限公司 | Audio processing method and device and storage medium |
| CN113986191A (en) * | 2021-12-27 | 2022-01-28 | 广州酷狗计算机科技有限公司 | Audio playing method and device, terminal equipment and storage medium |
| CN113986191B (en) * | 2021-12-27 | 2022-06-07 | 广州酷狗计算机科技有限公司 | Audio playing method and device, terminal equipment and storage medium |
| CN114598985A (en) * | 2022-03-07 | 2022-06-07 | 安克创新科技股份有限公司 | Audio processing method and device |
| CN114598985B (en) * | 2022-03-07 | 2024-05-03 | 安克创新科技股份有限公司 | Audio processing method and device |
| CN115174959A (en) * | 2022-06-21 | 2022-10-11 | 咪咕文化科技有限公司 | Video 3D sound effect setting method and device |
| CN115174959B (en) * | 2022-06-21 | 2024-01-30 | 咪咕文化科技有限公司 | Video 3D sound effect setting method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112037738B (en) | 2024-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112037738B (en) | Music data processing method and device and computer storage medium | |
| JP5161109B2 (en) | Signal decoding method and apparatus | |
| Emmerson et al. | Electro-acoustic music | |
| US9942673B2 (en) | Method and arrangement for fitting a hearing system | |
| JP7436664B2 (en) | Method for constructing a listening scene and related devices | |
| Müller et al. | The boomRoom: mid-air direct interaction with virtual sound sources | |
| Laitinen et al. | Parametric time-frequency representation of spatial sound in virtual worlds | |
| CN106170991A (en) | For the enhanced Apparatus and method for of sound field | |
| Hassager et al. | The role of spectral detail in the binaural transfer function on perceived externalization in a reverberant environment | |
| Spagnol et al. | Distance rendering and perception of nearby virtual sound sources with a near-field filter model | |
| Varghese et al. | How visual cues for when to listen aid selective auditory attention | |
| CN114242025A (en) | Method and device for generating accompaniment and storage medium | |
| Worrall | Space in sound: sound of space | |
| JP2011244197A (en) | Audio signal processing apparatus and method, program, and recording medium | |
| Cohen et al. | Spatial soundscape superposition and multimodal interaction | |
| CN114501297B (en) | Audio processing method and electronic equipment | |
| Neher | Towards a spatial ear trainer | |
| Henriksen | Space in electroacoustic music: composition, performance and perception of musical space | |
| Hermes | Towards Measuring Music Mix Quality: the factors contributing to the spectral clarity of single sounds | |
| Bologna et al. | A perceptual interface for vision substitution in a color matching experiment | |
| JP2004088608A (en) | Mixing device | |
| Lorenz | Impact of Head-Tracking on the listening experience of binaural music | |
| Munoz | Space Time Exploration of Musical Instruments | |
| Laamanen | Virtual Heritage: Audio design for immersive virtual environments using researched spatializers. | |
| Sato et al. | Subjective Evaluation on the Sense of" Being There" using Augmented reality Room Acoustic Estimator (ARAE) Platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |