WO2020062922A1 - Sound effect processing method and related product - Google Patents
Sound effect processing method and related product Download PDFInfo
- Publication number
- WO2020062922A1 WO2020062922A1 PCT/CN2019/090380 CN2019090380W WO2020062922A1 WO 2020062922 A1 WO2020062922 A1 WO 2020062922A1 CN 2019090380 W CN2019090380 W CN 2019090380W WO 2020062922 A1 WO2020062922 A1 WO 2020062922A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- target
- determining
- sound
- mono data
- dimensional coordinates
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
- H04S2400/01—Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
Definitions
- the present application relates to the technical field of audio playback, and in particular, to a sound effect processing method and related products.
- the sound volume is often based on the volume set by the user, and the sounding body uses a constant power corresponding to the volume set by the user to play the audio, so that the played sound meets the user's loudness requirements.
- a playback method is too simple in form, which often brings sensory fatigue to users.
- an embodiment of the present application provides a sound effect processing method, including:
- an embodiment of the present application provides a sound effect processing device, including:
- a determining unit configured to determine the three-dimensional coordinates of each of the plurality of sound sources corresponding to the electronic device and the mono data generated by each sound source to obtain a plurality of first three-dimensional coordinates and a plurality of mono data; The second three-dimensional coordinates of the target object corresponding to the electronic device;
- a synthesizing unit is configured to synthesize the plurality of mono data according to the plurality of first three-dimensional coordinates and the second three-dimensional coordinate to obtain target two-channel data.
- an embodiment of the present application provides an electronic device including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be processed by the foregoing. And the program includes instructions for some or all of the steps as described in the first aspect.
- an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program causes a computer to execute the program as described in the first aspect of the embodiment of the application. Describe some or all of the steps.
- an embodiment of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute Some or all of the steps described in the first aspect of the embodiment of the application.
- the computer program product can be a software installation package.
- FIG. 1A is a schematic structural diagram of an electronic device according to an embodiment of the present application.
- FIG. 1B is a schematic diagram of a scene of a coordinate axis of an electronic device according to an embodiment of the present application
- FIG. 2A is a schematic flowchart of a sound effect processing method according to an embodiment of the present application.
- FIG. 2B is a schematic diagram of a multi-channel dual-channel data scenario according to an embodiment of the present application.
- FIG. 3 is a schematic structural diagram of a sound effect processing device according to an embodiment of the present application.
- FIG. 4 is a schematic structural diagram of another electronic device according to an embodiment of the present application.
- the electronic devices involved in the embodiments of the present application may include various handheld devices (such as smart phones), vehicle-mounted devices, virtual reality (VR) / augmented reality (AR) devices with wireless communication functions, and may Wearable devices, computing devices or other processing devices connected to wireless modems, and various forms of user equipment (UE), mobile stations (MS), terminal devices, R & D / test platforms, Server and so on.
- UE user equipment
- MS mobile stations
- terminal devices R & D / test platforms, Server and so on.
- R & D / test platforms Server and so on.
- FIG. 1A is a schematic structural diagram of an electronic device according to an embodiment of the present application.
- the electronic device includes a control circuit and an input-output circuit, and the input-output circuit is connected to the control circuit.
- the control circuit may include a storage and processing circuit.
- the storage circuit in the storage and processing circuit may be a memory, such as a hard disk drive memory, a non-volatile memory (such as a flash memory or other electronic programmable read-only memory used to form a solid-state drive, etc.), a volatile memory (such as a static Or dynamic random access memory, etc.), this embodiment is not limited.
- the processing circuit in the storage and processing circuit can be used to control the operation of the electronic device.
- the processing circuit can be implemented based on one or more microprocessors, microcontrollers, digital signal processors, baseband processors, power management units, audio codec chips, application specific integrated circuits, display driver integrated circuits, and the like.
- the storage and processing circuit can be used to run software in an electronic device, for example, playing an incoming call alert ringing application, playing a short message alert ringing application, playing an alarm alert ringing application, playing a media file application, an Internet protocol voice ( Voice over Internet Protocol (VOIP) telephone calling applications, operating system functions, etc.
- These software can be used to perform some control operations, such as playing the call alert ring, playing the short message alert ring, playing the alarm alert ring, playing media files, making voice phone calls, and other functions in electronic devices.
- the examples are not limited.
- the input-output circuit can be used to enable the electronic device to implement data input and output, that is, to allow the electronic device to receive data from an external device and to allow the electronic device to output data from the electronic device to the external device.
- the input-output circuit may further include a sensor.
- the sensor may include an ambient light sensor, an infrared proximity sensor based on light and capacitance, an ultrasonic sensor, and a touch sensor (for example, a light touch sensor and / or a capacitive touch sensor, where the touch sensor may be part of a touch display screen, or Can be used independently as a touch sensor structure), acceleration sensor, gravity sensor, and other sensors.
- the input-output circuit may further include an audio component, and the audio component may be used to provide audio input and output functions for the electronic device.
- the audio component may also include a tone generator and other components for generating and detecting sound.
- the senor further includes a three-axis acceleration sensor for measuring a posture and an inclination angle of the electronic device.
- a three-axis acceleration sensor for measuring a posture and an inclination angle of the electronic device.
- it can also be used as a motion offset compensation calculation when the global positioning system (GPS) signal is not good, which can fully and accurately reflect the motion properties of the object.
- GPS global positioning system
- FIG. 1B is a schematic diagram of a scene where a three-dimensional acceleration sensor determines a coordinate axis of an electronic device.
- the x-axis, y-axis, and z-axis are relative to the body of the electronic device.
- the y-axis body is upward
- the x-axis body is right
- the z-axis is perpendicular to the front of the fuselage, and the center
- the horizontal component, vertical component, and vertical component are generally a unit of gravity (the size is 1g (m * m / s), the direction is perpendicular to the ground downward), and the projection on each axis. That is, the horizontal component is the corresponding value on the x-axis, the vertical component is the corresponding value on the y-axis, and the vertical component is the corresponding value on the z-axis.
- the x-axis defaults to 0, the y-axis defaults to 0, and the z-axis defaults to 9.81; place the electronic device on the desktop with the z-axis at -9.81; tilt the electronic device to the left, x The axis is positive; tilt the electronic device to the right and the x-axis is negative; tilt the electronic device upwards and the y-axis is negative; tilt the electronic device downwards and the y-axis is positive; set the z-axis to less than -3 In this case, the touch screen of the electronic device faces downward.
- the input-output circuit may also include one or more display screens.
- the display screen may include one or a combination of a liquid crystal display, an organic light emitting diode display, an electronic ink display, a plasma display, and a display using other display technologies.
- the display screen may include a touch sensor array (ie, the display screen may be a touch display screen).
- the touch sensor can be a capacitive touch sensor formed by a transparent array of touch sensor electrodes (such as indium tin oxide (ITO) electrodes), or it can be a touch sensor formed using other touch technologies, such as sonic touch, pressure-sensitive touch, resistance Touch, optical touch, etc. are not limited in the embodiments of the present application.
- the input-output circuit may further include a communication circuit for providing an electronic device with a capability to communicate with an external device.
- the communication circuit may include analog and digital input-output interface circuits, and wireless communication circuits based on radio frequency signals and / or optical signals.
- the wireless communication circuit in the communication circuit may include a radio frequency transceiver circuit, a power amplifier circuit, a low noise amplifier, a switch, a filter, and an antenna.
- the wireless communication circuit in the communication circuit may include a circuit for supporting near field communication (NFC) by transmitting and receiving a near field coupled electromagnetic signal.
- the communication circuit may include a near field communication antenna and a near field communication transceiver.
- the communication circuit may also include a cellular phone transceiver and antenna, a wireless local area network transceiver circuit and antenna, and the like.
- the input-output circuit may further include an input-output unit.
- the input-output unit may include a button, a joystick, a click wheel, a scroll wheel, a touch pad, a keypad, a keyboard, a camera, a light emitting diode, and other status indicators.
- the electronic device may further include a battery (not shown), and the battery is used to provide power to the electronic device.
- an embodiment of the present application provides a schematic flowchart of a sound effect processing method, which is applied to an electronic device. Specifically, as shown in FIG. 2A, the method includes:
- S201 Determine the three-dimensional coordinates of each sound source in the multiple sound sources corresponding to the electronic device and the mono data generated by each sound source to obtain multiple first three-dimensional coordinates and multiple mono data.
- the embodiments of the present application can be applied to a virtual reality / augmented reality scene, or a three-dimensional (3D) recording scene.
- the sound source may be a sounding body in a virtual scene, for example, an airplane in a game scene, and the sound source may be a fixed sound source or a mobile sound source.
- the sound source corresponding to the electronic device can use the above-mentioned coordinate axis as a reference to determine the first three-dimensional coordinate corresponding to the sound source, and when the sound source emits sound, it can obtain the source Of mono data.
- the electronic device may include multiple sound sources.
- the sound source of the game scene includes airplanes, guns, rivers, etc.
- the corresponding mono data is the gliding sound of the aircraft, the loading of guns, the sound of the fire, the sound of the water flowing in the river;
- the sound source of the game scene can also include game players,
- the corresponding mono data is footstep sounds, voice sounds, etc. of the game player, which is not limited here.
- step S201 may include: determining multiple reference objects corresponding to the electronic device; determining behavior information of each reference object in the multiple reference objects to obtain multiple behavior information; and according to the The plurality of behavior information determines a plurality of sound sources corresponding to the electronic device in the plurality of reference objects; determining a coordinate position corresponding to each sound source in the plurality of sound sources to obtain the plurality of first three-dimensional coordinates; The behavior information corresponding to each of the plurality of sound sources determines the mono data of the sound source to obtain the plurality of mono data.
- the reference object may be an object presented on a display page of the electronic device, for example, a house on the display page, a car in which a game player rides, or a firearm held.
- the reference object may also be an object that is not presented on the display page, such as a nearby game player, a gun, a vehicle, or the like.
- the behavior information is dynamic information of the reference object. It can be understood that different reference objects correspond to different types of behaviors. For example, guns can emit the sound of loading and firing guns, but not the sound of water flowing or talking. And each reference corresponds to a different sound when the behavior information is different. For example, the house will not emit sound under normal circumstances, but will emit a sound of bombardment when it is bombarded; the starting sound and driving sound will only occur when the car is moving. Sound; gamers produce voices when sending voices, footsteps when walking, and more. Therefore, in the possible examples described above, first determine multiple reference objects corresponding to the electronic device, and then obtain behavior information of each reference object, and determine whether the reference object is a sound source according to each behavior information, thereby improving the accuracy of determining the sound source. . Then, the coordinate position of each sound source is further determined to obtain a plurality of first three-dimensional coordinates, and then the mono data of the sound source is determined according to the behavior information corresponding to each sound source, which improves the accuracy of determining the mono data.
- This application does not limit how to determine the coordinate position corresponding to the sound source. For example, if it corresponds to a game scene and the game scene corresponds to a three-dimensional map, the coordinate position corresponding to different sound sources can be determined according to the map, that is, for characters Determining the first three-dimensional coordinates at a specific location can improve the accuracy of determining the first three-dimensional coordinates, facilitate the improvement of the 3D sound effect of the target two-channel data, and allow the user to be immersed in the game and feel the game world more realistic.
- the Determining the mono data of the sound source according to the behavior information corresponding to each of the multiple sound sources to obtain the multiple mono data includes: determining a sound type and a playback parameter corresponding to the target behavior information; The sound type and the playback parameter generate mono data of the target sound source.
- the sound type is the sound type of the target behavior information corresponding to the sound.
- the firearm includes the sound type of loading, firing, and hitting, and the playback parameters are loudness, frequency, and tone.
- step S201 takes the target sound source as an example, determines the sound type and playback parameters of the target sound source according to the target behavior information corresponding to the target sound source, and then generates the target sound source according to the sound type and playback parameters.
- Mono data can further improve the accuracy of determining mono data, improve the fit of mono data to application scenarios, and improve user experience.
- S202 Determine a second three-dimensional coordinate of a target object corresponding to the electronic device.
- the target object may be a game player corresponding to an electronic device in a game, a virtual reality or an augmented reality scene, or a target user corresponding to an electronic device in a 3D recording scene.
- the target object may also correspond to a three-dimensional position, that is, a second three-dimensional position.
- the first three-dimensional position is different from the second three-dimensional position.
- reference may be made to the method of determining the first three-dimensional coordinate, that is, the second three-dimensional coordinate of the target object is determined using the coordinate axis shown in FIG. 1B as a reference, and details are not described herein again.
- S203 Combining the plurality of mono data according to the plurality of first three-dimensional coordinates and the second three-dimensional coordinate to obtain target two-channel data.
- the mono data corresponding to each sound source can be synthesized to obtain the target two-channel data.
- a plurality of first three-dimensional coordinates, a second three-dimensional coordinate, and a plurality of mono data can be input into a Head Related Transformation Function (HRTF) model to obtain target binaural data.
- HRTF Head Related Transformation Function
- the electronic device can filter the audio data (sound from the sound source) using HRTF filters to obtain virtual surround sound, also called surround sound or panoramic sound, to achieve a three-dimensional stereo sound effect.
- HRTF Head-related Impulse Response
- BRIR Binaural Room Impulse Response
- the electronic device may be based on the spatial three-dimensional coordinate position (x, y, z) of the sound source, and the position may be any coordinate.
- the left and right channels are generated based on the mono data generated by the sound source.
- the principle of the left and right channels is based on the time difference between the sound source and the listener (X, Y, Z). And the phase pressure difference generates a two-channel sound.
- step S203 may include: determining the left ear three-dimensional coordinates and the right ear three-dimensional coordinates corresponding to the second three-dimensional coordinates; and according to the plurality of first three-dimensional coordinates, the left ear three-dimensional coordinates Determining the transmission paths between the plurality of sound sources and the target object by the coordinates and the three-dimensional coordinates of the right ear to obtain a plurality of transmission paths; determining each of the plurality of mono data according to the plurality of transmission paths The time and phase pressure of the mono data transmitted to the three-dimensional coordinates of the left ear and the three-dimensional coordinates of the right ear are obtained in multiple times and multiple phase pressures; and the multiple mono data are determined according to the multiple times.
- a plurality of time differences are obtained for the time difference corresponding to each mono data, and a phase pressure difference corresponding to each mono data in the plurality of mono data is determined according to the plurality of phase pressures to obtain a plurality of phase pressure differences; Determining the delay parameters corresponding to each of the plurality of mono data by the plurality of time differences and the plurality of phase pressure differences to obtain a plurality of delay parameters; and according to the plurality of delays Parameter, processing each mono data in the plurality of mono data to obtain corresponding left channel parameters and right channel parameters; corresponding to each mono data in the plurality of mono data And synthesizing the plurality of mono data to obtain the target two-channel data.
- the target object corresponds to a left ear three-dimensional coordinate and a right ear three-dimensional coordinate.
- This application does not limit how to determine the three-dimensional coordinates of the left ear and the three-dimensional coordinates of the right ear, and may be determined according to the 3D character model of the target object, that is, according to the second three-dimensional coordinates of the target object and the right ear that are preset in the 3D character model.
- the correlation between the three-dimensional coordinates, the second three-dimensional coordinates and the left ear three-dimensional coordinates is determined.
- the time difference and the phase pressure difference are respectively the time difference and the phase pressure difference transmitted to the left ear three-dimensional coordinates and the right ear three-dimensional coordinates. That is, the time difference and phase pressure difference between the mono data corresponding to the sound source and the left and right ears of the target object are transmitted.
- step S203 the left channel parameters and the right channel parameters of the mono data can be determined according to the delay parameters, and then synthesized to obtain the target two-channel data, which improves the playback effect of the audio data. Create immersive sensations to improve user experience.
- This application does not limit how to determine the time and phase pressure.
- This application takes the time and phase pressure corresponding to the three-dimensional coordinates of the target sound source to the left ear as an example.
- the time and phase pressure corresponding to the three-dimensional coordinates of the target sound source to the left ear are transmitted. Refer to this method for the method of determining the phase pressure and the monophonic data corresponding to other sound sources other than the target sound source among multiple sound sources to the left ear three-dimensional coordinates and the right ear three-dimensional coordinates.
- the plurality of sound sources includes a target sound source
- the plurality of first three-dimensional coordinates includes a target first three-dimensional coordinate corresponding to the target sound source
- the plurality of mono data includes the target sound Target mono data corresponding to the source
- Obtaining multiple time and multiple phase pressures with time and phase pressure includes: obtaining a cross-section using the target first three-dimensional coordinate and the left ear three-dimensional coordinate as an axis; determining the target first three-dimensional coordinate and the left ear three-dimensional Blocking objects between coordinates; determining a plurality of reference transmission paths for transmitting the target mono data to the three-dimensional coordinates of the left ear according to the cross section and the occluding objects; determining the reference transmission paths according to the plurality of reference transmission paths
- the propagation of the target mono data can include multiple reference transmissions. path.
- the first three-dimensional coordinates of the target and the three-dimensional coordinates of the left ear are used as the cross-sections. Since the sound propagation direction is fixed, the propagation trajectory will also have a certain symmetry along a certain symmetry axis, and multiple transmissions can be obtained path. The propagation of sound will be dispersed and transmitted when it encounters occluded objects, so that multiple corresponding reference transmission paths are determined according to the cross-section and the occluded objects in the application scene.
- the cross-section of a sound source data transmission is determined by using the first three-dimensional coordinates of the target and the three-dimensional coordinates of the left ear as axes, and then multiple reference transmission paths corresponding to the target mono data are determined according to the cross-section and the occluded object.
- the multiple reference transmission paths determine the phase pressure transmitted by the target sound source to the left ear of the target object. That is, the possible multiple reference transmission paths are determined according to the cross-sections corresponding to the first three-dimensional coordinates of the target and the three-dimensional coordinates of the left ear, and then the time and phase pressure are determined according to the multiple reference transmission paths, which improves the accuracy of determining the time and phase pressure. Sex.
- the determining the time and phase pressure of transmitting the target mono data to the three-dimensional coordinates of the left ear according to the multiple reference transmission paths includes: determining each of the multiple reference transmission paths The sound intensity and sound pressure corresponding to a transmission path are used to obtain a plurality of sound intensity and a plurality of sound pressures; and the target mono data is determined to be transmitted to the left ear three-dimensionally according to the sound intensity and the sound pressure. Coordinate time and phase pressure.
- the sound intensity refers to the sound energy of a unit area per unit time, which is perpendicular to the direction of propagation, and the unit is W / m2.
- Sound pressure is the increase in pressure due to the presence of sound waves, and the unit is Pa.
- This application does not limit how to determine the phase pressure based on multiple sound pressures.
- the preset weights can be determined according to the relative distance corresponding to the corresponding reference transmission path, and then multiple sound pressures and corresponding preset weights are weighted. Get the phase pressure.
- each sound source data has a corresponding sound pressure. Therefore, first determining the sound pressure corresponding to each reference transmission path to obtain multiple sound pressures, and then determining the phase pressure of the target mono data transmitted to the three-dimensional coordinates of the left ear based on the multiple sound pressures can improve the accuracy of determining the phase pressure.
- the three-dimensional coordinates of each sound source in a plurality of sound sources corresponding to the electronic device and the mono data generated by each sound source are determined to obtain a plurality of first three-dimensional coordinates and a plurality of sound sources.
- the channel data and the second three-dimensional coordinates of the target object corresponding to the electronic device are determined.
- the plurality of mono data is synthesized to obtain the target two-channel data.
- the target two-channel data corresponding to the multiple sound sources is generated, thereby improving the playback effect of the audio data and generating an immersive feeling.
- the method further includes: determining a target reverberation parameter corresponding to the target object; and processing the target two-channel data according to the target reverberation parameter to obtain reverberant two-channel data.
- the target reverb parameters include input volume, low frequency cut, high frequency cut, early reflection time, spatial breadth, diffusion degree, low mixing ratio, crossover point, reverberation time, high frequency attenuation point, dry sound adjustment, and reverberation.
- the volume, the amount of early reflected sound, the width of the sound field, the output sound field, and the tail sound are not limited here.
- the target reverberation parameter corresponding to the target object is determined, and then the target two-channel data is processed according to the target reverberation parameter to obtain the reverberant two-channel data.
- processing the target two-channel data according to the target object can further improve the playback effect of the audio data and improve the user experience.
- determining the target reverberation parameter corresponding to the target object includes: obtaining multiple historical reverberations corresponding to the target object that are stored in advance. Playing records; acquiring listening parameters corresponding to each historical reverb playing record in the plurality of historical reverb playing records to obtain a plurality of listening parameters; and determining a target reverb parameter corresponding to the target object according to the plurality of listening parameters.
- the listening parameters include audio type, playback duration, playback adjustment times, user mood parameters, and so on. It can be understood that obtaining multiple historical reverberation playback records corresponding to the target object in advance, obtaining listening parameters corresponding to each historical reverberation playback record, and then determining the target reverberation parameters based on the multiple listening parameters improves the determination of the target reverberation. The accuracy of the parameters is convenient for improving the playback effect.
- determining the target reverberation parameter corresponding to the target object according to the multiple listening parameters includes: according to the Multiple listening parameters determine the evaluation value corresponding to each historical reverb play record in the multiple historical reverb play records to obtain multiple evaluation values; and use the historical reverb record corresponding to the maximum value among the multiple evaluation values as A target historical reverberation record; and a reverberation parameter corresponding to the target historical reverberation record as a target reverberation parameter corresponding to the target object.
- the plurality of historical reverberation playback records includes a target historical reverberation playback record
- taking the target historical reverberation playback record as an example, first determine an application scenario of the electronic device, and determine that the application scenario corresponds to Preset mood parameters. Then, an evaluation value corresponding to the target historical reverberation record is determined according to a difference between the preset mood parameter and the user mood parameter.
- the preset mood parameters corresponding to the target object are different.
- the preset mood parameters of the target object are determined according to the application scenario corresponding to the electronic device.
- the evaluation value corresponding to the target historical reverberation record is determined according to the difference between the preset mood parameter and the user mood parameter.
- the evaluation value corresponding to the historical reverberation record is determined according to the listening parameters of each historical reverberation record, and then the historical reverberation record corresponding to the largest evaluation value is selected as the target historical reverberation record, and then according to the target historical reverberation record.
- the recorded listening parameters determine the target reverberation parameters, which improves the accuracy of determining the target reverberation parameters.
- FIG. 3 is a schematic structural diagram of a sound effect processing device according to an embodiment of the present application.
- the sound effect processing device 300 includes a determining unit 301 and a synthesizing unit 302, where:
- the determining unit 301 is configured to determine the three-dimensional coordinates of each sound source in the multiple sound sources corresponding to the electronic device and the mono data generated by each sound source to obtain multiple first three-dimensional coordinates and multiple mono data; The second three-dimensional coordinates of the target object corresponding to the electronic device;
- the synthesizing unit 302 is configured to synthesize the plurality of mono data according to the plurality of first three-dimensional coordinates and the second three-dimensional coordinate to obtain target binaural data.
- the determining unit 301 is further configured to determine the left ear three-dimensional coordinates and the right ear three-dimensional coordinates corresponding to the second three-dimensional coordinates; and determine the multiple according to the plurality of first three-dimensional coordinates, the left ear three-dimensional coordinates, and the right ear three-dimensional coordinates.
- a plurality of transmission paths are obtained from transmission paths between the sound sources and the target object; and each of the plurality of mono data is determined to be transmitted to the left ear three-dimensional coordinates according to the plurality of transmission paths.
- time and phase pressure of the three-dimensional coordinate of the right ear to obtain multiple times and multiple phase pressures; determining a time difference corresponding to each of the plurality of mono data according to the plurality of times to obtain multiple Time difference, determining a phase pressure difference corresponding to each of the plurality of mono data according to the plurality of phase pressures to obtain a plurality of phase pressure differences; and according to the plurality of time differences and the plurality of phases,
- the bit pressure difference determines a delay parameter corresponding to each of the plurality of mono data to obtain a plurality of delay parameters; according to the plurality of delay parameters, for each of the plurality of mono data,
- the channel data is processed to obtain corresponding left channel parameters and right channel parameters; the synthesizing unit 302 is specifically configured to use the left channel parameters and the right channels corresponding to each of the plurality of mono data Channel parameters, synthesizing the plurality of mono data to obtain target two-channel data.
- the plurality of sound sources includes a target sound source
- the plurality of first three-dimensional coordinates includes a target first three-dimensional coordinate corresponding to the target sound source
- the plurality of mono data includes all The target mono data corresponding to the target sound source
- the determining unit 301 is specifically configured to obtain a cross-section using the target first three-dimensional coordinate and the left ear three-dimensional coordinate as an axis; determining the target first three-dimensional coordinate and the left ear three-dimensional Blocking objects between coordinates; determining a plurality of reference transmission paths for transmitting the target mono data to the three-dimensional coordinates of the left ear according to the cross section and the occluding objects; determining the reference transmission paths according to the plurality of reference transmission paths The time and phase pressure of the target mono data transmitted to the three-dimensional coordinates of the left ear.
- the determining unit 301 is specifically configured to determine The sound intensity and sound pressure corresponding to each transmission path in the plurality of reference transmission paths obtain a plurality of sound intensity and a plurality of sound pressures; and determine the target mono data to be transmitted to the left according to the plurality of sound intensity.
- the phase pressure at which the target mono data is transmitted to the three-dimensional coordinate of the left ear is determined according to the plurality of sound pressures.
- the three-dimensional coordinates of each sound source in the plurality of sound sources corresponding to the electronic device and the mono data generated by each sound source are used to obtain multiple first three-dimensional coordinates and multiple mono channels.
- the determining unit 301 is specifically configured to determine multiple reference objects corresponding to the electronic device; determine behavior information of each reference object in the multiple reference objects to obtain multiple behavior information; and determine based on the multiple behavior information
- a plurality of sound sources corresponding to the electronic device in the plurality of reference objects determining a coordinate position corresponding to each sound source in the plurality of sound sources to obtain a plurality of first three-dimensional coordinates; and according to the plurality of sound sources,
- the behavior information corresponding to each sound source determines the mono data of the sound source to obtain a plurality of mono data.
- the multiple behavior information includes target behavior information corresponding to a target sound source; and determining the mono data of the sound source according to the behavior information corresponding to each sound source in the multiple sound sources
- the determining unit 301 is specifically configured to determine a sound type and a playback parameter corresponding to the target behavior information; and generate a target sound source according to the sound type and the playback parameter.
- the determination unit 301 is further configured to determine a target reverberation parameter corresponding to the target object; and the synthesis unit 302 is further configured to perform the target two-channel data on the target reverberation parameter according to the target reverberation parameter. Process it to get reverberant two-channel data.
- the determining unit 301 is specifically configured to obtain multiple historical reverb play records corresponding to the target object stored in advance; obtain each historical reverb play in the multiple historical reverb play records Recording corresponding listening parameters to obtain multiple listening parameters; and determining a target reverberation parameter corresponding to the target object according to the multiple listening parameters.
- the determining unit 301 is specifically configured to determine an evaluation value corresponding to each historical reverb play record in the multiple historical reverb play records according to the multiple listening parameters to obtain multiple evaluation values;
- the historical reverberation record corresponding to the maximum value among the multiple evaluation values is used as the target historical reverberation record, and the reverberation parameter corresponding to the target historical reverberation record is used as the target reverberation parameter corresponding to the target object.
- FIG. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
- the electronic device 400 includes a processor 410, a memory 420, a communication interface 430, and one or more programs 440.
- the one or more programs 440 are stored in the memory 420, and are configured by
- the processor 410 executes, and the program 440 includes instructions for performing the following steps:
- the program 440 in terms of synthesizing the plurality of mono data according to the plurality of first three-dimensional coordinates and the second three-dimensional coordinate to obtain target two-channel data, the program 440
- the instructions in are specifically used to perform the following operations:
- phase pressure difference corresponding to the channel data is used to obtain multiple phase pressure differences
- the plurality of sound sources includes a target sound source
- the plurality of first three-dimensional coordinates includes a target first three-dimensional coordinate corresponding to the target sound source
- the plurality of mono data includes all The target mono data corresponding to the target sound source; and determining the time and phase at which each mono data of the plurality of mono data is transmitted to the left ear three-dimensional coordinate according to the plurality of transmission paths
- the pressure obtains multiple time and multiple phase pressure aspects, and the instructions in the program 440 are specifically used to perform the following operations:
- Time and phase pressures for transmitting the target mono data to the three-dimensional coordinates of the left ear according to the multiple reference transmission paths are provided.
- the instructions in the program 440 are specifically used to do the following:
- the three-dimensional coordinates of each sound source and the mono data generated by each sound source in the plurality of sound sources corresponding to the electronic device 400 are obtained to obtain a plurality of first three-dimensional coordinates and a plurality of mono sounds.
- the instructions in the program 440 are specifically used to perform the following operations:
- the mono data of the sound source is determined according to the behavior information corresponding to each of the multiple sound sources to obtain multiple mono data.
- the multiple behavior information includes target behavior information corresponding to a target sound source; and determining the mono data of the sound source according to the behavior information corresponding to each sound source in the multiple sound sources
- the instructions in the program 440 are specifically used to perform the following operations:
- Mono data of the target sound source is generated according to the sound type and the playback parameter.
- the instructions in the program 440 are further used to perform the following operations:
- the instructions in the program 440 are specifically used to perform the following operations:
- a target reverberation parameter corresponding to the target object is determined according to the plurality of listening parameters.
- the instructions in the program 440 are specifically used to perform the following operations:
- the historical reverberation record corresponding to the maximum value among the multiple evaluation values is used as the target historical reverberation record, and the reverberation parameter corresponding to the target historical reverberation record is used as the target reverberation parameter corresponding to the target object.
- the sound processing method and the electronic device provided in the embodiments of the present application are similar to the method embodiments of the present application. Therefore, the implementation of the sound processing method and the electronic device can refer to the method implementation, the sound processing method and the electronic For the beneficial effects of the device, refer to the beneficial effects of the method. For brevity description, they are not repeated here.
- An embodiment of the present application further provides a computer storage medium, wherein the computer storage medium stores a computer program for causing a computer to execute a part or all of the steps of any method as described in the method embodiment, and the computer includes an electronic device. device.
- An embodiment of the present application further provides a computer program product.
- the computer program product includes a non-transitory computer-readable storage medium storing the computer program, and the computer program is operable to cause a computer to execute a part of any method as described in the method embodiments. Or all steps.
- the computer program product may be a software installation package, and the computer includes an electronic device.
- the disclosed device may be implemented in other ways.
- the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. In actual implementation, there may be another division manner. For example, multiple units or components may be combined or integrated into Another system, or some features, can be ignored or not implemented.
- the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be electrical or other forms.
- the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, which may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objective of the solution of this embodiment.
- each functional unit in each embodiment of the present application may be integrated into one processing unit, or each of the units may exist separately physically, or two or more units may be integrated into one unit.
- the above integrated unit may be implemented in the form of hardware or in the form of a software program mode.
- the integrated unit When the integrated unit is implemented in the form of a software program and sold or used as an independent product, it can be stored in a computer-readable memory.
- the technical solution of the present application essentially or part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory.
- a computer device which may be a personal computer, a server, or a network device, etc.
- the foregoing memory includes: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk, and other media that can store program codes.
- the program may be stored in a computer-readable memory, and the memory may include a flash disk. , ROM, RAM, disk or disc, etc.
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Stereophonic System (AREA)
Abstract
Disclosed are a sound effect processing method and a related product. The method comprises: determining a three-dimensional coordinate of each sound source from among a plurality of sound sources corresponding to an electronic device, and single-track data generated by each sound source, so as to obtain a plurality of first three-dimensional coordinates and a plurality of pieces of single-track data; determining a second three-dimensional coordinate of a target object corresponding to the electronic device; and according to the plurality of first three-dimensional coordinates and the second three-dimensional coordinate, synthesizing the plurality of pieces of single-track data to obtain target dual-track data. By means of the present application, a playing effect of audio data can be improved.
Description
本申请要求于2018年9月25日提交中国专利局、申请号为201811118269.4、申请名称为“3D音效处理方法及相关产品”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims the priority of a Chinese patent application filed on September 25, 2018 with the Chinese Patent Office, application number 201811118269.4, and application name "3D Sound Processing Method and Related Products", the entire contents of which are incorporated herein by reference. .
本申请涉及音频播放技术领域,具体涉及一种音效处理方法及相关产品。The present application relates to the technical field of audio playback, and in particular, to a sound effect processing method and related products.
随着电子设备功能的多样化发展以及其便携特性,越来越多的人喜欢通过电子设备来进行一些娱乐活动。尤其是,可以根据用户需要随时随地的听歌、看视频。With the diversified development of electronic device functions and its portable characteristics, more and more people like to use electronic devices for some entertainment activities. In particular, you can listen to songs and watch videos anytime, anywhere according to user needs.
现有的音频播放方式中,常以用户设定的音量为基础,由发声体采用对应用户设定音量的恒定功率来播放音频,使得播放的声音满足用户的响度需求。然而,这样的播放方式,形式过于单一,往往会给用户带来感官上的疲惫。In the existing audio playback methods, the sound volume is often based on the volume set by the user, and the sounding body uses a constant power corresponding to the volume set by the user to play the audio, so that the played sound meets the user's loudness requirements. However, such a playback method is too simple in form, which often brings sensory fatigue to users.
发明内容Summary of the Invention
第一方面,本申请实施例提供一种音效处理方法,包括:In a first aspect, an embodiment of the present application provides a sound effect processing method, including:
确定电子设备对应的多个声源中每一声源的三维坐标、以及每一声源产生的单声道数据得到多个第一三维坐标和多个单声道数据;Determining the three-dimensional coordinates of each of the plurality of sound sources corresponding to the electronic device and the mono data generated by each sound source to obtain a plurality of first three-dimensional coordinates and a plurality of mono data;
确定所述电子设备对应的目标对象的第二三维坐标;Determining a second three-dimensional coordinate of a target object corresponding to the electronic device;
根据所述多个第一三维坐标和所述第二三维坐标,对所述多个单声道数据进行合成得到目标双声道数据。Synthesizing the plurality of mono data according to the plurality of first three-dimensional coordinates and the second three-dimensional coordinate to obtain target two-channel data.
第二方面,本申请实施例提供一种音效处理装置,包括:In a second aspect, an embodiment of the present application provides a sound effect processing device, including:
确定单元,用于确定电子设备对应的多个声源中每一声源的三维坐标、以及每一声源产生的单声道数据得到多个第一三维坐标和多个单声道数据;确定所述电子设备对应的目标对象的第二三维坐标;A determining unit, configured to determine the three-dimensional coordinates of each of the plurality of sound sources corresponding to the electronic device and the mono data generated by each sound source to obtain a plurality of first three-dimensional coordinates and a plurality of mono data; The second three-dimensional coordinates of the target object corresponding to the electronic device;
合成单元,用于根据所述多个第一三维坐标和所述第二三维坐标,对所述多个单声道数据进行合成得到目标双声道数据。A synthesizing unit is configured to synthesize the plurality of mono data according to the plurality of first three-dimensional coordinates and the second three-dimensional coordinate to obtain target two-channel data.
第三方面,本申请实施例提供一种电子设备,包括处理器、存储器、通信 接口以及一个或多个程序,其中,上述一个或多个程序被存储在上述存储器中,并且被配置由上述处理器执行,所述程序包括用于如第一方面中所描述的部分或全部步骤的指令。According to a third aspect, an embodiment of the present application provides an electronic device including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be processed by the foregoing. And the program includes instructions for some or all of the steps as described in the first aspect.
第四方面,本申请实施例提供了一种计算机可读存储介质,其中,所述计算机可读存储介质存储计算机程序,其中,所述计算机程序使得计算机执行如本申请实施例第一方面中所描述的部分或全部步骤。In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program causes a computer to execute the program as described in the first aspect of the embodiment of the application. Describe some or all of the steps.
第五方面,本申请实施例提供了一种计算机程序产品,其中,所述计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质,所述计算机程序可操作来使计算机执行如本申请实施例第一方面中所描述的部分或全部步骤。该计算机程序产品可以为一个软件安装包。In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute Some or all of the steps described in the first aspect of the embodiment of the application. The computer program product can be a software installation package.
为了更清楚地说明本申请实施例中的技术方案,下面将对实施例中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to explain the technical solutions in the embodiments of the present application more clearly, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. Those of ordinary skill in the art can obtain other drawings according to the drawings without paying creative labor.
图1A为本申请实施例提供的一种电子设备的结构示意图;FIG. 1A is a schematic structural diagram of an electronic device according to an embodiment of the present application; FIG.
图1B为本申请实施例提供的一种电子设备的坐标轴的场景示意图;FIG. 1B is a schematic diagram of a scene of a coordinate axis of an electronic device according to an embodiment of the present application; FIG.
图2A为本申请实施例提供的一种音效处理方法的流程示意图;2A is a schematic flowchart of a sound effect processing method according to an embodiment of the present application;
图2B为本申请实施例提供的一种多路双声道数据的场景示意图;FIG. 2B is a schematic diagram of a multi-channel dual-channel data scenario according to an embodiment of the present application; FIG.
图3为本申请实施例提供的一种音效处理装置的结构示意图;3 is a schematic structural diagram of a sound effect processing device according to an embodiment of the present application;
图4为本申请实施例提供的另一种电子设备的结构示意图。FIG. 4 is a schematic structural diagram of another electronic device according to an embodiment of the present application.
为了使本技术领域的人员更好地理解本申请方案,下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述。In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application.
本申请实施例所涉及到的电子设备可以包括各种具有无线通信功能的手持设备(如智能手机)、车载设备、虚拟现实(virtual reality,VR)/增强现实(augmented reality,AR)设备,可穿戴设备、计算设备或连接到无线调制解调器的其他处理设备,以及各种形式的用户设备(user equipment,UE),移动台(mobile station,MS),终端设备(terminal device)、研发/测试平台、服务 器等等。为方便描述,上面提到的设备统称为电子设备。The electronic devices involved in the embodiments of the present application may include various handheld devices (such as smart phones), vehicle-mounted devices, virtual reality (VR) / augmented reality (AR) devices with wireless communication functions, and may Wearable devices, computing devices or other processing devices connected to wireless modems, and various forms of user equipment (UE), mobile stations (MS), terminal devices, R & D / test platforms, Server and so on. For ease of description, the devices mentioned above are collectively referred to as electronic devices.
请参阅图1A,图1A是本申请实施例提供了一种电子设备的结构示意图,电子设备包括控制电路和输入-输出电路,输入输出电路与控制电路连接。Please refer to FIG. 1A, which is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device includes a control circuit and an input-output circuit, and the input-output circuit is connected to the control circuit.
其中,控制电路可以包括存储和处理电路。该存储和处理电路中的存储电路可以是存储器,例如硬盘驱动存储器,非易失性存储器(例如闪存或用于形成固态驱动器的其它电子可编程只读存储器等),易失性存储器(例如静态或动态随机存取存储器等)等,本申请实施例不作限制。存储和处理电路中的处理电路可以用于控制电子设备的运转。该处理电路可以基于一个或多个微处理器,微控制器,数字信号处理器,基带处理器,功率管理单元,音频编解码器芯片,专用集成电路,显示驱动器集成电路等来实现。The control circuit may include a storage and processing circuit. The storage circuit in the storage and processing circuit may be a memory, such as a hard disk drive memory, a non-volatile memory (such as a flash memory or other electronic programmable read-only memory used to form a solid-state drive, etc.), a volatile memory (such as a static Or dynamic random access memory, etc.), this embodiment is not limited. The processing circuit in the storage and processing circuit can be used to control the operation of the electronic device. The processing circuit can be implemented based on one or more microprocessors, microcontrollers, digital signal processors, baseband processors, power management units, audio codec chips, application specific integrated circuits, display driver integrated circuits, and the like.
存储和处理电路可用于运行电子设备中的软件,例如,播放来电提示响铃应用程序、播放短消息提示响铃应用程序、播放闹钟提示响铃应用程序、播放媒体文件应用程序、互联网协议语音(voice over internet protocol,VOIP)电话呼叫应用程序、操作***功能等。这些软件可以用于执行一些控制操作,例如,播放来电提示响铃、播放短消息提示响铃、播放闹钟提示响铃、播放媒体文件、进行语音电话呼叫以及电子设备中的其它功能等,本申请实施例不作限制。The storage and processing circuit can be used to run software in an electronic device, for example, playing an incoming call alert ringing application, playing a short message alert ringing application, playing an alarm alert ringing application, playing a media file application, an Internet protocol voice ( Voice over Internet Protocol (VOIP) telephone calling applications, operating system functions, etc. These software can be used to perform some control operations, such as playing the call alert ring, playing the short message alert ring, playing the alarm alert ring, playing media files, making voice phone calls, and other functions in electronic devices. The examples are not limited.
其中,输入-输出电路可用于使电子设备实现数据的输入和输出,即允许电子设备从外部设备接收数据和允许电子设备将数据从电子设备输出至外部设备。Among them, the input-output circuit can be used to enable the electronic device to implement data input and output, that is, to allow the electronic device to receive data from an external device and to allow the electronic device to output data from the electronic device to the external device.
输入-输出电路可以进一步包括传感器。传感器可以包括环境光传感器,基于光和电容的红外接近传感器,超声波传感器,触摸传感器(例如,基于光触摸传感器和/或电容式触摸传感器,其中,触摸传感器可以是触控显示屏的一部分,也可以作为一个触摸传感器结构独立使用),加速度传感器,重力传感器,和其它传感器等。输入-输出电路还可以进一步包括音频组件,音频组件可以用于为电子设备提供音频输入和输出功能。音频组件还可以包括音调发生器以及其它用于产生和检测声音的组件。The input-output circuit may further include a sensor. The sensor may include an ambient light sensor, an infrared proximity sensor based on light and capacitance, an ultrasonic sensor, and a touch sensor (for example, a light touch sensor and / or a capacitive touch sensor, where the touch sensor may be part of a touch display screen, or Can be used independently as a touch sensor structure), acceleration sensor, gravity sensor, and other sensors. The input-output circuit may further include an audio component, and the audio component may be used to provide audio input and output functions for the electronic device. The audio component may also include a tone generator and other components for generating and detecting sound.
在本申请实施例,传感器还包括三轴加速度传感器,用于测量电子设备的姿态和倾斜角。除了自动切换水平、垂直显示视角外,还可在全球定位***(global positioning system,GPS)信号不好时,用作运动偏移补偿计算,能够全面准确反映物体的运动性质。In the embodiment of the present application, the sensor further includes a three-axis acceleration sensor for measuring a posture and an inclination angle of the electronic device. In addition to automatically switching the horizontal and vertical display angles, it can also be used as a motion offset compensation calculation when the global positioning system (GPS) signal is not good, which can fully and accurately reflect the motion properties of the object.
请参照图1B,图1B为三维加速度传感器确定电子设备的坐标轴的场景示意图。如图1B所示,x轴、y轴、z轴均是相对电子设备机身位置的,通常y 轴向机身向上,x轴向机身向右,z轴垂直机身正面,与地心引力同向。横向分量、纵向分量、竖向分量一般是一个单位的地心引力(大小1g(m*m/s),方向垂直地面向下),在各轴上的投影。即横向分量为x轴上对应的数值,纵向分量为y轴上对应的数值,竖向分量为z轴上对应的数值。Please refer to FIG. 1B, which is a schematic diagram of a scene where a three-dimensional acceleration sensor determines a coordinate axis of an electronic device. As shown in Figure 1B, the x-axis, y-axis, and z-axis are relative to the body of the electronic device. Usually, the y-axis body is upward, the x-axis body is right, and the z-axis is perpendicular to the front of the fuselage, and the center The same direction of gravity. The horizontal component, vertical component, and vertical component are generally a unit of gravity (the size is 1g (m * m / s), the direction is perpendicular to the ground downward), and the projection on each axis. That is, the horizontal component is the corresponding value on the x-axis, the vertical component is the corresponding value on the y-axis, and the vertical component is the corresponding value on the z-axis.
例如:将电子设备平放在桌面上,x轴默认为0,y轴默认0,z轴默认9.81;将电子设备朝下放在桌面上,z轴为-9.81;将电子设备向左倾斜,x轴为正值;将电子设备向右倾斜,x轴为负值;将电子设备向上倾斜,y轴为负值;将电子设备向下倾斜,y轴为正值;将z轴小于-3的情况,视为电子设备的触控显示屏朝下。For example: place the electronic device on the desktop, the x-axis defaults to 0, the y-axis defaults to 0, and the z-axis defaults to 9.81; place the electronic device on the desktop with the z-axis at -9.81; tilt the electronic device to the left, x The axis is positive; tilt the electronic device to the right and the x-axis is negative; tilt the electronic device upwards and the y-axis is negative; tilt the electronic device downwards and the y-axis is positive; set the z-axis to less than -3 In this case, the touch screen of the electronic device faces downward.
输入-输出电路还可以包括一个或多个显示屏。显示屏可以包括液晶显示屏,有机发光二极管显示屏,电子墨水显示屏,等离子显示屏,使用其它显示技术的显示屏中一种或者几种的组合。显示屏可以包括触摸传感器阵列(即,显示屏可以是触控显示屏)。触摸传感器可以是由透明的触摸传感器电极(例如氧化铟锡(ITO)电极)阵列形成的电容式触摸传感器,或者可以是使用其它触摸技术形成的触摸传感器,例如音波触控,压敏触摸,电阻触摸,光学触摸等,本申请实施例不作限制。The input-output circuit may also include one or more display screens. The display screen may include one or a combination of a liquid crystal display, an organic light emitting diode display, an electronic ink display, a plasma display, and a display using other display technologies. The display screen may include a touch sensor array (ie, the display screen may be a touch display screen). The touch sensor can be a capacitive touch sensor formed by a transparent array of touch sensor electrodes (such as indium tin oxide (ITO) electrodes), or it can be a touch sensor formed using other touch technologies, such as sonic touch, pressure-sensitive touch, resistance Touch, optical touch, etc. are not limited in the embodiments of the present application.
输入-输出电路还可以进一步包括通信电路,用于为电子设备提供与外部设备通信的能力。通信电路可以包括模拟和数字输入-输出接口电路,和基于射频信号和/或光信号的无线通信电路。通信电路中的无线通信电路可以包括射频收发器电路、功率放大器电路、低噪声放大器、开关、滤波器和天线。举例来说,通信电路中的无线通信电路可以包括用于通过发射和接收近场耦合电磁信号来支持近场通信(near field communication,NFC)的电路。例如,通信电路可以包括近场通信天线和近场通信收发器。通信电路还可以包括蜂窝电话收发器和天线,无线局域网收发器电路和天线等。The input-output circuit may further include a communication circuit for providing an electronic device with a capability to communicate with an external device. The communication circuit may include analog and digital input-output interface circuits, and wireless communication circuits based on radio frequency signals and / or optical signals. The wireless communication circuit in the communication circuit may include a radio frequency transceiver circuit, a power amplifier circuit, a low noise amplifier, a switch, a filter, and an antenna. For example, the wireless communication circuit in the communication circuit may include a circuit for supporting near field communication (NFC) by transmitting and receiving a near field coupled electromagnetic signal. For example, the communication circuit may include a near field communication antenna and a near field communication transceiver. The communication circuit may also include a cellular phone transceiver and antenna, a wireless local area network transceiver circuit and antenna, and the like.
输入-输出电路还可以进一步包括输入-输出单元。输入-输出单元可以包括按钮,操纵杆,点击轮,滚动轮,触摸板,小键盘,键盘,照相机,发光二极管和其它状态指示器等。The input-output circuit may further include an input-output unit. The input-output unit may include a button, a joystick, a click wheel, a scroll wheel, a touch pad, a keypad, a keyboard, a camera, a light emitting diode, and other status indicators.
其中,电子设备还可以进一步包括电池(未图示),电池用于给电子设备提供电能。The electronic device may further include a battery (not shown), and the battery is used to provide power to the electronic device.
下面对本申请实施例进行详细介绍。The embodiments of the present application are described in detail below.
请参照图2A,本申请实施例提供一种音效处理方法的流程示意图,该方法 应用于电子设备。具体的,如图2A所示,该方法包括:Referring to FIG. 2A, an embodiment of the present application provides a schematic flowchart of a sound effect processing method, which is applied to an electronic device. Specifically, as shown in FIG. 2A, the method includes:
S201:确定电子设备对应的多个声源中每一声源的三维坐标、以及每一声源产生的单声道数据得到多个第一三维坐标和多个单声道数据。S201: Determine the three-dimensional coordinates of each sound source in the multiple sound sources corresponding to the electronic device and the mono data generated by each sound source to obtain multiple first three-dimensional coordinates and multiple mono data.
本申请实施例可应用于虚拟现实/增强现实场景,或者,三维(3 Dimensions,3D)录音场景。本申请实施例中,声源可以为虚拟场景中的一个发声体,例如,游戏场景中的一个飞机,声源可以为固定声源,或者,移动声源。如图1B所示的电子设备的坐标轴,则电子设备对应的声源可以上述坐标轴为基准,确定该声源对应的第一三维坐标,并在声源发出声音时,则可以获取生源产生的单声道数据。The embodiments of the present application can be applied to a virtual reality / augmented reality scene, or a three-dimensional (3D) recording scene. In the embodiment of the present application, the sound source may be a sounding body in a virtual scene, for example, an airplane in a game scene, and the sound source may be a fixed sound source or a mobile sound source. As shown in the coordinate axis of the electronic device shown in FIG. 1B, the sound source corresponding to the electronic device can use the above-mentioned coordinate axis as a reference to determine the first three-dimensional coordinate corresponding to the sound source, and when the sound source emits sound, it can obtain the source Of mono data.
电子设备可包括多个声源。例如,游戏场景的声源包括飞机、***、河水等,对应的单声道数据为飞机的滑翔声,***上膛、发射的声音,河水的水流声;游戏场景的声源也可包括游戏玩家,对应的单声道数据为游戏玩家的脚步声、语音声等,在此不做限定。The electronic device may include multiple sound sources. For example, the sound source of the game scene includes airplanes, guns, rivers, etc. The corresponding mono data is the gliding sound of the aircraft, the loading of guns, the sound of the fire, the sound of the water flowing in the river; the sound source of the game scene can also include game players, The corresponding mono data is footstep sounds, voice sounds, etc. of the game player, which is not limited here.
在一个可能的示例中,步骤S201的具体实施方式可包括:确定电子设备对应的多个参考对象;确定所述多个参考对象中每一参考对象的行为信息得到多个行为信息;根据所述多个行为信息确定所述多个参考对象中与所述电子设备对应的多个声源;确定所述多个声源中每一声源对应的坐标位置得到所述多个第一三维坐标;根据所述多个声源中每一声源对应的行为信息确定该声源的单声道数据得到所述多个单声道数据。In a possible example, the specific implementation of step S201 may include: determining multiple reference objects corresponding to the electronic device; determining behavior information of each reference object in the multiple reference objects to obtain multiple behavior information; and according to the The plurality of behavior information determines a plurality of sound sources corresponding to the electronic device in the plurality of reference objects; determining a coordinate position corresponding to each sound source in the plurality of sound sources to obtain the plurality of first three-dimensional coordinates; The behavior information corresponding to each of the plurality of sound sources determines the mono data of the sound source to obtain the plurality of mono data.
其中,参考对象可以是电子设备的显示页面中所呈现的对象,例如:显示页面中的房子、游戏玩家所乘坐的汽车、所持的枪械。参考对象也可以是未呈现于显示页面的对象,例如:邻近的游戏玩家、枪炮、车辆等。The reference object may be an object presented on a display page of the electronic device, for example, a house on the display page, a car in which a game player rides, or a firearm held. The reference object may also be an object that is not presented on the display page, such as a nearby game player, a gun, a vehicle, or the like.
行为信息是参考对象的动态信息。可以理解,不同的参考对象对应于不同的行为类型,例如,***可发出上膛、打枪的声音,而不会发出水流声、说话声。且每一参考处于不同的行为信息时对应不同的声音,例如:房子在一般情况下不会发出声音,而在被炮击时会发出炮击声;汽车在开动时,才会发生启动声和行驶的声音;游戏玩家在发送语音时产生语音声、在步行时产生脚步声等等。因此,在上述可能的示例中,先确定电子设备对应的多个参考对象,然后获取各个参考对象的行为信息,根据各个行为信息确定该参考对象是否为声源,提高了确定声源的准确性。然后,进一步确定各个声源的坐标位置得到多个第一三维坐标,再根据各个声源对应的行为信息确定该声源的单声道数据, 提高了确定单声道数据的准确性。The behavior information is dynamic information of the reference object. It can be understood that different reference objects correspond to different types of behaviors. For example, guns can emit the sound of loading and firing guns, but not the sound of water flowing or talking. And each reference corresponds to a different sound when the behavior information is different. For example, the house will not emit sound under normal circumstances, but will emit a sound of bombardment when it is bombarded; the starting sound and driving sound will only occur when the car is moving. Sound; gamers produce voices when sending voices, footsteps when walking, and more. Therefore, in the possible examples described above, first determine multiple reference objects corresponding to the electronic device, and then obtain behavior information of each reference object, and determine whether the reference object is a sound source according to each behavior information, thereby improving the accuracy of determining the sound source. . Then, the coordinate position of each sound source is further determined to obtain a plurality of first three-dimensional coordinates, and then the mono data of the sound source is determined according to the behavior information corresponding to each sound source, which improves the accuracy of determining the mono data.
本申请对于如何确定声源对应的坐标位置不做限定,举例来说,若对应游戏场景,且该游戏场景对应一个三维地图,则可根据该地图确定不同声源对应的坐标位置,即针对角色具***置确定第一三维坐标,可提高确定第一三维坐标的准确性,便于提高目标双声道数据的3D音效,让用户在游戏时,能够身临其境,感觉游戏世界更为逼真。This application does not limit how to determine the coordinate position corresponding to the sound source. For example, if it corresponds to a game scene and the game scene corresponds to a three-dimensional map, the coordinate position corresponding to different sound sources can be determined according to the map, that is, for characters Determining the first three-dimensional coordinates at a specific location can improve the accuracy of determining the first three-dimensional coordinates, facilitate the improvement of the 3D sound effect of the target two-channel data, and allow the user to be immersed in the game and feel the game world more realistic.
本申请对于如何根据行为信息确定单声道数据不做限定,若所述多个行为信息包括所述多个声源中的目标声源对应的目标行为信息,在一个可能的示例中,所述根据所述多个声源中每一声源对应的行为信息确定该声源的单声道数据得到所述多个单声道数据包括:确定所述目标行为信息对应的声音类型和播放参数;根据所述声音类型和所述播放参数生成所述目标声源的单声道数据。This application does not limit how to determine mono data according to behavior information. If the plurality of behavior information includes target behavior information corresponding to a target sound source among the multiple sound sources, in a possible example, the Determining the mono data of the sound source according to the behavior information corresponding to each of the multiple sound sources to obtain the multiple mono data includes: determining a sound type and a playback parameter corresponding to the target behavior information; The sound type and the playback parameter generate mono data of the target sound source.
其中,声音类型为该声音对应的目标行为信息的发声类型,例如:***包括上膛、发射、击中的声音类型,播放参数为响度、频率、音调等。The sound type is the sound type of the target behavior information corresponding to the sound. For example, the firearm includes the sound type of loading, firing, and hitting, and the playback parameters are loudness, frequency, and tone.
可以理解,步骤S201的具体实施方式以目标声源为例,根据目标声源对应的目标行为信息确定目标声源的声音类型和播放参数,然后再根据声音类型和播放参数生成该目标声源的单声道数据,可进一步提高了确定单声道数据的准确性,提高了单声道数据与应用场景的贴合性,便于提高用户体验。It can be understood that the specific implementation of step S201 takes the target sound source as an example, determines the sound type and playback parameters of the target sound source according to the target behavior information corresponding to the target sound source, and then generates the target sound source according to the sound type and playback parameters. Mono data can further improve the accuracy of determining mono data, improve the fit of mono data to application scenarios, and improve user experience.
S202:确定所述电子设备对应的目标对象的第二三维坐标。S202: Determine a second three-dimensional coordinate of a target object corresponding to the electronic device.
在本申请实施例中,目标对象可以是游戏、虚拟现实或增强现实场景中电子设备对应的游戏玩家,也可以是3D录音场景中电子设备对应的目标用户等。目标对象也可以对应一个三维位置,即第二三维位置,当然,第一三维位置与第二三维位置为不同的位置。对于确定第二三维坐标的方法可以参照确定第一三维坐标的方法,即以图1B中所示的坐标轴为基准,确定所述目标对象的第二三维坐标,在此不再赘述。In the embodiment of the present application, the target object may be a game player corresponding to an electronic device in a game, a virtual reality or an augmented reality scene, or a target user corresponding to an electronic device in a 3D recording scene. The target object may also correspond to a three-dimensional position, that is, a second three-dimensional position. Of course, the first three-dimensional position is different from the second three-dimensional position. For the method of determining the second three-dimensional coordinate, reference may be made to the method of determining the first three-dimensional coordinate, that is, the second three-dimensional coordinate of the target object is determined using the coordinate axis shown in FIG. 1B as a reference, and details are not described herein again.
S203:根据所述多个第一三维坐标和所述第二三维坐标,对所述多个单声道数据进行合成得到目标双声道数据。S203: Combining the plurality of mono data according to the plurality of first three-dimensional coordinates and the second three-dimensional coordinate to obtain target two-channel data.
其中,在知晓各个声源对应的第一三维坐标、目标对象的第二三维坐标,则可以将各个声源对应的单声道数据进行合成得到目标双声道数据。具体地,可以将多个第一三维坐标、第二三维坐标和多个单声道数据输入到头相关变换函数(Head Related Transfer Function,HRTF)模型中得到目标双声道数据。Where the first three-dimensional coordinates corresponding to each sound source and the second three-dimensional coordinates of the target object are known, the mono data corresponding to each sound source can be synthesized to obtain the target two-channel data. Specifically, a plurality of first three-dimensional coordinates, a second three-dimensional coordinate, and a plurality of mono data can be input into a Head Related Transformation Function (HRTF) model to obtain target binaural data.
具体实现中,电子设备可对音频数据(声源发出的声音)使用HRTF滤波 器进行滤波得到虚拟环绕声,也称之为环绕声,或者全景声,实现一种三维立体音效。HRTF在时间域所对应的名称是头相关的脉冲响应(Head Related Impulse Response,HRIR),或者将音频数据与双耳房间脉冲响应(Binaural Room Impulse Response,BRIR)做卷积,双耳房间脉冲响应由三个部分组成:直达声,早期反射声和混响。In specific implementation, the electronic device can filter the audio data (sound from the sound source) using HRTF filters to obtain virtual surround sound, also called surround sound or panoramic sound, to achieve a three-dimensional stereo sound effect. The corresponding name of HRTF in the time domain is Head-related Impulse Response (HRIR), or convolution of audio data with Binaural Room Impulse Response (BRIR), Binaural Room Impulse Response Consists of three parts: direct sound, early reflections, and reverb.
举例来说,电子设备可根据声源的空间三维坐标位置(x,y,z),该位置可以是任意坐标。再根据声源产生的单声道数据生成左右声道,左右声道的产生原理是依据声源距离听者(X,Y,Z),单点数据传输左耳和传输到右耳数据的时间差以及相位压力差生成双声道声音。For example, the electronic device may be based on the spatial three-dimensional coordinate position (x, y, z) of the sound source, and the position may be any coordinate. The left and right channels are generated based on the mono data generated by the sound source. The principle of the left and right channels is based on the time difference between the sound source and the listener (X, Y, Z). And the phase pressure difference generates a two-channel sound.
在一个可能的示例中,步骤S203的具体实施方式可包括:确定所述第二三维坐标对应的左耳三维坐标和右耳三维坐标;根据所述多个第一三维坐标、所述左耳三维坐标和所述右耳三维坐标确定所述多个声源与所述目标对象之间的传输路径得到多个传输路径;根据所述多个传输路径确定所述多个单声道数据中每一单声道数据传输至所述左耳三维坐标、所述右耳三维坐标的时间和相位压力得到多个时间和多个相位压力;根据所述多个时间确定所述多个单声道数据中每一单声道数据对应的时间差得到多个时间差,根据所述多个相位压力确定所述多个单声道数据中每一单声道数据对应的相位压力差得到多个相位压力差;根据所述多个时间差和所述多个相位压力差确定所述多个单声道数据中每一单声道数据对应的延迟参数得到多个延迟参数;根据所述多个延迟参数,对所述多个单声道数据中每一单声道数据进行处理得到对应的左声道参数和右声道参数;按照所述多个单声道数据中每一单声道数据对应的左声道参数和右声道参数,对所述多个单声道数据进行合成得到所述目标双声道数据。In a possible example, the specific implementation of step S203 may include: determining the left ear three-dimensional coordinates and the right ear three-dimensional coordinates corresponding to the second three-dimensional coordinates; and according to the plurality of first three-dimensional coordinates, the left ear three-dimensional coordinates Determining the transmission paths between the plurality of sound sources and the target object by the coordinates and the three-dimensional coordinates of the right ear to obtain a plurality of transmission paths; determining each of the plurality of mono data according to the plurality of transmission paths The time and phase pressure of the mono data transmitted to the three-dimensional coordinates of the left ear and the three-dimensional coordinates of the right ear are obtained in multiple times and multiple phase pressures; and the multiple mono data are determined according to the multiple times. A plurality of time differences are obtained for the time difference corresponding to each mono data, and a phase pressure difference corresponding to each mono data in the plurality of mono data is determined according to the plurality of phase pressures to obtain a plurality of phase pressure differences; Determining the delay parameters corresponding to each of the plurality of mono data by the plurality of time differences and the plurality of phase pressure differences to obtain a plurality of delay parameters; and according to the plurality of delays Parameter, processing each mono data in the plurality of mono data to obtain corresponding left channel parameters and right channel parameters; corresponding to each mono data in the plurality of mono data And synthesizing the plurality of mono data to obtain the target two-channel data.
其中,目标对象对应一个左耳三维坐标和一个右耳三维坐标。本申请对于如何确定左耳三维坐标和右耳三维坐标不做限定,可根据所述目标对象的3D人物模型进行确定,即根据3D人物模型中预先设置的目标对象的第二三维坐标和右耳三维坐标,第二三维坐标和左耳三维坐标之间的关联关系进行确定。The target object corresponds to a left ear three-dimensional coordinate and a right ear three-dimensional coordinate. This application does not limit how to determine the three-dimensional coordinates of the left ear and the three-dimensional coordinates of the right ear, and may be determined according to the 3D character model of the target object, that is, according to the second three-dimensional coordinates of the target object and the right ear that are preset in the 3D character model. The correlation between the three-dimensional coordinates, the second three-dimensional coordinates and the left ear three-dimensional coordinates is determined.
时间差和相位压力差分别为传输至左耳三维坐标、右耳三维坐标的时间之差和相位压力之差。也就是说,声源对应的单声道数据传输至目标对象的左耳、右耳的时间差和相位压力差。The time difference and the phase pressure difference are respectively the time difference and the phase pressure difference transmitted to the left ear three-dimensional coordinates and the right ear three-dimensional coordinates. That is, the time difference and phase pressure difference between the mono data corresponding to the sound source and the left and right ears of the target object are transmitted.
可以理解,由于左耳和右耳之间存在一定距离,声音在空气中传播时存在压力差。因此,通过实施步骤S203的具体实施方式,可根据延迟参数确定单 声道数据的左声道参数和右声道参数,再进行合成得到目标双声道数据,提高了音频数据的播放效果,可产生沉浸感受,便于提高用户体验。It can be understood that due to the distance between the left and right ears, there is a pressure difference when the sound travels through the air. Therefore, by implementing the specific implementation of step S203, the left channel parameters and the right channel parameters of the mono data can be determined according to the delay parameters, and then synthesized to obtain the target two-channel data, which improves the playback effect of the audio data. Create immersive sensations to improve user experience.
本申请对于如何确定时间和相位压力的方法不做限定,本申请以目标声源传输至左耳三维坐标对应的时间和相位压力为例,该目标声源传输至左耳三维坐标对应的时间和相位压力,以及多个声源中除了目标声源之外的其它声源对应的单声道数据确定传输至左耳三维坐标、右耳三维坐标对应的时间和相位压力的方法可参照此方法。This application does not limit how to determine the time and phase pressure. This application takes the time and phase pressure corresponding to the three-dimensional coordinates of the target sound source to the left ear as an example. The time and phase pressure corresponding to the three-dimensional coordinates of the target sound source to the left ear are transmitted. Refer to this method for the method of determining the phase pressure and the monophonic data corresponding to other sound sources other than the target sound source among multiple sound sources to the left ear three-dimensional coordinates and the right ear three-dimensional coordinates.
具体的,若所述多个声源包括目标声源,所述多个第一三维坐标包括所述目标声源对应的目标第一三维坐标,所述多个单声道数据包括所述目标声源对应的目标单声道数据;在一个可能的示例中,所述根据所述多个传输路径确定所述多个单声道数据中每一单声道数据传输至所述左耳三维坐标的时间和相位压力得到多个时间和多个相位压力包括:以所述目标第一三维坐标与所述左耳三维坐标为轴线得到横截面;确定所述目标第一三维坐标与所述左耳三维坐标之间的遮挡物体;根据所述横截面和所述遮挡物体确定所述目标单声道数据传输至所述左耳三维坐标的多个参考传输路径;根据所述多个参考传输路径确定所述目标单声道数据传输至所述左耳三维坐标的时间和相位压力。Specifically, if the plurality of sound sources includes a target sound source, the plurality of first three-dimensional coordinates includes a target first three-dimensional coordinate corresponding to the target sound source, and the plurality of mono data includes the target sound Target mono data corresponding to the source; in a possible example, determining, according to the plurality of transmission paths, each mono data of the plurality of mono data transmitted to the three-dimensional coordinates of the left ear Obtaining multiple time and multiple phase pressures with time and phase pressure includes: obtaining a cross-section using the target first three-dimensional coordinate and the left ear three-dimensional coordinate as an axis; determining the target first three-dimensional coordinate and the left ear three-dimensional Blocking objects between coordinates; determining a plurality of reference transmission paths for transmitting the target mono data to the three-dimensional coordinates of the left ear according to the cross section and the occluding objects; determining the reference transmission paths according to the plurality of reference transmission paths The time and phase pressure of the target mono data transmitted to the three-dimensional coordinates of the left ear.
由于在现实环境中,声音是沿着各个方向进行传播,当然,在传播过程中,也会出现反射、折射、干涉、衍射等现象,因此,目标单声道数据的传播可包括多个参考传输路径。如图2B所示,以目标第一三维坐标与左耳三维坐标为轴线作横截面,由于声音传播方向一定,传播轨迹也会沿着一定的对称轴具备一定的对称性,可得到多个传输路径。而声音的传播在遇到遮挡物体时会分散传输,从而根据横截面和应用场景中的遮挡物体确定对应的多个参考传输路径。In the real environment, sound travels in all directions. Of course, reflection, refraction, interference, diffraction and other phenomena also occur during the propagation process. Therefore, the propagation of the target mono data can include multiple reference transmissions. path. As shown in FIG. 2B, the first three-dimensional coordinates of the target and the three-dimensional coordinates of the left ear are used as the cross-sections. Since the sound propagation direction is fixed, the propagation trajectory will also have a certain symmetry along a certain symmetry axis, and multiple transmissions can be obtained path. The propagation of sound will be dispersed and transmitted when it encounters occluded objects, so that multiple corresponding reference transmission paths are determined according to the cross-section and the occluded objects in the application scene.
可以理解,以目标第一三维坐标与左耳三维坐标为轴线确定一个声源数据传输的横截面,再根据该横截面和遮挡物体确定目标单声道数据对应的多个参考传输路径,再由多个参考传输路径确定目标声源传输至目标对象的左耳的相位压力。也就是说,根据目标第一三维坐标与左耳三维坐标对应的横截面确定可能的多个参考传输路径,再根据多个参考传输路径确定时间和相位压力,提高了确定时间和相位压力的准确性。It can be understood that the cross-section of a sound source data transmission is determined by using the first three-dimensional coordinates of the target and the three-dimensional coordinates of the left ear as axes, and then multiple reference transmission paths corresponding to the target mono data are determined according to the cross-section and the occluded object. The multiple reference transmission paths determine the phase pressure transmitted by the target sound source to the left ear of the target object. That is, the possible multiple reference transmission paths are determined according to the cross-sections corresponding to the first three-dimensional coordinates of the target and the three-dimensional coordinates of the left ear, and then the time and phase pressure are determined according to the multiple reference transmission paths, which improves the accuracy of determining the time and phase pressure. Sex.
在一个可能的示例中,所述根据所述多个参考传输路径确定所述目标单声道数据传输至所述左耳三维坐标的时间和相位压力包括:确定所述多个参考传输路径中每一传输路径对应的声强和声压得到多个声强和多个声压;根据所述 多个声强和所述多个声压确定所述目标单声道数据传输至所述左耳三维坐标的时间和相位压力。In a possible example, the determining the time and phase pressure of transmitting the target mono data to the three-dimensional coordinates of the left ear according to the multiple reference transmission paths includes: determining each of the multiple reference transmission paths The sound intensity and sound pressure corresponding to a transmission path are used to obtain a plurality of sound intensity and a plurality of sound pressures; and the target mono data is determined to be transmitted to the left ear three-dimensionally according to the sound intensity and the sound pressure. Coordinate time and phase pressure.
其中,声强是指单位时间内,声波通过垂直于传播方向单位面积的声能量,单位为W/m2。声压是由于声波的存在而引起的压力增值,单位为Pa。Among them, the sound intensity refers to the sound energy of a unit area per unit time, which is perpendicular to the direction of propagation, and the unit is W / m2. Sound pressure is the increase in pressure due to the presence of sound waves, and the unit is Pa.
本申请对于如何根据多个声压确定相位压力的方法不做限定,可依据对应参考传输路径对应的相对距离确定预设权值,再将多个声压和对应的预设权值进行加权计算得到相位压力。This application does not limit how to determine the phase pressure based on multiple sound pressures. The preset weights can be determined according to the relative distance corresponding to the corresponding reference transmission path, and then multiple sound pressures and corresponding preset weights are weighted. Get the phase pressure.
可以理解,每一声源数据具有对应的声压。因此,先确定各个参考传输路径对应的声压以得到多个声压,再根据多个声压确定目标单声道数据传输至左耳三维坐标的相位压力,可提高确定相位压力的准确性。It can be understood that each sound source data has a corresponding sound pressure. Therefore, first determining the sound pressure corresponding to each reference transmission path to obtain multiple sound pressures, and then determining the phase pressure of the target mono data transmitted to the three-dimensional coordinates of the left ear based on the multiple sound pressures can improve the accuracy of determining the phase pressure.
在如图2A所示的音效处理方法中,确定电子设备对应的多个声源中每一声源的三维坐标、以及每一声源产生的单声道数据得到多个第一三维坐标和多个单声道数据,并确定了电子设备对应的目标对象的第二三维坐标。然后,根据所述多个第一三维坐标和所述第二三维坐标,对所述多个单声道数据进行合成得到目标双声道数据。如此,在确定了多个声源的第一三维坐标以及目标对象的第二三维坐标之后,生成多个声源对应的目标双声道数据,从而提高音频数据的播放效果,可产生沉浸感受,便于提高用户体验。In the sound effect processing method shown in FIG. 2A, the three-dimensional coordinates of each sound source in a plurality of sound sources corresponding to the electronic device and the mono data generated by each sound source are determined to obtain a plurality of first three-dimensional coordinates and a plurality of sound sources. The channel data and the second three-dimensional coordinates of the target object corresponding to the electronic device are determined. Then, according to the plurality of first three-dimensional coordinates and the second three-dimensional coordinate, the plurality of mono data is synthesized to obtain the target two-channel data. In this way, after the first three-dimensional coordinates of the multiple sound sources and the second three-dimensional coordinates of the target object are determined, the target two-channel data corresponding to the multiple sound sources is generated, thereby improving the playback effect of the audio data and generating an immersive feeling. Facilitate user experience.
在一个可能的示例中,所述方法还包括:确定所述目标对象对应的目标混响参数;根据所述目标混响参数对所述目标双声道数据进行处理得到混响双声道数据。In a possible example, the method further includes: determining a target reverberation parameter corresponding to the target object; and processing the target two-channel data according to the target reverberation parameter to obtain reverberant two-channel data.
其中,目标混响参数包括输入音量、低频切除、高频切除、早反射时间、空间广度、扩散程度、低混比例、分频点、残响时间、高频衰点、干声调节、混响音量、早反射声音量、声场宽度、输出声场、尾音等,在此不做限定。Among them, the target reverb parameters include input volume, low frequency cut, high frequency cut, early reflection time, spatial breadth, diffusion degree, low mixing ratio, crossover point, reverberation time, high frequency attenuation point, dry sound adjustment, and reverberation. The volume, the amount of early reflected sound, the width of the sound field, the output sound field, and the tail sound are not limited here.
可以理解,确定目标对象对应的目标混响参数,然后根据目标混响参数对目标双声道数据进行处理得到混响双声道数据。如此,依据目标对象对目标双声道数据进行处理,可进一步提高音频数据的播放效果,便于提高用户体验。It can be understood that the target reverberation parameter corresponding to the target object is determined, and then the target two-channel data is processed according to the target reverberation parameter to obtain the reverberant two-channel data. In this way, processing the target two-channel data according to the target object can further improve the playback effect of the audio data and improve the user experience.
本申请对于如何确定目标混响参数不做限定,在一个可能的示例中,所述确定所述目标对象对应的目标混响参数包括:获取预先存储的所述目标对象对应的多个历史混响播放记录;获取所述多个历史混响播放记录中每一历史混响播放记录对应的收听参数得到多个收听参数;根据所述多个收听参数确定所述目标对象对应的目标混响参数。This application does not limit how to determine the target reverberation parameter. In a possible example, determining the target reverberation parameter corresponding to the target object includes: obtaining multiple historical reverberations corresponding to the target object that are stored in advance. Playing records; acquiring listening parameters corresponding to each historical reverb playing record in the plurality of historical reverb playing records to obtain a plurality of listening parameters; and determining a target reverb parameter corresponding to the target object according to the plurality of listening parameters.
其中,收听参数包括音频类型、播放时长、播放调整次数、用户心情参数等。可以理解,获取预先存储的与目标对象对应的多个历史混响播放记录,获取各个历史混响播放记录对应的收听参数,再根据多个收听参数确定目标混响参数,提高了确定目标混响参数的准确性,便于提高播放效果。The listening parameters include audio type, playback duration, playback adjustment times, user mood parameters, and so on. It can be understood that obtaining multiple historical reverberation playback records corresponding to the target object in advance, obtaining listening parameters corresponding to each historical reverberation playback record, and then determining the target reverberation parameters based on the multiple listening parameters improves the determination of the target reverberation. The accuracy of the parameters is convenient for improving the playback effect.
本申请对于如何根据多个收听参数确定目标混响参数不做限定,在一个可能的示例中,所述根据所述多个收听参数确定所述目标对象对应的目标混响参数包括:根据所述多个收听参数确定所述多个历史混响播放记录中每一历史混响播放记录对应的评价值得到多个评价值;将所述多个评价值中的最大值对应的历史混响记录作为目标历史混响记录;将所述目标历史混响记录对应的混响参数作为所述目标对象对应的目标混响参数。This application does not limit how to determine the target reverberation parameter according to multiple listening parameters. In a possible example, determining the target reverberation parameter corresponding to the target object according to the multiple listening parameters includes: according to the Multiple listening parameters determine the evaluation value corresponding to each historical reverb play record in the multiple historical reverb play records to obtain multiple evaluation values; and use the historical reverb record corresponding to the maximum value among the multiple evaluation values as A target historical reverberation record; and a reverberation parameter corresponding to the target historical reverberation record as a target reverberation parameter corresponding to the target object.
其中,若所述多个历史混响播放记录包括目标历史混响播放记录,则以所述目标历史混响播放记录为例,先确定所述电子设备的应用场景,以及确定所述应用场景对应的预设心情参数。再根据所述预设心情参数和所述用户心情参数之间的差值确定所述目标历史混响记录对应的评价值。Wherein, if the plurality of historical reverberation playback records includes a target historical reverberation playback record, taking the target historical reverberation playback record as an example, first determine an application scenario of the electronic device, and determine that the application scenario corresponds to Preset mood parameters. Then, an evaluation value corresponding to the target historical reverberation record is determined according to a difference between the preset mood parameter and the user mood parameter.
可以理解,在不同的应用场景中,目标对象对应的预设心情参数不同。如此,根据电子设备对应的应用场景确定目标对象的预设心情参数。然后,根据预设心情参数和用户心情参数之间的差值确定目标历史混响记录对应的评价值。在本申请中,根据每一历史混响记录的收听参数确定对应历史混响记录的评价值,然后选取最大的评价值对应的历史混响记录作为目标历史混响记录,再根据目标历史混响记录的收听参数确定目标混响参数,提高了确定目标混响参数的准确性。It can be understood that, in different application scenarios, the preset mood parameters corresponding to the target object are different. In this way, the preset mood parameters of the target object are determined according to the application scenario corresponding to the electronic device. Then, the evaluation value corresponding to the target historical reverberation record is determined according to the difference between the preset mood parameter and the user mood parameter. In this application, the evaluation value corresponding to the historical reverberation record is determined according to the listening parameters of each historical reverberation record, and then the historical reverberation record corresponding to the largest evaluation value is selected as the target historical reverberation record, and then according to the target historical reverberation record. The recorded listening parameters determine the target reverberation parameters, which improves the accuracy of determining the target reverberation parameters.
请参照图3,图3是本申请实施例提供的一种音效处理装置的结构示意图,如图3所示,上述音效处理装置300包括确定单元301和合成单元302,其中:Please refer to FIG. 3. FIG. 3 is a schematic structural diagram of a sound effect processing device according to an embodiment of the present application. As shown in FIG. 3, the sound effect processing device 300 includes a determining unit 301 and a synthesizing unit 302, where:
确定单元301用于确定电子设备对应的多个声源中每一声源的三维坐标、以及每一声源产生的单声道数据得到多个第一三维坐标和多个单声道数据;确定所述电子设备对应的目标对象的第二三维坐标;The determining unit 301 is configured to determine the three-dimensional coordinates of each sound source in the multiple sound sources corresponding to the electronic device and the mono data generated by each sound source to obtain multiple first three-dimensional coordinates and multiple mono data; The second three-dimensional coordinates of the target object corresponding to the electronic device;
合成单元302用于根据所述多个第一三维坐标和所述第二三维坐标,对所述多个单声道数据进行合成得到目标双声道数据。The synthesizing unit 302 is configured to synthesize the plurality of mono data according to the plurality of first three-dimensional coordinates and the second three-dimensional coordinate to obtain target binaural data.
在一个可能的示例中,在所述根据所述多个第一三维坐标和所述第二三维坐标,对所述多个单声道数据进行合成得到目标双声道数据方面,所述确定单元301还用于确定所述第二三维坐标对应的左耳三维坐标和右耳三维坐标;根 据所述多个第一三维坐标、所述左耳三维坐标和所述右耳三维坐标确定所述多个声源与所述目标对象之间的传输路径得到多个传输路径;根据所述多个传输路径确定所述多个单声道数据中每一单声道数据传输至所述左耳三维坐标、所述右耳三维坐标的时间和相位压力得到多个时间和多个相位压力;根据所述多个时间确定所述多个单声道数据中每一单声道数据对应的时间差得到多个时间差,根据所述多个相位压力确定所述多个单声道数据中每一单声道数据对应的相位压力差得到多个相位压力差;根据所述多个时间差和所述多个相位压力差确定所述多个单声道数据中每一单声道数据对应的延迟参数得到多个延迟参数;根据所述多个延迟参数,对所述多个单声道数据中每一单声道数据进行处理得到对应的左声道参数和右声道参数;所述合成单元302具体用于按照所述多个单声道数据中每一单声道数据对应的左声道参数和右声道参数,对所述多个单声道数据进行合成得到目标双声道数据。In a possible example, in the aspect of synthesizing the plurality of mono data according to the plurality of first three-dimensional coordinates and the second three-dimensional coordinate to obtain target two-channel data, the determining unit 301 is further configured to determine the left ear three-dimensional coordinates and the right ear three-dimensional coordinates corresponding to the second three-dimensional coordinates; and determine the multiple according to the plurality of first three-dimensional coordinates, the left ear three-dimensional coordinates, and the right ear three-dimensional coordinates. A plurality of transmission paths are obtained from transmission paths between the sound sources and the target object; and each of the plurality of mono data is determined to be transmitted to the left ear three-dimensional coordinates according to the plurality of transmission paths. And time and phase pressure of the three-dimensional coordinate of the right ear to obtain multiple times and multiple phase pressures; determining a time difference corresponding to each of the plurality of mono data according to the plurality of times to obtain multiple Time difference, determining a phase pressure difference corresponding to each of the plurality of mono data according to the plurality of phase pressures to obtain a plurality of phase pressure differences; and according to the plurality of time differences and the plurality of phases, The bit pressure difference determines a delay parameter corresponding to each of the plurality of mono data to obtain a plurality of delay parameters; according to the plurality of delay parameters, for each of the plurality of mono data, The channel data is processed to obtain corresponding left channel parameters and right channel parameters; the synthesizing unit 302 is specifically configured to use the left channel parameters and the right channels corresponding to each of the plurality of mono data Channel parameters, synthesizing the plurality of mono data to obtain target two-channel data.
在一个可能的示例中,所述多个声源包括目标声源,所述多个第一三维坐标包括所述目标声源对应的目标第一三维坐标,所述多个单声道数据包括所述目标声源对应的目标单声道数据;In a possible example, the plurality of sound sources includes a target sound source, the plurality of first three-dimensional coordinates includes a target first three-dimensional coordinate corresponding to the target sound source, and the plurality of mono data includes all The target mono data corresponding to the target sound source;
在所述根据所述多个传输路径确定所述多个单声道数据中每一单声道数据传输至所述左耳三维坐标、所述右耳三维坐标的时间和相位压力得到多个时间和多个相位压力方面,所述确定单元301具体用于以所述目标第一三维坐标与所述左耳三维坐标为轴线得到横截面;确定所述目标第一三维坐标与所述左耳三维坐标之间的遮挡物体;根据所述横截面和所述遮挡物体确定所述目标单声道数据传输至所述左耳三维坐标的多个参考传输路径;根据所述多个参考传输路径确定所述目标单声道数据传输至所述左耳三维坐标的时间和相位压力。The time and phase pressure of each mono data transmission to the left ear three-dimensional coordinate and the right ear three-dimensional coordinate in the plurality of mono data determined according to the plurality of transmission paths to obtain a plurality of times With respect to multiple phase pressures, the determining unit 301 is specifically configured to obtain a cross-section using the target first three-dimensional coordinate and the left ear three-dimensional coordinate as an axis; determining the target first three-dimensional coordinate and the left ear three-dimensional Blocking objects between coordinates; determining a plurality of reference transmission paths for transmitting the target mono data to the three-dimensional coordinates of the left ear according to the cross section and the occluding objects; determining the reference transmission paths according to the plurality of reference transmission paths The time and phase pressure of the target mono data transmitted to the three-dimensional coordinates of the left ear.
在一个可能的示例中,在所述根据所述多个参考传输路径确定所述目标单声道数据传输至所述左耳三维坐标的时间和相位压力方面,所述确定单元301具体用于确定所述多个参考传输路径中每一传输路径对应的声强和声压得到多个声强和多个声压;根据所述多个声强确定所述目标单声道数据传输至所述左耳三维坐标的时间,根据所述多个声压确定所述目标单声道数据传输至所述左耳三维坐标的相位压力。In a possible example, in determining the time and phase pressure for transmitting the target mono data to the three-dimensional coordinates of the left ear according to the multiple reference transmission paths, the determining unit 301 is specifically configured to determine The sound intensity and sound pressure corresponding to each transmission path in the plurality of reference transmission paths obtain a plurality of sound intensity and a plurality of sound pressures; and determine the target mono data to be transmitted to the left according to the plurality of sound intensity. At the time of the three-dimensional coordinate of the ear, the phase pressure at which the target mono data is transmitted to the three-dimensional coordinate of the left ear is determined according to the plurality of sound pressures.
在一个可能的示例中,在所述确定电子设备对应的多个声源中每一声源的三维坐标、以及每一声源产生的单声道数据得到多个第一三维坐标和多个单声道数据方面,所述确定单元301具体用于确定电子设备对应的多个参考对象; 确定所述多个参考对象中每一参考对象的行为信息得到多个行为信息;根据所述多个行为信息确定所述多个参考对象中与所述电子设备对应的多个声源;确定所述多个声源中每一声源对应的坐标位置得到多个第一三维坐标;根据所述多个声源中每一声源对应的行为信息确定该声源的单声道数据得到多个单声道数据。In a possible example, the three-dimensional coordinates of each sound source in the plurality of sound sources corresponding to the electronic device and the mono data generated by each sound source are used to obtain multiple first three-dimensional coordinates and multiple mono channels. In terms of data, the determining unit 301 is specifically configured to determine multiple reference objects corresponding to the electronic device; determine behavior information of each reference object in the multiple reference objects to obtain multiple behavior information; and determine based on the multiple behavior information A plurality of sound sources corresponding to the electronic device in the plurality of reference objects; determining a coordinate position corresponding to each sound source in the plurality of sound sources to obtain a plurality of first three-dimensional coordinates; and according to the plurality of sound sources, The behavior information corresponding to each sound source determines the mono data of the sound source to obtain a plurality of mono data.
在一个可能的示例中,所述多个行为信息包括目标声源对应的目标行为信息;在所述根据所述多个声源中每一声源对应的行为信息确定该声源的单声道数据得到所述多个单声道数据方面,所述确定单元301具体用于确定所述目标行为信息对应的声音类型和播放参数;根据所述声音类型和所述播放参数生成所述目标声源的单声道数据。In a possible example, the multiple behavior information includes target behavior information corresponding to a target sound source; and determining the mono data of the sound source according to the behavior information corresponding to each sound source in the multiple sound sources In terms of obtaining the plurality of mono data, the determining unit 301 is specifically configured to determine a sound type and a playback parameter corresponding to the target behavior information; and generate a target sound source according to the sound type and the playback parameter. Mono data.
在一个可能的示例中,所述确定单元301还用于确定所述目标对象对应的目标混响参数;所述合成单元302还用于根据所述目标混响参数对所述目标双声道数据进行处理得到混响双声道数据。In a possible example, the determination unit 301 is further configured to determine a target reverberation parameter corresponding to the target object; and the synthesis unit 302 is further configured to perform the target two-channel data on the target reverberation parameter according to the target reverberation parameter. Process it to get reverberant two-channel data.
在一个可能的示例中,所述确定单元301具体用于获取预先存储的所述目标对象对应的多个历史混响播放记录;获取所述多个历史混响播放记录中每一历史混响播放记录对应的收听参数得到多个收听参数;根据所述多个收听参数确定所述目标对象对应的目标混响参数。In a possible example, the determining unit 301 is specifically configured to obtain multiple historical reverb play records corresponding to the target object stored in advance; obtain each historical reverb play in the multiple historical reverb play records Recording corresponding listening parameters to obtain multiple listening parameters; and determining a target reverberation parameter corresponding to the target object according to the multiple listening parameters.
在一个可能的示例中,所述确定单元301具体用于根据所述多个收听参数确定所述多个历史混响播放记录中每一历史混响播放记录对应的评价值得到多个评价值;将所述多个评价值中的最大值对应的历史混响记录作为目标历史混响记录,将所述目标历史混响记录对应的混响参数作为所述目标对象对应的目标混响参数。In a possible example, the determining unit 301 is specifically configured to determine an evaluation value corresponding to each historical reverb play record in the multiple historical reverb play records according to the multiple listening parameters to obtain multiple evaluation values; The historical reverberation record corresponding to the maximum value among the multiple evaluation values is used as the target historical reverberation record, and the reverberation parameter corresponding to the target historical reverberation record is used as the target reverberation parameter corresponding to the target object.
请参照图4,图4是本申请实施例提供的一种电子设备的结构示意图。如图4所示,该电子设备400包括处理器410、存储器420、通信接口430以及一个或多个程序440,其中,上述一个或多个程序440被存储在上述存储器420中,并且被配置由上述处理器410执行,上述程序440包括用于执行以下步骤的指令:Please refer to FIG. 4, which is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in FIG. 4, the electronic device 400 includes a processor 410, a memory 420, a communication interface 430, and one or more programs 440. The one or more programs 440 are stored in the memory 420, and are configured by The processor 410 executes, and the program 440 includes instructions for performing the following steps:
确定电子设备400对应的多个声源中每一声源的三维坐标、以及每一声源产生的单声道数据得到多个第一三维坐标和多个单声道数据;Determining the three-dimensional coordinates of each of the plurality of sound sources corresponding to the electronic device 400 and the mono data generated by each sound source to obtain a plurality of first three-dimensional coordinates and a plurality of mono data;
确定所述电子设备400对应的目标对象的第二三维坐标;Determining a second three-dimensional coordinate of a target object corresponding to the electronic device 400;
根据所述多个第一三维坐标和所述第二三维坐标,对所述多个单声道数据 进行合成得到目标双声道数据。And synthesizing the plurality of mono data according to the plurality of first three-dimensional coordinates and the second three-dimensional coordinate to obtain target binaural data.
在一个可能的示例中,在所述根据所述多个第一三维坐标和所述第二三维坐标,对所述多个单声道数据进行合成得到目标双声道数据方面,所述程序440中的指令具体用于执行以下操作:In a possible example, in terms of synthesizing the plurality of mono data according to the plurality of first three-dimensional coordinates and the second three-dimensional coordinate to obtain target two-channel data, the program 440 The instructions in are specifically used to perform the following operations:
确定所述第二三维坐标对应的左耳三维坐标和右耳三维坐标;Determining a left ear three-dimensional coordinate and a right ear three-dimensional coordinate corresponding to the second three-dimensional coordinate;
根据所述多个第一三维坐标、所述左耳三维坐标和所述右耳三维坐标确定所述多个声源与所述目标对象之间的传输路径得到多个传输路径;Determining transmission paths between the plurality of sound sources and the target object according to the plurality of first three-dimensional coordinates, the left ear three-dimensional coordinates, and the right ear three-dimensional coordinates to obtain a plurality of transmission paths;
根据所述多个传输路径确定所述多个单声道数据中每一单声道数据传输至所述左耳三维坐标、所述右耳三维坐标的时间和相位压力得到多个时间和多个相位压力;Determining, according to the plurality of transmission paths, the time and phase pressure of each of the plurality of mono data transmitted to the left ear three-dimensional coordinate and the right ear three-dimensional coordinate to obtain multiple times and multiple Phase pressure
根据所述多个时间确定所述多个单声道数据中每一单声道数据对应的时间差得到多个时间差,根据所述多个相位压力确定所述多个单声道数据中每一单声道数据对应的相位压力差得到多个相位压力差;Determining a time difference corresponding to each of the plurality of mono data according to the plurality of times to obtain a plurality of time differences, and determining each of the plurality of mono data according to the plurality of phase pressures The phase pressure difference corresponding to the channel data is used to obtain multiple phase pressure differences;
根据所述多个时间差和所述多个相位压力差确定所述多个单声道数据中每一单声道数据对应的延迟参数得到多个延迟参数;Determining, according to the plurality of time differences and the plurality of phase pressure differences, a delay parameter corresponding to each of the plurality of mono data to obtain a plurality of delay parameters;
根据所述多个延迟参数,对所述多个单声道数据中每一单声道数据进行处理得到对应的左声道参数和右声道参数;Processing each of the plurality of mono data according to the plurality of delay parameters to obtain corresponding left channel parameters and right channel parameters;
按照所述多个单声道数据中每一单声道数据对应的左声道参数和右声道参数,对所述多个单声道数据进行合成得到目标双声道数据。Synthesizing the plurality of mono data according to the left channel parameter and the right channel parameter corresponding to each of the plurality of mono data to obtain the target two-channel data.
在一个可能的示例中,所述多个声源包括目标声源,所述多个第一三维坐标包括所述目标声源对应的目标第一三维坐标,所述多个单声道数据包括所述目标声源对应的目标单声道数据;在所述根据所述多个传输路径确定所述多个单声道数据中每一单声道数据传输至所述左耳三维坐标的时间和相位压力得到多个时间和多个相位压力方面,所述程序440中的指令具体用于执行以下操作:In a possible example, the plurality of sound sources includes a target sound source, the plurality of first three-dimensional coordinates includes a target first three-dimensional coordinate corresponding to the target sound source, and the plurality of mono data includes all The target mono data corresponding to the target sound source; and determining the time and phase at which each mono data of the plurality of mono data is transmitted to the left ear three-dimensional coordinate according to the plurality of transmission paths The pressure obtains multiple time and multiple phase pressure aspects, and the instructions in the program 440 are specifically used to perform the following operations:
以所述目标第一三维坐标与所述左耳三维坐标为轴线得到横截面;Obtain a cross section by using the first three-dimensional coordinates of the target and the three-dimensional coordinates of the left ear as axes;
确定所述目标第一三维坐标与所述左耳三维坐标之间的遮挡物体;Determining an occlusion object between the target first three-dimensional coordinate and the left ear three-dimensional coordinate;
根据所述横截面和所述遮挡物体确定所述目标单声道数据传输至所述左耳三维坐标的多个参考传输路径;Determining a plurality of reference transmission paths for transmitting the target mono data to the three-dimensional coordinates of the left ear according to the cross section and the occluded object;
根据所述多个参考传输路径确定所述目标单声道数据传输至所述左耳三维坐标的时间和相位压力。Time and phase pressures for transmitting the target mono data to the three-dimensional coordinates of the left ear according to the multiple reference transmission paths.
在一个可能的示例中,在所述根据所述多个参考传输路径确定所述目标单 声道数据传输至所述左耳三维坐标的时间和相位压力方面,所述程序440中的指令具体用于执行以下操作:In a possible example, in determining the time and phase pressure for transmitting the target mono data to the three-dimensional coordinates of the left ear according to the multiple reference transmission paths, the instructions in the program 440 are specifically used To do the following:
确定所述多个参考传输路径中每一传输路径对应的声强和声压得到多个声强和多个声压;Determining a sound intensity and a sound pressure corresponding to each transmission path in the plurality of reference transmission paths to obtain a plurality of sound intensity and a plurality of sound pressures;
根据所述多个声强确定所述目标单声道数据传输至所述左耳三维坐标的时间,根据所述多个声压确定所述目标单声道数据传输至所述左耳三维坐标的相位压力。Determining the time at which the target mono data is transmitted to the left ear three-dimensional coordinates according to the plurality of sound intensities, and determining the time at which the target mono data is transmitted to the left ear three-dimensional coordinates according to the plurality of sound pressures Phase pressure.
在一个可能的示例中,在所述确定电子设备400对应的多个声源中每一声源的三维坐标、以及每一声源产生的单声道数据得到多个第一三维坐标和多个单声道数据方面,所述程序440中的指令具体用于执行以下操作:In a possible example, the three-dimensional coordinates of each sound source and the mono data generated by each sound source in the plurality of sound sources corresponding to the electronic device 400 are obtained to obtain a plurality of first three-dimensional coordinates and a plurality of mono sounds. In terms of data, the instructions in the program 440 are specifically used to perform the following operations:
确定所述电子设备400对应的多个参考对象;Determining a plurality of reference objects corresponding to the electronic device 400;
确定所述多个参考对象中每一参考对象的行为信息得到多个行为信息;Determining behavior information of each reference object in the multiple reference objects to obtain multiple behavior information;
根据所述多个行为信息确定所述多个参考对象中与所述电子设备对应的多个声源;Determining a plurality of sound sources corresponding to the electronic device among the plurality of reference objects according to the plurality of behavior information;
确定所述多个声源中每一声源对应的坐标位置得到多个第一三维坐标;Determining coordinate positions corresponding to each of the plurality of sound sources to obtain a plurality of first three-dimensional coordinates;
根据所述多个声源中每一声源对应的行为信息确定该声源的单声道数据得到多个单声道数据。The mono data of the sound source is determined according to the behavior information corresponding to each of the multiple sound sources to obtain multiple mono data.
在一个可能的示例中,所述多个行为信息包括目标声源对应的目标行为信息;在所述根据所述多个声源中每一声源对应的行为信息确定该声源的单声道数据得到所述多个单声道数据方面,所述程序440中的指令具体用于执行以下操作:In a possible example, the multiple behavior information includes target behavior information corresponding to a target sound source; and determining the mono data of the sound source according to the behavior information corresponding to each sound source in the multiple sound sources In terms of obtaining the plurality of mono data, the instructions in the program 440 are specifically used to perform the following operations:
确定所述目标行为信息对应的声音类型和播放参数;Determining a sound type and a playback parameter corresponding to the target behavior information;
根据所述声音类型和所述播放参数生成所述目标声源的单声道数据。Mono data of the target sound source is generated according to the sound type and the playback parameter.
在一个可能的示例中,所述程序440中的指令还用于执行以下操作:In a possible example, the instructions in the program 440 are further used to perform the following operations:
确定所述目标对象对应的目标混响参数;Determining a target reverberation parameter corresponding to the target object;
根据所述目标混响参数对所述目标双声道数据进行处理得到混响双声道数据。Processing the target two-channel data according to the target reverberation parameter to obtain reverberant two-channel data.
在一个可能的示例中,在所述根据所述目标混响参数对所述目标双声道数据进行处理得到混响双声道数据方面,所述程序440中的指令具体用于执行以下操作:In a possible example, in terms of processing the target two-channel data according to the target reverberation parameter to obtain reverberant two-channel data, the instructions in the program 440 are specifically used to perform the following operations:
获取预先存储的所述目标对象对应的多个历史混响播放记录;Acquiring multiple historical reverberation playback records corresponding to the target object stored in advance;
获取所述多个历史混响播放记录中每一历史混响播放记录对应的收听参数得到多个收听参数;Acquiring listening parameters corresponding to each historical reverb play record in the multiple historical reverb play records to obtain multiple listening parameters;
根据所述多个收听参数确定所述目标对象对应的目标混响参数。A target reverberation parameter corresponding to the target object is determined according to the plurality of listening parameters.
在一个可能的示例中,在所述根据所述多个收听参数确定所述目标对象对应的目标混响参数方面,所述程序440中的指令具体用于执行以下操作:In a possible example, in determining the target reverberation parameter corresponding to the target object according to the multiple listening parameters, the instructions in the program 440 are specifically used to perform the following operations:
根据所述多个收听参数确定所述多个历史混响播放记录中每一历史混响播放记录对应的评价值得到多个评价值;Determining, according to the plurality of listening parameters, an evaluation value corresponding to each historical reverb play record in the multiple historical reverb play records to obtain multiple evaluation values;
将所述多个评价值中的最大值对应的历史混响记录作为目标历史混响记录,将所述目标历史混响记录对应的混响参数作为所述目标对象对应的目标混响参数。The historical reverberation record corresponding to the maximum value among the multiple evaluation values is used as the target historical reverberation record, and the reverberation parameter corresponding to the target historical reverberation record is used as the target reverberation parameter corresponding to the target object.
基于同一发明构思,本申请实施例中提供的音效处理方法和电子设备解决问题的原理与本申请方法实施例相似,因此音效处理方法和电子设备的实施可以参见方法的实施,音效处理方法和电子设备的有益效果可以参见方法的有益效果,为简洁描述,在这里不再赘述。Based on the same inventive concept, the sound processing method and the electronic device provided in the embodiments of the present application are similar to the method embodiments of the present application. Therefore, the implementation of the sound processing method and the electronic device can refer to the method implementation, the sound processing method and the electronic For the beneficial effects of the device, refer to the beneficial effects of the method. For brevity description, they are not repeated here.
本申请实施例还提供一种计算机存储介质,其中,该计算机存储介质存储用于存储计算机程序,该计算机程序使得计算机执行如方法实施例中记载的任一方法的部分或全部步骤,计算机包括电子设备。An embodiment of the present application further provides a computer storage medium, wherein the computer storage medium stores a computer program for causing a computer to execute a part or all of the steps of any method as described in the method embodiment, and the computer includes an electronic device. device.
本申请实施例还提供一种计算机程序产品,计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质,计算机程序可操作来使计算机执行如方法实施例中记载的任一方法的部分或全部步骤。该计算机程序产品可以为一个软件安装包,计算机包括电子设备。An embodiment of the present application further provides a computer program product. The computer program product includes a non-transitory computer-readable storage medium storing the computer program, and the computer program is operable to cause a computer to execute a part of any method as described in the method embodiments. Or all steps. The computer program product may be a software installation package, and the computer includes an electronic device.
需要说明的是,对于前述的各方法实施例,为了简单描述,故将其都表述为一系列的动作组合,但是本领域技术人员应该知悉,本申请并不受所描述的动作顺序的限制,因为依据本申请,某些步骤可以采用其他顺序或者同时进行。其次,本领域技术人员也应该知悉,说明书中所描述的实施例均属于优选实施例,所涉及的动作和模式并不一定是本申请所必须的。It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all described as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action order. Because according to the present application, certain steps may be performed in another order or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modes involved are not necessarily required for this application.
在上述实施例中,对各个实施例的描述都各有侧重,某个实施例中没有详述的部分,可以参见其他实施例的相关描述。In the above embodiments, the description of each embodiment has its own emphasis. For a part that is not described in detail in one embodiment, reference may be made to related descriptions in other embodiments.
在本申请所提供的几个实施例中,应该理解到,所揭露的装置,可通过其它的方式实现。例如,以上所描述的装置实施例仅仅是示意性的,例如单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多 个单元或组件可以结合或者可以集成到另一个***,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,装置或单元的间接耦合或通信连接,可以是电性或其它的形式。In the several embodiments provided in this application, it should be understood that the disclosed device may be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. In actual implementation, there may be another division manner. For example, multiple units or components may be combined or integrated into Another system, or some features, can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be electrical or other forms.
作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, which may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objective of the solution of this embodiment.
另外,在本申请各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现,也可以采用软件程序模式的形式实现。In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each of the units may exist separately physically, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software program mode.
集成的单元如果以软件程序模式的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储器中。基于这样的理解,本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储器中,包括若干指令用以使得一台计算机设备(可为个人计算机、服务器或者网络设备等)执行本申请各个实施例方法的全部或部分步骤。而前述的存储器包括:U盘、只读存储器(read-only memory,ROM)、随机存取存储器(random access memory,RAM)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。When the integrated unit is implemented in the form of a software program and sold or used as an independent product, it can be stored in a computer-readable memory. Based on such an understanding, the technical solution of the present application essentially or part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory, Several instructions are included to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods in the embodiments of the present application. The foregoing memory includes: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk, and other media that can store program codes.
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成,该程序可以存储于一计算机可读存储器中,存储器可以包括:闪存盘、ROM、RAM、磁盘或光盘等。A person of ordinary skill in the art may understand that all or part of the steps in the various methods of the foregoing embodiments may be completed by a program instructing related hardware. The program may be stored in a computer-readable memory, and the memory may include a flash disk. , ROM, RAM, disk or disc, etc.
以上对本申请实施例进行了详细介绍,本文中应用了具体个例对本申请的原理及实施方式进行了阐述,以上实施例的说明只是用于帮助理解本申请的方法及其核心思想;同时,对于本领域的一般技术人员,依据本申请的思想,在具体实施方式及应用范围上均会有改变之处,综上,本说明书内容不应理解为对本申请的限制。The embodiments of the present application have been described in detail above. Specific examples have been used in this document to explain the principles and implementation of the present application. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present application. Persons of ordinary skill in the art may change the specific implementation manner and the scope of application according to the idea of the present application. In summary, the content of this specification should not be construed as a limitation on the present application.
Claims (20)
- 一种音效处理方法,其特征在于,包括:A sound effect processing method, comprising:确定电子设备对应的多个声源中每一声源的三维坐标、以及每一声源产生的单声道数据,得到多个第一三维坐标和多个单声道数据;Determining the three-dimensional coordinates of each of the plurality of sound sources corresponding to the electronic device and the mono data generated by each sound source to obtain a plurality of first three-dimensional coordinates and a plurality of mono data;确定所述电子设备对应的目标对象的第二三维坐标;Determining a second three-dimensional coordinate of a target object corresponding to the electronic device;根据所述多个第一三维坐标和所述第二三维坐标,对所述多个单声道数据进行合成,得到目标双声道数据。Synthesizing the plurality of mono data according to the plurality of first three-dimensional coordinates and the second three-dimensional coordinate to obtain target binaural data.
- 根据权利要求1所述的方法,其特征在于,所述根据所述多个第一三维坐标和所述第二三维坐标,对所述多个单声道数据进行合成,得到目标双声道数据,包括:The method according to claim 1, wherein the synthesizing the plurality of mono data according to the plurality of first three-dimensional coordinates and the second three-dimensional coordinate to obtain target two-channel data ,include:确定所述第二三维坐标对应的左耳三维坐标和右耳三维坐标;Determining a left ear three-dimensional coordinate and a right ear three-dimensional coordinate corresponding to the second three-dimensional coordinate;根据所述多个第一三维坐标、所述左耳三维坐标和所述右耳三维坐标确定所述多个声源与所述目标对象之间的传输路径,得到多个传输路径;Determining transmission paths between the plurality of sound sources and the target object according to the plurality of first three-dimensional coordinates, the left ear three-dimensional coordinates, and the right ear three-dimensional coordinates to obtain a plurality of transmission paths;根据所述多个传输路径确定所述多个单声道数据中每一单声道数据传输至所述左耳三维坐标、所述右耳三维坐标的时间和相位压力,得到多个时间和多个相位压力;Determining the time and phase pressure for transmitting each of the plurality of mono data to the three-dimensional coordinate of the left ear and the three-dimensional coordinate of the right ear according to the plurality of transmission paths to obtain a plurality of time and multiple Phase pressure根据所述多个时间确定所述多个单声道数据中每一单声道数据对应的时间差,得到多个时间差,根据所述多个相位压力确定所述多个单声道数据中每一单声道数据对应的相位压力差,得到多个相位压力差;Determining a time difference corresponding to each of the plurality of mono data according to the plurality of times, obtaining a plurality of time differences, and determining each of the plurality of mono data according to the plurality of phase pressures The phase pressure difference corresponding to the mono data to obtain multiple phase pressure differences;根据所述多个时间差和所述多个相位压力差确定所述多个单声道数据中每一单声道数据对应的延迟参数,得到多个延迟参数;Determining, according to the plurality of time differences and the plurality of phase pressure differences, a delay parameter corresponding to each of the plurality of mono data to obtain a plurality of delay parameters;根据所述多个延迟参数,对所述多个单声道数据中每一单声道数据进行处理,得到对应的左声道参数和右声道参数;Processing each of the plurality of mono data according to the plurality of delay parameters to obtain corresponding left channel parameters and right channel parameters;按照所述多个单声道数据中每一单声道数据对应的左声道参数和右声道参数,对所述多个单声道数据进行合成,得到目标双声道数据。Synthesizing the plurality of mono data according to the left channel parameter and the right channel parameter corresponding to each of the plurality of mono data to obtain the target two-channel data.
- 根据权利要求2所述的方法,其特征在于,所述多个声源包括目标声源,所述多个第一三维坐标包括所述目标声源对应的目标第一三维坐标,所述多个单声道数据包括所述目标声源对应的目标单声道数据;The method according to claim 2, wherein the plurality of sound sources include a target sound source, the plurality of first three-dimensional coordinates include a target first three-dimensional coordinate corresponding to the target sound source, and the plurality of The mono data includes target mono data corresponding to the target sound source;所述根据所述多个传输路径确定所述多个单声道数据中每一单声道数据传输至所述左耳三维坐标的时间和相位压力,包括:The determining the time and phase pressure of transmitting each of the plurality of mono data to the three-dimensional coordinate of the left ear according to the plurality of transmission paths includes:以所述目标第一三维坐标与所述左耳三维坐标为轴线,得到横截面;Obtain the cross section by taking the first three-dimensional coordinates of the target and the three-dimensional coordinates of the left ear as axes;确定所述目标第一三维坐标与所述左耳三维坐标之间的遮挡物体;Determining an occlusion object between the target first three-dimensional coordinate and the left ear three-dimensional coordinate;根据所述横截面和所述遮挡物体确定所述目标单声道数据传输至所述左耳三维坐标的多个参考传输路径;Determining a plurality of reference transmission paths for transmitting the target mono data to the three-dimensional coordinates of the left ear according to the cross section and the occluded object;根据所述多个参考传输路径确定所述目标单声道数据传输至所述左耳三维坐标的时间和相位压力。Time and phase pressures for transmitting the target mono data to the three-dimensional coordinates of the left ear according to the multiple reference transmission paths.
- 根据权利要求3所述的方法,其特征在于,所述根据所述多个参考传输路径确定所述目标单声道数据传输至所述左耳三维坐标的时间和相位压力,包括:The method according to claim 3, wherein determining the time and phase pressure for transmitting the target mono data to the three-dimensional coordinate of the left ear according to the multiple reference transmission paths comprises:确定所述多个参考传输路径中每一传输路径对应的声强和声压,得到多个声强和多个声压;Determining a sound intensity and a sound pressure corresponding to each of the plurality of reference transmission paths to obtain a plurality of sound intensity and a plurality of sound pressures;根据所述多个声强确定所述目标单声道数据传输至所述左耳三维坐标的时间,根据所述多个声压确定所述目标单声道数据传输至所述左耳三维坐标的相位压力。Determining the time at which the target mono data is transmitted to the left ear three-dimensional coordinates according to the plurality of sound intensities, and determining the time at which the target mono data is transmitted to the left ear three-dimensional coordinates according to the plurality of sound pressures Phase pressure.
- 根据权利要求1-4任一项所述的方法,其特征在于,所述确定电子设备对应的多个声源中每一声源的三维坐标、以及每一声源产生的单声道数据,得到多个第一三维坐标和多个单声道数据,包括:The method according to any one of claims 1-4, wherein the determining the three-dimensional coordinates of each sound source among the plurality of sound sources corresponding to the electronic device and the mono data generated by each sound source obtains multiple First three-dimensional coordinates and multiple mono data, including:确定电子设备对应的多个参考对象;Determining multiple reference objects corresponding to the electronic device;确定所述多个参考对象中每一参考对象的行为信息,得到多个行为信息;Determining behavior information of each reference object in the multiple reference objects to obtain multiple behavior information;根据所述多个行为信息确定所述多个参考对象中与所述电子设备对应的多个声源;Determining a plurality of sound sources corresponding to the electronic device among the plurality of reference objects according to the plurality of behavior information;确定所述多个声源中每一声源对应的坐标位置,得到多个第一三维坐标;Determining coordinate positions corresponding to each of the plurality of sound sources to obtain a plurality of first three-dimensional coordinates;根据所述多个声源中每一声源对应的行为信息确定该声源的单声道数据,得到多个单声道数据。Mono data of the sound source is determined according to behavior information corresponding to each of the multiple sound sources to obtain multiple mono data.
- 根据权利要求5所述的方法,其特征在于,所述多个行为信息包括所述多个声源中的目标声源对应的目标行为信息;The method according to claim 5, wherein the plurality of behavior information includes target behavior information corresponding to a target sound source among the plurality of sound sources;所述根据所述多个声源中每一声源对应的行为信息确定该声源的单声道数据,得到多个单声道数据,包括:The determining the mono data of the sound source according to the behavior information corresponding to each of the multiple sound sources, and obtaining the multiple mono data includes:确定所述目标行为信息对应的声音类型和播放参数;Determining a sound type and a playback parameter corresponding to the target behavior information;根据所述声音类型和所述播放参数生成所述目标声源的单声道数据。Mono data of the target sound source is generated according to the sound type and the playback parameter.
- 根据权利要求1-6任一项所述的方法,其特征在于,所述方法还包括:The method according to any one of claims 1-6, further comprising:确定所述目标对象对应的目标混响参数;Determining a target reverberation parameter corresponding to the target object;根据所述目标混响参数对所述目标双声道数据进行处理,得到混响双声道数据。And processing the target two-channel data according to the target reverberation parameter to obtain reverberated two-channel data.
- 根据权利要求7所述的方法,其特征在于,所述确定所述目标对象对应的目标混响参数,包括:The method according to claim 7, wherein the determining a target reverberation parameter corresponding to the target object comprises:获取预先存储的所述目标对象对应的多个历史混响播放记录;Acquiring multiple historical reverberation playback records corresponding to the target object stored in advance;获取所述多个历史混响播放记录中每一历史混响播放记录对应的收听参数,得到多个收听参数;Acquiring listening parameters corresponding to each historical reverberation playback record in the multiple historical reverberation playback records to obtain multiple listening parameters;根据所述多个收听参数确定所述目标对象对应的目标混响参数。A target reverberation parameter corresponding to the target object is determined according to the plurality of listening parameters.
- 根据权利要求8所述的方法,其特征在于,所述根据所述多个收听参数确定所述目标对象对应的目标混响参数,包括:The method according to claim 8, wherein determining the target reverberation parameter corresponding to the target object according to the plurality of listening parameters comprises:根据所述多个收听参数确定所述多个历史混响播放记录中每一历史混响播放记录对应的评价值,得到多个评价值;Determining an evaluation value corresponding to each historical reverb play record in the multiple historical reverb play records according to the multiple listening parameters, and obtaining multiple evaluation values;将所述多个评价值中的最大值对应的历史混响记录作为目标历史混响记录,将所述目标历史混响记录对应的混响参数作为所述目标对象对应的目标混响参数。The historical reverberation record corresponding to the maximum value among the multiple evaluation values is used as the target historical reverberation record, and the reverberation parameter corresponding to the target historical reverberation record is used as the target reverberation parameter corresponding to the target object.
- 一种音效处理装置,其特征在于,包括:A sound effect processing device, comprising:确定单元,用于确定电子设备对应的多个声源中每一声源的三维坐标、以及每一声源产生的单声道数据,得到多个第一三维坐标和多个单声道数据;确定所述电子设备对应的目标对象的第二三维坐标;A determining unit, configured to determine the three-dimensional coordinates of each of the plurality of sound sources corresponding to the electronic device and the mono data generated by each sound source, to obtain a plurality of first three-dimensional coordinates and a plurality of mono data; The second three-dimensional coordinate of the target object corresponding to the electronic device;合成单元,用于根据所述多个第一三维坐标和所述第二三维坐标,对所述多个单声道数据进行合成,得到目标双声道数据。A synthesizing unit is configured to synthesize the plurality of mono data according to the plurality of first three-dimensional coordinates and the second three-dimensional coordinate to obtain target two-channel data.
- 根据权利要求10所述的装置,其特征在于,所述确定单元还用于确定所述第二三维坐标对应的左耳三维坐标和右耳三维坐标;根据所述多个第一三维坐标、所述左耳三维坐标和所述右耳三维坐标确定所述多个声源与所述目标对象之间的传输路径,得到多个传输路径;根据所述多个传输路径确定所述多个单声道数据中每一单声道数据传输至所述左耳三维坐标、所述右耳三维坐标的时间和相位压力,得到多个时间和多个相位压力;根据所述多个时间确定所述多个单声道数据中每一单声道数据对应的时间差,得到多个时间差,根据所述多个相位压力确定所述多个单声道数据中每一单声道数据对应的相位压力差,得到多个相位压力差;根据所述多个时间差和所述多个相位压力差确定所述多个单声道数据中每一单声道数据对应的延迟参数,得到多个延迟参数;根据所 述多个延迟参数,对所述多个单声道数据中每一单声道数据进行处理,得到对应的左声道参数和右声道参数;The device according to claim 10, wherein the determining unit is further configured to determine a left ear three-dimensional coordinate and a right ear three-dimensional coordinate corresponding to the second three-dimensional coordinate; The three-dimensional coordinates of the left ear and the three-dimensional coordinates of the right ear determine transmission paths between the plurality of sound sources and the target object to obtain a plurality of transmission paths; and determine the plurality of mono sounds according to the plurality of transmission paths. Time and phase pressure of each mono data in the channel data transmitted to the three-dimensional coordinates of the left ear and the three-dimensional coordinates of the right ear to obtain multiple times and multiple phase pressures; determine the multiple based on the multiple times A time difference corresponding to each mono data in the mono data to obtain a plurality of time differences, and determine a phase pressure difference corresponding to each mono data in the plurality of mono data according to the plurality of phase pressures, Obtaining a plurality of phase pressure differences; determining a delay parameter corresponding to each of the plurality of mono data according to the plurality of time differences and the plurality of phase pressure differences, and obtaining a plurality of delay parameters The delay parameters of said plurality of said plurality of single data channels each single channel data is processed to obtain parameters corresponding to the left channel and a right channel parameter;所述合成单元具体用于按照所述多个单声道数据中每一单声道数据对应的左声道参数和右声道参数,对所述多个单声道数据进行合成,得到目标双声道数据。The synthesizing unit is specifically configured to synthesize the plurality of mono data according to a left channel parameter and a right channel parameter corresponding to each of the plurality of mono data to obtain a target dual Channel data.
- 根据权利要求11所述的装置,其特征在于,所述多个声源包括目标声源,所述多个第一三维坐标包括所述目标声源对应的目标第一三维坐标,所述多个单声道数据包括所述目标声源对应的目标单声道数据;所述确定单元具体用于以所述目标第一三维坐标与所述左耳三维坐标为轴线,得到横截面;确定所述目标第一三维坐标与所述左耳三维坐标之间的遮挡物体;根据所述横截面和所述遮挡物体确定所述目标单声道数据传输至所述左耳三维坐标的多个参考传输路径;根据所述多个参考传输路径确定所述目标单声道数据传输至所述左耳三维坐标的时间和相位压力。The device according to claim 11, wherein the plurality of sound sources include a target sound source, the plurality of first three-dimensional coordinates include a target first three-dimensional coordinate corresponding to the target sound source, and the plurality of The mono data includes target mono data corresponding to the target sound source; the determining unit is specifically configured to obtain a cross section by using the target first three-dimensional coordinate and the left ear three-dimensional coordinate as an axis; determining the An occlusion object between a first three-dimensional coordinate of the target and the three-dimensional coordinate of the left ear; determining a plurality of reference transmission paths for transmitting the target mono data to the three-dimensional coordinate of the left ear according to the cross section and the occluded object Determining, according to the multiple reference transmission paths, the time and phase pressure at which the target mono data is transmitted to the three-dimensional coordinates of the left ear.
- 根据权利要求12所述的装置,其特征在于,所述确定单元具体用于确定所述多个参考传输路径中每一传输路径对应的声强和声压,得到多个声强和多个声压;根据所述多个声强确定所述目标单声道数据传输至所述左耳三维坐标的时间,根据所述多个声压确定所述目标单声道数据传输至所述左耳三维坐标的相位压力。The device according to claim 12, wherein the determining unit is specifically configured to determine a sound intensity and a sound pressure corresponding to each transmission path in the multiple reference transmission paths, to obtain multiple sound intensities and multiple sounds Pressure; determining the time when the target mono data is transmitted to the left ear three-dimensional coordinates according to the plurality of sound intensities, and determining the target mono data transmission to the left ear three-dimensional coordinates according to the plurality of sound pressures Phase pressure of the coordinates.
- 根据权利要求10-13任一项所述的装置,其特征在于,所述确定单元具体用于确定电子设备对应的多个参考对象;确定所述多个参考对象中每一参考对象的行为信息,得到多个行为信息;根据所述多个行为信息确定所述多个参考对象中与所述电子设备对应的多个声源;确定所述多个声源中每一声源对应的坐标位置,得到多个第一三维坐标;根据所述多个声源中每一声源对应的行为信息确定该声源的单声道数据,得到多个单声道数据。The device according to any one of claims 10-13, wherein the determining unit is specifically configured to determine a plurality of reference objects corresponding to the electronic device; determine behavior information of each reference object in the plurality of reference objects To obtain a plurality of behavior information; determine a plurality of sound sources corresponding to the electronic device in the plurality of reference objects according to the plurality of behavior information; determine a coordinate position corresponding to each sound source in the plurality of sound sources, Obtaining multiple first three-dimensional coordinates; determining mono data of the sound source according to behavior information corresponding to each of the multiple sound sources, and obtaining multiple mono data.
- 根据权利要求14所述的装置,其特征在于,所述多个行为信息包括目标声源对应的目标行为信息;所述确定单元具体用于确定所述目标行为信息对应的声音类型和播放参数;根据所述声音类型和所述播放参数生成所述目标声源的单声道数据。The device according to claim 14, wherein the plurality of behavior information includes target behavior information corresponding to a target sound source; and the determining unit is specifically configured to determine a sound type and a playback parameter corresponding to the target behavior information; Mono data of the target sound source is generated according to the sound type and the playback parameter.
- 根据权利要求10-15任一项所述的装置,其特征在于,所述确定单元还用于确定所述目标对象对应的目标混响参数;The device according to any one of claims 10 to 15, wherein the determining unit is further configured to determine a target reverberation parameter corresponding to the target object;所述合成单元还用于根据所述目标混响参数对所述目标双声道数据进行处 理,得到混响双声道数据。The synthesis unit is further configured to process the target two-channel data according to the target reverberation parameter to obtain reverberated two-channel data.
- 根据权利要求16所述的装置,其特征在于,所述确定单元具体用于获取预先存储的所述目标对象对应的多个历史混响播放记录;获取所述多个历史混响播放记录中每一历史混响播放记录对应的收听参数,得到多个收听参数;根据所述多个收听参数确定所述目标对象对应的目标混响参数。The device according to claim 16, wherein the determining unit is specifically configured to obtain a plurality of historical reverb play records corresponding to the target object stored in advance; and acquire each of the plurality of historical reverb play records. A listening parameter corresponding to a historical reverberation playback record is used to obtain multiple listening parameters; and a target reverberation parameter corresponding to the target object is determined according to the multiple listening parameters.
- 一种电子设备,其特征在于,包括处理器、存储器、通信接口以及一个或多个程序,其中,所述一个或多个程序被存储在所述存储器中,并且被配置由所述处理器执行,所述程序包括用于执行权利要求1-9任一项方法中的步骤的指令。An electronic device, comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor The program includes instructions for performing the steps in the method of any one of claims 1-9.
- 一种计算机可读存储介质,其特征在于,其用于存储计算机程序,其中,所述计算机程序使得计算机执行如权利要求1-9任一项所述的方法。A computer-readable storage medium, which is used to store a computer program, wherein the computer program causes a computer to execute the method according to any one of claims 1-9.
- 一种计算机程序产品,其特征在于,所述计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质,所述计算机程序可操作来使计算机执行如权利要求1-9任一项所述的方法。A computer program product, characterized in that the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute the computer program according to any one of claims 1-9. The method described.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811118269.4 | 2018-09-25 | ||
CN201811118269.4A CN109246580B (en) | 2018-09-25 | 2018-09-25 | 3D sound effect processing method and related product |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020062922A1 true WO2020062922A1 (en) | 2020-04-02 |
Family
ID=65056811
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2019/090380 WO2020062922A1 (en) | 2018-09-25 | 2019-06-06 | Sound effect processing method and related product |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN109246580B (en) |
WO (1) | WO2020062922A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114594480A (en) * | 2022-03-11 | 2022-06-07 | 北京女娲补天科技信息技术有限公司 | Throwing item testing method and device based on sound wave positioning |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108924705B (en) * | 2018-09-25 | 2021-07-02 | Oppo广东移动通信有限公司 | 3D sound effect processing method and related product |
CN109246580B (en) * | 2018-09-25 | 2022-02-11 | Oppo广东移动通信有限公司 | 3D sound effect processing method and related product |
CN113467603B (en) * | 2020-03-31 | 2024-03-08 | 抖音视界有限公司 | Audio processing method and device, readable medium and electronic equipment |
CN116389982A (en) * | 2023-05-19 | 2023-07-04 | 零束科技有限公司 | Audio processing method, device, electronic equipment and storage medium |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060133628A1 (en) * | 2004-12-01 | 2006-06-22 | Creative Technology Ltd. | System and method for forming and rendering 3D MIDI messages |
CN103218198A (en) * | 2011-08-12 | 2013-07-24 | 索尼电脑娱乐公司 | Sound localization for user in motion |
CN104869524A (en) * | 2014-02-26 | 2015-08-26 | 腾讯科技(深圳)有限公司 | Processing method and device for sound in three-dimensional virtual scene |
CN109246580A (en) * | 2018-09-25 | 2019-01-18 | Oppo广东移动通信有限公司 | 3D sound effect treatment method and Related product |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105159655B (en) * | 2014-05-28 | 2020-04-24 | 腾讯科技(深圳)有限公司 | Behavior event playing method and device |
CN106448687B (en) * | 2016-09-19 | 2019-10-18 | 中科超影(北京)传媒科技有限公司 | Audio production and decoded method and apparatus |
CN107360494A (en) * | 2017-08-03 | 2017-11-17 | 北京微视酷科技有限责任公司 | A kind of 3D sound effect treatment methods, device, system and sound system |
CN107608519A (en) * | 2017-09-26 | 2018-01-19 | 深圳传音通讯有限公司 | A kind of sound method of adjustment and virtual reality device |
-
2018
- 2018-09-25 CN CN201811118269.4A patent/CN109246580B/en active Active
-
2019
- 2019-06-06 WO PCT/CN2019/090380 patent/WO2020062922A1/en active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060133628A1 (en) * | 2004-12-01 | 2006-06-22 | Creative Technology Ltd. | System and method for forming and rendering 3D MIDI messages |
CN103218198A (en) * | 2011-08-12 | 2013-07-24 | 索尼电脑娱乐公司 | Sound localization for user in motion |
CN104869524A (en) * | 2014-02-26 | 2015-08-26 | 腾讯科技(深圳)有限公司 | Processing method and device for sound in three-dimensional virtual scene |
CN109246580A (en) * | 2018-09-25 | 2019-01-18 | Oppo广东移动通信有限公司 | 3D sound effect treatment method and Related product |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114594480A (en) * | 2022-03-11 | 2022-06-07 | 北京女娲补天科技信息技术有限公司 | Throwing item testing method and device based on sound wave positioning |
Also Published As
Publication number | Publication date |
---|---|
CN109246580A (en) | 2019-01-18 |
CN109246580B (en) | 2022-02-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10911882B2 (en) | Methods and systems for generating spatialized audio | |
WO2020062922A1 (en) | Sound effect processing method and related product | |
US9693170B2 (en) | Multidimensional virtual learning system and method | |
US11109177B2 (en) | Methods and systems for simulating acoustics of an extended reality world | |
EP3629145B1 (en) | Method for processing 3d audio effect and related products | |
CN109254752B (en) | 3D sound effect processing method and related product | |
WO2020063028A1 (en) | 3d sound effect processing method and related product | |
WO2020063037A1 (en) | 3d sound effect processing method and related product | |
CN109104687B (en) | Sound effect processing method and related product | |
WO2020063027A1 (en) | 3d sound effect processing method and related product | |
CN115834775A (en) | Online call management device and storage medium storing online call management program | |
CN109243413B (en) | 3D sound effect processing method and related product | |
WO2022227921A1 (en) | Audio processing method and apparatus, wireless headset, and computer readable medium | |
CN117676002A (en) | Audio processing method and electronic equipment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19866526 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 19866526 Country of ref document: EP Kind code of ref document: A1 |