WO2014178479A1 - 헤드 마운트 디스플레이 및 이를 이용한 오디오 콘텐츠 제공 방법 - Google Patents
헤드 마운트 디스플레이 및 이를 이용한 오디오 콘텐츠 제공 방법 Download PDFInfo
- Publication number
- WO2014178479A1 WO2014178479A1 PCT/KR2013/004990 KR2013004990W WO2014178479A1 WO 2014178479 A1 WO2014178479 A1 WO 2014178479A1 KR 2013004990 W KR2013004990 W KR 2013004990W WO 2014178479 A1 WO2014178479 A1 WO 2014178479A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- hmd
- audio signal
- virtual
- present
- sound
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
- H04S7/303—Tracking of listener position or orientation
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/02—Casings; Cabinets ; Supports therefor; Mountings therein
- H04R1/028—Casings; Cabinets ; Supports therefor; Mountings therein associated with devices performing functions other than acoustics, e.g. electric candles
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S1/00—Two-channel systems
- H04S1/002—Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution
- H04S1/005—For headphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S5/00—Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
- H04S7/303—Tracking of listener position or orientation
- H04S7/304—For headphones
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2499/00—Aspects covered by H04R or H04S not otherwise provided for in their subgroups
- H04R2499/10—General applications
- H04R2499/15—Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
- H04S2400/11—Positioning of individual sound objects, e.g. moving airplane, within a sound field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
- H04S2400/15—Aspects of sound capture and related signal processing for recording or reproduction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
- H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
Definitions
- the present invention relates to a head mounted display (HMD) and a method for providing audio content using the same, and more particularly, an HMD for augmenting and providing a virtual audio signal adaptively according to a listening environment of a live audio signal. And an audio content providing method using the same.
- HMD head mounted display
- Head mounted display refers to various digital devices that can be worn on the head like glasses to provide multimedia content.
- various wearable computers have been developed, and the HMD is also widely used.
- HMD can provide various convenience to users by combining with augmented reality technology and N screen technology beyond simple display function.
- augmented reality technologies are mostly visual aspects of synthesizing virtual images with real images of the real world.
- the HMD when equipped with an audio output unit, it can provide not only the existing visually oriented augmented reality but also the auditory centric augmented reality. At this time, a technique for realistically augmenting a virtual audio signal to a user is needed.
- the present invention has an object to provide augmented reality audio to the user wearing the HMD.
- One object of the present invention is to provide a user by harmoniously mixing a real sound and a virtual audio signal.
- Another object of the present invention is to generate a new audio content in real time by separating the sound source of the received real sound.
- an audio content providing method using a head mounted display receiving a real sound using a microphone unit; Obtaining a virtual audio signal; Extracting a spatial acoustic parameter using the received real sound; Filtering the virtual audio signal using the extracted spatial acoustic parameters; And outputting the filtered virtual audio signal.
- HMD head mounted display
- a head mounted display according to an embodiment of the present invention, the processor for controlling the operation of the HMD; A microphone unit for receiving real sound; And an audio output unit outputting a sound based on a command of the processor, wherein the processor receives the real sound using the microphone unit, obtains a virtual audio signal, and uses the received real sound. And extracting a spatial acoustic parameter, filtering the virtual audio signal using the extracted spatial acoustic parameter, and outputting the filtered virtual audio signal to the audio output unit.
- HMD head mounted display
- the audio content may be provided based on the location of the user.
- the present invention may enable a user to listen to the audio content in a realistic sense.
- new audio content when recording real sound, can be generated by simultaneously recording a virtual audio signal together in real time.
- FIG. 1 is a block diagram illustrating an HMD according to an embodiment of the present invention.
- FIG. 2 is a flowchart illustrating a method of providing audio content according to an embodiment of the present invention.
- FIG. 3 is a flowchart illustrating a method of providing audio content according to another embodiment of the present invention.
- FIG. 4 is a flowchart illustrating a method of generating audio content according to an embodiment of the present invention.
- 5 to 8 illustrate in detail a method for providing audio content according to an embodiment of the present invention.
- FIG 9 illustrates in detail a method for generating audio content according to an embodiment of the present invention.
- FIG 10 and 11 are views showing the output of the audio signal of the same content in different environments according to an embodiment of the present invention.
- FIG. 1 is a block diagram illustrating an HMD 100 according to an embodiment of the present invention.
- the HMD 100 of the present invention includes a processor 110, a display unit 120, an audio output unit 130, a communication unit 140, a sensor unit 150, and a storage unit 160. It may include.
- the display unit 120 outputs an image on the display screen.
- the display unit 120 outputs content executed in the processor 110 or outputs an image based on a control command of the processor 110.
- the display unit 120 may display an image based on a control command of the external digital device 200 connected to the HMD 100.
- the display unit 120 may display content being executed by the external digital device 200 connected to the HMD 100.
- the HMD 100 may receive data from the external digital device 200 through the communication unit 140 and output an image based on the received data.
- the audio output unit 130 includes audio output means such as speakers and earphones, and a control module for controlling them.
- the audio output unit 130 outputs a voice based on content executed in the processor 110 or a control command of the processor 110.
- the audio output unit 130 of the HMD 100 may include a left channel output unit (not shown) and a right channel output unit (not shown).
- the left channel output unit and the right channel output unit output the left channel and the right channel of the audio signal, respectively.
- the audio output unit 130 may output the audio signal of the external digital device 200 connected to the HMD 100.
- the communication unit 140 may communicate with the external digital device 200 or the server using various protocols to transmit / receive data.
- the communication unit 140 may access a server or a cloud through a network, and transmit / receive digital data, for example, content.
- the HMD 100 may connect to the external digital device 200 using the communication unit 140.
- the HMD 100 may receive display output information of content being executed by the connected external digital device 200 in real time, and output an image to the display unit 120 using the received information.
- the HMD 100 may receive an audio signal of the content being executed by the connected external digital device 200 in real time and output the received audio signal to the audio output unit 130.
- the sensor unit 150 may transmit a user input or an environment recognized by the HMD 100 to the processor 110 using at least one sensor mounted on the HMD 100.
- the sensor unit 150 may include a plurality of sensing means.
- the plurality of sensing means includes a gravity sensor, a geomagnetic sensor, a motion sensor, a gyro sensor, an acceleration sensor, an infrared sensor, an inclination sensor, an illuminance sensor, a proximity sensor, an altitude sensor, an olfactory sensor, a temperature Sensing means such as a sensor, a depth sensor, a pressure sensor, a bending sensor, an audio sensor, a video sensor, a global positioning system (GPS) sensor, a touch sensor, and the like may be included.
- GPS global positioning system
- the sensor unit 150 collectively refers to the various sensing means described above, and may sense various inputs of the user and the environment of the user, and may transmit a sensing result to allow the processor 110 to perform an operation accordingly.
- the above-described sensors may be included in the HMD 100 as a separate element or integrated into at least one or more elements.
- the sensor unit 150 may include a microphone unit 152.
- the microphone unit 152 receives the real sound around the HMD 100 and delivers it to the processor 110.
- the microphone unit 152 may convert the real sound into an audio signal and transmit the converted sound to the processor 110.
- the microphone unit 152 may include a microphone array having a plurality of microphones.
- the storage unit 160 may store digital data including various contents such as video, audio, photographs, documents, applications, and the like.
- the storage unit 150 may include various digital data storage media such as flash memory, random access memory (RAM), and solid state drive (SSD).
- the storage unit 150 may store the content received by the communication unit 140 from the external digital device 200 or the server.
- the processor 110 of the present invention may execute contents of the HMD 100 itself or contents received through data communication. It can also run various applications and process data inside the device. In addition, the processor 110 may control each unit of the HMD 100 described above, and may control data transmission and reception between the units.
- the HMD 100 may be connected to at least one external digital device 200 and operate based on a control command of the connected external digital device 200.
- the external digital device 200 includes various types of digital devices capable of controlling the HMD 100.
- the external digital device 200 includes a smart phone, a PC, a personal digital assistant (PDA), a notebook, a tablet PC, a media player, and the like, and various types of digital devices capable of controlling the operation of the HMD. It includes.
- the HMD 100 transmits / receives data with the external digital device 200 using various wired / wireless communication means.
- usable wireless communication means include NFC (Near Field Communication), Zigbee, infrared communication, Bluetooth, Wi-Fi, the present invention is not limited to this.
- the HMD 100 may be connected to the external digital device 200 to perform communication by using any one of the above-mentioned communication means or a combination thereof.
- the HMD 100 shown in FIG. 1 is a block diagram according to an embodiment of the present invention, in which blocks shown separately represent logically distinguishing elements of a device. Therefore, the elements of the above-described device may be mounted in one chip or in a plurality of chips according to the design of the device.
- FIG. 2 is a flowchart illustrating a method of providing audio content according to an embodiment of the present invention.
- Each step of FIG. 2 described below may be performed by the HMD of the present invention. That is, the processor 110 of the HMD 100 shown in FIG. 1 may control each step of FIG. 2.
- the HMD 100 is controlled by the external digital device 200 according to another embodiment of the present invention, the HMD 100 based on the control command of the corresponding external digital device 200, each step of FIG. Can be performed.
- the HMD of the present invention receives the real sound using the microphone unit (S210).
- said microphone unit comprises a single microphone or microphone array.
- the microphone unit converts the received real sound into an audio signal and delivers it to the processor.
- the HMD of the present invention acquires a virtual audio signal (S220).
- the virtual audio signal includes augmented reality audio information for providing to the user wearing the HMD according to an embodiment of the present invention.
- the virtual audio signal may be obtained based on the real sound received in step S210. That is, the HMD may analyze the received real sound to obtain a virtual audio signal corresponding to the real sound.
- the HMD may obtain the virtual audio signal from a storage unit or from a server through a communication unit.
- the HMD of the present invention extracts the spatial acoustic parameters using the received real sound (S230).
- the spatial acoustic parameters are information representing room acoustics of the environment in which the real sound is received, and various characteristics related to room acoustics such as reverberation time, transmission frequency characteristics, cutoff performance, etc. May contain information.
- the spatial acoustic parameter may include the following information.
- the spatial acoustic parameter may include a room impulse response (RIR).
- the room shock response is the sound pressure measured at the position of the listener when the sound source is excited as an impulse function.
- Techniques for modeling indoor shock response include a variety of models, including an all-zero model based on finite impulse response (FIR) and a pole-zero model based on finite impulse response (IIR).
- the HMD of the present invention filters the virtual audio signal using the extracted spatial acoustic parameters (S240).
- the HMD of the present invention may generate a filter using at least one of the spatial acoustic parameters extracted in step S230.
- the HMD filters the virtual audio signal using the generated filter, thereby applying the characteristics of the spatial acoustic parameters extracted in operation S230 to the virtual audio signal. Therefore, the HMD of the present invention can provide the virtual audio signal to the user with the same effect as the environment in which the real sound is received.
- the HMD of the present invention outputs the filtered virtual audio signal (S250).
- the HMD of the present invention can output the filtered virtual audio signal to an audio output unit.
- the HMD can adjust the reproduction property of the virtual audio signal using the real sound received in operation S210.
- This playback attribute includes at least one of a playback pitch and a playback tempo.
- the HMD may acquire the position of the virtual sound source of the virtual audio signal. The position of the virtual sound source may be specified by a user wearing the HMD or acquired together as additional data when the virtual audio signal is acquired.
- the HMD of the present invention may convert the virtual audio signal into a 3D audio signal based on the acquired position of the virtual sound source, and output the converted 3D audio signal.
- the 3D audio signal includes a binaural audio signal having a 3D effect.
- the HMD may generate Head Related Transfer Function (HRTF) information based on the location information of the virtual sound source, and convert the virtual audio signal into a 3D audio signal using the generated HRTF information.
- the HRTF refers to a transfer function between a sound wave coming from a sound source having an arbitrary position and a sound wave reaching the eardrum, and varying in accordance with azimuth information and altitude information of the sound source. If an audio signal having no directivity (i.e., directivity) is filtered with a HRTF in a specific direction, the user wearing the HMD feels as if the sound is heard in the specific direction.
- the HMD may perform the operation of converting the virtual audio signal into the 3D audio signal before or after step S240.
- the HMD may generate a filter in which the spatial acoustic parameter extracted in step S230 and the HRTF are integrated, and filter the virtual audio signal through the integrated filter and output the filtered signal.
- FIG. 3 is a flowchart illustrating a method of providing audio content according to another embodiment of the present invention.
- Each step of FIG. 3 described below may be performed by the HMD of the present invention. That is, the processor 110 of the HMD 100 shown in FIG. 1 may control each step of FIG. 3.
- the same or corresponding parts as those of the embodiment of FIG. 2 will be omitted.
- the HMD of the present invention acquires position information of the HMD (S310).
- the HMD may include a GPS sensor, and may acquire location information of the HMD using the GPS sensor.
- the HMD may obtain location information based on a network service such as Wi-Fi.
- the HMD of the present invention acquires audio content of at least one sound source using the obtained location information (S320).
- the audio content includes augmented reality audio content for providing to a user wearing the HMD according to an embodiment of the present invention.
- the HMD may acquire audio content of a sound source located around the HMD from a server or a cloud based on the location information of the HMD. That is, when the HMD transmits the location information to the server or the cloud, the server or the cloud searches for audio content of a sound source located near the HMD using the location information as query information.
- the server or cloud can send the retrieved audio content to the HMD.
- a plurality of sound sources may exist around where the HMD is located, and the HMD may acquire audio contents of the plurality of sound sources together.
- the HMD of the present invention acquires the spatial acoustic parameter for the audio content using the obtained position information (S330).
- the spatial sound parameter is information for realistically outputting the audio content according to a real environment, and may include various kinds of characteristic information as described above with reference to step S230 of FIG. 2.
- the spatial acoustic parameter may be determined based on the distance information between the sound source and the HMD and the obstacle information.
- the obstacle information is information on various obstacles (eg, buildings, etc.) that prevent sound transmission between the sound source and the HMD, and may be obtained from map data based on the location information of the HMD.
- the spatial acoustic parameters may be predicted based on the distance information and the obstacle information, and the HMD may obtain the predicted values as the spatial acoustic parameters.
- the HMD of the present invention can obtain a plurality of spatial acoustic parameters respectively corresponding to the plurality of sound sources.
- the HMD of the present invention filters the audio content using the obtained spatial acoustic parameters (S340).
- the HMD of the present invention may generate a filter using at least one of the spatial acoustic parameters obtained in operation S330.
- the HMD filters the audio content using the generated filter, thereby applying the characteristics of the spatial acoustic parameters obtained in operation S330 to the audio content. Therefore, the HMD of the present invention can provide the audio content to the user with the same effect as the environment in which the real sound is received. If the HMD acquires audio content of a plurality of sound sources, the HMD may filter the obtained plurality of audio contents with corresponding spatial acoustic parameters, respectively.
- the HMD of the present invention outputs the filtered audio content (S350).
- the HMD of the present invention can output the filtered audio content to an audio output unit.
- the HMD can acquire direction information of the sound source based on the HMD.
- the direction information includes azimuth information of a sound source based on the HMD.
- the HMD may acquire the direction information by using the position information of the sound source and the sensing value of the gyro sensor of the HMD.
- the HMD of the present invention can convert audio content into a 3D audio signal based on the obtained direction information and distance information between the sound source and the HMD, and output the converted 3D audio signal. More specifically, the HMD may generate Head Related Transfer Function (HRTF) information based on the direction information and distance information, and convert the audio content into a 3D audio signal using the generated HRTF information.
- HRTF Head Related Transfer Function
- the HMD may perform the operation before or after the step S340 of converting the audio content into the 3D audio signal.
- the HMD may generate a filter in which the spatial acoustic parameter extracted in step S330 and the HRTF are integrated, and filter and output audio content with the integrated filter.
- the HMD may further acquire time information for providing audio content. Even in the same place, different sound sources may exist depending on time.
- the HMD of the present invention can obtain viewpoint information through a user's input and the like, and can obtain audio content using the viewpoint information. That is, the HMD may acquire audio content of at least one sound source by using the viewpoint information and the position information of the HMD together. Therefore, the HMD of the present invention can obtain a sound source of a specific time zone of a specific place and provide it to the user.
- FIG. 4 is a flowchart illustrating a method of generating audio content according to an embodiment of the present invention.
- Each step of FIG. 4 described below may be performed by the HMD of the present invention. That is, the processor 110 of the HMD 100 shown in FIG. 1 may control each step of FIG. 4.
- the present invention is not limited thereto, and each step of FIG. 4 may be performed by various types of portable devices including an HMD.
- the same or corresponding parts as those of the embodiment of FIG. 2 will be omitted.
- the HMD of the present invention receives the real sound using the microphone unit (S410).
- said microphone unit comprises a single microphone or microphone array.
- the microphone unit converts the received real sound into an audio signal and delivers it to the processor.
- the HMD of the present invention acquires a virtual audio signal corresponding to real sound (S420).
- the virtual audio signal includes augmented reality audio information for providing to the user wearing the HMD according to an embodiment of the present invention.
- the virtual audio signal may be obtained based on the real sound received in step S410. That is, the HMD may analyze the received real sound to obtain a virtual audio signal corresponding to the real sound.
- the HMD may obtain the virtual audio signal from a storage unit or from a server through a communication unit.
- the HMD of the present invention separates the received real sound into at least one sound source signal (S430).
- the received real sound may include one or more sound source signals, and the HMD separates the real sound into at least one sound source signal based on the position of the individual sound sources.
- the microphone unit of the HMD may include a microphone array, and the sound source signal may be separated by using a time difference, a sound pressure difference, and the like of the real sound received by each microphone of the microphone array.
- the HMD of the present invention sets a sound source signal to be replaced among the separated at least one sound source signal (S440).
- the HMD may replace some or all of the plurality of sound source signals included in the real sound with a virtual audio signal to record.
- the user may select the sound source signal to be replaced using various interfaces.
- the HMD may display a visual object corresponding to each of the extracted sound source signals with the display unit, and the user may select a specific visual object among the displayed visual objects to select the replacement sound source signal.
- the HMD sets the sound source signal selected by the user as the sound source signal to be replaced.
- the HMD of the present invention replaces the set sound source signal with a virtual audio signal to record (S450).
- the HMD of the present invention may record an audio signal including the received real sound, but replace the audio signal with the virtual audio signal except for the set sound source signal. Accordingly, the HMD of the present invention can generate new audio content in which the received real sound and the virtual audio signal are combined.
- the HMD may adjust and record the reproduction property of the virtual audio signal based on the real sound received in operation S410. This playback attribute includes at least one of a playback pitch and a playback tempo.
- the HMD may acquire the position of the virtual sound source of the virtual audio signal.
- the position of the virtual sound source may be specified by a user wearing the HMD or acquired together as additional data when the virtual audio signal is acquired.
- the position of the virtual sound source may be determined based on the position of the object corresponding to the sound source signal to be replaced.
- the HMD of the present invention may convert the virtual audio signal into a 3D audio signal based on the acquired position of the virtual sound source, and output the converted 3D audio signal. More specifically, the HMD may generate Head Related Transfer Function (HRTF) information based on the location information of the virtual sound source, and convert the virtual audio signal into a 3D audio signal using the generated HRTF information.
- HRTF Head Related Transfer Function
- An object of the present invention is to provide a user with a natural and realistic sound by applying an artificially synthesized reverberation effect to a virtual audio signal recorded in a specific environment.
- 5 to 8 specifically illustrate an audio content providing method according to an embodiment of the present invention.
- FIG. 5 shows how the HMD 100 of the present invention receives real sound and extracts spatial acoustic parameters.
- the HMD 100 may include a microphone unit, and may receive real sound through the microphone unit.
- the real sound received by the HMD 100 may include one or more sound source signals.
- the user 10 wearing the HMD 100 listens to a string quartet indoors.
- the real sound received by the HMD 100 may include sound source signals 50a, 50b, 50c, and 50d of respective instruments playing string quartet.
- the HMD 100 extracts the spatial acoustic parameters of the indoor space using the received real sound.
- the spatial acoustic parameters may include various parameters such as reverberation time and indoor shock response.
- the HMD 100 generates a filter using at least one of the extracted spatial acoustic parameters.
- FIG. 6 shows how the HMD 100 of the present invention outputs the virtual audio signal 60 in the environment of FIG. 5 in which real sound is received.
- the HMD 100 of the present invention may acquire the virtual audio signal 60.
- the virtual audio signal 60 includes augmented reality audio information for providing to the user 10 wearing the HMD 100.
- the virtual audio signal 60 may be obtained based on the real sound received by the HMD 100.
- the HMD 100 may acquire a virtual audio signal 60, for example, a flute performance of the same song, based on a string quartet included in real sound.
- the HMD 100 of the present invention may obtain the virtual audio signal 60 from a storage unit or from a server through a communication unit.
- the HMD 100 filters the virtual audio signal 60 using the spatial acoustic parameters obtained in FIG. 5.
- the HMD 100 filters the virtual audio signal 60 using the spatial acoustic parameters obtained in the indoor space in which the string quartet is played, thereby applying the spatial acoustic parameter characteristics of the indoor space to the virtual audio signal 60.
- the HMD 100 of the present invention can provide the user 10 with a flute performance, which is a virtual audio signal 60, as if played in the same indoor space as the actual string quartet.
- the HMD 100 outputs the filtered virtual audio signal 60 to the audio output unit.
- the HMD 100 may adjust the reproduction property of the virtual audio signal 60 using the received real sound.
- the HMD 100 may adjust the flute performance of the virtual audio signal 60 to maintain the same tempo and pitch as the string quartet actually played.
- the HMD 100 may adjust the reproduction portion of the flute performance based on the string quartet actually played, thereby synchronizing the reproduction of the flute performance with the actual string quartet.
- the HMD 100 may acquire the position of the virtual sound source of the virtual audio signal 60.
- the location of the virtual sound source may be specified by the user 10 wearing the HMD 100 or may be acquired together as additional data when the virtual audio signal 60 is acquired.
- the HMD 100 of the present invention may convert the virtual audio signal 60 into a 3D audio signal based on the obtained position of the virtual sound source, and output the converted 3D audio signal.
- the HMD 100 may generate HRTF information based on the position of the virtual sound source, and convert the virtual audio signal 60 into a 3D audio signal using the generated HRTF information.
- the HMD 100 may cause the sound image of the virtual audio signal 60 to be positioned at the position of the virtual sound source.
- the virtual sound source of the virtual audio signal 60 is set to be located at the right rear side of the string quartet player.
- the HMD 100 can provide the user 10 so that the flute playing sounds as if performed on the right back side of string quartet players.
- FIGS. 7 and 8 show how the HMD 100 of the present invention outputs the virtual audio signal 60 in the outdoor space.
- the same or corresponding parts as those of the embodiments of FIGS. 5 and 6 will be omitted.
- the HMD 100 of the present invention may extract a spatial sound parameter by receiving real sound in an outdoor space.
- the real sound received by the HMD 100 may include sound source signals 52a, 52b, 52c, and 52d of each instrument that plays a string quartet in an outdoor space.
- the HMD 100 extracts the spatial acoustic parameters of the outdoor space using the received real sound.
- the HMD 100 generates a filter using at least one of the extracted spatial acoustic parameters.
- the HMD 100 of the present invention may output a virtual audio signal 60 in the environment of FIG. 7 in which real sound is received.
- the HMD 100 filters the virtual audio signal 60 using the spatial acoustic parameters obtained in FIG. 7. That is, the HMD 100 filters the virtual audio signal 60 using the spatial acoustic parameters obtained in the outdoor space where the string quartet is played, thereby applying the spatial acoustic parameter characteristics of the outdoor space to the virtual audio signal 60. have.
- the HMD 100 of the present invention can provide the user 10 with a flute performance, which is a virtual audio signal 60, as if played in the same outdoor space as the actual string quartet.
- the HMD 100 outputs the filtered virtual audio signal 60 to the audio output unit. If the virtual sound source of the virtual audio signal 60 is set to the left of the string quartet players as shown in FIG. 8, the HMD 100 may cause the user 10 to sound like the flute performance is performed on the left of the string quartet players. ) Can be provided.
- FIG. 9 specifically illustrates an audio content generation method according to an embodiment of the present invention.
- the HMD 100 generates audio content in the same environment as that of FIGS. 5 and 6.
- the generation of the audio content may be performed by various types of portable devices as well as the HMD 100.
- the same or corresponding parts as those of FIGS. 5 and 6 will be omitted.
- the HMD 100 receives a real sound using a microphone unit and obtains a virtual audio signal 60 corresponding to the received real sound.
- the virtual audio signal 60 includes augmented reality audio information for providing to the user 10 wearing the HMD 100.
- the virtual audio signal 60 may be obtained based on the real sound received by the HMD 100.
- the HMD 100 of the present invention separates the received real sound into at least one sound source signal 50a, 50b, 50c, 50d.
- the microphone unit of the HMD 100 may include a microphone array, and each of the sound source signals 50a, 50b, 50c, and 50d included in the real sound using a signal received by each microphone of the microphone array. Can be separated.
- the HMD 100 may separate the real sound based on the position of the sound source of each sound source signal 50a, 50b, 50c, 50d.
- the HMD 100 of the present invention sets a sound source signal to replace among the separated sound source signals 50a, 50b, 50c, and 50d.
- the HMD 100 may set the substitute sound source signal through various methods. For example, the HMD 100 may set the sound source signal selected by the user 10 wearing the HMD 100 as the replacement sound source signal.
- the HMD 100 may provide various interfaces for selecting a sound source signal, and may set a sound source signal to replace the sound source signal selected through the interface. In the embodiment of FIG. 9, the user 10 selects a sound source signal to replace the sound source signal 50d among the separated sound source signals 50a, 50b, 50c, and 50d.
- the HMD 100 of the present invention records an audio signal including the received real sound. At this time, the HMD 100 replaces the set sound source signal 50d with the virtual audio signal 60 to record. That is, the HMD 100 bypasses the sound source signal 50d of the received real sound, and instead records the virtual audio signal 60 together with the remaining sound source signals 50a, 50b, and 50c. Accordingly, the HMD 100 of the present invention may generate new audio content in which some sound source signals 50a, 50b, 50c and the virtual audio signal 60 are combined among the received real sounds.
- the HMD 100 of the present invention may adjust and record the reproduction property of the virtual audio signal 60 based on the received real sound. For example, the HMD 100 may adjust the flute performance of the virtual audio signal 60 to maintain the same tempo and pitch as the string quartet actually played. In addition, the HMD 100 may adjust the reproduction portion of the flute performance based on the string quartet actually played, thereby synchronizing the reproduction of the flute performance with the actual string quartet.
- the HMD 100 may obtain the position of the virtual sound source of the virtual audio signal 60.
- the location of the virtual sound source may be specified by the user 10 wearing the HMD 100 or may be acquired together as additional data when the virtual audio signal 60 is acquired.
- the position of the virtual sound source may be determined based on the position of the object corresponding to the sound source signal 50d to be replaced.
- the HMD 100 of the present invention may convert the virtual audio signal 60 into a 3D audio signal based on the acquired position of the virtual sound source, and record the converted 3D audio signal. A detailed embodiment of the conversion to the 3D audio signal is as described above with reference to FIG. 6.
- the HMD 100 may extract spatial acoustic parameters from the received real sound and record the filtered virtual audio signal 60 using the spatial acoustic parameters. Specific embodiments of the extraction of the spatial acoustic parameters and the filtering of the virtual audio signal 60 have been described above with reference to FIGS. 5 and 6.
- the user may be provided with the content 30 using the HMD 100.
- the content 30 includes various types of content such as movies, music, documents, video calls, navigation, and the like.
- the HMD 100 may output the image data to the display unit 120.
- the voice data of the content 30 may be output to the audio output unit of the HMD 100.
- the HMD 100 may receive real sounds around the HMD 100 and extract spatial acoustic parameters based on the received real sounds.
- the HMD 100 of the present invention may filter the audio signal of the content 30 by using the extracted spatial acoustic parameters and output the filtered audio signal.
- the HMD 100 outputs the same movie.
- the HMD 100 may output audio signals of the content 30 differently in the indoor space of FIG. 10 and the external space of FIG. 11. That is, the HMD 100 of the present invention may adaptively filter and output the audio signal of the content 30 when the environment for outputting the content 30 changes. Therefore, a user wearing the HMD 100 of the present invention can be immersed in the content 30 even in a changing listening environment.
- 12 to 14 specifically illustrate a method for providing audio content according to another embodiment of the present invention.
- the HMD 100 of the present invention provides the user 10 with audio content as augmented reality.
- the same or corresponding parts as those of the embodiment of FIGS. 5 to 8 will be omitted.
- a user 10 is walking in an external space (eg, a street around a time square) while wearing the HMD 100 of the present invention.
- the HMD 100 may include a GPS sensor, and may acquire location information of the HMD 100 using the GPS sensor.
- the HMD 100 may obtain location information based on a network service such as Wi-Fi.
- Fig. 13 shows map data 25 corresponding to the position information detected by the HMD 100 of the present invention.
- the map data 25 includes audio content 62a, 62b, 62c information of a sound source located around the HMD 100.
- the HMD 100 of the present invention obtains at least one audio content of the audio contents 62a, 62b, 62c.
- the HMD 100 may acquire audio contents 62a, 62b, and 62c of the plurality of sound sources together.
- the HMD 100 may acquire location information of sound sources of the audio contents 62a, 62b, and 62c together.
- the HMD 100 may further obtain view information for providing audio content.
- the HMD 100 may obtain audio content by using the location information of the HMD 100 together with the viewpoint information. That is, the audio content acquired by the HMD 100 at the same location may vary depending on the viewpoint information. For example, if the time information acquired by the HMD 100 is evening of December 31, 2012, the HMD 100 may obtain a Happy New Year concert of December 31, 2012 as audio content. On the other hand, if the time information acquired by the HMD 100 is December 31, 2011, the HMD 100 may acquire the Happy New Year concert of December 31, 2011 as audio content.
- the HMD 100 of the present invention may obtain spatial acoustic parameters for the audio contents 62a, 62b, 62c by using the acquired position information.
- the spatial acoustic parameters are information for realistically outputting the audio contents 62a, 62b, and 62c according to the actual environment, and may include various kinds of characteristic information as described above.
- the spatial acoustic parameter may be determined based on distance information between the sound source of each audio content 62a, 62b, 62c and the HMD 100.
- the spatial acoustic parameters may be determined based on obstacle information between the sound source of each audio content 62a, 62b, 62c and the HMD 100.
- the obstacle information is information on various obstacles (for example, buildings, etc.) that interfere with sound transmission between each sound source and the HMD 100, and may be obtained from the map data 25.
- the HMD 100 acquires audio contents 62a, 62b, 62c of a plurality of sound sources together, the distance and obstacle information between each sound source and the listener may be different. Therefore, the HMD 100 of the present invention may obtain a plurality of spatial acoustic parameters corresponding to the plurality of sound sources, respectively.
- the HMD 100 of the present invention filters the audio content 62a, 62b, 62c using the obtained spatial acoustic parameters. If the HMD 100 acquires only some of the audio contents of the plurality of audio contents 62a, 62b, and 62c, the HMD 100 obtains a spatial parameter corresponding to the obtained audio contents and filters the corresponding audio contents. can do.
- the HMD 100 of the present invention outputting the filtered audio content.
- the HMD 100 of the present invention outputs the filtered audio contents 62a 'and 62b' to the audio output unit.
- the HMD 100 may output the image content 36 corresponding to the filtered audio content 62a ′, 62b ′ to the display unit.
- the HMD 100 may provide previously filtered concert live near the Times Square where the user is located as the filtered audio content 62a ', 62b'.
- the HMD 100 filters the audio content 62a 'and 62b' filtered based on the position of the sound source of the obtained audio content 62a and 62b, the distance between the HMD 100, and the obstacle information. to provide. Therefore, a user wearing the HMD 100 can listen to the audio contents 62a and 62b as if they are listening to a concert in the field.
- the HMD 100 may obtain direction information of each sound source based on the HMD 100.
- the direction information includes azimuth information of a sound source based on the HMD 100.
- the HMD 100 may obtain the direction information by using the position information of the sound source and the sensing value of the gyro sensor of the HMD 100.
- the HMD 100 of the present invention converts the filtered audio content 62a ', 62b' into a 3D audio signal based on the obtained direction information and distance information between the sound source and the HMD 100, and converts the converted 3D audio. Can output a signal. More specifically, the HMD 100 generates HRTF information based on the direction information and distance information, and converts the filtered audio content 62a ', 62b' into a 3D audio signal using the generated HRTF information. Can be.
- the HMD described in the present invention can be changed and replaced with various devices according to the purpose of the present invention.
- the HMD of the present invention includes various devices capable of providing a display by being worn by a user, such as eye mounted display (EMD), eyeglass, eyepiece, eye wear, HWD (Head Worn Display), and used in the present invention. It is not limited to the said term.
- EMD eye mounted display
- HWD Head Worn Display
- the present invention may be applied, in whole or in part, to various digital devices including HMDs.
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Stereophonic System (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Description
Claims (8)
- 헤드 마운트 디스플레이(Head Mounted Display, HMD)를 이용한 오디오 콘텐츠 제공 방법으로써,마이크 유닛을 이용하여 리얼 사운드를 수신하는 단계;가상 오디오 신호를 획득하는 단계;상기 수신된 리얼 사운드를 이용하여 공간 음향 파라미터를 추출하는 단계;상기 추출된 공간 음향 파라미터를 이용하여 상기 가상 오디오 신호를 필터링 하는 단계; 및상기 필터링 된 가상 오디오 신호를 출력하는 단계;를 포함하는 것을 특징으로 하는 헤드 마운트 디스플레이(Head Mounted Display, HMD)를 이용한 오디오 콘텐츠 제공 방법.
- 제 1항에 있어서,상기 공간 음향 파라미터는 상기 수신된 리얼 사운드를 이용하여 추출된 잔향 시간 및 실내 충격 응답(Room Impulse Response, RIR) 중 적어도 하나를 포함하는 것을 특징으로 하는 헤드 마운트 디스플레이(Head Mounted Display, HMD)를 이용한 오디오 콘텐츠 제공 방법.
- 제 1항에 있어서,상기 가상 오디오 신호는 상기 수신된 리얼 사운드를 기초로 획득되는 것을 특징으로 하는 헤드 마운트 디스플레이(Head Mounted Display, HMD)를 이용한 오디오 콘텐츠 제공 방법.
- 제 1항에 있어서,상기 가상 오디오 신호를 출력하는 단계는,상기 수신된 리얼 사운드를 이용하여 상기 가상 오디오 신호의 재생 속성을 조정하는 것을 특징으로 하는 헤드 마운트 디스플레이(Head Mounted Display, HMD)를 이용한 오디오 콘텐츠 제공 방법.
- 제 4항에 있어서,상기 재생 속성은 재생 피치 및 재생 템포 중 적어도 하나를 포함하는 것을 특징으로 하는 헤드 마운트 디스플레이(Head Mounted Display, HMD)를 이용한 오디오 콘텐츠 제공 방법.
- 제 1항에 있어서,상기 가상 오디오 신호의 가상 음원의 위치를 획득하는 단계를 더 포함하며,상기 가상 오디오 신호를 출력하는 단계는, 상기 획득된 가상 음원의 위치에 기초하여 변환된 3D(3 Dimensional) 오디오 신호로 출력하는 것을 특징으로 하는 헤드 마운트 디스플레이(Head Mounted Display, HMD)를 이용한 오디오 콘텐츠 제공 방법.
- 제 6항에 있어서,상기 가상 음원의 위치에 기초하여 HRTF(Head Related Transfer Function) 정보를 생성하는 단계;상기 생성된 HRTF 정보를 이용하여 상기 가상 오디오 신호를 3D 오디오 신호로 변환하는 단계;를 더 포함하는 것을 특징으로 하는 헤드 마운트 디스플레이(Head Mounted Display, HMD)를 이용한 오디오 콘텐츠 제공 방법.
- 헤드 마운트 디스플레이(Head Mounted Display, HMD)로써,상기 HMD의 작동을 제어하는 프로세서;리얼 사운드를 수신하는 마이크 유닛; 및상기 프로세서의 명령에 기초하여 사운드를 출력하는 오디오 출력 유닛을 포함하되,상기 프로세서는,상기 마이크 유닛을 이용하여 리얼 사운드를 수신하고,가상 오디오 신호를 획득하고,상기 수신된 리얼 사운드를 이용하여 공간 음향 파라미터를 추출하고,상기 추출된 공간 음향 파라미터를 이용하여 상기 가상 오디오 신호를 필터링 하고,상기 필터링 된 가상 오디오 신호를 상기 오디오 출력 유닛으로 출력하는 것을 특징으로 하는 헤드 마운트 디스플레이(HMD).
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/787,897 US20160088417A1 (en) | 2013-04-30 | 2013-06-05 | Head mounted display and method for providing audio content by using same |
KR1020157031067A KR20160005695A (ko) | 2013-04-30 | 2013-06-05 | 헤드 마운트 디스플레이 및 이를 이용한 오디오 콘텐츠 제공 방법 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR20130048208 | 2013-04-30 | ||
KR10-2013-0048208 | 2013-04-30 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2014178479A1 true WO2014178479A1 (ko) | 2014-11-06 |
Family
ID=51843592
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/KR2013/004990 WO2014178479A1 (ko) | 2013-04-30 | 2013-06-05 | 헤드 마운트 디스플레이 및 이를 이용한 오디오 콘텐츠 제공 방법 |
Country Status (3)
Country | Link |
---|---|
US (1) | US20160088417A1 (ko) |
KR (1) | KR20160005695A (ko) |
WO (1) | WO2014178479A1 (ko) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR101627647B1 (ko) * | 2014-12-04 | 2016-06-07 | 가우디오디오랩 주식회사 | 바이노럴 렌더링을 위한 오디오 신호 처리 장치 및 방법 |
WO2016144459A1 (en) * | 2015-03-06 | 2016-09-15 | Microsoft Technology Licensing, Llc | Real-time remodeling of user voice in an immersive visualization system |
WO2017026559A1 (ko) * | 2015-08-13 | 2017-02-16 | 주식회사 넥스트이온 | 디스플레이 장치에 표시되는 영상의 방향 변화에 따라 소리의 위상을 전환시키는 방법 및 시스템 |
US10031718B2 (en) | 2016-06-14 | 2018-07-24 | Microsoft Technology Licensing, Llc | Location based audio filtering |
CN109076305A (zh) * | 2016-02-02 | 2018-12-21 | Dts(英属维尔京群岛)有限公司 | 增强现实耳机环境渲染 |
Families Citing this family (36)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10129682B2 (en) * | 2012-01-06 | 2018-11-13 | Bacch Laboratories, Inc. | Method and apparatus to provide a virtualized audio file |
US10173129B2 (en) * | 2013-06-09 | 2019-01-08 | Sony Interactive Entertainment Inc. | Methods for rendering interactive content to a head mounted display |
US9226090B1 (en) * | 2014-06-23 | 2015-12-29 | Glen A. Norris | Sound localization for an electronic call |
KR102226817B1 (ko) * | 2014-10-01 | 2021-03-11 | 삼성전자주식회사 | 콘텐츠 재생 방법 및 그 방법을 처리하는 전자 장치 |
KR102524641B1 (ko) * | 2016-01-22 | 2023-04-21 | 삼성전자주식회사 | Hmd 디바이스 및 그 제어 방법 |
US20170303052A1 (en) * | 2016-04-18 | 2017-10-19 | Olive Devices LLC | Wearable auditory feedback device |
US10469976B2 (en) * | 2016-05-11 | 2019-11-05 | Htc Corporation | Wearable electronic device and virtual reality system |
US10451719B2 (en) * | 2016-06-22 | 2019-10-22 | Loose Cannon Systems, Inc. | System and method to indicate relative location of nodes in a group |
US9906885B2 (en) * | 2016-07-15 | 2018-02-27 | Qualcomm Incorporated | Methods and systems for inserting virtual sounds into an environment |
US10445936B1 (en) * | 2016-08-01 | 2019-10-15 | Snap Inc. | Audio responsive augmented reality |
US10089063B2 (en) * | 2016-08-10 | 2018-10-02 | Qualcomm Incorporated | Multimedia device for processing spatialized audio based on movement |
US10896544B2 (en) * | 2016-10-07 | 2021-01-19 | Htc Corporation | System and method for providing simulated environment |
US10848899B2 (en) * | 2016-10-13 | 2020-11-24 | Philip Scott Lyren | Binaural sound in visual entertainment media |
WO2018093193A1 (en) * | 2016-11-17 | 2018-05-24 | Samsung Electronics Co., Ltd. | System and method for producing audio data to head mount display device |
GB2557594B (en) * | 2016-12-09 | 2020-01-01 | Sony Interactive Entertainment Inc | Image processing system and method |
EP3343957B1 (en) * | 2016-12-30 | 2022-07-06 | Nokia Technologies Oy | Multimedia content |
WO2018147701A1 (ko) * | 2017-02-10 | 2018-08-16 | 가우디오디오랩 주식회사 | 오디오 신호 처리 방법 및 장치 |
DE102017207581A1 (de) * | 2017-05-05 | 2018-11-08 | Sivantos Pte. Ltd. | Hörsystem sowie Hörvorrichtung |
GB201709851D0 (en) * | 2017-06-20 | 2017-08-02 | Nokia Technologies Oy | Processing audio signals |
US11361771B2 (en) | 2017-09-22 | 2022-06-14 | Lg Electronics Inc. | Method for transmitting/receiving audio data and device therefor |
CA3078420A1 (en) | 2017-10-17 | 2019-04-25 | Magic Leap, Inc. | Mixed reality spatial audio |
EP3503102A1 (en) | 2017-12-22 | 2019-06-26 | Nokia Technologies Oy | An apparatus and associated methods for presentation of captured spatial audio content |
CN110164464A (zh) * | 2018-02-12 | 2019-08-23 | 北京三星通信技术研究有限公司 | 音频处理方法及终端设备 |
CN111713091A (zh) * | 2018-02-15 | 2020-09-25 | 奇跃公司 | 混合现实虚拟混响 |
US10628988B2 (en) * | 2018-04-13 | 2020-04-21 | Aladdin Manufacturing Corporation | Systems and methods for item characteristic simulation |
EP3804132A1 (en) | 2018-05-30 | 2021-04-14 | Magic Leap, Inc. | Index scheming for filter parameters |
GB2575511A (en) | 2018-07-13 | 2020-01-15 | Nokia Technologies Oy | Spatial audio Augmentation |
GB2575509A (en) * | 2018-07-13 | 2020-01-15 | Nokia Technologies Oy | Spatial audio capture, transmission and reproduction |
US10871939B2 (en) * | 2018-11-07 | 2020-12-22 | Nvidia Corporation | Method and system for immersive virtual reality (VR) streaming with reduced audio latency |
GB201819896D0 (en) * | 2018-12-06 | 2019-01-23 | Bae Systems Plc | Tracking system |
GB2582952B (en) * | 2019-04-10 | 2022-06-15 | Sony Interactive Entertainment Inc | Audio contribution identification system and method |
JP7446420B2 (ja) | 2019-10-25 | 2024-03-08 | マジック リープ, インコーポレイテッド | 反響フィンガプリント推定 |
KR20210123198A (ko) * | 2020-04-02 | 2021-10-13 | 주식회사 제이렙 | 증강 현실 기반의 전기 음향과 건축 음향 통합 시뮬레이션 장치 |
US11470439B1 (en) | 2021-06-02 | 2022-10-11 | Meta Platforms Technologies, Llc | Adjustment of acoustic map and presented sound in artificial reality systems |
CN114286278B (zh) * | 2021-12-27 | 2024-03-15 | 北京百度网讯科技有限公司 | 音频数据处理方法、装置、电子设备及存储介质 |
CN114363794B (zh) * | 2021-12-27 | 2023-10-24 | 北京百度网讯科技有限公司 | 音频处理方法、装置、电子设备和计算机可读存储介质 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR970078522A (ko) * | 1996-05-13 | 1997-12-12 | 김광호 | 디지탈필터를 이용한 음장모델링장치 |
JP2002140076A (ja) * | 2000-11-03 | 2002-05-17 | Junichi Kakumoto | 音響信号の臨場感伝達方式 |
JP2004077277A (ja) * | 2002-08-19 | 2004-03-11 | Fujitsu Ltd | 音源位置の可視化表示方法および音源位置表示装置 |
JP2005080124A (ja) * | 2003-09-02 | 2005-03-24 | Japan Science & Technology Agency | リアルタイム音響再現システム |
JP2012150278A (ja) * | 2011-01-19 | 2012-08-09 | Kitakyushu Foundation For The Advancement Of Industry Science And Technology | 仮想空間のビジュアル変化に対応した音響効果の自動生成システム |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
AU5362694A (en) * | 1992-11-06 | 1994-06-08 | Virtual Vision, Inc. | Head mounted video display system with portable video interface unit |
WO2012130297A1 (en) * | 2011-03-30 | 2012-10-04 | Phonak Ag | Wireless sound transmission system and method |
US9384737B2 (en) * | 2012-06-29 | 2016-07-05 | Microsoft Technology Licensing, Llc | Method and device for adjusting sound levels of sources based on sound source priority |
-
2013
- 2013-06-05 US US14/787,897 patent/US20160088417A1/en not_active Abandoned
- 2013-06-05 KR KR1020157031067A patent/KR20160005695A/ko not_active Application Discontinuation
- 2013-06-05 WO PCT/KR2013/004990 patent/WO2014178479A1/ko active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR970078522A (ko) * | 1996-05-13 | 1997-12-12 | 김광호 | 디지탈필터를 이용한 음장모델링장치 |
JP2002140076A (ja) * | 2000-11-03 | 2002-05-17 | Junichi Kakumoto | 音響信号の臨場感伝達方式 |
JP2004077277A (ja) * | 2002-08-19 | 2004-03-11 | Fujitsu Ltd | 音源位置の可視化表示方法および音源位置表示装置 |
JP2005080124A (ja) * | 2003-09-02 | 2005-03-24 | Japan Science & Technology Agency | リアルタイム音響再現システム |
JP2012150278A (ja) * | 2011-01-19 | 2012-08-09 | Kitakyushu Foundation For The Advancement Of Industry Science And Technology | 仮想空間のビジュアル変化に対応した音響効果の自動生成システム |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR101627647B1 (ko) * | 2014-12-04 | 2016-06-07 | 가우디오디오랩 주식회사 | 바이노럴 렌더링을 위한 오디오 신호 처리 장치 및 방법 |
WO2016089180A1 (ko) * | 2014-12-04 | 2016-06-09 | 가우디오디오랩 주식회사 | 바이노럴 렌더링을 위한 오디오 신호 처리 장치 및 방법 |
US9961466B2 (en) | 2014-12-04 | 2018-05-01 | Gaudi Audio Lab, Inc. | Audio signal processing apparatus and method for binaural rendering |
WO2016144459A1 (en) * | 2015-03-06 | 2016-09-15 | Microsoft Technology Licensing, Llc | Real-time remodeling of user voice in an immersive visualization system |
US9558760B2 (en) | 2015-03-06 | 2017-01-31 | Microsoft Technology Licensing, Llc | Real-time remodeling of user voice in an immersive visualization system |
CN107430868A (zh) * | 2015-03-06 | 2017-12-01 | 微软技术许可有限责任公司 | 沉浸式可视化***中用户语音的实时重构 |
US10176820B2 (en) | 2015-03-06 | 2019-01-08 | Microsoft Technology Licensing, Llc | Real-time remodeling of user voice in an immersive visualization system |
WO2017026559A1 (ko) * | 2015-08-13 | 2017-02-16 | 주식회사 넥스트이온 | 디스플레이 장치에 표시되는 영상의 방향 변화에 따라 소리의 위상을 전환시키는 방법 및 시스템 |
CN109076305A (zh) * | 2016-02-02 | 2018-12-21 | Dts(英属维尔京群岛)有限公司 | 增强现实耳机环境渲染 |
CN109076305B (zh) * | 2016-02-02 | 2021-03-23 | Dts(英属维尔京群岛)有限公司 | 增强现实耳机环境渲染 |
US10031718B2 (en) | 2016-06-14 | 2018-07-24 | Microsoft Technology Licensing, Llc | Location based audio filtering |
Also Published As
Publication number | Publication date |
---|---|
KR20160005695A (ko) | 2016-01-15 |
US20160088417A1 (en) | 2016-03-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2014178479A1 (ko) | 헤드 마운트 디스플레이 및 이를 이용한 오디오 콘텐츠 제공 방법 | |
CN110249640B (zh) | 用于虚拟现实(vr)、增强现实(ar)和混合现实(mr)***的分布式音频捕获技术 | |
EP2942980A1 (en) | Real-time control of an acoustic environment | |
TW201215179A (en) | Virtual spatial sound scape | |
WO2021143574A1 (zh) | 增强现实眼镜、基于增强现实眼镜的ktv实现方法与介质 | |
WO2014061931A1 (ko) | 음향 재생 장치 및 음향 재생 방법 | |
CN112312297B (zh) | 音频带宽减小 | |
KR20140129654A (ko) | 헤드 마운트 디스플레이 및 이를 이용한 오디오 콘텐츠 제공 방법 | |
EP2743917B1 (en) | Information system, information reproducing apparatus, information generating method, and storage medium | |
WO2018186693A1 (ko) | 영상 정보에 기반하여 가상 스피커를 재생하기 위한 음원 재생 장치 | |
CN108269460B (zh) | 一种电子屏幕的阅读方法、***及终端设备 | |
US20240007790A1 (en) | Method and device for sound processing for a synthesized reality setting | |
US11200739B2 (en) | Virtual scene | |
WO2022267468A1 (zh) | 一种声音处理方法及其装置 | |
CN106412751B (zh) | 一种可辨别方向的耳机及其实现方法 | |
EP3503558B1 (en) | Audio content format selection | |
CN112599144B (zh) | 音频数据处理方法、音频数据处理装置、介质与电子设备 | |
CN103974168A (zh) | 一种信息处理方法及电子设备 | |
JP6897565B2 (ja) | 信号処理装置、信号処理方法及びコンピュータプログラム | |
KR100954033B1 (ko) | 다시점 화상 시스템에서 시점 종속 다채널 오디오 처리방법 및 장치 | |
JP2013187841A (ja) | 電子機器及び出力制御方法並びにプログラム | |
JP3363921B2 (ja) | 音像定位装置 | |
KR20140129659A (ko) | 포터블 디바이스 및 이를 이용한 오디오 콘텐츠 생성 방법 | |
Spandonidis et al. | Design of Smart Glasses that Enable Computer Vision for the Improvement of Autonomy of the Visually Impaired. | |
KR101111734B1 (ko) | 복수 개의 음원을 구분하여 음향을 출력하는 방법 및 장치 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 13883552 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 20157031067 Country of ref document: KR Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 14787897 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 13883552 Country of ref document: EP Kind code of ref document: A1 |