US20190215598A1 - Signal processing device, signal processing method, and program - Google Patents
Signal processing device, signal processing method, and program Download PDFInfo
- Publication number
- US20190215598A1 US20190215598A1 US16/354,710 US201916354710A US2019215598A1 US 20190215598 A1 US20190215598 A1 US 20190215598A1 US 201916354710 A US201916354710 A US 201916354710A US 2019215598 A1 US2019215598 A1 US 2019215598A1
- Authority
- US
- United States
- Prior art keywords
- signal
- sound
- signal processing
- processing device
- acoustic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012545 processing Methods 0.000 title claims abstract description 233
- 238000003672 processing method Methods 0.000 title description 8
- 210000000613 ear canal Anatomy 0.000 claims abstract description 49
- 230000000694 effects Effects 0.000 claims description 42
- 230000000644 propagated effect Effects 0.000 claims description 3
- 230000001902 propagating effect Effects 0.000 abstract description 69
- 230000009467 reduction Effects 0.000 abstract description 42
- 238000000034 method Methods 0.000 description 87
- 230000008569 process Effects 0.000 description 74
- 238000005070 sampling Methods 0.000 description 41
- 230000006870 function Effects 0.000 description 33
- 238000010586 diagram Methods 0.000 description 31
- 230000010365 information processing Effects 0.000 description 16
- 238000012546 transfer Methods 0.000 description 14
- 238000004891 communication Methods 0.000 description 13
- 238000004364 calculation method Methods 0.000 description 10
- 238000003058 natural language processing Methods 0.000 description 9
- 238000004458 analytical method Methods 0.000 description 8
- 238000006243 chemical reaction Methods 0.000 description 8
- 210000003128 head Anatomy 0.000 description 7
- 230000007246 mechanism Effects 0.000 description 7
- 230000003321 amplification Effects 0.000 description 5
- 210000000988 bone and bone Anatomy 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000003199 nucleic acid amplification method Methods 0.000 description 5
- 238000002474 experimental method Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 210000003454 tympanic membrane Anatomy 0.000 description 4
- 238000005259 measurement Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 238000003780 insertion Methods 0.000 description 2
- 230000037431 insertion Effects 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000009413 insulation Methods 0.000 description 2
- 230000001151 other effect Effects 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000004397 blinking Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 210000005069 ears Anatomy 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- PWPJGUXAGUPAHP-UHFFFAOYSA-N lufenuron Chemical compound C1=C(Cl)C(OC(F)(F)C(C(F)(F)F)F)=CC(Cl)=C1NC(=O)NC(=O)C1=C(F)C=CC=C1F PWPJGUXAGUPAHP-UHFFFAOYSA-N 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 230000010363 phase shift Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/10—Earpieces; Attachments therefor ; Earphones; Monophonic headphones
- H04R1/1091—Details not provided for in groups H04R1/1008 - H04R1/1083
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10K—SOUND-PRODUCING DEVICES; METHODS OR DEVICES FOR PROTECTING AGAINST, OR FOR DAMPING, NOISE OR OTHER ACOUSTIC WAVES IN GENERAL; ACOUSTICS NOT OTHERWISE PROVIDED FOR
- G10K11/00—Methods or devices for transmitting, conducting or directing sound in general; Methods or devices for protecting against, or for damping, noise or other acoustic waves in general
- G10K11/16—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general
- G10K11/175—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound
- G10K11/178—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound by electro-acoustically regenerating the original acoustic waves in anti-phase
- G10K11/1783—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound by electro-acoustically regenerating the original acoustic waves in anti-phase handling or detecting of non-standard events or conditions, e.g. changing operating modes under specific operating conditions
- G10K11/17837—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound by electro-acoustically regenerating the original acoustic waves in anti-phase handling or detecting of non-standard events or conditions, e.g. changing operating modes under specific operating conditions by retaining part of the ambient acoustic environment, e.g. speech or alarm signals that the user needs to hear
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10K—SOUND-PRODUCING DEVICES; METHODS OR DEVICES FOR PROTECTING AGAINST, OR FOR DAMPING, NOISE OR OTHER ACOUSTIC WAVES IN GENERAL; ACOUSTICS NOT OTHERWISE PROVIDED FOR
- G10K11/00—Methods or devices for transmitting, conducting or directing sound in general; Methods or devices for protecting against, or for damping, noise or other acoustic waves in general
- G10K11/16—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general
- G10K11/175—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound
- G10K11/178—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound by electro-acoustically regenerating the original acoustic waves in anti-phase
- G10K11/1787—General system configurations
- G10K11/17879—General system configurations using both a reference signal and an error signal
- G10K11/17881—General system configurations using both a reference signal and an error signal the reference signal being an acoustic signal, e.g. recorded with a microphone
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/10—Earpieces; Attachments therefor ; Earphones; Monophonic headphones
- H04R1/1083—Reduction of ambient noise
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/20—Arrangements for obtaining desired frequency or directional characteristics
- H04R1/22—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired frequency characteristic only
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
- H04R3/04—Circuits for transducers, loudspeakers or microphones for correcting frequency response
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10K—SOUND-PRODUCING DEVICES; METHODS OR DEVICES FOR PROTECTING AGAINST, OR FOR DAMPING, NOISE OR OTHER ACOUSTIC WAVES IN GENERAL; ACOUSTICS NOT OTHERWISE PROVIDED FOR
- G10K2210/00—Details of active noise control [ANC] covered by G10K11/178 but not provided for in any of its subgroups
- G10K2210/10—Applications
- G10K2210/108—Communication systems, e.g. where useful sound is kept and noise is cancelled
- G10K2210/1081—Earphones, e.g. for telephones, ear protectors or headsets
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10K—SOUND-PRODUCING DEVICES; METHODS OR DEVICES FOR PROTECTING AGAINST, OR FOR DAMPING, NOISE OR OTHER ACOUSTIC WAVES IN GENERAL; ACOUSTICS NOT OTHERWISE PROVIDED FOR
- G10K2210/00—Details of active noise control [ANC] covered by G10K11/178 but not provided for in any of its subgroups
- G10K2210/30—Means
- G10K2210/301—Computational
- G10K2210/3014—Adaptive noise equalizers [ANE], i.e. where part of the unwanted sound is retained
Definitions
- the present disclosure relates to a signal processing device, a signal processing method, and a program.
- acoustic devices which are worn on heads of users for use such as earphones or headphones (which may hereinafter be referred to as “head mounted acoustic devices”)
- head mounted acoustic devices in addition to devices that simply output acoustic information, devices with functions in which use situations are considered have become widespread.
- a head mounted acoustic device capable of suppressing ambient sounds (so-called noise) coming from an external environment and enhancing a sound insulation effect using a so-called noise canceling technique is known.
- Patent Literature 1 discloses an example of an acoustic device using such a noise canceling technique.
- Patent Literature 1 JP 4882773B
- the present disclosure proposes a signal processing device, a signal processing method, and a program, which are capable of enabling a listener to listen to ambient sounds of the external environment in an appropriate manner while wearing a head mounted acoustic device.
- a signal processing device including: a first acquiring unit configured to acquire a sound collection result for a first sound propagating in an external space outside a mounting unit to be worn on an ear of a listener; a second acquiring unit configured to acquire a sound collection result for a second sound propagating in an internal space connected with an external ear canal inside the mounting unit; a first filter processing unit configured to generate a difference signal which is substantially equal to a difference between the first sound propagating directly from the external space toward an inside of the external ear canal and the first sound propagating from the external space to the internal space via the mounting unit on the basis of the sound collection result for the first sound; a subtracting unit configured to generate a subtraction signal obtained by subtracting a first signal component based on the sound collection result for the first sound and a second signal component based on an input acoustic signal to be output from an acoustic device from an inside of the mounting unit toward the internal space from the sound collection result for the second sound; a second filter
- a signal processing method including, by a processor: acquiring a sound collection result for a first sound propagating in an external space outside a mounting unit to be worn on an ear of a listener; acquiring a sound collection result for a second sound propagating in an internal space connected with an external ear canal inside the mounting unit; generating a difference signal which is substantially equal to a difference between the first sound propagating directly from the external space toward an inside of the external ear canal and the first sound propagating from the external space to the internal space via the mounting unit on the basis of the sound collection result for the first sound; generating a subtraction signal obtained by subtracting a first signal component based on the sound collection result for the first sound and a second signal component based on an input acoustic signal to be output from an acoustic device from an inside of the mounting unit toward the internal space from the sound collection result for the second sound; generating a noise reduction signal for reducing the subtraction signal on the basis of the subtraction signal; and
- a program causing a computer to execute: acquiring a sound collection result for a first sound propagating in an external space outside a mounting unit to be worn on an ear of a listener; acquiring a sound collection result for a second sound propagating in an internal space connected with an external ear canal inside the mounting unit; generating a difference signal which is substantially equal to a difference between the first sound propagating directly from the external space toward an inside of the external ear canal and the first sound propagating from the external space to the internal space via the mounting unit on the basis of the sound collection result for the first sound; generating a subtraction signal obtained by subtracting a first signal component based on the sound collection result for the first sound and a second signal component based on an input acoustic signal to be output from an acoustic device from an inside of the mounting unit toward the internal space from the sound collection result for the second sound; generating a noise reduction signal for reducing the subtraction signal on the basis of the subtraction signal; and adding the
- a signal processing device, a signal processing method, and a program which are capable of enabling a listener to listen to the ambient sounds of the external environment in an appropriate manner while wearing a head mounted acoustic device are provided.
- FIG. 1 is an explanatory diagram for describing an application example of a head mounted acoustic device to which a signal processing device according to an embodiment of the present disclosure is applied.
- FIG. 2 is an explanatory diagram for describing an example of a principle for implementing a hear-through effect.
- FIG. 3 is a diagram schematically illustrating an example of a propagation environment before an ambient sound is heard by a user in a case in which the user wears a canal type earphone.
- FIG. 4 is a diagram schematically illustrating an example of a propagation environment before an ambient sound is heard by the user in a case in which the user does not wear a head mounted acoustic device.
- FIG. 5 is a block diagram illustrating an example of a basic functional configuration of a signal processing device according to an embodiment of the present disclosure.
- FIG. 6 is an explanatory diagram for describing a mechanism of the occurrence of a phenomenon in which vibration of a voice uttered by the user propagates within an internal space.
- FIG. 7 is a block diagram illustrating an example of a functional configuration of a signal processing device according to a first embodiment of the present disclosure.
- FIG. 8 is an explanatory diagram for describing an example of a configuration of the signal processing device according to the embodiment.
- FIG. 9 is a block diagram illustrating an example of a functional configuration of a signal processing device according to a second embodiment of the present disclosure.
- FIG. 10 is an explanatory diagram for describing an example of a configuration for further reducing a delay amount in the signal processing device according to the embodiment.
- FIG. 11 is a diagram illustrating an example of a functional configuration of a monitor canceller.
- FIG. 12 is a block diagram illustrating an example of a functional configuration of a signal processing device according to a modified example of the embodiment.
- FIG. 13 is a diagram illustrating an example of a functional configuration of a signal processing device according to a third embodiment of the present disclosure.
- FIG. 14 is a block diagram illustrating another example of a functional configuration of the signal processing device according to the embodiment.
- FIG. 15 is an explanatory diagram for describing an application example of a signal processing device according to the embodiment.
- FIG. 16 is a diagram illustrating an example of a hardware configuration of a signal processing device according to embodiments of the present disclosure.
- a head mounted acoustic device such as earphones or headphones which are worn on the heads of the users when used, in addition to devices that simply output acoustic information, devices with functions in which use situations are considered have become widespread.
- a head mounted acoustic device capable of suppressing ambient sounds (so-called noise) coming from an external environment and enhancing a sound insulation effect using a so-called noise canceling technique is known.
- UIs user interfaces
- voice input that enable users to operate information processing devices by interacting with the devices by voice by applying a voice recognition technique
- FIG. 1 is an explanatory diagram for describing an application example of a head mounted acoustic device to which a signal processing device according to an embodiment of the present disclosure is applied.
- FIG. 1 illustrates an example of a situation in which the user uses a portable information processing device such as a smartphone while wearing a head mounted acoustic device 51 in a so-called public place such as a case in which the user goes out.
- a state in which the user is able to hear a so-called ambient sound coming from an external environment even while the user is wearing the head mounted acoustic device 51 in a manner similar to that in a case in which the user does not wear the head mounted acoustic device 51 is also referred to as a “hear-through state.”
- an effect of enabling the user to hear a so-called ambient sound coming from an external environment even while the user is wearing the head mounted acoustic device in a manner similar to that in a case in which the user does not wear the head mounted acoustic device 51 is also referred to as a “hear-through effect.”
- the user is able to check a sound output indicating notification of content of e-mails or news while checking a surrounding situation and wearing the head mounted acoustic device even in a public place.
- the user is also able to perform a phone call with another user by means of a so-called phone call function while checking a surrounding situation in motion.
- a technique based on the premise of the use of a head mounted acoustic device having high hermeticity such as a so-called canal type earphone is important. This is because there are cases in which, in a situation in which a head mounted acoustic device having relatively low hermeticity such as a so-called open air headphone is used, influence of so-called sound leakage is large, and use in public places is not necessarily preferable.
- FIG. 2 is an explanatory diagram for describing an example of the principle for implementing the hear-through effect and illustrates an example of a schematic functional configuration of the head mounted acoustic device 51 in a case in which the head mounted acoustic device 51 is configured as a so-called FF type NC earphone.
- the head mounted acoustic device 51 includes, for example, a microphone 71 , a filter circuit 72 , a power amplifier 73 , and a speaker 74 .
- reference numeral F schematically indicates a transfer function of a propagation environment before a sound N from a sound source S reaches (that is, leaks into) the user's ear (that is, the inside of the external ear canal) via the housing of the head mounted acoustic device 51 .
- Reference numeral F′ schematically indicates the transfer function of the propagation environment before the sound N from the sound source S reaches the microphone 71 .
- FIG. 3 schematically illustrates an example of the propagation environment before the sound N from the sound source S is heard by the user U in a case in which the user U wears a so-called canal type earphone as the head mounted acoustic device 51 .
- reference numeral UA schematically indicates a space in the external ear canal of a user U (hereinafter also referred to simply as an “external ear canal”).
- reference numerals F and F′ in FIG. 3 correspond to reference numerals F and F′ illustrated in FIG. 2 , respectively.
- FIG. 3 schematically illustrates an example of the propagation environment before the sound N from the sound source S is heard by the user U in a case in which the user U wears a so-called canal type earphone as the head mounted acoustic device 51 .
- reference numeral UA schematically indicates a space in the external ear canal of a user U (hereinafter also referred to simply as an “external ear canal”).
- a space connected to the external ear canal UA inside the head mounted acoustic device 51 is also referred to as an “internal space.”
- a space outside the head mounted acoustic device 51 is also referred to as an “external space.”
- the sound N from the sound source S propagating via the propagation environment F N may leak into the ear U′ of the user (specifically, the internal space connected to the external ear canal UA). Therefore, in the NC earphone, the influence of the sound N is mitigated by adding a signal having a reverse phase (a noise reduction signal) to the sound N propagating via the propagation environment F.
- a signal having a reverse phase a noise reduction signal
- the sound N from the sound source S of the external environment reaches the microphone 71 via the propagation environment F′ and is collected by the microphone 71 .
- the filter circuit 72 generates a signal having a reverse phase (noise reduction signal) to that of the sound N propagating via the propagation environment F on the basis of the sound N collected by the microphone 71 .
- the noise reduction signal generated by the filter circuit 72 undergoes gain adjustment performed by the power amplifier 73 and is then output toward the ear U′ of the user through the speaker 74 . Accordingly, a component of the sound N propagating to the ear U′ of the user via the propagation environment F is canceled by a component of the noise reduction signal output from the speaker 74 , and the sound N is suppressed.
- transfer functions based on device characteristics of the microphone 71 , the power amplifier 73 , and the speaker 74 are indicated by M, A, and H, respectively.
- a filter coefficient when the filter circuit 72 generates the noise reduction signal on the basis of an acoustic signal collected by the microphone 71 is indicated by a.
- so-called noise canceling is implemented by designing the filter coefficient ⁇ of the filter circuit 72 so that a relational expression indicated by (Formula 1) below is satisfied.
- the user U hears the sound N from the sound source S of the external environment in a manner substantially equivalent to the case in which the head mounted acoustic device 51 is not worn.
- FIG. 4 sis a diagram schematically illustrating an example of the propagation environment before the sound N from the sound source S is heard by the user U in a case in which the user U does not wear the head mounted acoustic device 51 .
- reference numeral G schematically indicates a transfer function of a propagation environment before the sound N from the sound source S directly reaches the inside of the external ear canal UA of the user U.
- the filter coefficient of the filter circuit 72 in the case of implementing the hear-through effect is indicated by ⁇
- each of the noise canceling and the hear-through effect is implemented by adding a sound wave of the sound N propagated to the inside of the external ear canal UA via the head mounted acoustic device 51 and a sound wave of the sound N output from the speaker 74 in the air as illustrated in FIG. 2 . Therefore, it is understood that it is preferable that a delay amount before the sound N from the sound source S is collected by the microphone 71 and output from the speaker 74 via the filter circuit 72 and the power amplifier 73 , including a conversion process performed by an AD converter (ADC) or a DA converter (DAC), be suppressed to be about 100 ⁇ s or less.
- ADC AD converter
- DAC DA converter
- the filter circuit 72 of the filter coefficient ⁇ is constituted as a digital filter by installing the ADC and the DAC. This is because if the filter circuit 72 is constituted as a digital filter, it is possible to easily implement a filter process which is smaller in variation than in an analog filter and is unable to be implemented by an analog filter.
- the processing load is increased by the filtering process such as decimation and interpolation, and a delay occurs accordingly.
- the sound output from the speaker 74 and the sound N from the sound source S propagating via the propagation environment F in FIG. 2 are added in the space in the external ear canal UA (that is, a space near the eardrum), and an added sound is recognized by the user as one sound. Therefore, it is generally known that if the delay amount exceeds 10 ms, it is recognized as if an echo occurs, or it is recognized as if a sound is heard twice. Even in a case in which the delay amount is less than 10 ms, the frequency characteristic may be influenced by mutual interference of sounds or it may be difficult to implement the hear-through effect and the noise canceling.
- a delay of 1 ms is assumed to occurs between the sound output from the speaker 74 and the sound N from the sound source S propagating via the propagation environment F.
- an acoustic signal of a band near 1 kHz undergoes phase shift corresponding to one cycle (that is, 360°) and then added.
- an acoustic signal of ae band near 500 Hz has a reverse phase and then is cancelled.
- a so-called dip occurs.
- the delay amount is suppressed to be 100 ⁇ s, it is possible to increase a frequency band at which the dip occurs due to a reverse phase relation up to 5 kHz.
- the human external ear canal is known to have resonance points near about 3 kHz to 4 kHz although there are individual differences. For this reason, in the frequency band exceeding 4 kHz, it corresponds to the so-called individual difference part, and thus the appropriate hear-through effect is considered to be obtained by suppressing the delay amount to be 100 ⁇ s or less and adjusting the frequency band at which the dip occurs to be around 5 kHz.
- each of reference numerals 51 a and 51 b indicates the head mounted acoustic device 51 .
- reference numeral 51 a indicates the head mounted acoustic device 51 worn on the right ear
- reference numeral 51 b indicates the head mounted acoustic device 51 attached to the left ear.
- the head mounted acoustic devices 51 a and 51 b are not particularly distinguished, there are also referred to as a “head mounted acoustic device 51 ” as described above.
- FIG. 5 each of reference numerals 51 a and 51 b indicates the head mounted acoustic device 51 .
- reference numeral 51 a indicates the head mounted acoustic device 51 worn on the right ear
- reference numeral 51 b indicates the head mounted acoustic device 51 attached to the left ear.
- the head mounted acoustic device 51 includes a mounting unit 510 , a driver 511 , and an external microphone 513 .
- the mounting unit 510 illustrates a part worn on the user U in the housing of the head mounted acoustic device 51 .
- the mounting unit 510 has an outer shape in which that it is worn on the ear of the user U such that at least a part thereof is insertable into the ear hole of the user U who is the wearer.
- an ear hole insertion portion having a shape insertable into the ear hole of the user U is formed in the mounting unit 510 , and the mounting unit 510 is worn on the ears of the user U such that the ear hole insertion portion is inserted into the ear hole.
- FIG. 3 illustrates a state in which the mounting unit 510 of the head mounted acoustic device 51 is worn on the ear of the user U.
- the space in the mounting unit 510 corresponds to the internal space.
- the driver 511 is a component for driving an acoustic device such as the speaker and causing the acoustic device to output the sound based on the acoustic signal.
- the driver 511 causes the speaker to output the sound based on the acoustic signal by vibrating a vibration plate of the speaker on the basis of an input analog acoustic signal (that is, a drive signal).
- the microphone amplifier 111 is a so-called amplifier for adjusting a gain of the acoustic signal.
- the ambient sound collected by the external microphone 513 undergoes gain adjustment (for example, amplification) performed by the microphone amplifier 111 and is then input to the HT filter 121 .
- the HT filter 121 corresponds to the filter circuit 72 (see FIG. 2 ) in the case of implementing the hear-through effect described above with reference to FIGS. 2 to 4 .
- the HT filter 121 performs signal processing based on the filter coefficient ⁇ described on the basis of (Formula 2) and (Formula 3) on the acoustic signal output from the microphone amplifier 111 (that is, the acoustic signal which has been collected by the external microphone 513 and has undergone the gain adjustment performed by the microphone amplifier 111 ).
- the acoustic signal output as a result of performing signal processing by the HT filter 121 is hereinafter also referred to as a “difference signal.”
- the ambient sound in a case in which the user directly hears it is simulated (that is, the hear-through effect is implemented) by adding the difference signal and the ambient sound propagating to the internal space via the mounting unit 510 of the head mounted acoustic device 51 (that is, the sound propagating via the propagation environment F in FIGS. 2 and 3 ).
- the HT filter 121 corresponds to an example of a “first filter processing unit.”
- the EQ 131 outputs the sound input which has undergone the equalizing process to the adding unit 123 .
- the adding unit 123 adds the difference signal output from the HT filter 121 to the sound input output from the EQ 131 (that is, the sound input that has undergone the equalizing process) and outputs the acoustic signal generated as the addition result to the power amplifier 141 .
- the power amplifier 141 is a so-called amplifier for adjusting the gain of the acoustic signal.
- the acoustic signal output from the adding unit 123 (that is, the addition result of the sound input and the difference signal) undergoes gain adjustment (that is, amplification) performed by the power amplifier 141 and is then output to the driver 511 .
- the driver 511 drives the speaker on the basis of the acoustic signal output from the power amplifier 141 , and thus the sound based on the acoustic signal is radiated into the internal space inside the mounting unit 510 (that is, the space connected to the external ear canal UA of the user U).
- the sound radiated into the internal space by the driver 511 driving the speaker is added to the ambient sound propagating to the internal space (that is, the sound propagating via the propagation environment F in FIGS. 2 and 3 ) via the mounting unit 510 of the head mounted acoustic device 51 and heard by the user U.
- the component of the difference signal included in the sound radiated from the driver 511 to the internal space is added to the ambient sound propagated to the internal space via the mounting unit 510 and heard by the user U.
- the user U is able to hear the ambient sound in a manner similar to that in the case in which the head mounted acoustic device 51 is not worn as illustrated in FIG. 4 in addition to the sound input such as the audio content.
- the operation of the signal processing device 80 described above is merely an example, and the signal processing device 80 need not necessarily faithfully reproduce the hear-through effect if the user U is able to hear the ambient sound in a state in which the user U is wearing the head mounted acoustic device 51 .
- the HT filter 121 may control a characteristic and a gain of the difference signal such that the user U feels the volume of the ambient sound higher than in the state in which the user U does not wear the head mounted acoustic device 51 .
- the HT filter 121 may control the characteristic and the gain of the difference signal so that the user U feels the volume of the ambient sound lower than in the state where the user U does not wear the head mounted acoustic device 51 .
- FIG. 6 is an explanatory diagram for describing a mechanism in which the vibration of the voice uttered by the user propagates in the internal space.
- the vibration of the voice uttered by the user U propagates to the external ear canal UA via bones or flesh in the head of the user U, so that the external ear canal wall is vibrated like a secondary speaker.
- the head mounted acoustic device 51 having the high hermeticity such as a canal type earphone is worn
- a degree of hermeticity of the space in the external ear canal UA is increased by the head mounted acoustic device 51 , and an escape route in the air is limited, and thus the vibration in the space is directly transferred to the eardrum.
- Signal processing devices were made in view of the problem as described above, and it is desirable to implement to implement the hear-through effect in a more appropriate manner (that is, in a manner in which the user has a less strange feeling).
- FIG. 7 is a block diagram illustrating an example of a functional configuration of the signal processing device according to the present embodiment.
- the signal processing device according to the present embodiment is also referred to as a “signal processing device 11 ” in order to be distinguished from the signal processing device 80 (see FIG. 5 ).
- illustration of the DAC and the ADC is omitted in the functional configuration illustrated in FIG. 7 .
- the acoustic signal collected by the internal microphone 515 includes a component of the sound output from the speaker on the basis of control performed by the driver 511 , a component of the ambient sound propagating to the internal space via the mounting unit 510 (the sound propagating via the propagation environment F in FIGS. 2 and 3 ), and a component of a voice of the user propagating to the external ear canal UA (the component of the voice illustrated in FIG. 6 ). Further, the sound collected by the internal microphone 515 (that is, the sound propagating to the internal space) corresponds to an example of a “second sound.”
- the microphone amplifier 151 is a so-called amplifier that adjusts the gain of the acoustic signal.
- the acoustic signal based on the sound collection result obtained by the internal microphone 515 (that is, the sound collection result for the sound propagating to the internal space) undergoes gain adjustment (for example, amplification) performed by the microphone amplifier 151 and is then input to the subtracting unit 171 .
- the EQ 132 is a component for performing the equalizing process on the sound input in accordance with the device characteristics of the internal microphone 515 and the microphone amplifier 151 . Specifically, in a case in which the transfer function based on the device characteristics of the internal microphone 515 and the microphone amplifier 151 is indicated by M, the EQ 132 applies a frequency characteristic which is “target characteristic—M” to the sound input.
- the transfer function M corresponding to the device characteristics of the internal microphone 515 and the microphone amplifier 151 may be calculated in advance on the basis of a result of a prior experiment or the like. Then, the EQ 132 outputs the sound input which has undergone the equalizing process to the subtracting unit 171 .
- the sound input which has undergone the equalizing process performed by EQ 132 corresponds to an example of a “second signal component.”
- the subtracting unit 171 subtracts the sound input output from the EQ 132 (that is, the sound input to which the frequency characteristic which is “target characteristic—M” is applied) from the acoustic signal output from the microphone amplifier 151 , and outputs the acoustic signal generated as a subtraction result to the occlusion canceller 161 .
- the acoustic signal output as the subtraction result obtained by the subtracting unit 171 corresponds to the acoustic signal in which the component of the sound input among the components of the acoustic signal collected by the internal microphone 515 is suppressed.
- the acoustic signal includes a component in which the difference signal and the ambient sound propagating to the internal space via the mounting unit 510 are added (hereinafter also referred to as an “ambient sound component”) and the component of the voice of the user U propagating to the external ear canal UA via bones or flesh of the head of the user U (hereinafter also referred to simply as a “voice component”).
- an ambient sound component a component in which the difference signal and the ambient sound propagating to the internal space via the mounting unit 510 are added
- voice component the component of the voice of the user U propagating to the external ear canal UA via bones or flesh of the head of the user U
- the occlusion canceller 161 corresponds to a so-called filter processing unit operating on a principle similar to that of so-called feed-back (FB) type NC filter.
- the occlusion canceller 161 generates an acoustic signal for suppressing the component of the acoustic signal to a predetermined volume (hereinafter also referred to as a “noise reduction signal”) on the acoustic signal output from the subtracting unit 171 .
- the acoustic signal output from the subtracting unit 171 includes the ambient sound component and the voice component, and the low frequency side of the voice component is amplified due to a property of a propagation path. Therefore, for example, in order to enable the user U to hear the voice component in a manner similar to that in the case in which the user U does not wear the head mounted acoustic device 51 , the occlusion canceller 161 may generate the noise reduction signal for suppressing the low frequency side of the voice component among the voice components of the acoustic signal acquired from the subtracting unit 171 . Further, the occlusion canceller 161 corresponds to an example of a “second signal processing unit.”
- the occlusion canceller 161 generates the noise reduction signal on the basis of the acoustic signal output from the subtracting unit 171 . Then, the occlusion canceller 161 outputs the generated noise reduction signal to the adding unit 123 .
- the EQ 131 performs the equalizing process on the sound input, similarly to the EQ 131 described above with reference to FIG. 5 .
- the EQ 131 further performs the equalizing process on the sound input in accordance with to a characteristic to be applied to the output sound depending on a structure or the like of the speaker driven by the driver 511 and the transfer function of the space from the speaker to the internal microphone 515 .
- a characteristic to be applied to the output sound depending on a structure or the like of the speaker driven by the driver 511 and the transfer function of the space from the speaker to the internal microphone 515 For example, a function obtained by multiplying the transfer function corresponding to the characteristic applies to the output sound depending on the structure or the like of the speaker driven by the driver 511 by the transfer function of the space from the speaker to the internal microphone 515 is indicated by H.
- the EQ 131 applies a frequency characteristic which is “target characteristic 1/H to the sound input.
- the EQ 131 outputs the sound input which has undergone the equalizing process to the adding unit 123 .
- the adding unit 123 adds the difference signal output from the HT filter 121 and the noise reduction signal output from the occlusion canceller 161 to the sound input output from the EQ 131 (that is, the sound input after the equalizing process). Then, the adding unit 123 outputs the acoustic signal generated as an addition result to the power amplifier 141 .
- the acoustic signal output from the adding unit 123 undergoes gain adjustment (for example, amplification) performed by the power amplifier 141 and is then output to the driver 511 .
- the driver 511 drives the speaker on the basis of the acoustic signal output from the power amplifier 141 , and thus the sound based on the acoustic signal is radiated into the internal space in the mounting unit 510 (that is, the space connected with the external ear canal UA of the user U Space).
- the example of the functional configuration of the signal processing device 11 according to the present embodiment has been described above with reference to FIG. 7 .
- the configuration of the signal processing device 11 is not necessarily limited to the example illustrated in FIG. 7 as long as the operations of the components of the signal processing device 11 described above can be implemented.
- FIG. 8 is an explanatory diagram for describing an example of the configuration of the signal processing device 11 according to the present embodiment.
- the head mounted acoustic device 51 and the signal processing device 11 are configured as different devices.
- FIG. 8 illustrates an example of a configuration in a case in which the head mounted acoustic device 51 and the signal processing device 11 are installed in the same housing.
- a configuration for example, a signal processing unit
- corresponding to the signal processing device 11 is installed in the mounting unit 510 of the head mounted acoustic device 51 .
- the signal processing device 11 may be configured as an independent device or may be configured as a part of an information processing device such as a so-called smartphone or the like. Further, at least some components of the signal processing device 11 may be installed in an external device (for example, a server or the like) different from the signal processing device 11 . In this case, it is preferable that a delay amount before the ambient sound propagating via the external environment is collected by the external microphone 513 and output from the speaker of the head mounted acoustic device 51 via the HT filter 121 and the power amplifier 141 , including the conversion process performed by the ADC and the DAC, be suppressed to be about 100 ⁇ s or less.
- the signal processing device 11 generates the noise reduction signal for suppressing at least some components among the voice components of the user U on the basis of the sound collection result obtained by the internal microphone 515 (that is, the sound collection result for the sound propagating to the internal space). Then, the signal processing device 11 adds the generated difference signal and the noise reduction signal to the input sound input, and outputs the added acoustic signal. Accordingly, the driver 511 of the head mounted acoustic device 51 drives the speaker on the basis of the acoustic signal output from the signal processing device 11 , and thus the sound based on the acoustic signal is radiated into the internal space.
- the sound radiated into the internal space when the driver 511 drives the speaker includes a component based on the noise reduction signal generated by the occlusion canceller 161 .
- the component on the basis of the noise reduction signal is added to the voice component of the user U propagating to the external ear canal UA in the internal space on the basis of an utterance of the user U. Accordingly, at least some components among the voice components (for example, the component on the lower frequency side among the voice components) is suppressed, and the suppressed voice component reaches the eardrum of the user U and is heard by the user U.
- the signal processing device 11 of the present embodiment it is possible to implement the hear-through effect in a manner in which the user U has no strange feeling in his/her voice being heard.
- the hear-through effect is implemented in a manner in which the user U has no strange feeling in his/her voice being heard by providing the occlusion canceller 161 .
- the acoustic signal to be processed by the occlusion canceller 161 includes the component of the difference signal output from the speaker of the head mounted acoustic device 51 .
- the signal processing device according to the present embodiment was made in view of the problem described above, and it is desirable to implement the hear-through effect in a more natural manner (that is, in a manner in which the user has a less strange feeling) than the signal processing device 11 according to the first embodiment.
- the signal processing device according to the present embodiment is also referred to as a “signal processing device 12 ” in order to be distinguished from the signal processing device 11 according to the first embodiment.
- FIG. 9 is a block diagram illustrating an example of a functional configuration of a signal processing device according to the present embodiment. Further, similarly to the examples illustrated in FIGS. 5 and 7 , in order to facilitate understanding of description, illustration of the DAC and the ADC is omitted in the functional configuration illustrated in FIG. 9 .
- the signal processing device 12 according to the present embodiment differs from the signal processing device 11 according to the first embodiment (see FIG. 7 ) in that a monitor canceller 181 and a subtracting unit 191 are provided. Therefore, in the following description, the functional configuration of the signal processing device 12 according to the present embodiment will be described focusing on a difference with the signal processing device 11 according to the first embodiment described above (see FIG. 7 ).
- the monitor canceller 181 and the subtracting unit 191 are configured to suppress a component corresponding to the difference signal among components in the acoustic signal output from the microphone amplifier 151 (that is, the acoustic signal on the basis of the sound collection result of the internal microphone 515 ).
- the ambient sound collected by the external microphone 513 undergoes gain adjustment (for example, amplification) performed by the microphone amplifier 111 and is then input to the HT filter 121 and the monitor canceller 181 .
- gain adjustment for example, amplification
- the monitor canceller 181 performs the signal processing based on the filter coefficient ⁇ described on the basis of (Formula 2) and (Formula 3) on the acoustic signal output from the microphone amplifier 111 , and generates the difference signal.
- the monitor canceller 181 performs a filter process on the generated difference signal on the basis of the transfer function corresponding to each characteristic so that influences of the device characteristic of each of the power amplifier 141 , the driver 511 , and the microphone amplifier 151 and a spatial characteristic in the internal space are reflected. This is because a characteristic of a route from the occlusion canceller 161 to the occlusion canceller 161 via the power amplifier 141 , the driver 511 , and the microphone amplifier 151 is not reflected in the acoustic signal output from the microphone amplifier 111 .
- an infinite impulse response filter an IIR filter
- a finite impulse response filter a finite impulse response filter
- a simple process for a delay component may be mainly allocated to the FIR filter
- a process related to frequency characteristic may be mainly allocated to the IIR filter.
- the configuration in which the IIR filter and the FIR filter are installed is merely an example, and the configuration of the monitor canceller 181 is not necessarily limited.
- the FIR filter may be installed in the monitor canceller 181 , and both of the simple process for the delay component and the process related to the frequency characteristic may be executed by the FIR filter.
- the filter process may be implemented only by the IIR filter.
- a method for reducing the influence of the delay component for example, a method of employing the ADC and the DAC or employing a low-delay device as a filter (for example, a decimation filter) used for bit rate conversion may be used.
- a device having a smaller driving delay that is, a more responsive device
- a sound system such as the driver 511 (and the speaker), the external microphone 513 , or the internal microphone 515 .
- a sound speed delay between the speaker and the internal microphone 515 may be reduced by bringing the speaker driven by the driver 511 and the internal microphone 515 closer to each other in the internal space.
- the device characteristic of each of the power amplifier 141 , the driver 511 , and the microphone amplifier 151 and the spatial characteristic in the internal space may be derived in advance using, for example, a time stretched pulse (TSP) or the like.
- each characteristic may be calculated on the basis of measurement results of the acoustic signal (TSP) input from the power amplifier 141 (specifically, the DAC) and the acoustic signal output from the microphone amplifier 151 .
- TSP time stretched pulse
- the device characteristics of each of the power amplifier 141 , the driver 511 , and the microphone amplifier 151 and the spatial characteristic in the internal space may be individually measured, and the respective measurement results may be convoluted.
- the filter characteristic of the monitor canceller 181 may be adjusted in advance on the basis of the prior measurement result of each characteristic described above.
- the monitor canceller 181 corresponds to an example of a “third filter processing unit.” Further, the acoustic signal which has undergone the filter process performed by the monitor canceller 181 corresponds to a “first signal component.”
- the monitor canceller 181 outputs the difference signal which has undergone various kinds of filter processes to the subtracting unit 191 .
- the subtracting unit 191 subtracts the difference signal output from the monitor canceller 181 from the acoustic signal output from the microphone amplifier 151 , and outputs the acoustic signal generated as a subtraction result to the subtracting unit 171 positioned at a subsequent stage.
- the acoustic signal output as the subtraction result obtained by the subtracting unit 171 corresponds to an acoustic signal in which the component corresponding to the difference signal among the components of the acoustic signal collected by the internal microphone 515 is suppressed.
- a subsequent process is similar to that of the signal processing device 11 according to the first embodiment.
- the component of the sound input output from the EQ 132 is subtracted from the acoustic signal output from the subtracting unit 191 through the subtracting unit 171 , and the resulting acoustic signal is then input to the occlusion canceller 161 .
- the acoustic signal input to the occlusion canceller 161 is an acoustic signal in which the component corresponding to a difference signal and the component corresponding to the sound input among the components of the acoustic signal collected by the internal microphone 515 are suppressed (that is, the voice component).
- the signal processing device 12 it is possible to exclude the component of the difference signal from a processing target from which the occlusion canceller 161 generates the noise reduction signal. In other words, in the signal processing device 12 according to the present embodiment, it is possible to prevent the component of the difference signal from being suppressed by the noise reduction signal. Therefore, the signal processing device 12 according to the present embodiment is able to implement the hear-through effect in a more natural manner (that is, a manner in which the user U has a less strange feeling) than in the signal processing device 11 according to the first embodiment.
- a route indicated by reference numeral R 11 that is, a route on which the acoustic signal based on the sound collection result of the external microphone 513 is radiated into the internal space via the microphone amplifier 111 , the HT filter 121 , the power amplifier 141 , and the driver 511 is focus on.
- the route R 11 in order to implement the hear-through effect in a preferable manner (specifically, in order to adjust the frequency band at which the dip occurs to be around 5 kHz), it is preferable to suppress the delay amount to be 100 ⁇ s or less.
- the delay amount of the route R 11 is also referred to as a “delay amount D_HTF.”
- a route indicated by reference numeral R 13 that is, a route on which the acoustic signal based on the sound collection result of the external microphone 513 reaches the subtracting unit 191 via the monitor canceller 181 is focused on.
- the monitor canceller 181 generates the difference signal, similarly to the HT filter 121 .
- a propagation delay will occur (propagates between the speaker and the internal microphone 515 ) before the driver 511 drives the speaker on the basis of the difference signal, and so the acoustic signal based on the sound including the component of the difference signal radiated into the internal space propagates in the space inside the internal space and is collected by the internal microphone 515 .
- a delay amount of the propagation delay in the internal space is also referred to as a “delay amount D_ACO.”
- the delay amount of the route R 13 is necessary to cause the delay amount of the route R 13 to be equal to or less than a value obtained by adding the delay amount D_HTF (100 ⁇ s) and the delay amount D_ACO.
- a distance between the speaker driven by the driver 511 and the internal microphone 515 is about 3 to 4 cm even in a case of a relatively long headphone such as a so-called overhead type headphone.
- the delay amount of the route R 13 is set to D_HTC, it is necessary to satisfy a relation of the delay amount D_HTC ⁇ D_HTF+D_ACO and satisfy a relation of D_HTF ⁇ 100 ⁇ s and D_ACO ⁇ 100 ⁇ s.
- FIG. 10 is an explanatory diagram for describing an example of a configuration for further reducing the delay amount in the signal processing device 12 according to the present embodiment (that is, satisfying the delay condition described above).
- an ADC and a DAC that perform a conversion process between an analog signal and a digital signal and a filter that converts a sampling rate of a digital signal are explicitly illustrated for the signal processing device 12 illustrated in FIG. 9 .
- FIG. 10 explicitly illustrates ADCs 112 and 152 , a DAC 142 , decimation filters 113 and 153 , and interpolation filters 133 , 134 , and 143 for the functional configuration of the signal processing device 12 illustrated in FIG. 9 .
- the ADCs 112 and 152 are components for converting an analog acoustic signal into a digital signal.
- the ADCs 112 and 152 perform conversion into a digital signal by performing delta-sigma modulation on the analog acoustic signal.
- the DAC 142 is a component for converting a digital signal into an analog acoustic signal.
- the decimation filters 113 and 153 are components for down-sampling a sampling rate of an input digital signal to a predetermined sampling rate lower than the sampling rate.
- the interpolation filters 133 , 134 , and 143 are components for up-sampling the sampling rate of the input digital signal to a predetermined sampling rate higher than the sampling rate.
- the analog acoustic signal output on the basis of the sound collection result of the external microphone 513 undergoes gain adjustment performed by the microphone amplifier 111 and then converted into a digital signal through the ADC 112 .
- the ADC 112 performs sampling on the input analog signal at the sampling rate of 64 Fs to be converted into a digital signal.
- the ADC 112 outputs the converted digital signal to the decimation filter 113 .
- the decimation filter 113 down-samples the sampling rate of the digital signal output from the ADC 112 from 64 Fs to 8 Fs.
- the components positioned at a stage subsequent to the decimation filter 113 for example, the HT filter 121 and the monitor canceller 181 ) perform various kinds of processes on the digital signal whose sampling rate is down-sampled to 8 Fs.
- the analog acoustic signal output on the basis of the sound collection result of the internal microphone 515 undergoes gain adjustment performed by the microphone amplifier 151 and converted into a digital signal through the ADC 152 .
- the ADC 152 performs sampling on the input analog signal at the sampling rate of 64 Fs to be converted into a digital signal.
- the ADC 152 outputs the converted digital signal to the decimation filter 153 .
- the decimation filter 153 down-samples the sampling rate of the digital signal output from the ADC 152 from 64 Fs to 8 Fs.
- the component positioned at a stage subsequent to the decimation filter 153 (for example, the occlusion canceller 161 ) perform various kinds of processes on the digital signal whose sampling rate is down-sampled to 8 Fs.
- the sound input (the digital signal of 1 Fs) which has undergone the equalizing process performed by the EQ 132 is up-sampled to the sampling rate of 8 Fs by the interpolation filter 134 and then input to the subtracting unit 171 .
- the sound input (the digital signal of 1 Fs) which has undergone the equalizing process performed by the EQ 131 is up-sampled to the sampling rate of 8 Fs by the interpolation filter 133 and then input to the adding unit 123 .
- the addition unit 123 adds the difference signal output from the HT filter 121 , the sound input output from the interpolation filter 133 , and the noise reduction signal output from the occlusion canceller 161 .
- all of the difference signal, the sound input, and the noise reduction signal added by the adding unit 123 are digital signals of 8 Fs.
- the digital signal of 8 Fs output as the addition result of the adding unit 123 is up-sampled to a digital signal of 64 Fs by the interpolation filter 143 , converted into an analog acoustic signal by the DAC 142 , and input to the power amplifier 141 .
- the analog acoustic signal undergoes gain adjustment performed by the power amplifier 141 and then input to the driver 511 . Accordingly, when the driver 511 drives the speaker on the basis of the inputted analog acoustic signal, the speaker radiates the sound based on the analog acoustic signal into the internal space.
- the signal processing device 12 down-samples the digital signal of 64 Fs obtained by converting the collected analogue acoustic signal to about 8 Fs higher than the sampling rate (1 Fs) of the sound input.
- the HT filter 121 executes each calculation (that is, the filter process) on the digital signal of 8 Fs, and thus it is possible to reduce a delay of one sampling unit.
- the signal processing device 12 illustrated in FIG. 10 since the digital signal of 64 Fs is down-sampled to the digital signal of 8 Fs, it is possible to suppress the delay amount of the processes related to the down-sampling (that is, the processes of the ADC 112 and the ADC 152 ) to be smaller than in the case of down-sampling to the digital signal of 1 Fs. This similarly applies to the processes related to the up-sampling. In other words, in the signal processing device 12 illustrated in FIG.
- down-sampling to the digital signal of the lower sampling rate may be further performed, and then the digital signal may be a processing target of at least some calculations of the HT filter 121 , the monitor canceller 181 , and the occlusion canceller 161 .
- FIG. 11 is a diagram illustrating an example of a functional configuration of the monitor canceller 181 .
- the monitor canceller 181 illustrated in FIG. 11 is configured so that various kinds of filter processes are executed on the digital signal of 1 Fs after the digital signal of 8 Fs is down-sampled to the digital signal of 1 Fs.
- the monitor canceller 181 illustrated in FIG. 11 includes a decimation filter 183 , an IIR filter 184 , an FIR filter 185 , and an interpolation filter 186 .
- the decimation filter 183 down-samples the digital signal of 8 Fs input to the monitor canceller 181 into a digital signal of 1 Fs and outputs the digital signal down-sampled to 1 Fs to the IIR filter 184 positioned at a subsequent stage.
- the IIR filter 184 and the FIR filter 185 are components for executing the filter process performed by the monitor canceller 181 described above with reference to FIG. 9 .
- the process related to the frequency characteristic is mainly allocated to the IIR filter 184
- the simple process for the delay component is allocated to the FIR filter 185 .
- the IIR filter 184 and the FIR filter 185 execute various kinds of filter processes on the digital signal of 1 Fs.
- the digital signal (that is, the digital signal of 1 Fs) which has undergone various kinds of filter processes performed by the IIR filter 184 and the FIR filter 185 is up-sampled to the digital signal of 8 Fs through the interpolation filter 186 . Then, the digital signal up-sampled to 8 Fs is output to the subtracting unit 191 (see FIG. 10 ) positioned at a stage subsequent to the monitor canceller 181 .
- resources for the calculations may be reduced by reducing the sampling rate locally for at least some calculations among various kinds of calculations (for example, the calculations in the HT filter 121 , the monitor canceller 181 , and the occlusion canceller 161 ).
- a calculation in which the sampling rate is locally reduced among various kinds of calculations in the signal processing device 12 may be appropriately decided on the basis of a checking result of checking efficiency of resource reduction associated with the down-sampling through a prior experiment or the like.
- the example of the mechanism for reducing the delay amount of each route (for example, the routes R 11 and R 13 illustrated in FIGS. 9 and 10 ) in the signal processing device 12 according to the present embodiment and implementing the hear-through effect in a more appropriate manner has been described above with reference to FIGS. 9 and 10 .
- the example of the mechanism for reducing the delay amount through the signal processing device 12 illustrated in FIG. 9 has been described above, but it will be appreciated that it is possible to reduce the delay amount on the basis of a similar mechanism even in the signal processing device 80 illustrated in FIG. 5 or the signal processing device 11 illustrated in FIG. 7 .
- FIG. 12 is a block diagram illustrating an example of a functional configuration of a signal processing device according to a modified example of the present embodiment.
- the signal processing device according to the modified example is also referred to as a “signal processing device 13 ” to be distinguished from the signal processing device 12 according to the present embodiment described above with reference to FIGS. 9 and 10 .
- the ADC and the DAC that perform the conversion process between the analog signal and the digital signal and the filter that converts the sampling rate of the digital signal are explicitly illustrated.
- the signal processing device 13 according to the modified example differs from the signal processing device 12 according to the above embodiment (see FIG. 10 ) in that a monitor canceller 181 ′ is provided instead of the monitor canceller 181 illustrated in FIG. 12 . Therefore, the present description will proceed, particularly, focusing on a configuration of the monitor canceller 181 ′, and the remaining components are similar to those of the signal processing device 12 according to the above embodiment, and thus detailed description thereof is omitted.
- the monitor canceller 181 ′ is positioned at a stage subsequent to the HT filter 121 and processes the difference signal output from the HT filter 121 . Due to this configuration, the monitor canceller 181 ′ need not perform the process related to the generation of the difference signal (that is, the process based on (Formula 2) and (Formula 3) described above, unlike the monitor canceller 181 described above with reference to FIG. 9 .
- the monitor canceller 181 ′ performs the filter process based on the transfer function corresponding to each characteristic on the inputted difference signal so that the influences of the device characteristic of each of the power amplifier 141 , the driver 511 , and the microphone amplifier 151 and the spatial characteristic in the internal space are reflected.
- the monitor canceller 181 ′ outputs the difference signal which has undergone the filter process to the subtracting unit 191 positioned at a subsequent stage.
- a subsequent process is similar to that of the signal processing device 12 according to the above embodiment (see FIGS. 9 and 10 ).
- the signal processing device 13 according to the modified example can communalize the process related to the generation of the difference signal in the HT filter 121 and the monitor canceller 181 of the signal processing device 12 illustrated in FIGS. 9 and 10 as the process of the HT filter 121 . Therefore, as compared with the signal processing device 12 according to the above-described embodiment, the signal processing device 13 according to the modified example is able to reduce the resources for the calculation related to the generation of the difference signal, and thus it is possible to reduce the circuit size.
- the signal processing device 13 according to the modified example of the present embodiment has been described above with reference to FIG. 12 .
- the signal processing device 12 subtracts the component corresponding to the difference signal from the acoustic signal based on the sound collection result of the internal microphone 515 in addition to the component of the sound input.
- the signal processing device 12 according to the present embodiment it is possible to exclude the component of the difference signal from the processing target from which the occlusion canceller 161 generates the noise reduction signal.
- the signal processing device 12 according to the present embodiment it is possible to prevent the component of the difference signal from being suppressed by the noise reduction signal. Therefore, the signal processing device 12 according to the present embodiment is able to implement the hear-through effect in a more natural manner (that is, a manner in which the user U has a less strange feeling) than in the signal processing device 11 according to the first embodiment.
- the noise reduction signal for suppressing the voice component of the user propagating to the external ear canal UA is generated using the sound collection result of collecting the sound propagating in the internal space through the internal microphone 515 . Due to this configuration, the acoustic signal based on the sound collection result of the internal microphone 515 (that is, the sound propagating in the internal space) includes the voice component (that is, the voice component of the user U propagating to the external ear canal UA via the bones or fresh of the head of the user U) as described above.
- an example of a signal processing device which is capable of using the voice component included in the acoustic signal based on the sound collection result obtained by the internal microphone 515 as a voice input (for example, a transmission signal in a voice call) will be described.
- FIG. 13 is a block diagram illustrating an example of a functional configuration of a signal processing device according to the present embodiment.
- the signal processing device illustrated in FIG. 13 is also referred to as a “signal processing device 14 a ” to be distinguished from the signal processing device according to each embodiment.
- illustration of the DAC and the ADC is omitted in order to facilitate understanding of the description.
- the signal processing device 14 a according to the present embodiment differs from the signal processing device 13 according to the second embodiment (see FIG. 9 ) in that a noise gate 411 , an EQ 412 , and a compressor 413 are provided.
- a noise gate 411 an EQ 412
- a compressor 413 a compressor
- an acoustic signal passing through the node n 11 is split, and some split acoustic signals are input to the noise gate 411 .
- the noise gate 411 is a component for performing a so-called noise gate process on the input acoustic signal. Specifically, as the noise gate process, the noise gate 411 performs a process of lowering a level of an output signal at which a level of an input acoustic signal is equal to or less than a certain level (that is, closes a gate) and causing the level of the output signal to an original level (that is, opens the gate) if it exceeds the certain level.
- a certain level that is, closes a gate
- parameters in the noise gate process such as an attenuation rate of the output level, opening and closing envelopes of the gate, and a frequency band at which the gate responds are appropriately set so that an articulation rate of an uttered sound (that is, a voice component included in an input acoustic signal) is improved.
- the noise gate 411 outputs the acoustic signal which has undergone the noise gate process to the EQ 412 positioned at a subsequent stage.
- the EQ 412 is a component for performing the equalizing process on the acoustic signal output from the noise gate 411 .
- the low-frequency side of the voice component included in the acoustic signal split at the node n 11 (that is, the acoustic signal based on the sound collection result of the internal microphone 515 ) is amplified, and the sound based on the acoustic signal (that is, the voice component) is heard by the listener as if it is muffled.
- the EQ 412 improves the articulation rate of the sound to be heard by correcting the frequency characteristic of the acoustic signal so that the sound based on the acoustic signal is heard naturally by the listener (that is, so that a more natural frequency characteristic balance is obtained).
- the target characteristic that enables the EQ 412 to perform the equalizing process on the input acoustic signal may be decided on the basis of a result of a prior experiment or the like in advance.
- the EQ 412 outputs the acoustic signal which has undergone the equalizing process (that is, the acoustic signal including the voice component) to the compressor 413 positioned at a subsequent stage.
- the compressor 413 is a component for performing a process for adjusting a time amplitude on the input acoustic signal as a so-called compressor process.
- the voice component included in the input acoustic signal propagates to the external ear canal UA via the bones or fresh of the head of the user U and causes the external ear canal wall to vibrate like a secondary speaker, and the vibration reaches the internal microphone 515 via the external ear canal UA.
- the propagation path in which the voice component reaches the internal microphone 515 has non-linearity slightly as compared with the air propagation such as the propagation in the external environment.
- a difference in a magnitude of an uttered voice which varies depending on a magnitude of a generated voice is larger than in a case in which a normal voice propagating via the air is collected, and thus the listener may be unable to hear the voice collected without change.
- the compressor 413 arranges a time axis amplitude of the acoustic signal based on the sound collection result obtained by the internal microphone 515 (specifically, the acoustic signal output from the EQ 412 ) so that the difference in the magnitude of the uttered voice is suppressed.
- the compressor 413 performs the compressor process on the input acoustic signal, and outputs the acoustic signal which has undergone the compressor process (that is, the acoustic signal including the voice component) as a voice signal.
- the configuration of the signal processing device 14 a illustrated in FIG. 13 is merely an example, and the configuration is not particularly limited as long as it is possible to output the acoustic signal including the voice component collected by the internal microphone 515 as the voice signal.
- FIG. 14 is a block diagram illustrating another example of a functional configuration of the signal processing device according to the present embodiment.
- the signal processing device illustrated in FIG. 14 is also referred to as a “signal processing device 14 b ” to be distinguished from the signal processing device described above with reference to FIG. 13 .
- the signal processing device illustrated in FIG. 14 is not distinguished from the signal processing device described above with reference to FIG. 13 , it is also referred to simply as “signal processing device 14 .”
- an acoustic signal passing through the node n 12 is split, and some split acoustic signals are input to the noise gate 411 .
- the acoustic signal passing through the node n 12 corresponds to an acoustic signal obtained by further subtracting the component of the sound input from the acoustic signal passing through the node n 11 . Therefore, in the signal processing device 14 b illustrated in FIG. 14 , it is possible to output the acoustic signal in which components other than the voice components are further suppressed in the acoustic signals based on the sound collection result of the internal microphone 515 as the voice signal as compared with the signal processing device 14 a illustrated in FIG. 13 .
- the acoustic signal obtained by subtracting the difference signal from the acoustic signal based on the sound collection result of the internal microphone 515 through the subtracting unit 191 is output as the voice signal.
- the acoustic signal in which the component corresponding to the ambient sound among the components included in the acoustic signal based on the sound collection result of the internal microphone 515 is suppressed is output as the voice signal.
- FIG. 15 is an explanatory diagram for describing an application example of the signal processing device 14 according to the present embodiment.
- FIG. 15 illustrates an example of a functional configuration of an information processing system which is capable of executing various kinds of processes on the basis of instruction content indicated by the voice input by using the voice signal output from the signal processing device 14 as the voice input.
- the information processing system illustrated in FIG. 15 includes a head mounted acoustic device 51 , a signal processing device 14 , an analyzing unit 61 , a control unit 63 , and a processing executing unit 65 . Since the head mounted acoustic device 51 and the signal processing device 14 are similar to those in the example illustrated in FIG. 13 or FIG. 14 , detailed description thereof will be omitted.
- the analyzing unit 61 is a component for acquiring the voice signal (that is, the voice output) output from the signal processing device 14 as the voice input and performing various kinds of analysis on the voice input so that the control unit 63 to be described later is able to recognize content indicated by the voice input (that is, the instruction content given from the user U).
- the analyzing unit 61 includes a voice recognizing unit 611 and a natural language processing unit 613 .
- the voice recognizing unit 611 converts the voice input acquired from the signal processing device 14 into character information by analyzing the voice input on the basis of a so-called voice recognition technique. Then, the voice recognizing unit 611 outputs a result of analysis based on the voice recognition technique, that is, the character information obtained by converting the voice input to the natural language processing unit 613 .
- the natural language processing unit 613 acquires the character information obtained by converting the voice input from the voice recognizing unit 611 as the result of analyzing the voice input obtained from the signal processing device 14 on the basis of the voice recognition technique.
- the natural language processing unit 613 performs analysis based on a so-called natural language processing technique (for example, lexical analysis (morphological analysis), syntax analysis, semantic analysis, or the like) on the acquired character information.
- a so-called natural language processing technique for example, lexical analysis (morphological analysis), syntax analysis, semantic analysis, or the like
- the natural language processing unit 613 outputs information indicating a result of performing natural language processing on the character information obtained by converting the voice input acquired from the signal processing device 14 to the control unit 63 .
- the control unit 63 acquires information indicating a result of analyzing the voice input acquired from the signal processing device 14 (that is, a result of performing natural language processing on the character information obtained by converting the voice input) from the analyzing unit 61 .
- the control unit 63 recognizes the instruction content given from the user U which is based on the voice input on the basis of the acquired analysis result.
- the control unit 63 specifies a target function (for example, an application) on the basis of the recognized instruction content given from the user U and instructs the processing executing unit 65 to execute the specified function.
- a target function for example, an application
- the processing executing unit 65 is a component for executing various kinds of functions. On the basis of the instruction given from the control unit 63 , The processing executing unit 65 reads various kinds of data for executing a target function (for example, a library for executing an application or data of content) and executes the function on the basis of the read data. Further, a storage destination of data for executing various kinds of functions through the processing executing unit 65 is not particularly limited as long as the data is stored at a position at which it is readable by the processing executing unit 65 .
- a target function for example, a library for executing an application or data of content
- the processing executing unit 65 may also input acoustic information based on a result of executing the function instructed from the control unit 63 (for example, audio content reproduced on the basis of an instruction) to the signal processing device 14 .
- the processing executing unit 65 may generate voice information indicating content to be presented to the user U on the basis of the result of executing the function instructed from the control unit 63 on the basis of a so-called voice synthesis technique and input the generated audio information to the signal processing device 14 .
- the user U is able to recognize results of executing various kinds of functions on the basis of the instruction content given from the user U as the acoustic information (voice information) output through the head mounted acoustic device 51 .
- the user U is able to instruct the information processing system to execute various kinds of functions by voice in the state in which the user UE wear the head mounted acoustic device 51 and hear the acoustic information based on the result of executing the functions through the head mounted acoustic device 51 .
- the user U is able to give an instruction to reproduce desired audio content by voice and hear a result of reproducing audio content through the head mounted acoustic device 51 .
- the user is able to instruct the information processing system to read desired character information (for example, a delivered e-mail, news, information uploaded to a network, or the like) and hear a result of reading the character information through the head mounted acoustic device 51 .
- desired character information for example, a delivered e-mail, news, information uploaded to a network, or the like
- the information processing system illustrated in FIG. 15 may be used for a so-called voice call.
- the voice signal output from the signal processing device 14 may be used as a transmission signal, and a received signal may be input to the signal processing device 14 as the sound input.
- the configuration of the information processing system illustrated in FIG. 15 is merely an example, and the configuration illustrated in FIG. 15 is not necessarily limited as long as it is possible to implement the processes of the components of the information processing system described above.
- at least some of the analyzing unit 61 , the control unit 63 , and the processing executing unit 65 may be installed in an external device (for example, a server) connected via a network.
- FIG. 16 is a diagram illustrating an example of the hardware configuration of the signal processing device 10 according to each embodiment of the present disclosure.
- the signal processing device 10 includes a processor 901 , a memory 903 , a storage 905 , an operation device 907 , a notifying device 909 , an acoustic device 911 , a sound collecting device 913 , and a bus 917 . Further, the signal processing device 10 may include a communication device 915 .
- the processor 901 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), or a system on chip (SoC), and executes various processes of the signal processing device 10 .
- the processor 901 may be constituted by, for example, an electronic circuit that executes various kinds of calculation processes.
- the components of the signal processing devices 11 to 14 may be implemented by the processor 901 .
- the memory 903 includes a random access memory (RAM) and a read only memory (ROM), and stores programs and data executed by the processor 901 .
- the storage 905 may include a storage medium such as a semiconductor memory or a hard disk.
- the operation device 907 has a function of generating an input signal that enables the user to perform a desired operation.
- the operation device 907 may be configured as, for example, a touch panel.
- the operation device 907 may be configured with an input unit that enables the user to input information such as a button, a switch, and a keyboard, an input control circuit that generates an input signal on the basis of an input performed by the user and supplies the input signal to the processor 901 , and the like.
- the notifying device 909 is an example of an output device and may be a device such as a liquid crystal display (LCD) device, an organic EL (organic light emitting diode) display, or the like. In this case, the notifying device 909 is able to notify the user of predetermined information by displaying a screen.
- LCD liquid crystal display
- organic EL organic light emitting diode
- the notifying device 909 described above is merely an example, and a form of the notifying device 909 is not particularly limited as long as it is possible to notify the user of predetermined information.
- the notifying device 909 may be a device that notifies the user predetermined information by means of a lighting or blinking pattern such as a light emitting diode (LED).
- the notifying device 909 may be a device that notifies the user of predetermined information through vibration such as a so-called vibrator.
- the acoustic device 911 is a device that notifies the user of predetermined information by outputting a predetermined acoustic signal as in a speaker or the like.
- the speaker driven by the driver 511 may be configured with the acoustic device 911 .
- the sound collecting device 913 is a device that collects a voice uttered by the user or a sound coming from a surrounding environment and acquires them as acoustic information (acoustic signal) as in a microphone. Further, the sound collecting device 913 may acquire data indicating an analogue acoustic signal indicating the collected voice or sound as the acoustic information or may convert the analog acoustic signal into a digital acoustic signal, and acquire data indicating the converted digital acoustic signal as the acoustic information.
- Each of the external microphone 513 and the internal microphone 515 in the head mounted acoustic device 51 described above may be implemented by the sound collecting device 913 .
- the communication device 915 is a communication unit installed in the signal processing device 10 , and performs communication with an external device via a network.
- the communication device 915 is a communication interface for wired or wireless communication.
- the communication device 915 may include a communication antenna, a radio frequency (RF) circuit, a baseband processor, and the like.
- RF radio frequency
- the communication device 915 has a function of performing various kinds of signal processing on a signal received from the external device and is able to supply a digital signal generated from the received analog signal to the processor 901 .
- the bus 917 connects the processor 901 , the memory 903 , the storage 905 , the operation device 907 , the notifying device 909 , the acoustic device 911 , the sound collecting device 913 , and the communication device 915 with one another.
- the bus 917 may include a plurality of types of buses.
- a program causing hardware such as a processor, a memory, and a storage which are installed in a computer to perform functions similar to those of the components of the signal processing device 10 .
- a computer readable storage medium having the program stored therein may also be provided.
- the signal processing device 10 As described above, the signal processing device 10 according to each embodiment of the present disclosure (that is, the signal processing devices 11 to 14 described above) generates the difference signal on the basis of the sound collection result for the ambient sound propagating in the external space outside the mounting unit 510 of the head mounted acoustic device 51 . Further, the signal processing device 10 generates the noise reduction signal for suppressing the voice component propagating to the internal space on the basis of the sound collection result for the sound propagating to the internal space inside the mounting unit 510 . Then, the signal processing device 10 adds the generated difference signal and the noise reduction signal to the input sound input, and outputs the acoustic signal generated on the basis of the addition result to the driver 511 of the head mounted acoustic device 51 . Accordingly, the driver 511 is driven in accordance with the acoustic signal, and the sound based on the acoustic signal is radiated into the internal space.
- the component of the difference signal included in the sound radiated into the internal space and the ambient sound propagating to the internal space via the mounting unit 510 (that is, the sound propagating via the propagation environment F in FIGS. 2 and 3 ) are added in the internal space, and the addition result is heard by the user U, and thus the hear-through effect can be implemented.
- the noise reduction signal included in the sound radiated into the internal space and the voice component propagating to the external ear canal UA via the bones or fresh of the head of the user U are added, and the addition result is heard by the user U, and thus the user U is able to hear his/her voice in a more natural manner (that is, the user U has no strange feeling).
- a series of processes (that is, signal processing such as various kinds of filter processes) executed by the signal processing device 10 according to each embodiment of the present disclosure described above corresponds to an example of a “signal processing method.”
- present technology may also be configured as below.
- a signal processing device including:
- a first acquiring unit configured to acquire a sound collection result for a first sound propagating in an external space outside a mounting unit to be worn on an ear of a listener
- a second acquiring unit configured to acquire a sound collection result for a second sound propagating in an internal space connected with an external ear canal inside the mounting unit
- a first filter processing unit configured to generate a difference signal which is substantially equal to a difference between the first sound propagating directly from the external space toward an inside of the external ear canal and the first sound propagating from the external space to the internal space via the mounting unit on the basis of the sound collection result for the first sound;
- a subtracting unit configured to generate a subtraction signal obtained by subtracting a first signal component based on the sound collection result for the first sound and a second signal component based on an input acoustic signal to be output from an acoustic device from an inside of the mounting unit toward the internal space from the sound collection result for the second sound;
- a second filter processing unit configured to generate a noise reduction signal for reducing the subtraction signal on the basis of the subtraction signal
- an adding unit configured to add the difference signal and the noise reduction signal to the input acoustic signal to generate a drive signal for driving the acoustic device.
- the signal processing device including:
- a third filter processing unit configured to apply, to the acoustic signal based on the sound collection result for the first sound, a characteristic corresponding to at least a transfer function of a route on which the acoustic signal output from the acoustic device is collected as the second sound via the internal space, and output the acoustic signal based on the sound collection result for the first sound as the first signal component.
- the third filter processing unit generates the first signal component using the sound collection result for the first sound as an input signal.
- the third filter processing unit generates the first signal component using the difference signal output from the first filter processing unit as an input signal.
- the third filter processing unit includes a fourth filter processing unit configured to process a delay component in the acoustic signal based on the input sound collection result for the first sound and a fifth filter processing unit configured to process a frequency component.
- the fourth filter processing unit includes an infinite impulse response filter.
- the fifth filter processing unit includes a finite impulse response filter.
- the signal processing device including: a first equalization processing unit configured to equalize the input acoustic signal to a first target characteristic and output the equalized acoustic signal to the adding unit; and
- a second equalization processing unit configured to equalize the input acoustic signal to a second target characteristic and output the equalized acoustic signal to the subtracting unit as the second signal component.
- the signal processing device including: a voice signal output unit configured to output a signal component based on a result of subtracting the first signal component from the sound collection result for the second sound as a voice signal.
- the voice signal output unit outputs the subtraction signal as the voice
- the signal processing device including:
- the signal processing device including:
- a signal processing device including:
- an acquiring unit configured to acquire a sound collection result for a sound propagating in an external space outside a mounting unit to be worn on an ear of a listener
- a filter processing unit configured to generate a difference signal which is substantially equal to a difference between the sound directly propagating from the external space toward an inside of an external ear canal and the sound propagating from the external space to the inside of the external ear canal via the mounting unit on the basis of the sound collection result for the sound;
- an adding unit configured to add the difference signal to an input acoustic signal to be output from an acoustic device from an inside of the mounting unit toward the inside of the external ear canal to generate a drive signal for driving the acoustic device
- a delay amount before the sound propagating in the external space is collected, and then the sound based on the drive signal obtained by adding the difference signal based on the sound is output from the acoustic device is 100 ⁇ s or less.
- the signal processing device including:
- an AD converting unit configured to perform AD conversion of converting the sound collection result for the sound propagating in the external space into a first digital signal at a first sampling rate
- a decimation filter configured to generate a second digital signal by down-sampling the first digital signal to a third sampling rate which is lower than the first sampling rate and higher than a second sampling rate for sampling the input acoustic signal
- an interpolation filter configured to up-sample the digital signal sampled at the third sampling rate to the first sampling rate
- a DA converting unit configured to perform DA conversion of converting an output result of the interpolation filter into an analog acoustic signal
- the filter processing unit generates the difference signal using the second digital signal as an input signal.
- a signal processing method including, by a processor:
- a difference signal which is substantially equal to a difference between the first sound propagating directly from the external space toward an inside of the external ear canal and the first sound propagating from the external space to the internal space via the mounting unit on the basis of the sound collection result for the first sound;
- a difference signal which is substantially equal to a difference between the first sound propagating directly from the external space toward an inside of the external ear canal and the first sound propagating from the external space to the internal space via the mounting unit on the basis of the sound collection result for the first sound;
Landscapes
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Otolaryngology (AREA)
- Headphones And Earphones (AREA)
- Soundproofing, Sound Blocking, And Sound Damping (AREA)
- Circuit For Audible Band Transducer (AREA)
Abstract
Description
- This application is a continuation of and claims the benefit under 35 U.S.C. § 120 of U.S. patent application Ser. No. 15/565,524, titled “SIGNAL PROCESSING DEVICE, SIGNAL PROCESSING METHOD, AND PROGRAM,” filed Oct. 10, 2017, which is a National Stage of International Application No. PCT/JP2016/056504, filed in the Japanese Patent Office as a Receiving office on Mar. 2, 2016, which claims priority to Japanese Patent Application Number 2015-084817, filed in the Japanese Patent Office on Apr. 17, 2015, each of which is hereby incorporated by reference in its entirety.
- The present disclosure relates to a signal processing device, a signal processing method, and a program.
- In recent years, as acoustic devices which are worn on heads of users for use such as earphones or headphones (which may hereinafter be referred to as “head mounted acoustic devices”), in addition to devices that simply output acoustic information, devices with functions in which use situations are considered have become widespread. As a specific example, a head mounted acoustic device capable of suppressing ambient sounds (so-called noise) coming from an external environment and enhancing a sound insulation effect using a so-called noise canceling technique is known. Patent Literature 1 discloses an example of an acoustic device using such a noise canceling technique.
- Patent Literature 1: JP 4882773B
- Meanwhile, as information processing devices which are configured to be carried by users such as so-called smartphones, tablet terminals, and wearable terminals have become more widespread, use situations of the head mounted acoustic devices are no longer limited to listening to so-called audio content but have been further diversified.
- With the diversification of the use situations, desirable use situations in which listeners (users) are able to listen to ambient sounds coming from the external environment while wearing head mounted acoustic devices can be considered as well.
- In this regard, the present disclosure proposes a signal processing device, a signal processing method, and a program, which are capable of enabling a listener to listen to ambient sounds of the external environment in an appropriate manner while wearing a head mounted acoustic device.
- According to the present disclosure, there is provided a signal processing device, including: a first acquiring unit configured to acquire a sound collection result for a first sound propagating in an external space outside a mounting unit to be worn on an ear of a listener; a second acquiring unit configured to acquire a sound collection result for a second sound propagating in an internal space connected with an external ear canal inside the mounting unit; a first filter processing unit configured to generate a difference signal which is substantially equal to a difference between the first sound propagating directly from the external space toward an inside of the external ear canal and the first sound propagating from the external space to the internal space via the mounting unit on the basis of the sound collection result for the first sound; a subtracting unit configured to generate a subtraction signal obtained by subtracting a first signal component based on the sound collection result for the first sound and a second signal component based on an input acoustic signal to be output from an acoustic device from an inside of the mounting unit toward the internal space from the sound collection result for the second sound; a second filter processing unit configured to generate a noise reduction signal for reducing the subtraction signal on the basis of the subtraction signal; and an adding unit configured to add the difference signal and the noise reduction signal to the input acoustic signal to generate a drive signal for driving the acoustic device.
- Further, according to the present disclosure, there is provided a signal processing method, including, by a processor: acquiring a sound collection result for a first sound propagating in an external space outside a mounting unit to be worn on an ear of a listener; acquiring a sound collection result for a second sound propagating in an internal space connected with an external ear canal inside the mounting unit; generating a difference signal which is substantially equal to a difference between the first sound propagating directly from the external space toward an inside of the external ear canal and the first sound propagating from the external space to the internal space via the mounting unit on the basis of the sound collection result for the first sound; generating a subtraction signal obtained by subtracting a first signal component based on the sound collection result for the first sound and a second signal component based on an input acoustic signal to be output from an acoustic device from an inside of the mounting unit toward the internal space from the sound collection result for the second sound; generating a noise reduction signal for reducing the subtraction signal on the basis of the subtraction signal; and adding the difference signal and the noise reduction signal to the input acoustic signal and to generate a drive signal for driving the acoustic device.
- Further, according to the present disclosure, there is provided a program causing a computer to execute: acquiring a sound collection result for a first sound propagating in an external space outside a mounting unit to be worn on an ear of a listener; acquiring a sound collection result for a second sound propagating in an internal space connected with an external ear canal inside the mounting unit; generating a difference signal which is substantially equal to a difference between the first sound propagating directly from the external space toward an inside of the external ear canal and the first sound propagating from the external space to the internal space via the mounting unit on the basis of the sound collection result for the first sound; generating a subtraction signal obtained by subtracting a first signal component based on the sound collection result for the first sound and a second signal component based on an input acoustic signal to be output from an acoustic device from an inside of the mounting unit toward the internal space from the sound collection result for the second sound; generating a noise reduction signal for reducing the subtraction signal on the basis of the subtraction signal; and adding the difference signal and the noise reduction signal to the input acoustic signal and to generate a drive signal for driving the acoustic device.
- As described above, according to the present disclosure, a signal processing device, a signal processing method, and a program, which are capable of enabling a listener to listen to the ambient sounds of the external environment in an appropriate manner while wearing a head mounted acoustic device are provided.
- Note that the effects described above are not necessarily limitative. With or in the place of the above effects, there may be achieved any one of the effects described in this specification or other effects that may be grasped from this specification.
-
FIG. 1 is an explanatory diagram for describing an application example of a head mounted acoustic device to which a signal processing device according to an embodiment of the present disclosure is applied. -
FIG. 2 is an explanatory diagram for describing an example of a principle for implementing a hear-through effect. -
FIG. 3 is a diagram schematically illustrating an example of a propagation environment before an ambient sound is heard by a user in a case in which the user wears a canal type earphone. -
FIG. 4 is a diagram schematically illustrating an example of a propagation environment before an ambient sound is heard by the user in a case in which the user does not wear a head mounted acoustic device. -
FIG. 5 is a block diagram illustrating an example of a basic functional configuration of a signal processing device according to an embodiment of the present disclosure. -
FIG. 6 is an explanatory diagram for describing a mechanism of the occurrence of a phenomenon in which vibration of a voice uttered by the user propagates within an internal space. -
FIG. 7 is a block diagram illustrating an example of a functional configuration of a signal processing device according to a first embodiment of the present disclosure. -
FIG. 8 is an explanatory diagram for describing an example of a configuration of the signal processing device according to the embodiment. -
FIG. 9 is a block diagram illustrating an example of a functional configuration of a signal processing device according to a second embodiment of the present disclosure. -
FIG. 10 is an explanatory diagram for describing an example of a configuration for further reducing a delay amount in the signal processing device according to the embodiment. -
FIG. 11 is a diagram illustrating an example of a functional configuration of a monitor canceller. -
FIG. 12 is a block diagram illustrating an example of a functional configuration of a signal processing device according to a modified example of the embodiment. -
FIG. 13 is a diagram illustrating an example of a functional configuration of a signal processing device according to a third embodiment of the present disclosure. -
FIG. 14 is a block diagram illustrating another example of a functional configuration of the signal processing device according to the embodiment. -
FIG. 15 is an explanatory diagram for describing an application example of a signal processing device according to the embodiment. -
FIG. 16 is a diagram illustrating an example of a hardware configuration of a signal processing device according to embodiments of the present disclosure. - Hereinafter, (a) preferred embodiment(s) of the present disclosure will be described in detail with reference to the appended drawings. In this specification and the appended drawings, structural elements that have substantially the same function and structure are denoted with the same reference numerals, and repeated explanation of these structural elements is omitted.
- The description will proceed in the following order.
- 2. Principle for implementing hear-through effect
- 2.2. Basic functional configuration
- 4.1. Schematic functional configuration
4.2. Configuration example for reducing delay amount
4.3. Modified example - 6. Hardware configuration
- In order to facilitate understanding of characteristics of a signal processing device related to the present disclosure, first, an application example of a head mounted acoustic device such as an earphone or a headphone to which the signal processing device can be applied will be described, and then a problem of the signal processing device according to the present disclosure will be described.
- As the head mounted acoustic devices such as earphones or headphones which are worn on the heads of the users when used, in addition to devices that simply output acoustic information, devices with functions in which use situations are considered have become widespread. As a specific example, a head mounted acoustic device capable of suppressing ambient sounds (so-called noise) coming from an external environment and enhancing a sound insulation effect using a so-called noise canceling technique is known.
- Meanwhile, as information processing devices which are configured to be carried by users such as so-called smartphones, tablet terminals, and wearable terminals have become widespread, the use situations of head mounted acoustic devices are no longer limited to listening to so-called audio content but have been further diversified.
- For example, in recent years, user interfaces (UIs) that enable users to recognize notification information without checking a screen or the like when an information processing device reads out the information by voice through a speech synthesis technology have become widespread. As another example, interactive UIs based on voice input that enable users to operate information processing devices by interacting with the devices by voice by applying a voice recognition technique have also become widespread.
- In order to cause such a UI to be usable even in so-called public places, a situation in which the user constantly wears a head mounted acoustic device is also considered. For example,
FIG. 1 is an explanatory diagram for describing an application example of a head mounted acoustic device to which a signal processing device according to an embodiment of the present disclosure is applied. In other words,FIG. 1 illustrates an example of a situation in which the user uses a portable information processing device such as a smartphone while wearing a head mountedacoustic device 51 in a so-called public place such as a case in which the user goes out. - As described above, there are cases in which it is desirable for the user to be able to hear so-called ambient sounds coming from an external environment in addition to acoustic information (for example, audio content) output from the information processing device while constantly wearing the head mounted
acoustic device 51. In these cases, it is more preferable for the user to be able to hear the ambient sounds coming from the external environment in a manner similar to that in a case in which the user does not wear the head mountedacoustic device 51. - In the following description, a state in which the user is able to hear a so-called ambient sound coming from an external environment even while the user is wearing the head mounted
acoustic device 51 in a manner similar to that in a case in which the user does not wear the head mountedacoustic device 51 is also referred to as a “hear-through state.” Similarly, an effect of enabling the user to hear a so-called ambient sound coming from an external environment even while the user is wearing the head mounted acoustic device in a manner similar to that in a case in which the user does not wear the head mountedacoustic device 51 is also referred to as a “hear-through effect.” - If the hear-through state described above is implemented, for example, the user is able to check a sound output indicating notification of content of e-mails or news while checking a surrounding situation and wearing the head mounted acoustic device even in a public place. As another example, the user is also able to perform a phone call with another user by means of a so-called phone call function while checking a surrounding situation in motion.
- On the other hand, in order to cause the user to experience a more natural hear-through effect, a technique based on the premise of the use of a head mounted acoustic device having high hermeticity (in other words, a high shielding property against the external environment) such as a so-called canal type earphone is important. This is because there are cases in which, in a situation in which a head mounted acoustic device having relatively low hermeticity such as a so-called open air headphone is used, influence of so-called sound leakage is large, and use in public places is not necessarily preferable.
- On the other hand, in situations in which a head mounted acoustic device having high hermeticity such as a canal type earphone is used, ambient sounds coming from an external environment which leak into the ear (the so-called external ear canal) of the user via the head mounted acoustic device are at least partially shielded. Therefore, the user is likely to hear ambient sounds coming from an external environment in different manner from the state in which the user does not wear the head mounted acoustic device, or the user may hardly hear the ambient sounds.
- In this regard, in the present disclosure, an example of a technique for implementing the hear-through state described above in a situation in which a head mounted acoustic device having high hermeticity such as a so-called canal type earphone is used will be described.
- First, an example of a principle for implementing the hear-through effect will be described in comparison with an example of a so-called feed-forward (FF) type noise canceling (NC) earphone (or headphone). For example,
FIG. 2 is an explanatory diagram for describing an example of the principle for implementing the hear-through effect and illustrates an example of a schematic functional configuration of the head mountedacoustic device 51 in a case in which the head mountedacoustic device 51 is configured as a so-called FF type NC earphone. - As illustrated in
FIG. 2 , the head mountedacoustic device 51 includes, for example, amicrophone 71, afilter circuit 72, apower amplifier 73, and aspeaker 74. InFIG. 2 , reference numeral F schematically indicates a transfer function of a propagation environment before a sound N from a sound source S reaches (that is, leaks into) the user's ear (that is, the inside of the external ear canal) via the housing of the head mountedacoustic device 51. Reference numeral F′ schematically indicates the transfer function of the propagation environment before the sound N from the sound source S reaches themicrophone 71. - Here,
FIG. 3 is referred to.FIG. 3 schematically illustrates an example of the propagation environment before the sound N from the sound source S is heard by the user U in a case in which the user U wears a so-called canal type earphone as the head mountedacoustic device 51. InFIG. 3 , reference numeral UA schematically indicates a space in the external ear canal of a user U (hereinafter also referred to simply as an “external ear canal”). Further, reference numerals F and F′ inFIG. 3 correspond to reference numerals F and F′ illustrated inFIG. 2 , respectively. In the following description, as illustrated inFIG. 3 , when the head mountedacoustic device 51 is worn on the ear of the user U, a space connected to the external ear canal UA inside the head mountedacoustic device 51 is also referred to as an “internal space.” Further, when the head mountedacoustic device 51 is worn on the ear of the user U, a space outside the head mountedacoustic device 51 is also referred to as an “external space.” - As illustrated in
FIGS. 2 and 3 , the sound N from the sound source S propagating via the propagation environment F N may leak into the ear U′ of the user (specifically, the internal space connected to the external ear canal UA). Therefore, in the NC earphone, the influence of the sound N is mitigated by adding a signal having a reverse phase (a noise reduction signal) to the sound N propagating via the propagation environment F. - Specifically, for example, the sound N from the sound source S of the external environment reaches the
microphone 71 via the propagation environment F′ and is collected by themicrophone 71. Thefilter circuit 72 generates a signal having a reverse phase (noise reduction signal) to that of the sound N propagating via the propagation environment F on the basis of the sound N collected by themicrophone 71. - The noise reduction signal generated by the
filter circuit 72 undergoes gain adjustment performed by thepower amplifier 73 and is then output toward the ear U′ of the user through thespeaker 74. Accordingly, a component of the sound N propagating to the ear U′ of the user via the propagation environment F is canceled by a component of the noise reduction signal output from thespeaker 74, and the sound N is suppressed. - Here, transfer functions based on device characteristics of the
microphone 71, thepower amplifier 73, and thespeaker 74 are indicated by M, A, and H, respectively. Further, a filter coefficient when thefilter circuit 72 generates the noise reduction signal on the basis of an acoustic signal collected by themicrophone 71 is indicated by a. At this time, in the NC earphone, so-called noise canceling is implemented by designing the filter coefficient α of thefilter circuit 72 so that a relational expression indicated by (Formula 1) below is satisfied. -
[Math. 1] -
F′AHMαN+FN≈0 (Formula 1) - On the other hand, in the hear-through state, as illustrated in
FIG. 3 , in the state in which the head mountedacoustic device 51 is worn, the user U hears the sound N from the sound source S of the external environment in a manner substantially equivalent to the case in which the head mountedacoustic device 51 is not worn. - For example,
FIG. 4 sis a diagram schematically illustrating an example of the propagation environment before the sound N from the sound source S is heard by the user U in a case in which the user U does not wear the head mountedacoustic device 51. InFIG. 4 , reference numeral G schematically indicates a transfer function of a propagation environment before the sound N from the sound source S directly reaches the inside of the external ear canal UA of the user U. - In other words, in a case in which the hear-through effect is implemented on the basis of the head mounted
acoustic device 51 illustrated inFIG. 2 , it is preferable to generate the sound to be output from thespeaker 74 so that the situation illustrated inFIG. 3 (the situation in which the head mountedacoustic device 51 is worn) and the situation illustrated inFIG. 4 (the situation in which the head mountedacoustic device 51 is not worn) are equalized. - Specifically, if the filter coefficient of the
filter circuit 72 in the case of implementing the hear-through effect is indicated by γ, it is possible to implement the hear-through effect ideally by designing the filter coefficient γ so that relational expressions indicated by (Formula 2) and (Formula 3) below are satisfied. -
- Further, each of the noise canceling and the hear-through effect is implemented by adding a sound wave of the sound N propagated to the inside of the external ear canal UA via the head mounted
acoustic device 51 and a sound wave of the sound N output from thespeaker 74 in the air as illustrated inFIG. 2 . Therefore, it is understood that it is preferable that a delay amount before the sound N from the sound source S is collected by themicrophone 71 and output from thespeaker 74 via thefilter circuit 72 and thepower amplifier 73, including a conversion process performed by an AD converter (ADC) or a DA converter (DAC), be suppressed to be about 100 μs or less. - Here, the reason for suppressing the delay amount to be 100 μs or less will be described in further detail. In the case of implementing the hear-through effect on the basis of a sound collection result of the
microphone 71 installed in the housing in the head mountedacoustic device 51 having the high hermeticity (for example, a canal type earphone or an overhead type headphone), it is preferable to constitute thefilter circuit 72 of the filter coefficient γ as a digital filter by installing the ADC and the DAC. This is because if thefilter circuit 72 is constituted as a digital filter, it is possible to easily implement a filter process which is smaller in variation than in an analog filter and is unable to be implemented by an analog filter. - On the other hand, in the case in which the ADC and the DAC are installed, the processing load is increased by the filtering process such as decimation and interpolation, and a delay occurs accordingly.
- As described above, in
FIG. 2 , the sound output from thespeaker 74 and the sound N from the sound source S propagating via the propagation environment F inFIG. 2 are added in the space in the external ear canal UA (that is, a space near the eardrum), and an added sound is recognized by the user as one sound. Therefore, it is generally known that if the delay amount exceeds 10 ms, it is recognized as if an echo occurs, or it is recognized as if a sound is heard twice. Even in a case in which the delay amount is less than 10 ms, the frequency characteristic may be influenced by mutual interference of sounds or it may be difficult to implement the hear-through effect and the noise canceling. - As a concrete example, in
FIG. 2 , a delay of 1 ms is assumed to occurs between the sound output from thespeaker 74 and the sound N from the sound source S propagating via the propagation environment F. In this case, an acoustic signal of a band near 1 kHz undergoes phase shift corresponding to one cycle (that is, 360°) and then added. On the other hand, an acoustic signal of ae band near 500 Hz has a reverse phase and then is cancelled. In other words, in a case in which signals with a delay of 1 ms are added simply, a so-called dip occurs. On the other hand, if the delay amount is suppressed to be 100 μs, it is possible to increase a frequency band at which the dip occurs due to a reverse phase relation up to 5 kHz. - Generally, the human external ear canal is known to have resonance points near about 3 kHz to 4 kHz although there are individual differences. For this reason, in the frequency band exceeding 4 kHz, it corresponds to the so-called individual difference part, and thus the appropriate hear-through effect is considered to be obtained by suppressing the delay amount to be 100 μs or less and adjusting the frequency band at which the dip occurs to be around 5 kHz.
- Next, an example of the basic functional configuration of the signal processing device for implementing the hear-through effect will be described with reference to
FIG. 5 .FIG. 5 is a block diagram illustrating an example of a basic functional configuration of asignal processing device 80 according to an embodiment of the present disclosure. As described above, thesignal processing device 80 practically includes a DAC and an ADC in order to convert each acoustic signal into a digital signal and perform various kinds of filter processes, but in the example illustrated inFIG. 5 , in order to facilitate understanding of the description, description of the DAC and the ADC is omitted. - In
FIG. 5 , each ofreference numerals acoustic device 51. In other words,reference numeral 51 a indicates the head mountedacoustic device 51 worn on the right ear, andreference numeral 51 b indicates the head mountedacoustic device 51 attached to the left ear. In a case in which the head mountedacoustic devices acoustic device 51” as described above. In the example illustrated inFIG. 5 , since the head mountedacoustic devices acoustic device 51 a side, and illustration of the head mountedacoustic device 51 b is omitted. - As illustrated in
FIG. 5 , the head mountedacoustic device 51 includes a mountingunit 510, adriver 511, and anexternal microphone 513. - The mounting
unit 510 illustrates a part worn on the user U in the housing of the head mountedacoustic device 51. - For example, in a case in which the head mounted
acoustic device 51 is configured as a so-called canal type earphone, the mountingunit 510 has an outer shape in which that it is worn on the ear of the user U such that at least a part thereof is insertable into the ear hole of the user U who is the wearer. Specifically, in this case, an ear hole insertion portion having a shape insertable into the ear hole of the user U is formed in the mountingunit 510, and the mountingunit 510 is worn on the ears of the user U such that the ear hole insertion portion is inserted into the ear hole. For example, the example illustrated inFIG. 3 illustrates a state in which the mountingunit 510 of the head mountedacoustic device 51 is worn on the ear of the user U. - In a case in which the mounting
unit 510 is worn on the user U, the space in the mounting unit 510 (that is, the space connected to the external ear canal UA of the user U) corresponds to the internal space. - The
driver 511 is a component for driving an acoustic device such as the speaker and causing the acoustic device to output the sound based on the acoustic signal. As a specific example, thedriver 511 causes the speaker to output the sound based on the acoustic signal by vibrating a vibration plate of the speaker on the basis of an input analog acoustic signal (that is, a drive signal). - The
external microphone 513 is a sound collecting device that directly collects a sound (a so-called ambient sound) propagating via an external space outside the mountingunit 510 for enabling the head mountedacoustic device 51 to be worn on the user U. For example, theexternal microphone 513 may be configured as a so-called micro electro mechanical systems (MEMS) microphone which is formed on the basis of the MEMS technology. An installation position of theexternal microphone 513 is not particularly limited as long as it is able to collect the sound propagating via the external space. As a specific example, theexternal microphone 513 may be installed in the mounting unit of the head mountedacoustic device 51 or may be installed at a position different from the mounting unit. The sound (that is, the ambient sound) collected by theexternal microphone 513 corresponds to an example of a “first sound.” - The
signal processing device 80 illustrated inFIG. 5 is a component for executing various signal processing (for example, the filter process described above with reference toFIGS. 2 to 4 ) in order to implement the hear-through effect. As illustrated inFIG. 5 , thesignal processing device 80 includes amicrophone amplifier 111, anHT filter 121, an addingunit 123, apower amplifier 141, and an equalizer (EQ) 131. - The
microphone amplifier 111 is a so-called amplifier for adjusting a gain of the acoustic signal. The ambient sound collected by theexternal microphone 513 undergoes gain adjustment (for example, amplification) performed by themicrophone amplifier 111 and is then input to theHT filter 121. - The
HT filter 121 corresponds to the filter circuit 72 (seeFIG. 2 ) in the case of implementing the hear-through effect described above with reference toFIGS. 2 to 4 . In other words, theHT filter 121 performs signal processing based on the filter coefficient γ described on the basis of (Formula 2) and (Formula 3) on the acoustic signal output from the microphone amplifier 111 (that is, the acoustic signal which has been collected by theexternal microphone 513 and has undergone the gain adjustment performed by the microphone amplifier 111). At this time, the acoustic signal output as a result of performing signal processing by theHT filter 121 is hereinafter also referred to as a “difference signal.” In other words, the ambient sound in a case in which the user directly hears it is simulated (that is, the hear-through effect is implemented) by adding the difference signal and the ambient sound propagating to the internal space via the mountingunit 510 of the head mounted acoustic device 51 (that is, the sound propagating via the propagation environment F inFIGS. 2 and 3 ). TheHT filter 121 corresponds to an example of a “first filter processing unit.” - The
HT filter 121 outputs the difference signal generated as a result of performing signal processing on the acoustic signal output from themicrophone amplifier 111 to the addingunit 123. - The
EQ 131 performs a so-called equalizing process on the acoustic signal input to the signal processing device 80 (hereinafter also referred to as a “sound input”) such as audio content or a received signal in a voice call. As a specific example, in a case of feeding back the sound collection result for the ambient sound as in the case of implementing the noise canceling and the hear-through effect, a gain of a low-frequency side component tends to increases due to a sound characteristic of the ambient sound. Therefore, theEQ 131 corrects the sound characteristic (for example, frequency characteristic) of the sound input so that the sound component on the low frequency side to be superimposed on the basis of the feedback is suppressed from the sound input in advance. The sound input corresponds to an example of an “input acoustic signal.” - Then, the
EQ 131 outputs the sound input which has undergone the equalizing process to the addingunit 123. - The adding
unit 123 adds the difference signal output from theHT filter 121 to the sound input output from the EQ 131 (that is, the sound input that has undergone the equalizing process) and outputs the acoustic signal generated as the addition result to thepower amplifier 141. - The
power amplifier 141 is a so-called amplifier for adjusting the gain of the acoustic signal. The acoustic signal output from the adding unit 123 (that is, the addition result of the sound input and the difference signal) undergoes gain adjustment (that is, amplification) performed by thepower amplifier 141 and is then output to thedriver 511. Then, thedriver 511 drives the speaker on the basis of the acoustic signal output from thepower amplifier 141, and thus the sound based on the acoustic signal is radiated into the internal space inside the mounting unit 510 (that is, the space connected to the external ear canal UA of the user U). - The sound radiated into the internal space by the
driver 511 driving the speaker is added to the ambient sound propagating to the internal space (that is, the sound propagating via the propagation environment F inFIGS. 2 and 3 ) via the mountingunit 510 of the head mountedacoustic device 51 and heard by the user U. At this time, the component of the difference signal included in the sound radiated from thedriver 511 to the internal space is added to the ambient sound propagated to the internal space via the mountingunit 510 and heard by the user U. In other words, the user U is able to hear the ambient sound in a manner similar to that in the case in which the head mountedacoustic device 51 is not worn as illustrated inFIG. 4 in addition to the sound input such as the audio content. - It should be noted that the operation of the
signal processing device 80 described above is merely an example, and thesignal processing device 80 need not necessarily faithfully reproduce the hear-through effect if the user U is able to hear the ambient sound in a state in which the user U is wearing the head mountedacoustic device 51. As a specific example, theHT filter 121 may control a characteristic and a gain of the difference signal such that the user U feels the volume of the ambient sound higher than in the state in which the user U does not wear the head mountedacoustic device 51. Similarly, theHT filter 121 may control the characteristic and the gain of the difference signal so that the user U feels the volume of the ambient sound lower than in the state where the user U does not wear the head mountedacoustic device 51. On the basis of this configuration, for example, thesignal processing device 80 may control the volume of the ambient sound heard by the user U in accordance with an input state of the sound input or a type of sound input (for example, audio content, a received signal of a voice call, or the like). - As described above, the example of the basic functional configuration of the signal processing device for implementing the hear-through effect has been described above with reference to
FIG. 5 . - On the other hand, in a case in which the user U is wearing the head mounted
acoustic device 51 having the high hermeticity such as a so-called canal type earphone, the user U may have a strange feeling with how a voice uttered by the user U is heard, and this point is similar in the example illustrated inFIG. 5 . This is because that the vibration of the voice uttered by the user propagates within the internal space. In this regard, a mechanism in which the vibration of the voice uttered by the user propagates in the internal space will be described with reference toFIG. 6 .FIG. 6 is an explanatory diagram for describing a mechanism in which the vibration of the voice uttered by the user propagates in the internal space. - As illustrated in
FIG. 6 , the vibration of the voice uttered by the user U propagates to the external ear canal UA via bones or flesh in the head of the user U, so that the external ear canal wall is vibrated like a secondary speaker. Here, in a case in which the head mountedacoustic device 51 having the high hermeticity such as a canal type earphone is worn, a degree of hermeticity of the space in the external ear canal UA is increased by the head mountedacoustic device 51, and an escape route in the air is limited, and thus the vibration in the space is directly transferred to the eardrum. At this time, the vibration of the voice uttered by the user U propagating in the internal space is transferred to the eardrum as if the low frequency is amplified, and thus the user U hears his/her voice as if it is muffled, and the user U has a strange feeling accordingly. - Signal processing devices according to embodiments of the present disclosure were made in view of the problem as described above, and it is desirable to implement to implement the hear-through effect in a more appropriate manner (that is, in a manner in which the user has a less strange feeling).
- First, an example of a functional configuration of a signal processing device according to a first embodiment of the present disclosure will be described with reference to
FIG. 7 .FIG. 7 is a block diagram illustrating an example of a functional configuration of the signal processing device according to the present embodiment. In the following description, the signal processing device according to the present embodiment is also referred to as a “signal processing device 11” in order to be distinguished from the signal processing device 80 (seeFIG. 5 ). Further, similarly to the example illustrated inFIG. 5 , in order to facilitate understanding of description, illustration of the DAC and the ADC is omitted in the functional configuration illustrated inFIG. 7 . - As illustrated in
FIG. 7 , thesignal processing device 11 according to the present embodiment differs from the signal processing device 80 (seeFIG. 5 ) in that amicrophone amplifier 151, a subtractingunit 171, anocclusion canceller 161, and anEQ 132 are provided. As illustrated inFIG. 7 , the head mountedacoustic device 51 to which thesignal processing device 11 according to the present embodiment is applicable differs from the head mountedacoustic device 51 to which thesignal processing device 80 is applicable (seeFIG. 5 ) in that aninternal microphone 515 is provided. In this regard, in the following description, the functional configurations of thesignal processing device 11 according to the present embodiment and the head mountedacoustic device 51 to which thesignal processing device 11 is applicable will be described particularly focusing on a difference with those in the example illustrated inFIG. 5 . - The
internal microphone 515 is a sound collecting device that collects the sound propagating to the internal space inside the mountingunit 510 that enables the head mountedacoustic device 51 to be worn on the user U (that is, the space connected to the external ear canal UA of the user U). Similarly theexternal microphone 513, theinternal microphone 515 may be configured as, for example, a so-called MEMS microphone formed on the basis of MEMS technology. - For example, the
internal microphone 515 is installed in the mountingunit 510 to face the direction of the external ear canal UA. It will be appreciated that an installation position is not particularly limited as long as theinternal microphone 515 is capable of collecting the sound propagating to the internal space. - The acoustic signal collected by the
internal microphone 515 includes a component of the sound output from the speaker on the basis of control performed by thedriver 511, a component of the ambient sound propagating to the internal space via the mounting unit 510 (the sound propagating via the propagation environment F inFIGS. 2 and 3 ), and a component of a voice of the user propagating to the external ear canal UA (the component of the voice illustrated inFIG. 6 ). Further, the sound collected by the internal microphone 515 (that is, the sound propagating to the internal space) corresponds to an example of a “second sound.” - The
microphone amplifier 151 is a so-called amplifier that adjusts the gain of the acoustic signal. The acoustic signal based on the sound collection result obtained by the internal microphone 515 (that is, the sound collection result for the sound propagating to the internal space) undergoes gain adjustment (for example, amplification) performed by themicrophone amplifier 151 and is then input to thesubtracting unit 171. - The
EQ 132 is a component for performing the equalizing process on the sound input in accordance with the device characteristics of theinternal microphone 515 and themicrophone amplifier 151. Specifically, in a case in which the transfer function based on the device characteristics of theinternal microphone 515 and themicrophone amplifier 151 is indicated by M, theEQ 132 applies a frequency characteristic which is “target characteristic—M” to the sound input. The transfer function M corresponding to the device characteristics of theinternal microphone 515 and themicrophone amplifier 151 may be calculated in advance on the basis of a result of a prior experiment or the like. Then, theEQ 132 outputs the sound input which has undergone the equalizing process to thesubtracting unit 171. The sound input which has undergone the equalizing process performed byEQ 132 corresponds to an example of a “second signal component.” - The subtracting
unit 171 subtracts the sound input output from the EQ 132 (that is, the sound input to which the frequency characteristic which is “target characteristic—M” is applied) from the acoustic signal output from themicrophone amplifier 151, and outputs the acoustic signal generated as a subtraction result to theocclusion canceller 161. The acoustic signal output as the subtraction result obtained by the subtractingunit 171 corresponds to the acoustic signal in which the component of the sound input among the components of the acoustic signal collected by theinternal microphone 515 is suppressed. More specifically, the acoustic signal includes a component in which the difference signal and the ambient sound propagating to the internal space via the mountingunit 510 are added (hereinafter also referred to as an “ambient sound component”) and the component of the voice of the user U propagating to the external ear canal UA via bones or flesh of the head of the user U (hereinafter also referred to simply as a “voice component”). - The
occlusion canceller 161 corresponds to a so-called filter processing unit operating on a principle similar to that of so-called feed-back (FB) type NC filter. Theocclusion canceller 161 generates an acoustic signal for suppressing the component of the acoustic signal to a predetermined volume (hereinafter also referred to as a “noise reduction signal”) on the acoustic signal output from the subtractingunit 171. - As described above, the acoustic signal output from the subtracting
unit 171 includes the ambient sound component and the voice component, and the low frequency side of the voice component is amplified due to a property of a propagation path. Therefore, for example, in order to enable the user U to hear the voice component in a manner similar to that in the case in which the user U does not wear the head mountedacoustic device 51, theocclusion canceller 161 may generate the noise reduction signal for suppressing the low frequency side of the voice component among the voice components of the acoustic signal acquired from the subtractingunit 171. Further, theocclusion canceller 161 corresponds to an example of a “second signal processing unit.” - As described above, the
occlusion canceller 161 generates the noise reduction signal on the basis of the acoustic signal output from the subtractingunit 171. Then, theocclusion canceller 161 outputs the generated noise reduction signal to the addingunit 123. - The
EQ 131 performs the equalizing process on the sound input, similarly to theEQ 131 described above with reference toFIG. 5 . - The
EQ 131 according to the present embodiment further performs the equalizing process on the sound input in accordance with to a characteristic to be applied to the output sound depending on a structure or the like of the speaker driven by thedriver 511 and the transfer function of the space from the speaker to theinternal microphone 515. For example, a function obtained by multiplying the transfer function corresponding to the characteristic applies to the output sound depending on the structure or the like of the speaker driven by thedriver 511 by the transfer function of the space from the speaker to theinternal microphone 515 is indicated by H. In this case, theEQ 131 applies a frequency characteristic which is “target characteristic 1/H to the sound input. Further, it is preferable to calculate the transfer function corresponding to the characteristic to be applied to the output sound depending on the structure or the like of the speaker driven by thedriver 511 and the transfer function of the space from the speaker to theinternal microphone 515 in advance on the basis of a result of an experiment or the like. Then, theEQ 131 outputs the sound input which has undergone the equalizing process to the addingunit 123. - The adding
unit 123 adds the difference signal output from theHT filter 121 and the noise reduction signal output from theocclusion canceller 161 to the sound input output from the EQ 131 (that is, the sound input after the equalizing process). Then, the addingunit 123 outputs the acoustic signal generated as an addition result to thepower amplifier 141. - The acoustic signal output from the adding unit 123 (that is, the addition result of the sound input, the difference signal, and the noise reduction signal) undergoes gain adjustment (for example, amplification) performed by the
power amplifier 141 and is then output to thedriver 511. Then, thedriver 511 drives the speaker on the basis of the acoustic signal output from thepower amplifier 141, and thus the sound based on the acoustic signal is radiated into the internal space in the mounting unit 510 (that is, the space connected with the external ear canal UA of the user U Space). - The example of the functional configuration of the
signal processing device 11 according to the present embodiment has been described above with reference toFIG. 7 . The configuration of thesignal processing device 11 is not necessarily limited to the example illustrated inFIG. 7 as long as the operations of the components of thesignal processing device 11 described above can be implemented. - For example,
FIG. 8 is an explanatory diagram for describing an example of the configuration of thesignal processing device 11 according to the present embodiment. In the example illustrated inFIG. 7 , the head mountedacoustic device 51 and thesignal processing device 11 are configured as different devices. On the other hand,FIG. 8 illustrates an example of a configuration in a case in which the head mountedacoustic device 51 and thesignal processing device 11 are installed in the same housing. Specifically, in the example illustrated inFIG. 8 , a configuration (for example, a signal processing unit) corresponding to thesignal processing device 11 is installed in the mountingunit 510 of the head mountedacoustic device 51. - It will be appreciated that the
signal processing device 11 may be configured as an independent device or may be configured as a part of an information processing device such as a so-called smartphone or the like. Further, at least some components of thesignal processing device 11 may be installed in an external device (for example, a server or the like) different from thesignal processing device 11. In this case, it is preferable that a delay amount before the ambient sound propagating via the external environment is collected by theexternal microphone 513 and output from the speaker of the head mountedacoustic device 51 via theHT filter 121 and thepower amplifier 141, including the conversion process performed by the ADC and the DAC, be suppressed to be about 100 μs or less. - As described above, the
signal processing device 11 according to the present embodiment generates the noise reduction signal for suppressing at least some components among the voice components of the user U on the basis of the sound collection result obtained by the internal microphone 515 (that is, the sound collection result for the sound propagating to the internal space). Then, thesignal processing device 11 adds the generated difference signal and the noise reduction signal to the input sound input, and outputs the added acoustic signal. Accordingly, thedriver 511 of the head mountedacoustic device 51 drives the speaker on the basis of the acoustic signal output from thesignal processing device 11, and thus the sound based on the acoustic signal is radiated into the internal space. - The sound radiated into the internal space when the
driver 511 drives the speaker includes a component based on the noise reduction signal generated by theocclusion canceller 161. The component on the basis of the noise reduction signal is added to the voice component of the user U propagating to the external ear canal UA in the internal space on the basis of an utterance of the user U. Accordingly, at least some components among the voice components (for example, the component on the lower frequency side among the voice components) is suppressed, and the suppressed voice component reaches the eardrum of the user U and is heard by the user U. In other words, according to thesignal processing device 11 of the present embodiment, it is possible to implement the hear-through effect in a manner in which the user U has no strange feeling in his/her voice being heard. - Next, a signal processing device according to a second embodiment of the present disclosure will be described. In the first embodiment, the hear-through effect is implemented in a manner in which the user U has no strange feeling in his/her voice being heard by providing the
occlusion canceller 161. On the other hand, in thesignal processing device 11 according to the first embodiment, the acoustic signal to be processed by theocclusion canceller 161 includes the component of the difference signal output from the speaker of the head mountedacoustic device 51. For this reason, there are cases in which the hear-through effect is not sufficiently obtained (or an ambient sound having a different characteristic is heard by the user U) since the component of the difference signal is suppressed by the noise reduction signal which is generated by theocclusion canceller 161 on the basis of the acoustic signal. - In other words, the signal processing device according to the present embodiment was made in view of the problem described above, and it is desirable to implement the hear-through effect in a more natural manner (that is, in a manner in which the user has a less strange feeling) than the
signal processing device 11 according to the first embodiment. In the following description, the signal processing device according to the present embodiment is also referred to as a “signal processing device 12” in order to be distinguished from thesignal processing device 11 according to the first embodiment. - First, an example of a functional configuration of a
signal processing device 12 according to the present embodiment will be described with reference toFIG. 9 .FIG. 9 is a block diagram illustrating an example of a functional configuration of a signal processing device according to the present embodiment. Further, similarly to the examples illustrated inFIGS. 5 and 7 , in order to facilitate understanding of description, illustration of the DAC and the ADC is omitted in the functional configuration illustrated inFIG. 9 . - As illustrated in
FIG. 9 , thesignal processing device 12 according to the present embodiment differs from thesignal processing device 11 according to the first embodiment (seeFIG. 7 ) in that amonitor canceller 181 and asubtracting unit 191 are provided. Therefore, in the following description, the functional configuration of thesignal processing device 12 according to the present embodiment will be described focusing on a difference with thesignal processing device 11 according to the first embodiment described above (seeFIG. 7 ). - The
monitor canceller 181 and the subtractingunit 191 are configured to suppress a component corresponding to the difference signal among components in the acoustic signal output from the microphone amplifier 151 (that is, the acoustic signal on the basis of the sound collection result of the internal microphone 515). - In the
signal processing device 12 illustrated inFIG. 9 , the ambient sound collected by theexternal microphone 513 undergoes gain adjustment (for example, amplification) performed by themicrophone amplifier 111 and is then input to theHT filter 121 and themonitor canceller 181. - Similarly to the
HT filter 121, themonitor canceller 181 performs the signal processing based on the filter coefficient γ described on the basis of (Formula 2) and (Formula 3) on the acoustic signal output from themicrophone amplifier 111, and generates the difference signal. - Further, the
monitor canceller 181 performs a filter process on the generated difference signal on the basis of the transfer function corresponding to each characteristic so that influences of the device characteristic of each of thepower amplifier 141, thedriver 511, and themicrophone amplifier 151 and a spatial characteristic in the internal space are reflected. This is because a characteristic of a route from theocclusion canceller 161 to theocclusion canceller 161 via thepower amplifier 141, thedriver 511, and themicrophone amplifier 151 is not reflected in the acoustic signal output from themicrophone amplifier 111. - In the
monitor canceller 181, an infinite impulse response filter (an IIR filter) and a finite impulse response filter (a FIR filter) may be installed as a configuration for executing the filter process. In this case, for example, in the filter processes described above, a simple process for a delay component may be mainly allocated to the FIR filter, and a process related to frequency characteristic may be mainly allocated to the IIR filter. - It will be appreciated that the configuration in which the IIR filter and the FIR filter are installed is merely an example, and the configuration of the
monitor canceller 181 is not necessarily limited. As a specific example, the FIR filter may be installed in themonitor canceller 181, and both of the simple process for the delay component and the process related to the frequency characteristic may be executed by the FIR filter. - As another example, in a case in which the influence of the delay component is sufficiently small, the filter process may be implemented only by the IIR filter. As an example of a method for reducing the influence of the delay component, for example, a method of employing the ADC and the DAC or employing a low-delay device as a filter (for example, a decimation filter) used for bit rate conversion may be used. Further, a device having a smaller driving delay (that is, a more responsive device) may be employed as a sound system such as the driver 511 (and the speaker), the
external microphone 513, or theinternal microphone 515. Further, a sound speed delay between the speaker and theinternal microphone 515 may be reduced by bringing the speaker driven by thedriver 511 and theinternal microphone 515 closer to each other in the internal space. - The device characteristic of each of the
power amplifier 141, thedriver 511, and themicrophone amplifier 151 and the spatial characteristic in the internal space may be derived in advance using, for example, a time stretched pulse (TSP) or the like. In this case, for example, each characteristic may be calculated on the basis of measurement results of the acoustic signal (TSP) input from the power amplifier 141 (specifically, the DAC) and the acoustic signal output from themicrophone amplifier 151. As another example, the device characteristics of each of thepower amplifier 141, thedriver 511, and themicrophone amplifier 151 and the spatial characteristic in the internal space may be individually measured, and the respective measurement results may be convoluted. In other words, the filter characteristic of themonitor canceller 181 may be adjusted in advance on the basis of the prior measurement result of each characteristic described above. Themonitor canceller 181 corresponds to an example of a “third filter processing unit.” Further, the acoustic signal which has undergone the filter process performed by themonitor canceller 181 corresponds to a “first signal component.” - Then, the
monitor canceller 181 outputs the difference signal which has undergone various kinds of filter processes to thesubtracting unit 191. - The subtracting
unit 191 subtracts the difference signal output from themonitor canceller 181 from the acoustic signal output from themicrophone amplifier 151, and outputs the acoustic signal generated as a subtraction result to thesubtracting unit 171 positioned at a subsequent stage. At this time, the acoustic signal output as the subtraction result obtained by the subtractingunit 171 corresponds to an acoustic signal in which the component corresponding to the difference signal among the components of the acoustic signal collected by theinternal microphone 515 is suppressed. - A subsequent process is similar to that of the
signal processing device 11 according to the first embodiment. In other words, the component of the sound input output from theEQ 132 is subtracted from the acoustic signal output from the subtractingunit 191 through the subtractingunit 171, and the resulting acoustic signal is then input to theocclusion canceller 161. At this time, the acoustic signal input to theocclusion canceller 161 is an acoustic signal in which the component corresponding to a difference signal and the component corresponding to the sound input among the components of the acoustic signal collected by theinternal microphone 515 are suppressed (that is, the voice component). - With this configuration, in the
signal processing device 12 according to the present embodiment, it is possible to exclude the component of the difference signal from a processing target from which theocclusion canceller 161 generates the noise reduction signal. In other words, in thesignal processing device 12 according to the present embodiment, it is possible to prevent the component of the difference signal from being suppressed by the noise reduction signal. Therefore, thesignal processing device 12 according to the present embodiment is able to implement the hear-through effect in a more natural manner (that is, a manner in which the user U has a less strange feeling) than in thesignal processing device 11 according to the first embodiment. - The example of the functional configuration of the
signal processing device 12 according to the present embodiment has been described above with reference toFIG. 9 . - Next, an example of a mechanism of reducing the delay amount before the
signal processing device 12 according to the present embodiment adds the difference signal based on the sound collection result obtained byexternal microphone 513 and the noise reduction signal based on the sound collection result obtained by theinternal microphone 515 to the sound input and outputs the resulting signal through the speaker will be described. - First, in
FIG. 9 , a route indicated by reference numeral R11, that is, a route on which the acoustic signal based on the sound collection result of theexternal microphone 513 is radiated into the internal space via themicrophone amplifier 111, theHT filter 121, thepower amplifier 141, and thedriver 511 is focus on. As described above, in the route R11, in order to implement the hear-through effect in a preferable manner (specifically, in order to adjust the frequency band at which the dip occurs to be around 5 kHz), it is preferable to suppress the delay amount to be 100 μs or less. In the following description, the delay amount of the route R11 is also referred to as a “delay amount D_HTF.” - Next, a route indicated by reference numeral R13, that is, a route on which the acoustic signal based on the sound collection result of the
external microphone 513 reaches the subtractingunit 191 via themonitor canceller 181 is focused on. In the configuration illustrated inFIG. 9 , themonitor canceller 181 generates the difference signal, similarly to theHT filter 121. - Further, a propagation delay will occur (propagates between the speaker and the internal microphone 515) before the
driver 511 drives the speaker on the basis of the difference signal, and so the acoustic signal based on the sound including the component of the difference signal radiated into the internal space propagates in the space inside the internal space and is collected by theinternal microphone 515. In the following description, a delay amount of the propagation delay in the internal space is also referred to as a “delay amount D_ACO.” - In other words, in order to appropriately subtract the component of the difference signal from the acoustic signal collected by the
internal microphone 515 in thesubtracting unit 191, it is necessary to cause the delay amount of the route R13 to be equal to or less than a value obtained by adding the delay amount D_HTF (100 μs) and the delay amount D_ACO. - A distance between the speaker driven by the
driver 511 and theinternal microphone 515 is about 3 to 4 cm even in a case of a relatively long headphone such as a so-called overhead type headphone. - Here, if the distance between the speaker driven by the
driver 511 and theinternal microphone 515 is 3.4 cm, the delay amount D_ACO of the propagation delay in the internal space is 100 μs (=(0.034 m)/(sound speed=340 m/s). It will be appreciated that as the closer the distance between the speaker driven by thedriver 511 and theinternal microphone 515 is, the smaller the delay amount D_ACO is. - In this regard, in a case in which the delay amount of the route R13 is set to D_HTC, it is necessary to satisfy a relation of the delay amount D_HTC≤D_HTF+D_ACO and satisfy a relation of D_HTF≤100 μs and D_ACO≤100 μs.
- In this regard, an example of a configuration of a
signal processing device 12 that satisfies the delay condition described above will be described with reference toFIG. 10 .FIG. 10 is an explanatory diagram for describing an example of a configuration for further reducing the delay amount in thesignal processing device 12 according to the present embodiment (that is, satisfying the delay condition described above). In the example illustrated inFIG. 10 , an ADC and a DAC that perform a conversion process between an analog signal and a digital signal and a filter that converts a sampling rate of a digital signal are explicitly illustrated for thesignal processing device 12 illustrated inFIG. 9 . - Specifically,
FIG. 10 explicitly illustratesADCs DAC 142, decimation filters 113 and 153, andinterpolation filters signal processing device 12 illustrated inFIG. 9 . In the example illustrated inFIG. 10 , the sampling rate of the sound input input to thesignal processing device 12 is assumed to be 1 Fs (1 Fs=48 kHz). - The
ADCs ADCs DAC 142 is a component for converting a digital signal into an analog acoustic signal. - The decimation filters 113 and 153 are components for down-sampling a sampling rate of an input digital signal to a predetermined sampling rate lower than the sampling rate. The interpolation filters 133, 134, and 143 are components for up-sampling the sampling rate of the input digital signal to a predetermined sampling rate higher than the sampling rate.
- The analog acoustic signal output on the basis of the sound collection result of the
external microphone 513 undergoes gain adjustment performed by themicrophone amplifier 111 and then converted into a digital signal through theADC 112. In the example illustrated inFIG. 10 , theADC 112 performs sampling on the input analog signal at the sampling rate of 64 Fs to be converted into a digital signal. TheADC 112 outputs the converted digital signal to thedecimation filter 113. - The
decimation filter 113 down-samples the sampling rate of the digital signal output from theADC 112 from 64 Fs to 8 Fs. In other words, the components positioned at a stage subsequent to the decimation filter 113 (for example, theHT filter 121 and the monitor canceller 181) perform various kinds of processes on the digital signal whose sampling rate is down-sampled to 8 Fs. - Further, the analog acoustic signal output on the basis of the sound collection result of the
internal microphone 515 undergoes gain adjustment performed by themicrophone amplifier 151 and converted into a digital signal through theADC 152. In the example illustrated inFIG. 10 , theADC 152 performs sampling on the input analog signal at the sampling rate of 64 Fs to be converted into a digital signal. TheADC 152 outputs the converted digital signal to thedecimation filter 153. - The
decimation filter 153 down-samples the sampling rate of the digital signal output from theADC 152 from 64 Fs to 8 Fs. In other words, the component positioned at a stage subsequent to the decimation filter 153 (for example, the occlusion canceller 161) perform various kinds of processes on the digital signal whose sampling rate is down-sampled to 8 Fs. - The sound input (the digital signal of 1 Fs) which has undergone the equalizing process performed by the
EQ 132 is up-sampled to the sampling rate of 8 Fs by theinterpolation filter 134 and then input to thesubtracting unit 171. Similarly, the sound input (the digital signal of 1 Fs) which has undergone the equalizing process performed by theEQ 131 is up-sampled to the sampling rate of 8 Fs by theinterpolation filter 133 and then input to the addingunit 123. - Then, the
addition unit 123 adds the difference signal output from theHT filter 121, the sound input output from theinterpolation filter 133, and the noise reduction signal output from theocclusion canceller 161. At this time, all of the difference signal, the sound input, and the noise reduction signal added by the addingunit 123 are digital signals of 8 Fs. - Then, the digital signal of 8 Fs output as the addition result of the adding
unit 123 is up-sampled to a digital signal of 64 Fs by theinterpolation filter 143, converted into an analog acoustic signal by theDAC 142, and input to thepower amplifier 141. Then, the analog acoustic signal undergoes gain adjustment performed by thepower amplifier 141 and then input to thedriver 511. Accordingly, when thedriver 511 drives the speaker on the basis of the inputted analog acoustic signal, the speaker radiates the sound based on the analog acoustic signal into the internal space. - As described above, in the example illustrated in
FIG. 10 , thesignal processing device 12 down-samples the digital signal of 64 Fs obtained by converting the collected analogue acoustic signal to about 8 Fs higher than the sampling rate (1 Fs) of the sound input. - In other words, in the
signal processing device 12 illustrated inFIG. 10 , theHT filter 121, themonitor canceller 181, and theocclusion canceller 161 execute each calculation (that is, the filter process) on the digital signal of 8 Fs, and thus it is possible to reduce a delay of one sampling unit. - Further, in the
signal processing device 12 illustrated inFIG. 10 , since the digital signal of 64 Fs is down-sampled to the digital signal of 8 Fs, it is possible to suppress the delay amount of the processes related to the down-sampling (that is, the processes of theADC 112 and the ADC 152) to be smaller than in the case of down-sampling to the digital signal of 1 Fs. This similarly applies to the processes related to the up-sampling. In other words, in thesignal processing device 12 illustrated inFIG. 10 , since the digital signal of 8 Fs is up-sampled to the digital signal of 64 Fs, it is possible to suppress the delay amount of the processes related to the up-sampling (that is, the process of the DAC 142) to be smaller than in the case of up-sampling from the digital signal of 1 Fs. - Further, down-sampling to the digital signal of the lower sampling rate (for example, 1 Fs) may be further performed, and then the digital signal may be a processing target of at least some calculations of the
HT filter 121, themonitor canceller 181, and theocclusion canceller 161. - For example,
FIG. 11 is a diagram illustrating an example of a functional configuration of themonitor canceller 181. Themonitor canceller 181 illustrated inFIG. 11 is configured so that various kinds of filter processes are executed on the digital signal of 1 Fs after the digital signal of 8 Fs is down-sampled to the digital signal of 1 Fs. - More specifically, the
monitor canceller 181 illustrated inFIG. 11 includes adecimation filter 183, anIIR filter 184, anFIR filter 185, and aninterpolation filter 186. - The
decimation filter 183 down-samples the digital signal of 8 Fs input to themonitor canceller 181 into a digital signal of 1 Fs and outputs the digital signal down-sampled to 1 Fs to theIIR filter 184 positioned at a subsequent stage. - The
IIR filter 184 and theFIR filter 185 are components for executing the filter process performed by themonitor canceller 181 described above with reference toFIG. 9 . As described above, among the filter processes performed by themonitor canceller 181, the process related to the frequency characteristic is mainly allocated to theIIR filter 184, and the simple process for the delay component is allocated to theFIR filter 185. In the example illustrated inFIG. 11 , theIIR filter 184 and theFIR filter 185 execute various kinds of filter processes on the digital signal of 1 Fs. - The digital signal (that is, the digital signal of 1 Fs) which has undergone various kinds of filter processes performed by the
IIR filter 184 and theFIR filter 185 is up-sampled to the digital signal of 8 Fs through theinterpolation filter 186. Then, the digital signal up-sampled to 8 Fs is output to the subtracting unit 191 (seeFIG. 10 ) positioned at a stage subsequent to themonitor canceller 181. - As described above, in the
signal processing device 12 according to the present embodiment, resources for the calculations may be reduced by reducing the sampling rate locally for at least some calculations among various kinds of calculations (for example, the calculations in theHT filter 121, themonitor canceller 181, and the occlusion canceller 161). A calculation in which the sampling rate is locally reduced among various kinds of calculations in thesignal processing device 12 may be appropriately decided on the basis of a checking result of checking efficiency of resource reduction associated with the down-sampling through a prior experiment or the like. - The example of the mechanism for reducing the delay amount of each route (for example, the routes R11 and R13 illustrated in
FIGS. 9 and 10 ) in thesignal processing device 12 according to the present embodiment and implementing the hear-through effect in a more appropriate manner has been described above with reference toFIGS. 9 and 10 . The example of the mechanism for reducing the delay amount through thesignal processing device 12 illustrated inFIG. 9 has been described above, but it will be appreciated that it is possible to reduce the delay amount on the basis of a similar mechanism even in thesignal processing device 80 illustrated inFIG. 5 or thesignal processing device 11 illustrated inFIG. 7 . - Next, a modified example of the
signal processing device 12 according to the present embodiment will be described with reference toFIG. 12 .FIG. 12 is a block diagram illustrating an example of a functional configuration of a signal processing device according to a modified example of the present embodiment. The signal processing device according to the modified example is also referred to as a “signal processing device 13” to be distinguished from thesignal processing device 12 according to the present embodiment described above with reference toFIGS. 9 and 10 . In the example illustrated inFIG. 12 , similarly toFIG. 10 , the ADC and the DAC that perform the conversion process between the analog signal and the digital signal and the filter that converts the sampling rate of the digital signal are explicitly illustrated. - As illustrated in
FIG. 12 , thesignal processing device 13 according to the modified example differs from thesignal processing device 12 according to the above embodiment (seeFIG. 10 ) in that amonitor canceller 181′ is provided instead of themonitor canceller 181 illustrated inFIG. 12 . Therefore, the present description will proceed, particularly, focusing on a configuration of themonitor canceller 181′, and the remaining components are similar to those of thesignal processing device 12 according to the above embodiment, and thus detailed description thereof is omitted. - As illustrated in
FIG. 12 , themonitor canceller 181′ is positioned at a stage subsequent to theHT filter 121 and processes the difference signal output from theHT filter 121. Due to this configuration, themonitor canceller 181′ need not perform the process related to the generation of the difference signal (that is, the process based on (Formula 2) and (Formula 3) described above, unlike themonitor canceller 181 described above with reference toFIG. 9 . - In other words, the
monitor canceller 181′ performs the filter process based on the transfer function corresponding to each characteristic on the inputted difference signal so that the influences of the device characteristic of each of thepower amplifier 141, thedriver 511, and themicrophone amplifier 151 and the spatial characteristic in the internal space are reflected. - The
monitor canceller 181′ outputs the difference signal which has undergone the filter process to thesubtracting unit 191 positioned at a subsequent stage. A subsequent process is similar to that of thesignal processing device 12 according to the above embodiment (seeFIGS. 9 and 10 ). - With this configuration, the
signal processing device 13 according to the modified example can communalize the process related to the generation of the difference signal in theHT filter 121 and themonitor canceller 181 of thesignal processing device 12 illustrated inFIGS. 9 and 10 as the process of theHT filter 121. Therefore, as compared with thesignal processing device 12 according to the above-described embodiment, thesignal processing device 13 according to the modified example is able to reduce the resources for the calculation related to the generation of the difference signal, and thus it is possible to reduce the circuit size. - The
signal processing device 13 according to the modified example of the present embodiment has been described above with reference toFIG. 12 . - As described above, the
signal processing device 12 according to the present embodiment subtracts the component corresponding to the difference signal from the acoustic signal based on the sound collection result of theinternal microphone 515 in addition to the component of the sound input. With this configuration, in thesignal processing device 12 according to the present embodiment, it is possible to exclude the component of the difference signal from the processing target from which theocclusion canceller 161 generates the noise reduction signal. In other words, in thesignal processing device 12 according to the present embodiment, it is possible to prevent the component of the difference signal from being suppressed by the noise reduction signal. Therefore, thesignal processing device 12 according to the present embodiment is able to implement the hear-through effect in a more natural manner (that is, a manner in which the user U has a less strange feeling) than in thesignal processing device 11 according to the first embodiment. - Next, a signal processing device according to a third embodiment of the present disclosure will be described. As described above, in the signal processing device according to each embodiment of the present disclosure, the noise reduction signal for suppressing the voice component of the user propagating to the external ear canal UA is generated using the sound collection result of collecting the sound propagating in the internal space through the
internal microphone 515. Due to this configuration, the acoustic signal based on the sound collection result of the internal microphone 515 (that is, the sound propagating in the internal space) includes the voice component (that is, the voice component of the user U propagating to the external ear canal UA via the bones or fresh of the head of the user U) as described above. - In this regard, in the present embodiment, an example of a signal processing device which is capable of using the voice component included in the acoustic signal based on the sound collection result obtained by the
internal microphone 515 as a voice input (for example, a transmission signal in a voice call) will be described. - For example,
FIG. 13 is a block diagram illustrating an example of a functional configuration of a signal processing device according to the present embodiment. In the following description, the signal processing device illustrated inFIG. 13 is also referred to as a “signal processing device 14 a” to be distinguished from the signal processing device according to each embodiment. Further, in the functional configuration illustrated inFIG. 13 , illustration of the DAC and the ADC is omitted in order to facilitate understanding of the description. - As illustrated in
FIG. 13 , thesignal processing device 14 a according to the present embodiment differs from thesignal processing device 13 according to the second embodiment (seeFIG. 9 ) in that anoise gate 411, anEQ 412, and acompressor 413 are provided. In this regard, in the present description, the functional configuration of thesignal processing device 14 a according to the present embodiment will be described focusing on a difference with thesignal processing device 13 according to the second embodiment, and thus detailed description of the remaining parts will be omitted. - As illustrated in
FIG. 13 , in thesignal processing device 14 a, at a node positioned at a stage subsequent to thesubtracting unit 191 indicated by reference numeral n11 (that is, positioned between the subtractingunit 191 and the subtracting unit 171), an acoustic signal passing through the node n11 is split, and some split acoustic signals are input to thenoise gate 411. - The
noise gate 411 is a component for performing a so-called noise gate process on the input acoustic signal. Specifically, as the noise gate process, thenoise gate 411 performs a process of lowering a level of an output signal at which a level of an input acoustic signal is equal to or less than a certain level (that is, closes a gate) and causing the level of the output signal to an original level (that is, opens the gate) if it exceeds the certain level. As is commonly performed, parameters in the noise gate process such as an attenuation rate of the output level, opening and closing envelopes of the gate, and a frequency band at which the gate responds are appropriately set so that an articulation rate of an uttered sound (that is, a voice component included in an input acoustic signal) is improved. - Then, the
noise gate 411 outputs the acoustic signal which has undergone the noise gate process to theEQ 412 positioned at a subsequent stage. - The
EQ 412 is a component for performing the equalizing process on the acoustic signal output from thenoise gate 411. As described above, the low-frequency side of the voice component included in the acoustic signal split at the node n11 (that is, the acoustic signal based on the sound collection result of the internal microphone 515) is amplified, and the sound based on the acoustic signal (that is, the voice component) is heard by the listener as if it is muffled. For this reason, theEQ 412 improves the articulation rate of the sound to be heard by correcting the frequency characteristic of the acoustic signal so that the sound based on the acoustic signal is heard naturally by the listener (that is, so that a more natural frequency characteristic balance is obtained). - For example, the target characteristic that enables the
EQ 412 to perform the equalizing process on the input acoustic signal may be decided on the basis of a result of a prior experiment or the like in advance. - Then, the
EQ 412 outputs the acoustic signal which has undergone the equalizing process (that is, the acoustic signal including the voice component) to thecompressor 413 positioned at a subsequent stage. - The
compressor 413 is a component for performing a process for adjusting a time amplitude on the input acoustic signal as a so-called compressor process. - Specifically, as described above, the voice component included in the input acoustic signal propagates to the external ear canal UA via the bones or fresh of the head of the user U and causes the external ear canal wall to vibrate like a secondary speaker, and the vibration reaches the
internal microphone 515 via the external ear canal UA. As described above, the propagation path in which the voice component reaches theinternal microphone 515 has non-linearity slightly as compared with the air propagation such as the propagation in the external environment. - Therefore, a difference in a magnitude of an uttered voice which varies depending on a magnitude of a generated voice is larger than in a case in which a normal voice propagating via the air is collected, and thus the listener may be unable to hear the voice collected without change.
- In this regard, the
compressor 413 arranges a time axis amplitude of the acoustic signal based on the sound collection result obtained by the internal microphone 515 (specifically, the acoustic signal output from the EQ 412) so that the difference in the magnitude of the uttered voice is suppressed. - Then, the
compressor 413 performs the compressor process on the input acoustic signal, and outputs the acoustic signal which has undergone the compressor process (that is, the acoustic signal including the voice component) as a voice signal. - The configuration of the
signal processing device 14 a illustrated inFIG. 13 is merely an example, and the configuration is not particularly limited as long as it is possible to output the acoustic signal including the voice component collected by theinternal microphone 515 as the voice signal. - For example,
FIG. 14 is a block diagram illustrating another example of a functional configuration of the signal processing device according to the present embodiment. In the following description, the signal processing device illustrated inFIG. 14 is also referred to as a “signal processing device 14 b” to be distinguished from the signal processing device described above with reference toFIG. 13 . Further, in the case in which the signal processing device illustrated inFIG. 14 is not distinguished from the signal processing device described above with reference toFIG. 13 , it is also referred to simply as “signal processing device 14.” - As illustrated in
FIG. 14 , in thesignal processing device 14 b, at a node positioned at the stage subsequent to thesubtracting unit 171 indicated by reference numeral n12 (that is, positioned between the subtractingunit 171 and the occlusion canceller 161), an acoustic signal passing through the node n12 is split, and some split acoustic signals are input to thenoise gate 411. - Here, the acoustic signal passing through the node n12 corresponds to an acoustic signal obtained by further subtracting the component of the sound input from the acoustic signal passing through the node n11. Therefore, in the
signal processing device 14 b illustrated inFIG. 14 , it is possible to output the acoustic signal in which components other than the voice components are further suppressed in the acoustic signals based on the sound collection result of theinternal microphone 515 as the voice signal as compared with thesignal processing device 14 a illustrated inFIG. 13 . - The example of the functional configuration of the
signal processing device 14 according to the present embodiment has been described above with reference toFIGS. 13 and 14 . - As described above, in the
signal processing device 14 according to the present embodiment, the acoustic signal obtained by subtracting the difference signal from the acoustic signal based on the sound collection result of theinternal microphone 515 through the subtractingunit 191 is output as the voice signal. With this configuration, the acoustic signal in which the component corresponding to the ambient sound among the components included in the acoustic signal based on the sound collection result of theinternal microphone 515 is suppressed is output as the voice signal. In other words, according to thesignal processing device 14 of the present embodiment, it is possible to acquire a voice input having a higher S/N ratio (that is, a smaller noise) than in the case of collecting the voice of the user U using a microphone or the like in the external environment. - Next, an application example of the
signal processing device 14 according to the present embodiment will be described with reference toFIG. 15 .FIG. 15 is an explanatory diagram for describing an application example of thesignal processing device 14 according to the present embodiment. Specifically,FIG. 15 illustrates an example of a functional configuration of an information processing system which is capable of executing various kinds of processes on the basis of instruction content indicated by the voice input by using the voice signal output from thesignal processing device 14 as the voice input. - The information processing system illustrated in
FIG. 15 includes a head mountedacoustic device 51, asignal processing device 14, an analyzingunit 61, acontrol unit 63, and aprocessing executing unit 65. Since the head mountedacoustic device 51 and thesignal processing device 14 are similar to those in the example illustrated inFIG. 13 orFIG. 14 , detailed description thereof will be omitted. - The analyzing
unit 61 is a component for acquiring the voice signal (that is, the voice output) output from thesignal processing device 14 as the voice input and performing various kinds of analysis on the voice input so that thecontrol unit 63 to be described later is able to recognize content indicated by the voice input (that is, the instruction content given from the user U). The analyzingunit 61 includes avoice recognizing unit 611 and a naturallanguage processing unit 613. - The
voice recognizing unit 611 converts the voice input acquired from thesignal processing device 14 into character information by analyzing the voice input on the basis of a so-called voice recognition technique. Then, thevoice recognizing unit 611 outputs a result of analysis based on the voice recognition technique, that is, the character information obtained by converting the voice input to the naturallanguage processing unit 613. - The natural
language processing unit 613 acquires the character information obtained by converting the voice input from thevoice recognizing unit 611 as the result of analyzing the voice input obtained from thesignal processing device 14 on the basis of the voice recognition technique. The naturallanguage processing unit 613 performs analysis based on a so-called natural language processing technique (for example, lexical analysis (morphological analysis), syntax analysis, semantic analysis, or the like) on the acquired character information. - Then, the natural
language processing unit 613 outputs information indicating a result of performing natural language processing on the character information obtained by converting the voice input acquired from thesignal processing device 14 to thecontrol unit 63. - The
control unit 63 acquires information indicating a result of analyzing the voice input acquired from the signal processing device 14 (that is, a result of performing natural language processing on the character information obtained by converting the voice input) from the analyzingunit 61. Thecontrol unit 63 recognizes the instruction content given from the user U which is based on the voice input on the basis of the acquired analysis result. - The
control unit 63 specifies a target function (for example, an application) on the basis of the recognized instruction content given from the user U and instructs theprocessing executing unit 65 to execute the specified function. - The
processing executing unit 65 is a component for executing various kinds of functions. On the basis of the instruction given from thecontrol unit 63, Theprocessing executing unit 65 reads various kinds of data for executing a target function (for example, a library for executing an application or data of content) and executes the function on the basis of the read data. Further, a storage destination of data for executing various kinds of functions through theprocessing executing unit 65 is not particularly limited as long as the data is stored at a position at which it is readable by theprocessing executing unit 65. - At this time, the
processing executing unit 65 may also input acoustic information based on a result of executing the function instructed from the control unit 63 (for example, audio content reproduced on the basis of an instruction) to thesignal processing device 14. As another example, theprocessing executing unit 65 may generate voice information indicating content to be presented to the user U on the basis of the result of executing the function instructed from thecontrol unit 63 on the basis of a so-called voice synthesis technique and input the generated audio information to thesignal processing device 14. With this configuration, the user U is able to recognize results of executing various kinds of functions on the basis of the instruction content given from the user U as the acoustic information (voice information) output through the head mountedacoustic device 51. - In other words, according to the information processing system illustrated in
FIG. 15 , the user U is able to instruct the information processing system to execute various kinds of functions by voice in the state in which the user UE wear the head mountedacoustic device 51 and hear the acoustic information based on the result of executing the functions through the head mountedacoustic device 51. - As a specific example, the user U is able to give an instruction to reproduce desired audio content by voice and hear a result of reproducing audio content through the head mounted
acoustic device 51. - As another example, the user is able to instruct the information processing system to read desired character information (for example, a delivered e-mail, news, information uploaded to a network, or the like) and hear a result of reading the character information through the head mounted
acoustic device 51. - As another example, the information processing system illustrated in
FIG. 15 may be used for a so-called voice call. In this case, the voice signal output from thesignal processing device 14 may be used as a transmission signal, and a received signal may be input to thesignal processing device 14 as the sound input. - The configuration of the information processing system illustrated in
FIG. 15 is merely an example, and the configuration illustrated inFIG. 15 is not necessarily limited as long as it is possible to implement the processes of the components of the information processing system described above. As a specific example, at least some of the analyzingunit 61, thecontrol unit 63, and theprocessing executing unit 65 may be installed in an external device (for example, a server) connected via a network. - The example of the functional configuration of the information processing system using the voice signal output from the
signal processing device 14 as the voice input has been described above with reference toFIG. 15 as the application example of thesignal processing device 14 according to the present embodiment. - Next, an example of a hardware configuration of a
signal processing device 10 according to each embodiment of the present disclosure (that is, thesignal processing devices 11 to 14) will be described with reference toFIG. 16 .FIG. 16 is a diagram illustrating an example of the hardware configuration of thesignal processing device 10 according to each embodiment of the present disclosure. - As illustrated in
FIG. 16 , thesignal processing device 10 according to the present embodiment includes aprocessor 901, amemory 903, astorage 905, anoperation device 907, a notifyingdevice 909, anacoustic device 911, asound collecting device 913, and abus 917. Further, thesignal processing device 10 may include acommunication device 915. - The
processor 901 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), or a system on chip (SoC), and executes various processes of thesignal processing device 10. Theprocessor 901 may be constituted by, for example, an electronic circuit that executes various kinds of calculation processes. The components of thesignal processing devices 11 to 14 (particularly, theHT filter 121, theocclusion canceller 161, themonitor canceller 181, or the like) may be implemented by theprocessor 901. - The
memory 903 includes a random access memory (RAM) and a read only memory (ROM), and stores programs and data executed by theprocessor 901. Thestorage 905 may include a storage medium such as a semiconductor memory or a hard disk. - The
operation device 907 has a function of generating an input signal that enables the user to perform a desired operation. Theoperation device 907 may be configured as, for example, a touch panel. As another example, theoperation device 907 may be configured with an input unit that enables the user to input information such as a button, a switch, and a keyboard, an input control circuit that generates an input signal on the basis of an input performed by the user and supplies the input signal to theprocessor 901, and the like. - The notifying
device 909 is an example of an output device and may be a device such as a liquid crystal display (LCD) device, an organic EL (organic light emitting diode) display, or the like. In this case, the notifyingdevice 909 is able to notify the user of predetermined information by displaying a screen. - The example of the notifying
device 909 described above is merely an example, and a form of the notifyingdevice 909 is not particularly limited as long as it is possible to notify the user of predetermined information. As a specific example, the notifyingdevice 909 may be a device that notifies the user predetermined information by means of a lighting or blinking pattern such as a light emitting diode (LED). Further, the notifyingdevice 909 may be a device that notifies the user of predetermined information through vibration such as a so-called vibrator. - The
acoustic device 911 is a device that notifies the user of predetermined information by outputting a predetermined acoustic signal as in a speaker or the like. In the head mountedacoustic device 51, particularly, the speaker driven by thedriver 511 may be configured with theacoustic device 911. - The
sound collecting device 913 is a device that collects a voice uttered by the user or a sound coming from a surrounding environment and acquires them as acoustic information (acoustic signal) as in a microphone. Further, thesound collecting device 913 may acquire data indicating an analogue acoustic signal indicating the collected voice or sound as the acoustic information or may convert the analog acoustic signal into a digital acoustic signal, and acquire data indicating the converted digital acoustic signal as the acoustic information. Each of theexternal microphone 513 and theinternal microphone 515 in the head mountedacoustic device 51 described above may be implemented by thesound collecting device 913. - The
communication device 915 is a communication unit installed in thesignal processing device 10, and performs communication with an external device via a network. Thecommunication device 915 is a communication interface for wired or wireless communication. In a case in which thecommunication device 915 is configured as a wireless communication interface, thecommunication device 915 may include a communication antenna, a radio frequency (RF) circuit, a baseband processor, and the like. - The
communication device 915 has a function of performing various kinds of signal processing on a signal received from the external device and is able to supply a digital signal generated from the received analog signal to theprocessor 901. - The
bus 917 connects theprocessor 901, thememory 903, thestorage 905, theoperation device 907, the notifyingdevice 909, theacoustic device 911, thesound collecting device 913, and thecommunication device 915 with one another. Thebus 917 may include a plurality of types of buses. - Further, it is also possible to create a program causing hardware such as a processor, a memory, and a storage which are installed in a computer to perform functions similar to those of the components of the
signal processing device 10. Further, a computer readable storage medium having the program stored therein may also be provided. - As described above, the
signal processing device 10 according to each embodiment of the present disclosure (that is, thesignal processing devices 11 to 14 described above) generates the difference signal on the basis of the sound collection result for the ambient sound propagating in the external space outside the mountingunit 510 of the head mountedacoustic device 51. Further, thesignal processing device 10 generates the noise reduction signal for suppressing the voice component propagating to the internal space on the basis of the sound collection result for the sound propagating to the internal space inside the mountingunit 510. Then, thesignal processing device 10 adds the generated difference signal and the noise reduction signal to the input sound input, and outputs the acoustic signal generated on the basis of the addition result to thedriver 511 of the head mountedacoustic device 51. Accordingly, thedriver 511 is driven in accordance with the acoustic signal, and the sound based on the acoustic signal is radiated into the internal space. - With this configuration, the component of the difference signal included in the sound radiated into the internal space and the ambient sound propagating to the internal space via the mounting unit 510 (that is, the sound propagating via the propagation environment F in
FIGS. 2 and 3 ) are added in the internal space, and the addition result is heard by the user U, and thus the hear-through effect can be implemented. Further, the noise reduction signal included in the sound radiated into the internal space and the voice component propagating to the external ear canal UA via the bones or fresh of the head of the user U are added, and the addition result is heard by the user U, and thus the user U is able to hear his/her voice in a more natural manner (that is, the user U has no strange feeling). - A series of processes (that is, signal processing such as various kinds of filter processes) executed by the
signal processing device 10 according to each embodiment of the present disclosure described above corresponds to an example of a “signal processing method.” - The preferred embodiment(s) of the present disclosure has/have been described above with reference to the accompanying drawings, whilst the present disclosure is not limited to the above examples. A person skilled in the art may find various alterations and modifications within the scope of the appended claims, and it should be understood that they will naturally come under the technical scope of the present disclosure.
- Further, the effects described in this specification are merely illustrative or exemplified effects, and are not limitative. That is, with or in the place of the above effects, the technology according to the present disclosure may achieve other effects that are clear to those skilled in the art from the description of this specification.
- Additionally, the present technology may also be configured as below.
- (1)
- A signal processing device, including:
- a first acquiring unit configured to acquire a sound collection result for a first sound propagating in an external space outside a mounting unit to be worn on an ear of a listener;
- a second acquiring unit configured to acquire a sound collection result for a second sound propagating in an internal space connected with an external ear canal inside the mounting unit;
- a first filter processing unit configured to generate a difference signal which is substantially equal to a difference between the first sound propagating directly from the external space toward an inside of the external ear canal and the first sound propagating from the external space to the internal space via the mounting unit on the basis of the sound collection result for the first sound;
- a subtracting unit configured to generate a subtraction signal obtained by subtracting a first signal component based on the sound collection result for the first sound and a second signal component based on an input acoustic signal to be output from an acoustic device from an inside of the mounting unit toward the internal space from the sound collection result for the second sound;
- a second filter processing unit configured to generate a noise reduction signal for reducing the subtraction signal on the basis of the subtraction signal; and
- an adding unit configured to add the difference signal and the noise reduction signal to the input acoustic signal to generate a drive signal for driving the acoustic device.
- (2)
- The signal processing device according to (1), including:
- a third filter processing unit configured to apply, to the acoustic signal based on the sound collection result for the first sound, a characteristic corresponding to at least a transfer function of a route on which the acoustic signal output from the acoustic device is collected as the second sound via the internal space, and output the acoustic signal based on the sound collection result for the first sound as the first signal component.
- (3)
- The signal processing device according to (2),
- in which the third filter processing unit generates the first signal component using the sound collection result for the first sound as an input signal.
- (4)
- The signal processing device according to (2),
- in which the third filter processing unit generates the first signal component using the difference signal output from the first filter processing unit as an input signal.
- (5)
- The signal processing device according to any one of (2) to (4),
- in which the third filter processing unit includes a fourth filter processing unit configured to process a delay component in the acoustic signal based on the input sound collection result for the first sound and a fifth filter processing unit configured to process a frequency component.
- (6)
- The signal processing device according to (5),
- in which the fourth filter processing unit includes an infinite impulse response filter.
- (7)
- The signal processing device according to (5) or (6),
- in which the fifth filter processing unit includes a finite impulse response filter.
- (8)
- The signal processing device according to any one of (1) to (7), including: a first equalization processing unit configured to equalize the input acoustic signal to a first target characteristic and output the equalized acoustic signal to the adding unit; and
- a second equalization processing unit configured to equalize the input acoustic signal to a second target characteristic and output the equalized acoustic signal to the subtracting unit as the second signal component.
- (9)
- The signal processing device according to any one of (1) to (8), including: a voice signal output unit configured to output a signal component based on a result of subtracting the first signal component from the sound collection result for the second sound as a voice signal.
- (10)
- The signal processing device according to (9),
- in which the voice signal output unit outputs the subtraction signal as the voice
- The signal processing device according to any one of (1) to (10), including:
- at least one of a first sound collecting unit configured to collect the first sound and a second sound collecting unit configured to collect the second sound.
- (12)
- The signal processing device according to any one of (1) to (11), including:
- the acoustic device.
- (13)
- A signal processing device, including:
- an acquiring unit configured to acquire a sound collection result for a sound propagating in an external space outside a mounting unit to be worn on an ear of a listener;
- a filter processing unit configured to generate a difference signal which is substantially equal to a difference between the sound directly propagating from the external space toward an inside of an external ear canal and the sound propagating from the external space to the inside of the external ear canal via the mounting unit on the basis of the sound collection result for the sound; and
- an adding unit configured to add the difference signal to an input acoustic signal to be output from an acoustic device from an inside of the mounting unit toward the inside of the external ear canal to generate a drive signal for driving the acoustic device,
- in which a delay amount before the sound propagating in the external space is collected, and then the sound based on the drive signal obtained by adding the difference signal based on the sound is output from the acoustic device is 100 μs or less.
- (14)
- The signal processing device according to (13), including:
- an AD converting unit configured to perform AD conversion of converting the sound collection result for the sound propagating in the external space into a first digital signal at a first sampling rate;
- a decimation filter configured to generate a second digital signal by down-sampling the first digital signal to a third sampling rate which is lower than the first sampling rate and higher than a second sampling rate for sampling the input acoustic signal;
- an interpolation filter configured to up-sample the digital signal sampled at the third sampling rate to the first sampling rate; and
- a DA converting unit configured to perform DA conversion of converting an output result of the interpolation filter into an analog acoustic signal,
- in which the filter processing unit generates the difference signal using the second digital signal as an input signal.
- (15)
- A signal processing method, including, by a processor:
- acquiring a sound collection result for a first sound propagating in an external space outside a mounting unit to be worn on an ear of a listener;
- acquiring a sound collection result for a second sound propagating in an internal space connected with an external ear canal inside the mounting unit;
- generating a difference signal which is substantially equal to a difference between the first sound propagating directly from the external space toward an inside of the external ear canal and the first sound propagating from the external space to the internal space via the mounting unit on the basis of the sound collection result for the first sound;
- generating a subtraction signal obtained by subtracting a first signal component based on the sound collection result for the first sound and a second signal component based on an input acoustic signal to be output from an acoustic device from an inside of the mounting unit toward the internal space from the sound collection result for the second sound;
- generating a noise reduction signal for reducing the subtraction signal on the basis of the subtraction signal; and
- adding the difference signal and the noise reduction signal to the input acoustic signal and to generate a drive signal for driving the acoustic device.
- (16)
- A program causing a computer to execute:
- acquiring a sound collection result for a first sound propagating in an external space outside a mounting unit to be worn on an ear of a listener;
- acquiring a sound collection result for a second sound propagating in an internal space connected with an external ear canal inside the mounting unit;
- generating a difference signal which is substantially equal to a difference between the first sound propagating directly from the external space toward an inside of the external ear canal and the first sound propagating from the external space to the internal space via the mounting unit on the basis of the sound collection result for the first sound;
- generating a subtraction signal obtained by subtracting a first signal component based on the sound collection result for the first sound and a second signal component based on an input acoustic signal to be output from an acoustic device from an inside of the mounting unit toward the internal space from the sound collection result for the second sound;
- generating a noise reduction signal for reducing the subtraction signal on the basis of the subtraction signal; and
- adding the difference signal and the noise reduction signal to the input acoustic signal and to generate a drive signal for driving the acoustic device.
-
- 11 to 14 signal processing device
- 111 microphone amplifier
- 113 decimation filter
- 121 HT filter
- 123 adding unit
- 133 interpolation filter
- 134 interpolation filter
- 141 power amplifier
- 143 interpolation filter
- 151 microphone amplifier
- 153 decimation filter
- 161 occlusion canceller
- 171 subtracting unit
- 181 monitor canceller
- 183 decimation filter
- 184 IIR filter
- 185 FIR filter
- 186 interpolation filter
- 191 subtracting unit
- 411 noise gate
- 412 EQ
- 413 compressor
- 51 head mounted acoustic device
- 510 mounting unit
- 511 driver
- 513 external microphone
- 515 internal microphone
- 61 analyzing unit
- 611 voice recognizing unit
- 613 natural language processing unit
- 63 control unit
- 65 processing executing unit
Claims (8)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/354,710 US10667034B2 (en) | 2015-04-17 | 2019-03-15 | Signal processing device, signal processing method, and program |
Applications Claiming Priority (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2015084817 | 2015-04-17 | ||
JP2015-084817 | 2015-04-17 | ||
PCT/JP2016/056504 WO2016167040A1 (en) | 2015-04-17 | 2016-03-02 | Signal processing device, signal processing method, and program |
US201715565524A | 2017-10-10 | 2017-10-10 | |
US16/354,710 US10667034B2 (en) | 2015-04-17 | 2019-03-15 | Signal processing device, signal processing method, and program |
Related Parent Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/565,524 Continuation US10349163B2 (en) | 2015-04-17 | 2016-03-02 | Signal processing device, signal processing method, and program |
PCT/JP2016/056504 Continuation WO2016167040A1 (en) | 2015-04-17 | 2016-03-02 | Signal processing device, signal processing method, and program |
Publications (2)
Publication Number | Publication Date |
---|---|
US20190215598A1 true US20190215598A1 (en) | 2019-07-11 |
US10667034B2 US10667034B2 (en) | 2020-05-26 |
Family
ID=57126748
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/565,524 Active US10349163B2 (en) | 2015-04-17 | 2016-03-02 | Signal processing device, signal processing method, and program |
US16/354,710 Active US10667034B2 (en) | 2015-04-17 | 2019-03-15 | Signal processing device, signal processing method, and program |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/565,524 Active US10349163B2 (en) | 2015-04-17 | 2016-03-02 | Signal processing device, signal processing method, and program |
Country Status (5)
Country | Link |
---|---|
US (2) | US10349163B2 (en) |
EP (2) | EP3285497B1 (en) |
JP (1) | JP6604376B2 (en) |
CN (1) | CN107431852B (en) |
WO (1) | WO2016167040A1 (en) |
Families Citing this family (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107431852B (en) | 2015-04-17 | 2019-10-01 | 索尼公司 | Signal processing apparatus, signal processing method and computer readable storage medium |
KR20190128669A (en) * | 2017-03-09 | 2019-11-18 | 아브네라 코포레이션 | Real time sound processor |
WO2018163423A1 (en) | 2017-03-10 | 2018-09-13 | ヤマハ株式会社 | Headphone |
US10483931B2 (en) * | 2017-03-23 | 2019-11-19 | Yamaha Corporation | Audio device, speaker device, and audio signal processing method |
TWI648731B (en) * | 2017-07-24 | 2019-01-21 | 驊訊電子企業股份有限公司 | Active noise cancellation system |
CN111801727B (en) * | 2018-02-01 | 2024-07-23 | 思睿逻辑国际半导体有限公司 | Active Noise Cancellation (ANC) system with selectable sampling rate |
CN108206023A (en) * | 2018-04-10 | 2018-06-26 | 南京地平线机器人技术有限公司 | Sound processing apparatus and sound processing method |
US10149786B1 (en) * | 2018-06-08 | 2018-12-11 | Alvin J. Halfaker | Noise reduction earmuffs system and method |
US20220093120A1 (en) * | 2019-01-15 | 2022-03-24 | Nec Corporation | Information processing device, wearable device, information processing method, and storage medium |
CN111836147B (en) | 2019-04-16 | 2022-04-12 | 华为技术有限公司 | Noise reduction device and method |
EP3960259A4 (en) * | 2019-04-26 | 2023-05-17 | Sony Interactive Entertainment Inc. | Information processing system, information processing device, method for controlling information processing device, and program |
US11223891B2 (en) * | 2020-02-19 | 2022-01-11 | xMEMS Labs, Inc. | System and method thereof |
EP4097714A1 (en) | 2021-04-22 | 2022-12-07 | Google LLC | Reduced complexity implementation for acoustic noise canceling |
WO2022264535A1 (en) | 2021-06-18 | 2022-12-22 | ソニーグループ株式会社 | Information processing method and information processing system |
EP4358542A1 (en) | 2021-06-18 | 2024-04-24 | Sony Group Corporation | Information processing method, information processing system, data collection method, and data collection system |
WO2023107426A2 (en) * | 2021-12-07 | 2023-06-15 | Bose Corporation | Audio device having aware mode auto-leveler |
Family Cites Families (24)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7072476B2 (en) * | 1997-02-18 | 2006-07-04 | Matech, Inc. | Audio headset |
WO2006049260A1 (en) * | 2004-11-08 | 2006-05-11 | Nec Corporation | Signal processing method, signal processing device, and signal processing program |
GB2434708B (en) * | 2006-01-26 | 2008-02-27 | Sonaptic Ltd | Ambient noise reduction arrangements |
GB2446966B (en) * | 2006-04-12 | 2010-07-07 | Wolfson Microelectronics Plc | Digital circuit arrangements for ambient noise-reduction |
US8208644B2 (en) * | 2006-06-01 | 2012-06-26 | Personics Holdings Inc. | Earhealth monitoring system and method III |
JP5194434B2 (en) * | 2006-11-07 | 2013-05-08 | ソニー株式会社 | Noise canceling system and noise canceling method |
WO2008091874A2 (en) * | 2007-01-22 | 2008-07-31 | Personics Holdings Inc. | Method and device for acute sound detection and reproduction |
JP4882773B2 (en) | 2007-02-05 | 2012-02-22 | ソニー株式会社 | Signal processing apparatus and signal processing method |
JP2008258878A (en) * | 2007-04-04 | 2008-10-23 | Matsushita Electric Ind Co Ltd | Sound output device having microphone |
US9191740B2 (en) * | 2007-05-04 | 2015-11-17 | Personics Holdings, Llc | Method and apparatus for in-ear canal sound suppression |
WO2009006418A1 (en) * | 2007-06-28 | 2009-01-08 | Personics Holdings Inc. | Method and device for background noise mitigation |
JP4631939B2 (en) * | 2008-06-27 | 2011-02-16 | ソニー株式会社 | Noise reducing voice reproducing apparatus and noise reducing voice reproducing method |
CN201303410Y (en) * | 2008-11-12 | 2009-09-02 | 中北大学 | Novel earphone |
US8526628B1 (en) * | 2009-12-14 | 2013-09-03 | Audience, Inc. | Low latency active noise cancellation system |
WO2011161487A1 (en) * | 2010-06-21 | 2011-12-29 | Nokia Corporation | Apparatus, method and computer program for adjustable noise cancellation |
GB2492983B (en) * | 2011-07-18 | 2013-09-18 | Incus Lab Ltd | Digital noise-cancellation |
US8798283B2 (en) * | 2012-11-02 | 2014-08-05 | Bose Corporation | Providing ambient naturalness in ANR headphones |
US20140126733A1 (en) * | 2012-11-02 | 2014-05-08 | Daniel M. Gauger, Jr. | User Interface for ANR Headphones with Active Hear-Through |
US9050212B2 (en) | 2012-11-02 | 2015-06-09 | Bose Corporation | Binaural telepresence |
US20140126736A1 (en) | 2012-11-02 | 2014-05-08 | Daniel M. Gauger, Jr. | Providing Audio and Ambient Sound simultaneously in ANR Headphones |
KR101382553B1 (en) * | 2013-02-27 | 2014-04-07 | 한밭대학교 산학협력단 | Receiver for recognizing external environment |
CN103200480A (en) * | 2013-03-27 | 2013-07-10 | 北京昆腾微电子有限公司 | Headset and working method thereof |
CN103269465B (en) * | 2013-05-22 | 2016-09-07 | 歌尔股份有限公司 | The earphone means of communication under a kind of strong noise environment and a kind of earphone |
CN107431852B (en) | 2015-04-17 | 2019-10-01 | 索尼公司 | Signal processing apparatus, signal processing method and computer readable storage medium |
-
2016
- 2016-03-02 CN CN201680021159.9A patent/CN107431852B/en active Active
- 2016-03-02 JP JP2017512224A patent/JP6604376B2/en active Active
- 2016-03-02 US US15/565,524 patent/US10349163B2/en active Active
- 2016-03-02 EP EP16779832.1A patent/EP3285497B1/en active Active
- 2016-03-02 WO PCT/JP2016/056504 patent/WO2016167040A1/en active Application Filing
- 2016-03-02 EP EP19196604.3A patent/EP3614690A1/en not_active Withdrawn
-
2019
- 2019-03-15 US US16/354,710 patent/US10667034B2/en active Active
Also Published As
Publication number | Publication date |
---|---|
JPWO2016167040A1 (en) | 2018-02-08 |
EP3285497B1 (en) | 2021-10-27 |
EP3285497A4 (en) | 2019-03-27 |
US10349163B2 (en) | 2019-07-09 |
CN107431852A (en) | 2017-12-01 |
EP3285497A1 (en) | 2018-02-21 |
US20180115818A1 (en) | 2018-04-26 |
CN107431852B (en) | 2019-10-01 |
WO2016167040A1 (en) | 2016-10-20 |
EP3614690A1 (en) | 2020-02-26 |
JP6604376B2 (en) | 2019-11-13 |
US10667034B2 (en) | 2020-05-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10667034B2 (en) | Signal processing device, signal processing method, and program | |
CN103959813B (en) | Earhole Wearable sound collection device, signal handling equipment and sound collection method | |
JP6069829B2 (en) | Ear hole mounting type sound collecting device, signal processing device, and sound collecting method | |
US10622005B2 (en) | Method and device for spectral expansion for an audio signal | |
KR102004460B1 (en) | Digital hearing device using bluetooth circuit and digital signal processing | |
EP4047955A1 (en) | A hearing aid comprising a feedback control system | |
JP6197930B2 (en) | Ear hole mounting type sound collecting device, signal processing device, and sound collecting method | |
EP4132009A2 (en) | A hearing device comprising a feedback control system | |
JP6999187B2 (en) | Active noise elimination system for headphones | |
US11551704B2 (en) | Method and device for spectral expansion for an audio signal | |
EP4300992A1 (en) | A hearing aid comprising a combined feedback and active noise cancellation system | |
EP4099724A1 (en) | A low latency hearing aid | |
Zhuang et al. | A constrained optimal hear-through filter design approach for earphones | |
US11812224B2 (en) | Hearing device comprising a delayless adaptive filter | |
EP4329335A1 (en) | A method of reducing wind noise in a hearing device | |
EP4297435A1 (en) | A hearing aid comprising an active noise cancellation system | |
US20220406328A1 (en) | Hearing device comprising an adaptive filter bank | |
US20230169948A1 (en) | Signal processing device, signal processing program, and signal processing method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
AS | Assignment |
Owner name: SONY CORPORATION, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ASADA, KOHEI;YAMABE, YUSHI;HAYASHI, SHIGETOSHI;REEL/FRAME:048782/0949 Effective date: 20170727 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |