EP2611216B1 - Systèmes et procédés pour déterminer les fonctions de transfert liées à la tête - Google Patents
Systèmes et procédés pour déterminer les fonctions de transfert liées à la tête Download PDFInfo
- Publication number
- EP2611216B1 EP2611216B1 EP12152430.0A EP12152430A EP2611216B1 EP 2611216 B1 EP2611216 B1 EP 2611216B1 EP 12152430 A EP12152430 A EP 12152430A EP 2611216 B1 EP2611216 B1 EP 2611216B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- information
- input image
- hrtf
- reference images
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims description 56
- 230000006870 function Effects 0.000 title description 13
- 238000012546 transfer Methods 0.000 title description 10
- 238000004891 communication Methods 0.000 claims description 25
- 210000005069 ears Anatomy 0.000 claims description 15
- 210000003128 head Anatomy 0.000 description 55
- 210000005010 torso Anatomy 0.000 description 41
- 230000000875 corresponding effect Effects 0.000 description 32
- 238000005259 measurement Methods 0.000 description 16
- 238000012545 processing Methods 0.000 description 16
- 230000003287 optical effect Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 239000013598 vector Substances 0.000 description 5
- 230000008901 benefit Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- 241001077860 Helias Species 0.000 description 2
- 241000746998 Tragus Species 0.000 description 2
- 230000002596 correlated effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 210000000613 ear canal Anatomy 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 210000001699 lower leg Anatomy 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000000513 principal component analysis Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000011872 anthropometric measurement Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000012074 hearing test Methods 0.000 description 1
- 238000010348 incorporation Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 210000003454 tympanic membrane Anatomy 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/70—Adaptation of deaf aid to hearing loss, e.g. initial electronic fitting
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
- H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
Definitions
- This application relates generally to spatial hearing, and more specifically, systems and methods for determining head related transfer functions (HRTFs).
- HRTFs head related transfer functions
- HRTFs Spatial hearing involves measuring HRTFs of the subject being fitted. Determining HRTFs is a difficult task, and existing techniques for determining HRTFs are very time consuming, expensive, and cumbersome. HRTFs are typically measured in an anechoic chamber as the transfer function from a sound source positioned at different horizontal and vertical angels from the subject to small microphones positioned close to the subject's eardrums. Many measurement positions are obtained in order to get an accurate HRTF measurement.
- EP 1 667 487 relates to a sound image control device that localizes, using a sound transducer such as a speaker and a headphone, a sound image at a position other than where such sound transducer exists, and relates to a design tool for designing a sound image control device.
- DE 103 18 191 relates to a method for generating a transfer function.
- US 2010/0177906 relates to methods for distributed sensing and centralized reconstruction of two correlated signals, modeled as the input and output of an unknown sparse filtering operation.
- US 6,996,244 relates to estimation of HRTF for spatial sound representative.
- Applicant of the subject application determines that new systems and methods for determining HRTF would be desirable.
- a method for determining HRTF includes obtaining a plurality of reference images of different respective ears, one or more of the reference images associated with a corresponding pre-determined HRTF information, obtaining information regarding an input image that includes an image of an ear of a subject, comparing the information regarding the input image with information regarding the reference images using a processor, and selecting one of the pre-determined HRTF information based at least in part on a result of the act of comparing.
- At least one of the reference images may include an image of an ear, an image of a head, and an image of at least a part of a torso.
- the reference images may include different pairs of front and side reference images, each of the pairs being associated with a corresponding pre-determined HRTF information.
- the information regarding the input image may comprise image data for the input image.
- the information regarding the input image may comprise information extracted from the input image.
- the information extracted from the input image may comprise head size, head shape, torso size, torso shape, cavum concha height, cymba concha height, cavum concha width, fossa height, pinna height, pinna width, intertragal incisure width, cavum concha depth, or any combination thereof.
- the HRTF information may comprise one or more filter coefficients for a hearing aid.
- the act of comparing may comprise comparing the input image with the reference images to obtain different respective results representing different respective degrees of match
- the act of selecting the one of the pre-determined HRTF information may comprise selecting the HRTF information that corresponds with the one of the reference images providing the result representing the highest degree of match.
- the act of obtaining the plurality of reference images may be accomplished by accessing a database that stores the plurality of reference images.
- the act of obtaining the information regarding the input image may be performed by receiving the information regarding the input image through the Internet.
- the method may further include transmitting data regarding the selected pre-determined HRTF information to a device that is in communication with the processor.
- the method may further include using the selected pre-determined HRTF information to configure a hearing aid.
- a device for determining HRTF include a processor configured for obtaining a plurality of reference images of different respective ears, one or more of the reference images associated with a corresponding pre-determined HRTF information, obtaining information regarding an input image that includes an image of an ear of a subject, comparing the information regarding the input image with information regarding the reference images, and selecting one of the pre-determined HRTF information based at least in part on a result of the act of comparing.
- At least one of the reference images may include an image of an ear, an image of a head, and an image of at least a part of a torso.
- the reference images may include different pairs of front and side reference images, each of the pairs being associated with a corresponding pre-determined HRTF information.
- the information regarding the input image may comprise image data for the input image.
- the information regarding the input image may comprise information extracted from the input image.
- the information extracted from the input image may comprise head size, head shape, torso size, torso shape, cavum concha height, cymba concha height, cavum concha width, fossa height, pinna height, pinna width, intertragal incisure width, cavum concha depth, or any combination thereof.
- the HRTF information may comprise one or more filter coefficients for a hearing aid.
- the processor may be configured to perform the act of comparing by comparing the input image with the reference images to obtain different respective results representing different respective degrees of match, and wherein the processor may be configured to perform the act of selecting the one of the pre-determined HRTF information by selecting the HRTF information that corresponds with the one of the reference images providing the result representing the highest degree of match.
- the processor may be configured to perform the act of obtaining the plurality of reference images by accessing a database that stores the plurality of reference images.
- the processor may be configured to perform the act of obtaining the information regarding the input image by receiving the information regarding the input image through the Internet.
- the processor may be further configured for generating a signal to cause data regarding the selected pre-determined HRTF information to be transmitted to a device that is in communication with the processor.
- the selected pre-determined HRTF information may be for configuring a hearing aid.
- a machine readable non-transitory medium stores a set of instructions, an execution of which causes a method to be performed, the method comprising obtaining a plurality of reference images of different respective ears, one or more of the reference images associated with a corresponding pre-determined HRTF information, obtaining information regarding an input image that includes an image of an ear of a subject, comparing the information regarding the input image with information regarding the reference images, and selecting one of the pre-determined HRTF information based at least in part on a result of the act of comparing.
- a method for determining HRTF includes obtaining an input image of an ear of a subject, transmitting information regarding the input image to a device for comparing the information regarding the input image with information regarding a plurality of reference images of different respective ears, one or more of the reference images associated with a pre-determined HRTF information, and receiving, from the device, data regarding one of the HRTF information that is selected based at least in part on a result of the act of comparing.
- the input image may include an image of an ear, an image of a head, and an image of at least a part of a torso.
- the reference images may include different pairs of front and side reference images, each of the pairs being associated with a corresponding pre-determined HRTF information.
- the information regarding the input image may comprise image data for the input image.
- the information regarding the input image may comprise information extracted from the input image.
- the information extracted from the input image may comprise head size, head shape, torso size, torso shape, cavum concha height, cymba concha height, cavum concha width, fossa height, pinna height, pinna width, intertragal incisure width, cavum concha depth, or any combination thereof.
- the HRTF information may comprise one or more filter coefficients for a hearing aid.
- the selected HRTF information may be for configuring a hearing aid.
- the device may comprise a non-transitory medium that stores the plurality of reference images.
- the act of obtaining the input image may be performed by receiving the input image through a port at a device.
- a device for determining HRTF includes a processor configured for obtaining an input image of an ear of a subject, transmitting information regarding the input image to a device for comparing the information regarding the input image with information regarding a plurality of reference images of different respective ears, one or more of the reference images associated with a pre-determined HRTF information, and receiving, from the device, data regarding one of the HRTF information that is selected based at least in part on a result of the act of comparing.
- the input image may include an image of an ear, an image of a head, and an image of at least a part of a torso.
- the reference images may include different pairs of front and side reference images, each of the pairs being associated with a corresponding pre-determined HRTF information.
- the information regarding the input image may comprise image data for the input image.
- the information regarding the input image may comprise information extracted from the input image.
- the information extracted from the input image may comprise head size, head shape, torso size, torso shape, cavum concha height, cymba concha height, cavum concha width, fossa height, pinna height, pinna width, intertragal incisure width, cavum concha depth, or any combination thereof.
- the HRTF information may comprise one or more filter coefficients for a hearing aid.
- the selected HRTF information may be for configuring a hearing aid.
- the device may comprise a non-transitory medium that stores the plurality of reference images.
- the processor may be configured to perform the act of obtaining the input image by receiving the input image through a port at a device.
- a machine readable non-transitory medium stores a set of instructions, an execution of which causes a method to be performed, the method comprising obtaining an input image of an ear of a subject, transmitting information regarding the input image to a device for comparing the information regarding the input image with information regarding a plurality of reference images of different respective ears, one or more of the reference images associated with a pre-determined HRTF information, and receiving, from the device, data regarding one of the HRTF information that is selected based at least in part on a result of the act of comparing.
- FIG. 1 illustrates a system 10 for determining HRTF in accordance with some embodiments.
- the system 10 includes a first device 12 and a second device 14 that is in communication with the first device 12.
- the first device 12 includes a processor 16, and the second device 14 also includes a processor 18.
- the term "processor" may refer to one or more processing units, such as one or more processors.
- the first device 12 is configured (e.g., built and/or programmed) for obtaining an input image 20 of an ear of a subject, and transmitting the input image 20 to the second device 14.
- the second device 14 is configured to receive the input image 20, and process the input image 20 to determine HRTF for the subject's ear.
- one first device 12 is shown in the illustrated embodiments, in other embodiments, there may be a plurality of devices 12 that are communicatively coupled to the second device 14.
- the first device 12 may be different types of devices in different embodiments.
- the first device 12 may be a handheld communication device that has a camera, such as a phone (e.g., a smart phone, an iPhone), a blackberry, or iPad.
- the first device 12 may be a computer (such as a desktop, a laptop, a notebook, etc.). In such cases, the computer itself may be used to capture an image of the subject's ear as the input image if the computer has a camera. Alternatively, a digital camera may be used to capture an image of the subject' ear, and the image file may then be uploaded to the computer through an input port at the computer.
- the first device 12 may be a fitting device, or any of other types of devices, that is proprietary and/or custom-made.
- the second device 14 may also be different types of devices in different embodiments.
- the second device 14 may be a computer (such as a desktop, a laptop, etc.), or a plurality of computers.
- the second device 14 may be implemented as a database system.
- the second device 14 may be any types of device, as long as it is capable of receiving data and processing data.
- the second device 14 may have a non-transitory medium associated therewith for storing data. The non-transitory medium may be located in the second device 14, or may be communicatively coupled to the second device 14.
- the communication link 28 between the first device 12 and the second device may be implemented using different techniques in different embodiments.
- the communication link 28 may be a wire or a cable that is coupled between the first device 12 and the second device 14.
- the communication link 28 may be a wireless communication link.
- information may be transmitted between the first device 12 and the second device 14 via the Internet.
- information may be transmitted between the first device 12 and the second device 14 using a removable storage device, such as a memory card, a USB memory stick, etc.
- FIG. 2 illustrates a method 200 for determining HRTF using the system 10 of FIG. 1 in accordance with some embodiments.
- HRTF or similar terms (such as HRTFs, HRTF information, head related transfer function, etc.) may refer to head related transfer function(s) itself, or it may refer to information related to HRTF(s), such as one or more parameters or one or move values that are used to represent HRTF(s), any information that is derivable from HRTF, any information that is based at least in part on HRTF, or any information that is associated with HRTF.
- HRTF information may include one or more sets of filter coefficients for one or more input sources in a hearing instrument for restoring spatial sound cues, or may include any information that is used to represent or derive the one or more sets of filter coefficients.
- HRTF information may include filter(s) representing acoustic transfer functions that model transformations generated by a torso, head, and auricle of a listener on signal originating from a sound source, or any information that is used to represent or derive such filter(s).
- a sound source position may have an associated pair of HRTFs (a first HRTF for a right ear, and a second HRTF for a second ear).
- HRTF or similar terms (such as HRTFs, HRTF information, head related transfer function, etc.) may refer to HRTF information for a right ear, for a left ear, or for both the right and left ears.
- HRTF may be represented by a function that depends on a frequency of sound, and/or a position of the sound source (e.g., angle ⁇ representing azimuth, and angle ⁇ representing elevation of the sound source with respect to the listener).
- angle ⁇ representing azimuth
- angle ⁇ representing elevation of the sound source with respect to the listener
- the method 200 may be performed when a user is attempting to determine a HRTF of a subject, such as during a spatial hearing procedure. In other embodiments, the method 200 may be performed in other contexts or situations.
- a user of the first device 12 may use the first device 12 to obtain an input image 20 of an ear of the subject (Item 202).
- the first device 12 may be a handheld communication device, such as a phone (e.g., a smart phone, an iPhone), a blackberry, or iPad, etc.
- the user may use the first device 12 to take a picture of the subject's ear.
- the picture is then stored in a non-transitory medium in the first device 12 as the input image 20.
- the user may use a camera to take a picture of the subject's ear.
- the camera may be part of a computer, or may be communicatively coupled to a computer.
- the user may use a separate handheld camera to take a picture of the subject's ear, and the picture file from a media storage device inside the camera may then be transferred (e.g., uploaded) to the first device 12, such as a computer, a handheld communication device, or any of other devices that has communication capability.
- the act of obtaining the input image 20 may be considered as being performed by the first device 12 (e.g., the processor 16), which obtains the input image 20 by generating it, or by receiving it.
- FIG. 3 illustrates an example of an input image 20 for an ear of a subject.
- the input image 20 of the ear 270 may include the lobule 272, antitragus 274, cavum concha 276, cymba concha 278, scaphoid fossa 280, helix 282, antihelix 284, triangular fossa 286, crus helias 288, tragus 290, and the intertragal incisures 292.
- the input image 20 may not include all of the above features, and may include only a subset of the above features instead.
- the first device 12 transmits the input image 20 to the second device 14 for comparing the input image 20 with a plurality of reference images of different respective ears (Item 204).
- reference images or similar terms (such as “plurality of reference images”) may refer to a complete set of all available reference images, or alternatively, may refer to a subset of all available reference images.
- the act of transmitting may involve the processor 16 generating a control signal to cause the first device 12 to wirelessly transmit the input image 20 to the second device 14.
- the act of transmitting may involve the processor 16 generating a control signal to cause the first device 12 to transmit the input image 20 to the second device 14 using a wire or a cable.
- the input image 20 may be transmitted from the first device 12 to the second device 14 through the Internet.
- the second device 14 performs a method 250.
- the second device 14 obtains the input image 20 (Item 252). Such may be accomplished by the second device 14 (e.g., the processor 18 of the second device 14) receiving the input image 20 from the first device 12 in some embodiments.
- the second device 14 obtains reference images of different respective ears (Item 254).
- such may be accomplished by the processor 18 of the second device 14 accessing a non-transitory medium that stores the reference images.
- the act of obtaining reference images may be accomplished by the second device 14 (e.g., the processor 18 of the second device 14) receiving the reference images from another device (such as, from a computer, an external hard drive, a database, etc.).
- the reference images are stored in a non-transitory medium that is associated with the second device 14.
- the non-transitory medium may be inside the second device 14, or may be a storage device that is communicatively coupled to the second device 14. In either case, the non-transitory medium may be considered to be a part of the second device 14.
- FIG. 4 illustrates examples of reference images 300a-300i that may be stored at the second device 14.
- information that is "stored at" a device may refer to information that is physically located at the device, or may refer to information that may be accessed by the device.
- the term "store at” should not be limited to information that is physically stored at a device.
- each of the reference images 300 includes a picture of an ear.
- the features of the ears are different in different respective reference images 300.
- the lobule, antitragus, cavum concha, cymba concha, scaphoid fossa, helix, antihelix, triangular fossa, crus helias, tragus, or any combination of the foregoing may have size and/or shape that is different between two or more of the reference images 300.
- the reference images 300 stored at the second device 14 are collected over time from different fitting procedures of different subjects. Thus, as time progresses, the number of reference images 300 at the second device 14 may increase.
- the second device 14 may be accessed by different users (e.g., different operators at different fitting centres, hospitals, doctors' offices, etc.), who will upload the reference images 300, along with their corresponding HRTF information (e.g., filter coefficients, etc.), to the second device 14. For example, after a user has performed a session to determine HRTF (e.g., using traditional techniques) for a particular subject, the user may take a picture of the subject's ear(s), and transmit the picture along with the determined HRTF to the second device 14.
- HRTF filter coefficients, etc.
- one or more of the reference images 300 may be an image of a model ear.
- each reference image 300 may include an image of a head and an image of at least a part of a torso.
- each of the reference images 300 is associated with pre-determined HRTF.
- a reference image 300 may have one or more parameters for a HRTF that is associated with the image 300.
- the HRTF associated with the reference image 300 was previously obtained for the ear shown in the reference image 300.
- the HRTF associated with the reference image 300 is "pre-determined".
- the HRTF information may be stored at the second device 14 as part of the image file for the reference image 300. In other embodiments, the HRTF information may be stored at the second device 14 as separate files from the image files of the reference images 300.
- the second device 14 may also store meta data (e.g., pointers) for associating the HRTF information with respective reference images 300.
- the pre-determined HRTF for each reference image 300 may have multiple parameters that are stored in table form, with the file name of the reference image 300 being the index in the table.
- the parameters of the HRTF may be retrieved from the table by referencing the index (e.g., the file name of the corresponding reference image 300) at the table.
- Other techniques for associating the reference images 300 with their corresponding HRTF information may be used in other embodiments.
- the processor 18 of the second device 14 compares the input image 20 with the reference images 300 (Item 256).
- the input image and/or the reference images 300 may optionally be pre-processed.
- the processor 16 of the first device 12 and/or the processor 18 of the second device 14 may be configured to perform image processing to filter out noise in the input image 20, scaling the input image 20 so that the image of the ear has a certain height and width, adjusting a grey-scale of the input image 20, or a combination of the foregoing.
- the processor 18 of the second device 14 may be configured to perform image processing to filter out noise in the reference images 300, scaling the reference images 300 so that the respective images of the ears have a certain height and width, adjusting a grey-scale of the reference images 300, or a combination of the foregoing.
- the pre-processing of the reference images 300 may be performed when the reference images 300 are being stored at the second device 14. In other embodiments, the pre-processing of the reference images 300 may be performed during a process to determine HRTF. In such cases, the pre-processing of the reference images 300 may be performed in response to the processor 18 receiving an input image 20 from the first device 12.
- a degree of match may be determined between the input image 20 and each of the reference images 300.
- the processor 18 of the second device 14 selects the reference image 300 that provides the highest degree of match as the matching image.
- a cross-correlation value (which may be considered an example of a degree of match) may be computed using the input image 20 and each of the reference images 300.
- the processor 18 of the second device 14 selects the reference image 300 that provides the highest cross-correlation value as the matching image.
- a pattern matching technique may be used to determine how well the input image 20 matches a given reference image 300. Any of the pattern matching techniques known in the art may be used in different embodiments.
- different image processing techniques may be employed to efficiently search for a best match between the input image 20 and one of the reference images 300.
- the input image 20 may be processed to obtain an exterior profile (or outline) of the subject's ear.
- the exterior profile of the input image is then compared with exterior profiles of different respective reference images 300 to find a subset of reference images 300 that match the exterior profile of the subject's ear.
- the input image 20 is then compared with the subset of reference images 300 (now considering the features within the outline of the ear) to determine a reference image 300 that best matches the input image 20.
- deformation registration may be used to determine a reference image 300 that best matches the input image 20.
- Deformable image registration is a process or technique in which points in a first image of a first object are associated with corresponding points in a second image of a second object, wherein the first and second objects may have the same or different sizes and/or shapes.
- a deformation registration between two images may be represented by a matrix of vectors that maps points at a first image to points at a second image.
- the processor 18 of the second device 14 may calculate matrices representing deformation registration between the input image 20 and respective ones of the reference images 300.
- Each of the matrices may then be used to compute a score that represents a degree of match between the input image 20 and a corresponding one of the reference images 300.
- the score may be computed on the basis that a matrix with more vectors having non-zero values and/or vectors having higher non-zero values will have a relatively lower score. This is because more vectors mean that more points in one image need to be "moved" in order to reach the positions of the corresponding points in another image, and because a higher vector value means that the a point in one image needs to be moved further in order to reach the position of the corresponding point in another image.
- variants of Principal Component Analysis may be applied to the reference images 300 and used for classifying the input image 20.
- PCA Principal Component Analysis
- the processor 18 of the second device 14 After a reference image 300 has been determined to be the best match with the input image 20, the processor 18 of the second device 14 then obtains the HRTF that is associated with the matching reference image 300 (Item 258).
- the second device 14 is configured to obtain the HRTF by selecting (e.g., retrieving) the HRTF information that is associated with the best matching reference image 300.
- the act of obtaining HRTF by the processor 18 of the second device 14 may be considered performed when the processor 18 of the second device 14 has determined (e.g., selected) a best matching reference image 300.
- the act of obtaining HRTF by the processor 18 of the second device 14 may be considered performed when the processor 18 accesses a non-transitory medium to retrieve the stored corresponding HRTF.
- the HRTF information that may be obtained by the processor 18 may include one or more sets of filter coefficients for one or more input sources in a hearing instrument for restoring spatial sound cues, or may include any information that is used to represent or derive the one or more sets of filter coefficients.
- the HRTF information that may be obtained by the processor 18 may include filter configuration and coefficients for different devices to be fitted (depending on form factor, number of microphones, receiver type, etc).
- the processor 18 when selecting the reference image 300 that best matches the input image 20, the processor 18 also considers other information, such as head size of the subject, head shape of the subject, torso size of the subject, torso shape of the subject, or any combination thereof. For example, in some embodiments, when the first device 12 transmits the input image 20 to the second device 14, the first device 12 also transmits head and torso information for the subject to the second device 14. In such cases, each of the reference images 300 stored at the second device 14 is also associated with corresponding head and torso information.
- the second device 14 compares the input image 20 with the reference images 300
- the second device 14 also compares the head and torso information for the subject with the corresponding head and torso information that are associated with the respective reference images 300.
- the processor 18 of the second device 14 selects the pre-determined HRTF information that is associated a corresponding reference image 300 and corresponding head and torso information providing the best match with the input image 20 and the head and torso information for the subject.
- the processor 18 of the second device 14 After the processor 18 of the second device 14 has obtained the HRTF, the processor 18 then transmits the HRTF information to the first device 12 (Item 260).
- the processor 18 of the second device 14 may be configured to generate a signal to cause the second device 14 to transmit the HRTF information through a wireless network (e.g., through the Internet).
- the processor 18 of the second device 14 may be configured to generate a signal to cause the second device 14 to transmit the HRTF information through a wire or a cable.
- the first device 12 receives, from the second device 14, data regarding HRTFs that is obtained based on a result of the act of comparing (Item 206).
- the received data regarding the HRTF may be stored in a non-transitory medium that is associated with the first device 12. Additionally, or alternatively, the received data regarding the HRTF may be displayed on a screen (e.g., a screen at the first device 12) for presentation to the user of the first device 12.
- the first device 12 may transmit the HRTF information for incorporation into a hearing aid device that is for the subject. For example, if the HRTF information includes filter coefficients for a hearing aid, the HRTF information may be incorporated into such hearing aid (e.g., into a processor of the hearing aid).
- the selected HRTF for the subject may have an associated reference image 300 that does not provide the best match with the input image 20.
- the HRTF for the first "matching" reference image 300 may be selected by the second device 14, and the HRTF for the second "matching" reference image 300 may be selected by the second device 14, and the second device 14 then determine HRTF based on an interpolation between the two selected pre-determined HRTFs.
- the input image 20 may also include an image of the subject's head.
- the reference images 300 may also include corresponding images of different subjects' heads.
- the processor 16 of the first device 12, and/or the processor 18 of the second device 14 may process the input image 20 to determine information that is relevant for HRTF determination.
- the processor 16 and/or the processor 18 may perform image processing to determine head size (e.g., width), head shape, etc.
- the processor 18 may then select pre-determined HRTF (e.g., filter coefficients) that is stored at the second device 14 based at least in part on the determined head size and/or head shape.
- pre-determined HRTF e.g., filter coefficients
- the HRTF that is selected may be the HRTF that is associated with a reference image 300 having head size and/or head shape that best match the head size and/or head shape in the input image 20, and having ear feature(s) that best match the ear feature(s) in the input image 20.
- the input image 20 may also include an image of the subject's head, as well as at least a part of the subject's torso (e.g., the shoulders of the subject).
- the reference images 300 may also include corresponding images of different subjects' heads and their torsos.
- the processor 16 of the first device 12, and/or the processor 18 of the second device 14 may process the input image 20 to determine information that is relevant for HRTF determination.
- the processor 16 and/or the processor 18 may perform image processing to determine head size (e.g., width), head shape, torso width, torso depth, etc.
- the processor 18 may then select pre-determined HRTF (e.g., filter coefficients) that is stored at the second device 14 based at least in part on the determined head information and torso information.
- the HRTF that is selected may be the HRTF that is associated with a reference image 300 having head size and/or head shape that best match the head size and/or head shape in the input image 20, having torso size and/or torso shape that best match the torso size and/or torso shape in the input image 20, and having ear feature(s) that best match the ear feature(s) in the input image 20.
- pre-determined HRTF information is selected by the processor 18 of the second device 14 based on a comparison between the input image 20 and the reference images 300.
- pre-determined HRTF information may be selected by the processor 18 of the second device 14 based on a comparison between two input images 20 and two respective sets of reference images 300.
- the device 12 may be used to obtain two input images 20, with the first input image 20 being a front view of a subject that includes the head, ears, and at least part of the torso, and the second input image 20 being a side view of the subject that includes the head, the ear, and at least part of the torso.
- two sets of reference images 300 are stored at the second device 14, with the first set of reference images 300 being front images of different persons, and the second set of reference images 300 being side images of the different persons.
- Each front reference image 300 may include image of a head, images of the ears, and image of at least part of the torso, of a person.
- Each side reference image 300 may include image of the head, image of the ear, and image of at least part of the torso, of the same corresponding person.
- HRTF information stored at the second device 14.
- the processor 18 of the second device 14 selects HRTF information based on a first comparison of the front input image 20 with front reference images 300, and a second comparison of the side input image 20 with side reference images 300. In one implementation, the processor 18 selects the pre-determined HRTF information that is associated with the pair of front reference image 300 and side reference image 300 that best matches the pair of front input image 20 and side input image 20.
- the processor 18 of the second device 14 may be configured to compare information extracted the input image 20 with information extracted from the reference images 300, and select the pre-determined HRTF information based at least on a result of such comparison.
- the processor 16 of the first device 12 and/or the processor 18 of the second device 14 may be configured to perform image processing to extract certain information from the two input images 20 (front input image 20 and side input image 20).
- information that may be extracted from the input images 20 may include head size (width and/or depth), head shape, torso size (width and/or depth), torso shape, cavum concha height, cymba concha height, cavum concha width, fossa height, pinna height, pinna width, intertragal incisure width, cavum concha depth, or any combination of the foregoing.
- the processor 18 of the second device 12 may be configured to perform image processing to extract certain information from the reference images 300 (front reference images 300 and side reference images 300).
- information that may be extracted from the reference images 300 may include head size (width and/or depth), head shape, torso size (width and/or depth), torso shape, cavum concha height, cymba concha height, cavum concha width, fossa height, pinna height, pinna width, intertragal incisure width, cavum concha depth, or any combination of the foregoing.
- the processor 18 of the second device 14 selects HRTF information based on a first comparison of the front input image 20 with front reference images 300, and a second comparison of the side input image 20 with side reference images 300.
- the processor 18 selects the pre-determined HRTF information that is associated with the reference images 300 (pair of front and side reference images 300) providing information that best matches the information extracted from the pair of front input image 20 and side input image 20.
- information extracted from the input image 20 may be considered as an example of information regarding the input image 20.
- information regarding the input image 20 may be the input image 20 itself, in which case, the information may comprise image data for the input image 20.
- information extracted from the reference image 300 may be considered as an example of information regarding the reference image 300.
- information regarding the reference image 300 may be the reference image 300 itself, in which case, the information may comprise image data for the reference image 300.
- the act of comparing input image 20 with reference image(s) 300 may be performed based on classification technique.
- pre-determined HRTF information may be classified based on the information extracted from the reference images 300, wherein such information may include head size (width and/or depth), head shape, torso size (width and/or depth), torso shape, cavum concha height, cymba concha height, cavum concha width, fossa height, pinna height, pinna width, intertragal incisure width, cavum concha depth, or any combination of the foregoing.
- the HRTF information stored at the second device 14 may be classified based on information extracted from the pair of corresponding reference images 300.
- the processor 18 of the second device 14 may select one of the classified HRTF information based on a comparison of the information extracted front the input images 20 (front and side input images 20 for the subject), and information extracted from the reference images 300.
- the processor 18 selects one of the classified HRTF information that is associated with the reference images 300 (pair of front and side reference images 300) providing information that best matches the information extracted from the pair of front input image 20 and side input image 20.
- the HRTF for the first "matching" reference image information may be selected by the second device 14, and the HRTF for the second "matching" reference image information may be selected by the second device 14, and the second device 14 then determine HRTF based on an interpolation between the two selected pre-determined HRTFs.
- the processor 18 of the second device 14 may consider one or more parameters related to an ear canal when selecting HRTF information for the subject. For example, in some embodiments, when the pre-determined HRTF information is being stored at the second device 14, the second device 14 may also store feedback calibration measurements (if available) at the non-transitory medium, and the stored feedback calibration measurements may be linked with the corresponding HRTF information and/or the reference images 300. In one implementation, when different subjects are fitted with a hearing instrument, the feedback path from the receiver in the ear to the microphones on the instrument may be measured, and the measurement may provide a transfer function which may be used to increase the maximum stable gain of the hearing instrument by adaptive filtering.
- This transfer function may be correlated to the HRTF information in the sense that it embeds information on how sound changes when travelling from the microphone positioned outside the ear canal to the receiver inside the canal.
- the processor 18 of the second device 14 selects HRTF information based on a comparison between information regarding the input image 20 (or input images 20 if there are front and side input images 20) and information regarding the reference images 300, and a comparison between feedback measurement for the subject and feedback measurements for the different subjects stored at the second device.
- the processor 18 selects the pre-determined HRTF information that is associated a best matching information from the reference image 300 (or pair of reference images 300 if there are front and side reference images 300), and a best matching corresponding feedback measurement.
- the selection of the pre-determined HRTF by the second device 14 may be performed without using any images.
- feedback measurements for different subjects may be stored at the second device 14, and the corresponding HRTF information may also be stored at the second device 14 in a way that are associated with the corresponding feedback measurements.
- the processor 18 of the second device 14 obtains feedback measurement for the subject (e.g., by receiving such information from the first device 12), and selects HRTF information based on a comparison between the feedback measurement for the subject and feedback measurements for the different subjects stored at the second device 14.
- the processor 18 selects the pre-determined HRTF information that is associated with one of the pre-determined feedback measurements that best matches with the feedback measurement of the subject.
- the system 10 has been described with reference to involving the first device 12 and the second device 14.
- the system 10 may include only one device.
- the second device 14 may include a camera for capturing the input image 20 of an ear of the subject.
- the first device 12 and the method 200 may not be required for obtaining HRTF.
- the act 252 of obtaining the input image 20 is performed by the second device 14 generating an input image 20 of the ear of the subject using a camera.
- the second device 14 may obtain the input image 20 by receiving it through a port.
- a user may take a picture of the subject's ear using a digital camera, and the picture is then uploaded to the second device 14 (e.g., through the USB port, or through a media file input port).
- the second device 14 may not be required to perform the act 260 of transmitting the HRTF information.
- the obtained HRTF may be displayed at a screen (e.g., a screen at the second device 14, or a screen that is coupled to the second device 14).
- the methods 200, 250 have been described with reference to comparing a two dimensional input image 20 with two dimensional reference images 300.
- the input image 20 may be a three dimensional image
- each of the reference images 300 may be a three dimensional reference image.
- the first device 12 may be used to obtain two or more two dimensional images of a subject's ear from different respective directions. The two dimensional images may then be processed to reconstruct a three dimensional input image 20 of the subject's ear.
- the processor 16 of the first device 12 may be configured to perform the image processing to reconstruct the three dimensional input image 20.
- the processor 16 of the first device 12 may be configured to generate signals to cause the first device 12 to transmit the images to the second device 14.
- the second device 14 then perform the image processing on the images to reconstruct the three dimensional input image 20.
- the same image processing technique may be employed to generate each of the three dimensional reference images 300 that are stored at the second device 14 (e.g., at a non-transitory medium that is in the second device 14, or that is coupled to the second device 14).
- the second device 14 compares the three dimensional input image 20 with the three dimensional reference images 300.
- the second device 14 determines HRTF based on a result of the act of comparing the three dimensional input image 20 with the three dimensional reference images 300, and transmits the HRTF information to the first device 12.
- FIG. 5 is a block diagram that illustrates an embodiment of a computer system 1200 upon which one or more embodiments of the invention may be implemented.
- Computer system 1200 includes a bus 1202 or other communication mechanism for communicating information, and a processor 1204 coupled with the bus 1202 for processing information.
- the computer system 1200 may be used to implement the first device 12.
- the computer system 1200 may be used to implement the second device 14.
- the computer system 1200 also includes a main memory 1206, such as a random access memory (RAM) or other dynamic storage device, coupled to the bus 1202 for storing information and instructions to be executed by the processor 1204.
- the main memory 1206 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by the processor 1204.
- the computer system 1200 further includes a read only memory (ROM) 1208 or other static storage device coupled to the bus 1202 for storing static information and instructions for the processor 1204.
- ROM read only memory
- a data storage device 1210 such as a magnetic disk or optical disk, is provided and coupled to the bus 1202 for storing information and instructions.
- the computer system 1200 may be coupled via the bus 1202 to a display 1212, such as a cathode ray tube (CRT), or a flat panel, for displaying information to a user.
- a display 1212 such as a cathode ray tube (CRT), or a flat panel
- An input device 1214 is coupled to the bus 1202 for communicating information and command selections to processor 1204.
- cursor control 1216 is Another type of user input device, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 1204 and for controlling cursor movement on display 1212.
- This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane.
- the computer system 1200 may be used for performing various functions (e.g., calculation) in accordance with the embodiments described herein. According to one embodiment, such use is provided by computer system 1200 in response to processor 1204 executing one or more sequences of one or more instructions contained in the main memory 1206. Such instructions may be read into the main memory 1206 from another computer-readable medium, such as storage device 1210. Execution of the sequences of instructions contained in the main memory 1206 causes the processor 1204 to perform the processing act(s) described herein. One or more processors in a multi-processing arrangement may also be employed to execute the sequences of instructions contained in the main memory 1206. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention. Thus, embodiments described herein are not limited to any specific combination of hardware circuitry and software.
- Non-volatile media includes, for example, optical or magnetic disks, such as the storage device 1210.
- a non-volatile medium is an example of a non-transitory medium.
- Volatile media includes dynamic memory, such as the main memory 1206.
- a volatile medium is another example of a non-transitory medium.
- Transmission media includes coaxial cables, copper wire and fibre optics, including the wires that comprise the bus 1202. Transmission media can also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.
- Computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read.
- Various forms of computer-readable media may be involved in carrying one or more sequences of one or more instructions to the processor 1204 for execution.
- the instructions may initially be carried on a magnetic disk of a remote computer.
- the remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem.
- a modem local to the computer system 1200 can receive the data on the telephone line and use an infrared transmitter to convert the data to an infrared signal.
- An infrared detector coupled to the bus 1202 can receive the data carried in the infrared signal and place the data on the bus 1202.
- the bus 1202 carries the data to the main memory 1206, from which the processor 1204 retrieves and executes the instructions.
- the instructions received by the main memory 1206 may optionally be stored on the storage device 1210 either before or after execution by the processor 1204.
- the computer system 1200 also includes a communication interface 1218 coupled to the bus 1202.
- the communication interface 1218 provides a two-way data communication coupling to a network link 1220 that is connected to a local network 1222.
- the communication interface 1218 may be an integrated services digital network (ISDN) card or a modem to provide a data communication connection to a corresponding type of telephone line.
- ISDN integrated services digital network
- the communication interface 1218 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN.
- LAN local area network
- Wireless links may also be implemented.
- the communication interface 1218 sends and receives electrical, electromagnetic or optical signals that carry data streams representing various types of information.
- the network link 1220 typically provides data communication through one or more networks to other devices.
- the network link 1220 may provide a connection through local network 1222 to a host computer 1224 or to equipment 1226, which may be any device.
- the data streams transported over the network link 1220 can comprise electrical, electromagnetic or optical signals.
- the signals through the various networks and the signals on the network link 1220 and through the communication interface 1218, which carry data to and from the computer system 1200, are exemplary forms of carrier waves transporting the information.
- the computer system 1200 can send messages and receive data, including program code, through the network(s), the network link 1220, and the communication interface 1218.
Landscapes
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Neurosurgery (AREA)
- Otolaryngology (AREA)
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
Claims (12)
- Procédé (200) pour déterminer les fonctions de transfert liées à la tête (HRTF), comprenant:l'obtention (252) d'informations concernant une image d'entrée (20) qui comprend une image d'une oreille d'un sujet;caractérisé en ce que le procédé comprendl'obtention (254) d'une pluralité d'images de référence (300) de différentes oreilles respectives, l'une ou plusieurs des images de référence étant associées à une information de HRTF prédéterminée correspondante;la comparaison (256) des informations concernant l'image d'entrée avec des informations concernant les images de référence en utilisant un processeur;etla sélection de l'une des informations de HRTF prédéterminées sur la base d'au moins en partie un résultat de l'acte de comparaison,l'acte de comparaison (256) comprenant la comparaison de l'image d'entrée avec les images de référence pour obtenir différents résultats respectifs représentant des degrés respectifs de correspondance, etl'acte de sélection de l'une des informations de HRTF prédéterminées comprenant la sélection de l'information de HRTF qui correspond à l'une des images de référence fournissant le résultat représentant le plus haut degré de correspondance.
- Procédé selon la revendication 1, dans lequel au moins l'une des images de référence comprend une image d'une oreille, une image d'une tête, et une image d'au moins une partie d'un torse.
- Procédé selon la revendication 1 ou 2, dans lequel les images de référence comprennent différentes paires d'images de référence frontales et latérales, chacune des paires étant associée à une information de HRTF prédéterminée correspondante.
- Procédé selon l'une quelconque des revendications précédentes, dans lequel les informations concernant l'image d'entrée comprennent des données d'image pour l'image d'entrée.
- Procédé selon l'une quelconque des revendications précédentes, dans lequel les informations concernant l'image d'entrée comprennent des informations extraites de l'image d'entrée.
- Procédé selon la revendication 5, dans lequel les informations extraites de l'image d'entrée comprend la taille de la tête, la forme de la tête, la taille du torse, la forme du torse, la hauteur du cavum de la conque, la hauteur de la cymba de la conque, la largeur du cavum de la conque, la hauteur de la fosse, la hauteur du pavillon de l'oreille, la largeur de pavillon, la largeur d'incisure intertragale, la profondeur du cavum de la conque, ou toute combinaison de ceux-ci.
- Procédé selon l'une quelconque des revendications précédentes, dans lequel les informations de HRTF comprennent un ou plusieurs coefficients de filtre pour une prothèse auditive.
- Procédé selon l'une quelconque des revendications précédentes, dans lequel l'acte d'obtention de la pluralité d'images de référence est accompli en accédant à une base de données qui stocke la pluralité d'images de référence.
- Procédé selon l'une quelconque des revendications précédentes, dans lequel l'acte d'obtention des informations concernant l'image d'entrée est effectué par la réception des informations concernant l'image d'entrée par l'Internet.
- Procédé selon l'une quelconque des revendications précédentes, comprenant en outre la transmission de données concernant les informations de HRTF prédéterminées sélectionnées à un dispositif qui est en communication avec le processeur.
- Procédé selon l'une quelconque des revendications précédentes, comprenant en outre l'utilisation des informations de HRTF prédéterminées sélectionnées pour configurer une prothèse auditive.
- Dispositif pour déterminer HRTF, comprenant un processeur configuré pour exécuter un procédé selon l'une quelconque des revendications 1 à 11.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2012281419A JP5543571B2 (ja) | 2011-12-30 | 2012-12-25 | 頭部伝達関数を決定するシステムおよび方法 |
CN201210592852.5A CN103455824B (zh) | 2011-12-30 | 2012-12-31 | 用于确定头相关传递函数的***和方法 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DKPA201101008 | 2011-12-30 | ||
US13/342,792 US9030545B2 (en) | 2011-12-30 | 2012-01-03 | Systems and methods for determining head related transfer functions |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2611216A1 EP2611216A1 (fr) | 2013-07-03 |
EP2611216B1 true EP2611216B1 (fr) | 2015-12-16 |
Family
ID=45540789
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP12152430.0A Active EP2611216B1 (fr) | 2011-12-30 | 2012-01-25 | Systèmes et procédés pour déterminer les fonctions de transfert liées à la tête |
Country Status (1)
Country | Link |
---|---|
EP (1) | EP2611216B1 (fr) |
Families Citing this family (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9426589B2 (en) | 2013-07-04 | 2016-08-23 | Gn Resound A/S | Determination of individual HRTFs |
US9900722B2 (en) | 2014-04-29 | 2018-02-20 | Microsoft Technology Licensing, Llc | HRTF personalization based on anthropometric features |
US9609436B2 (en) | 2015-05-22 | 2017-03-28 | Microsoft Technology Licensing, Llc | Systems and methods for audio creation and delivery |
US10028070B1 (en) | 2017-03-06 | 2018-07-17 | Microsoft Technology Licensing, Llc | Systems and methods for HRTF personalization |
US10278002B2 (en) | 2017-03-20 | 2019-04-30 | Microsoft Technology Licensing, Llc | Systems and methods for non-parametric processing of head geometry for HRTF personalization |
US11205443B2 (en) | 2018-07-27 | 2021-12-21 | Microsoft Technology Licensing, Llc | Systems, methods, and computer-readable media for improved audio feature discovery using a neural network |
KR20210068409A (ko) | 2018-10-10 | 2021-06-09 | 소니그룹주식회사 | 정보 처리 장치, 정보 처리 방법 및 정보 처리 프로그램 |
GB2600932A (en) * | 2020-11-11 | 2022-05-18 | Sony Interactive Entertainment Inc | Audio personalisation method and system |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6996244B1 (en) * | 1998-08-06 | 2006-02-07 | Vulcan Patents Llc | Estimation of head-related transfer functions for spatial sound representative |
DE10318191A1 (de) * | 2003-04-22 | 2004-07-29 | Siemens Audiologische Technik Gmbh | Verfahren zur Erzeugung und Verwendung einer Übertragungsfunktion |
WO2005025270A1 (fr) * | 2003-09-08 | 2005-03-17 | Matsushita Electric Industrial Co., Ltd. | Outil de conception de dispositif de commande d'images audio et dispositif associe |
US8787501B2 (en) * | 2009-01-14 | 2014-07-22 | Qualcomm Incorporated | Distributed sensing of signals linked by sparse filtering |
-
2012
- 2012-01-25 EP EP12152430.0A patent/EP2611216B1/fr active Active
Also Published As
Publication number | Publication date |
---|---|
EP2611216A1 (fr) | 2013-07-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
DK2611216T3 (en) | Systems and methods for determining the head-related transfer functions | |
EP2611216B1 (fr) | Systèmes et procédés pour déterminer les fonctions de transfert liées à la tête | |
US10939225B2 (en) | Calibrating listening devices | |
US11706582B2 (en) | Calibrating listening devices | |
US11601775B2 (en) | Method for generating a customized/personalized head related transfer function | |
US11082791B2 (en) | Head-related impulse responses for area sound sources located in the near field | |
US9232310B2 (en) | Methods, apparatuses and computer program products for facilitating directional audio capture with multiple microphones | |
KR20060059866A (ko) | 음상 제어 장치의 설계 툴 및 음상 제어 장치 | |
CN114998888A (zh) | 用于产生头相关传递函数滤波器的布置 | |
US11943587B2 (en) | Generating a hearing assistance device shell | |
CN104469092B (zh) | 一种图像采集方法及电子设备 | |
AU2021442090A1 (en) | Error correction of head-related filters | |
JP2024508899A (ja) | オーディオ信号に方向性を適用する電子装置及びその方法 | |
Pausch et al. | Hybrid multi-harmonic model for the prediction of interaural time differences in individual behind-the-ear hearing-aid-related transfer functions | |
CN112330793A (zh) | 耳模三维模型的获取方法、耳机定制方法和计算设备 | |
WO2019174442A1 (fr) | Équipement d'adaptation, procédé de sortie vocale, dispositif, support de stockage et dispositif électronique | |
CN111246363A (zh) | 一种基于听觉匹配的虚拟声定制方法及装置 | |
CN111797675A (zh) | 一种人耳特征点的提取方法、装置和电子设备 | |
WO2020075427A1 (fr) | Système de traitement d'informations, procédé de traitement d'informations, et programme informatique | |
JP2020198545A (ja) | 情報処理装置、情報処理方法及びプログラム | |
US9794685B2 (en) | Video audio recording system, video audio recording device, and video audio recording method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
17P | Request for examination filed |
Effective date: 20140103 |
|
RBV | Designated contracting states (corrected) |
Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
17Q | First examination report despatched |
Effective date: 20140404 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
INTG | Intention to grant announced |
Effective date: 20150611 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 766058 Country of ref document: AT Kind code of ref document: T Effective date: 20160115 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602012013038 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 5 |
|
REG | Reference to a national code |
Ref country code: DK Ref legal event code: T3 Effective date: 20160301 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20151216 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160316 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 766058 Country of ref document: AT Kind code of ref document: T Effective date: 20151216 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160317 Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20160131 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160416 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160418 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 Ref country code: LU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160125 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602012013038 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
26N | No opposition filed |
Effective date: 20160919 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 6 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20160125 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 7 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20120125 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160131 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151216 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230525 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20240123 Year of fee payment: 13 Ref country code: GB Payment date: 20240122 Year of fee payment: 13 Ref country code: CH Payment date: 20240202 Year of fee payment: 13 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20240117 Year of fee payment: 13 Ref country code: DK Payment date: 20240117 Year of fee payment: 13 |