EP2611216B1 - Systeme und Verfahren zur Bestimmung von kopfbezogenen Übertragungsfunktionen - Google Patents

Systeme und Verfahren zur Bestimmung von kopfbezogenen Übertragungsfunktionen Download PDF

Info

Publication number
EP2611216B1
EP2611216B1 EP12152430.0A EP12152430A EP2611216B1 EP 2611216 B1 EP2611216 B1 EP 2611216B1 EP 12152430 A EP12152430 A EP 12152430A EP 2611216 B1 EP2611216 B1 EP 2611216B1
Authority
EP
European Patent Office
Prior art keywords
information
input image
hrtf
reference images
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP12152430.0A
Other languages
English (en)
French (fr)
Other versions
EP2611216A1 (de
Inventor
Thorkild Find Pedersen
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
GN Hearing AS
Original Assignee
GN Resound AS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US13/342,792 external-priority patent/US9030545B2/en
Application filed by GN Resound AS filed Critical GN Resound AS
Priority to JP2012281419A priority Critical patent/JP5543571B2/ja
Priority to CN201210592852.5A priority patent/CN103455824B/zh
Publication of EP2611216A1 publication Critical patent/EP2611216A1/de
Application granted granted Critical
Publication of EP2611216B1 publication Critical patent/EP2611216B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
    • H04R25/70—Adaptation of deaf aid to hearing loss, e.g. initial electronic fitting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
    • H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]

Definitions

  • This application relates generally to spatial hearing, and more specifically, systems and methods for determining head related transfer functions (HRTFs).
  • HRTFs head related transfer functions
  • HRTFs Spatial hearing involves measuring HRTFs of the subject being fitted. Determining HRTFs is a difficult task, and existing techniques for determining HRTFs are very time consuming, expensive, and cumbersome. HRTFs are typically measured in an anechoic chamber as the transfer function from a sound source positioned at different horizontal and vertical angels from the subject to small microphones positioned close to the subject's eardrums. Many measurement positions are obtained in order to get an accurate HRTF measurement.
  • EP 1 667 487 relates to a sound image control device that localizes, using a sound transducer such as a speaker and a headphone, a sound image at a position other than where such sound transducer exists, and relates to a design tool for designing a sound image control device.
  • DE 103 18 191 relates to a method for generating a transfer function.
  • US 2010/0177906 relates to methods for distributed sensing and centralized reconstruction of two correlated signals, modeled as the input and output of an unknown sparse filtering operation.
  • US 6,996,244 relates to estimation of HRTF for spatial sound representative.
  • Applicant of the subject application determines that new systems and methods for determining HRTF would be desirable.
  • a method for determining HRTF includes obtaining a plurality of reference images of different respective ears, one or more of the reference images associated with a corresponding pre-determined HRTF information, obtaining information regarding an input image that includes an image of an ear of a subject, comparing the information regarding the input image with information regarding the reference images using a processor, and selecting one of the pre-determined HRTF information based at least in part on a result of the act of comparing.
  • At least one of the reference images may include an image of an ear, an image of a head, and an image of at least a part of a torso.
  • the reference images may include different pairs of front and side reference images, each of the pairs being associated with a corresponding pre-determined HRTF information.
  • the information regarding the input image may comprise image data for the input image.
  • the information regarding the input image may comprise information extracted from the input image.
  • the information extracted from the input image may comprise head size, head shape, torso size, torso shape, cavum concha height, cymba concha height, cavum concha width, fossa height, pinna height, pinna width, intertragal incisure width, cavum concha depth, or any combination thereof.
  • the HRTF information may comprise one or more filter coefficients for a hearing aid.
  • the act of comparing may comprise comparing the input image with the reference images to obtain different respective results representing different respective degrees of match
  • the act of selecting the one of the pre-determined HRTF information may comprise selecting the HRTF information that corresponds with the one of the reference images providing the result representing the highest degree of match.
  • the act of obtaining the plurality of reference images may be accomplished by accessing a database that stores the plurality of reference images.
  • the act of obtaining the information regarding the input image may be performed by receiving the information regarding the input image through the Internet.
  • the method may further include transmitting data regarding the selected pre-determined HRTF information to a device that is in communication with the processor.
  • the method may further include using the selected pre-determined HRTF information to configure a hearing aid.
  • a device for determining HRTF include a processor configured for obtaining a plurality of reference images of different respective ears, one or more of the reference images associated with a corresponding pre-determined HRTF information, obtaining information regarding an input image that includes an image of an ear of a subject, comparing the information regarding the input image with information regarding the reference images, and selecting one of the pre-determined HRTF information based at least in part on a result of the act of comparing.
  • At least one of the reference images may include an image of an ear, an image of a head, and an image of at least a part of a torso.
  • the reference images may include different pairs of front and side reference images, each of the pairs being associated with a corresponding pre-determined HRTF information.
  • the information regarding the input image may comprise image data for the input image.
  • the information regarding the input image may comprise information extracted from the input image.
  • the information extracted from the input image may comprise head size, head shape, torso size, torso shape, cavum concha height, cymba concha height, cavum concha width, fossa height, pinna height, pinna width, intertragal incisure width, cavum concha depth, or any combination thereof.
  • the HRTF information may comprise one or more filter coefficients for a hearing aid.
  • the processor may be configured to perform the act of comparing by comparing the input image with the reference images to obtain different respective results representing different respective degrees of match, and wherein the processor may be configured to perform the act of selecting the one of the pre-determined HRTF information by selecting the HRTF information that corresponds with the one of the reference images providing the result representing the highest degree of match.
  • the processor may be configured to perform the act of obtaining the plurality of reference images by accessing a database that stores the plurality of reference images.
  • the processor may be configured to perform the act of obtaining the information regarding the input image by receiving the information regarding the input image through the Internet.
  • the processor may be further configured for generating a signal to cause data regarding the selected pre-determined HRTF information to be transmitted to a device that is in communication with the processor.
  • the selected pre-determined HRTF information may be for configuring a hearing aid.
  • a machine readable non-transitory medium stores a set of instructions, an execution of which causes a method to be performed, the method comprising obtaining a plurality of reference images of different respective ears, one or more of the reference images associated with a corresponding pre-determined HRTF information, obtaining information regarding an input image that includes an image of an ear of a subject, comparing the information regarding the input image with information regarding the reference images, and selecting one of the pre-determined HRTF information based at least in part on a result of the act of comparing.
  • a method for determining HRTF includes obtaining an input image of an ear of a subject, transmitting information regarding the input image to a device for comparing the information regarding the input image with information regarding a plurality of reference images of different respective ears, one or more of the reference images associated with a pre-determined HRTF information, and receiving, from the device, data regarding one of the HRTF information that is selected based at least in part on a result of the act of comparing.
  • the input image may include an image of an ear, an image of a head, and an image of at least a part of a torso.
  • the reference images may include different pairs of front and side reference images, each of the pairs being associated with a corresponding pre-determined HRTF information.
  • the information regarding the input image may comprise image data for the input image.
  • the information regarding the input image may comprise information extracted from the input image.
  • the information extracted from the input image may comprise head size, head shape, torso size, torso shape, cavum concha height, cymba concha height, cavum concha width, fossa height, pinna height, pinna width, intertragal incisure width, cavum concha depth, or any combination thereof.
  • the HRTF information may comprise one or more filter coefficients for a hearing aid.
  • the selected HRTF information may be for configuring a hearing aid.
  • the device may comprise a non-transitory medium that stores the plurality of reference images.
  • the act of obtaining the input image may be performed by receiving the input image through a port at a device.
  • a device for determining HRTF includes a processor configured for obtaining an input image of an ear of a subject, transmitting information regarding the input image to a device for comparing the information regarding the input image with information regarding a plurality of reference images of different respective ears, one or more of the reference images associated with a pre-determined HRTF information, and receiving, from the device, data regarding one of the HRTF information that is selected based at least in part on a result of the act of comparing.
  • the input image may include an image of an ear, an image of a head, and an image of at least a part of a torso.
  • the reference images may include different pairs of front and side reference images, each of the pairs being associated with a corresponding pre-determined HRTF information.
  • the information regarding the input image may comprise image data for the input image.
  • the information regarding the input image may comprise information extracted from the input image.
  • the information extracted from the input image may comprise head size, head shape, torso size, torso shape, cavum concha height, cymba concha height, cavum concha width, fossa height, pinna height, pinna width, intertragal incisure width, cavum concha depth, or any combination thereof.
  • the HRTF information may comprise one or more filter coefficients for a hearing aid.
  • the selected HRTF information may be for configuring a hearing aid.
  • the device may comprise a non-transitory medium that stores the plurality of reference images.
  • the processor may be configured to perform the act of obtaining the input image by receiving the input image through a port at a device.
  • a machine readable non-transitory medium stores a set of instructions, an execution of which causes a method to be performed, the method comprising obtaining an input image of an ear of a subject, transmitting information regarding the input image to a device for comparing the information regarding the input image with information regarding a plurality of reference images of different respective ears, one or more of the reference images associated with a pre-determined HRTF information, and receiving, from the device, data regarding one of the HRTF information that is selected based at least in part on a result of the act of comparing.
  • FIG. 1 illustrates a system 10 for determining HRTF in accordance with some embodiments.
  • the system 10 includes a first device 12 and a second device 14 that is in communication with the first device 12.
  • the first device 12 includes a processor 16, and the second device 14 also includes a processor 18.
  • the term "processor" may refer to one or more processing units, such as one or more processors.
  • the first device 12 is configured (e.g., built and/or programmed) for obtaining an input image 20 of an ear of a subject, and transmitting the input image 20 to the second device 14.
  • the second device 14 is configured to receive the input image 20, and process the input image 20 to determine HRTF for the subject's ear.
  • one first device 12 is shown in the illustrated embodiments, in other embodiments, there may be a plurality of devices 12 that are communicatively coupled to the second device 14.
  • the first device 12 may be different types of devices in different embodiments.
  • the first device 12 may be a handheld communication device that has a camera, such as a phone (e.g., a smart phone, an iPhone), a blackberry, or iPad.
  • the first device 12 may be a computer (such as a desktop, a laptop, a notebook, etc.). In such cases, the computer itself may be used to capture an image of the subject's ear as the input image if the computer has a camera. Alternatively, a digital camera may be used to capture an image of the subject' ear, and the image file may then be uploaded to the computer through an input port at the computer.
  • the first device 12 may be a fitting device, or any of other types of devices, that is proprietary and/or custom-made.
  • the second device 14 may also be different types of devices in different embodiments.
  • the second device 14 may be a computer (such as a desktop, a laptop, etc.), or a plurality of computers.
  • the second device 14 may be implemented as a database system.
  • the second device 14 may be any types of device, as long as it is capable of receiving data and processing data.
  • the second device 14 may have a non-transitory medium associated therewith for storing data. The non-transitory medium may be located in the second device 14, or may be communicatively coupled to the second device 14.
  • the communication link 28 between the first device 12 and the second device may be implemented using different techniques in different embodiments.
  • the communication link 28 may be a wire or a cable that is coupled between the first device 12 and the second device 14.
  • the communication link 28 may be a wireless communication link.
  • information may be transmitted between the first device 12 and the second device 14 via the Internet.
  • information may be transmitted between the first device 12 and the second device 14 using a removable storage device, such as a memory card, a USB memory stick, etc.
  • FIG. 2 illustrates a method 200 for determining HRTF using the system 10 of FIG. 1 in accordance with some embodiments.
  • HRTF or similar terms (such as HRTFs, HRTF information, head related transfer function, etc.) may refer to head related transfer function(s) itself, or it may refer to information related to HRTF(s), such as one or more parameters or one or move values that are used to represent HRTF(s), any information that is derivable from HRTF, any information that is based at least in part on HRTF, or any information that is associated with HRTF.
  • HRTF information may include one or more sets of filter coefficients for one or more input sources in a hearing instrument for restoring spatial sound cues, or may include any information that is used to represent or derive the one or more sets of filter coefficients.
  • HRTF information may include filter(s) representing acoustic transfer functions that model transformations generated by a torso, head, and auricle of a listener on signal originating from a sound source, or any information that is used to represent or derive such filter(s).
  • a sound source position may have an associated pair of HRTFs (a first HRTF for a right ear, and a second HRTF for a second ear).
  • HRTF or similar terms (such as HRTFs, HRTF information, head related transfer function, etc.) may refer to HRTF information for a right ear, for a left ear, or for both the right and left ears.
  • HRTF may be represented by a function that depends on a frequency of sound, and/or a position of the sound source (e.g., angle ⁇ representing azimuth, and angle ⁇ representing elevation of the sound source with respect to the listener).
  • angle ⁇ representing azimuth
  • angle ⁇ representing elevation of the sound source with respect to the listener
  • the method 200 may be performed when a user is attempting to determine a HRTF of a subject, such as during a spatial hearing procedure. In other embodiments, the method 200 may be performed in other contexts or situations.
  • a user of the first device 12 may use the first device 12 to obtain an input image 20 of an ear of the subject (Item 202).
  • the first device 12 may be a handheld communication device, such as a phone (e.g., a smart phone, an iPhone), a blackberry, or iPad, etc.
  • the user may use the first device 12 to take a picture of the subject's ear.
  • the picture is then stored in a non-transitory medium in the first device 12 as the input image 20.
  • the user may use a camera to take a picture of the subject's ear.
  • the camera may be part of a computer, or may be communicatively coupled to a computer.
  • the user may use a separate handheld camera to take a picture of the subject's ear, and the picture file from a media storage device inside the camera may then be transferred (e.g., uploaded) to the first device 12, such as a computer, a handheld communication device, or any of other devices that has communication capability.
  • the act of obtaining the input image 20 may be considered as being performed by the first device 12 (e.g., the processor 16), which obtains the input image 20 by generating it, or by receiving it.
  • FIG. 3 illustrates an example of an input image 20 for an ear of a subject.
  • the input image 20 of the ear 270 may include the lobule 272, antitragus 274, cavum concha 276, cymba concha 278, scaphoid fossa 280, helix 282, antihelix 284, triangular fossa 286, crus helias 288, tragus 290, and the intertragal incisures 292.
  • the input image 20 may not include all of the above features, and may include only a subset of the above features instead.
  • the first device 12 transmits the input image 20 to the second device 14 for comparing the input image 20 with a plurality of reference images of different respective ears (Item 204).
  • reference images or similar terms (such as “plurality of reference images”) may refer to a complete set of all available reference images, or alternatively, may refer to a subset of all available reference images.
  • the act of transmitting may involve the processor 16 generating a control signal to cause the first device 12 to wirelessly transmit the input image 20 to the second device 14.
  • the act of transmitting may involve the processor 16 generating a control signal to cause the first device 12 to transmit the input image 20 to the second device 14 using a wire or a cable.
  • the input image 20 may be transmitted from the first device 12 to the second device 14 through the Internet.
  • the second device 14 performs a method 250.
  • the second device 14 obtains the input image 20 (Item 252). Such may be accomplished by the second device 14 (e.g., the processor 18 of the second device 14) receiving the input image 20 from the first device 12 in some embodiments.
  • the second device 14 obtains reference images of different respective ears (Item 254).
  • such may be accomplished by the processor 18 of the second device 14 accessing a non-transitory medium that stores the reference images.
  • the act of obtaining reference images may be accomplished by the second device 14 (e.g., the processor 18 of the second device 14) receiving the reference images from another device (such as, from a computer, an external hard drive, a database, etc.).
  • the reference images are stored in a non-transitory medium that is associated with the second device 14.
  • the non-transitory medium may be inside the second device 14, or may be a storage device that is communicatively coupled to the second device 14. In either case, the non-transitory medium may be considered to be a part of the second device 14.
  • FIG. 4 illustrates examples of reference images 300a-300i that may be stored at the second device 14.
  • information that is "stored at" a device may refer to information that is physically located at the device, or may refer to information that may be accessed by the device.
  • the term "store at” should not be limited to information that is physically stored at a device.
  • each of the reference images 300 includes a picture of an ear.
  • the features of the ears are different in different respective reference images 300.
  • the lobule, antitragus, cavum concha, cymba concha, scaphoid fossa, helix, antihelix, triangular fossa, crus helias, tragus, or any combination of the foregoing may have size and/or shape that is different between two or more of the reference images 300.
  • the reference images 300 stored at the second device 14 are collected over time from different fitting procedures of different subjects. Thus, as time progresses, the number of reference images 300 at the second device 14 may increase.
  • the second device 14 may be accessed by different users (e.g., different operators at different fitting centres, hospitals, doctors' offices, etc.), who will upload the reference images 300, along with their corresponding HRTF information (e.g., filter coefficients, etc.), to the second device 14. For example, after a user has performed a session to determine HRTF (e.g., using traditional techniques) for a particular subject, the user may take a picture of the subject's ear(s), and transmit the picture along with the determined HRTF to the second device 14.
  • HRTF filter coefficients, etc.
  • one or more of the reference images 300 may be an image of a model ear.
  • each reference image 300 may include an image of a head and an image of at least a part of a torso.
  • each of the reference images 300 is associated with pre-determined HRTF.
  • a reference image 300 may have one or more parameters for a HRTF that is associated with the image 300.
  • the HRTF associated with the reference image 300 was previously obtained for the ear shown in the reference image 300.
  • the HRTF associated with the reference image 300 is "pre-determined".
  • the HRTF information may be stored at the second device 14 as part of the image file for the reference image 300. In other embodiments, the HRTF information may be stored at the second device 14 as separate files from the image files of the reference images 300.
  • the second device 14 may also store meta data (e.g., pointers) for associating the HRTF information with respective reference images 300.
  • the pre-determined HRTF for each reference image 300 may have multiple parameters that are stored in table form, with the file name of the reference image 300 being the index in the table.
  • the parameters of the HRTF may be retrieved from the table by referencing the index (e.g., the file name of the corresponding reference image 300) at the table.
  • Other techniques for associating the reference images 300 with their corresponding HRTF information may be used in other embodiments.
  • the processor 18 of the second device 14 compares the input image 20 with the reference images 300 (Item 256).
  • the input image and/or the reference images 300 may optionally be pre-processed.
  • the processor 16 of the first device 12 and/or the processor 18 of the second device 14 may be configured to perform image processing to filter out noise in the input image 20, scaling the input image 20 so that the image of the ear has a certain height and width, adjusting a grey-scale of the input image 20, or a combination of the foregoing.
  • the processor 18 of the second device 14 may be configured to perform image processing to filter out noise in the reference images 300, scaling the reference images 300 so that the respective images of the ears have a certain height and width, adjusting a grey-scale of the reference images 300, or a combination of the foregoing.
  • the pre-processing of the reference images 300 may be performed when the reference images 300 are being stored at the second device 14. In other embodiments, the pre-processing of the reference images 300 may be performed during a process to determine HRTF. In such cases, the pre-processing of the reference images 300 may be performed in response to the processor 18 receiving an input image 20 from the first device 12.
  • a degree of match may be determined between the input image 20 and each of the reference images 300.
  • the processor 18 of the second device 14 selects the reference image 300 that provides the highest degree of match as the matching image.
  • a cross-correlation value (which may be considered an example of a degree of match) may be computed using the input image 20 and each of the reference images 300.
  • the processor 18 of the second device 14 selects the reference image 300 that provides the highest cross-correlation value as the matching image.
  • a pattern matching technique may be used to determine how well the input image 20 matches a given reference image 300. Any of the pattern matching techniques known in the art may be used in different embodiments.
  • different image processing techniques may be employed to efficiently search for a best match between the input image 20 and one of the reference images 300.
  • the input image 20 may be processed to obtain an exterior profile (or outline) of the subject's ear.
  • the exterior profile of the input image is then compared with exterior profiles of different respective reference images 300 to find a subset of reference images 300 that match the exterior profile of the subject's ear.
  • the input image 20 is then compared with the subset of reference images 300 (now considering the features within the outline of the ear) to determine a reference image 300 that best matches the input image 20.
  • deformation registration may be used to determine a reference image 300 that best matches the input image 20.
  • Deformable image registration is a process or technique in which points in a first image of a first object are associated with corresponding points in a second image of a second object, wherein the first and second objects may have the same or different sizes and/or shapes.
  • a deformation registration between two images may be represented by a matrix of vectors that maps points at a first image to points at a second image.
  • the processor 18 of the second device 14 may calculate matrices representing deformation registration between the input image 20 and respective ones of the reference images 300.
  • Each of the matrices may then be used to compute a score that represents a degree of match between the input image 20 and a corresponding one of the reference images 300.
  • the score may be computed on the basis that a matrix with more vectors having non-zero values and/or vectors having higher non-zero values will have a relatively lower score. This is because more vectors mean that more points in one image need to be "moved" in order to reach the positions of the corresponding points in another image, and because a higher vector value means that the a point in one image needs to be moved further in order to reach the position of the corresponding point in another image.
  • variants of Principal Component Analysis may be applied to the reference images 300 and used for classifying the input image 20.
  • PCA Principal Component Analysis
  • the processor 18 of the second device 14 After a reference image 300 has been determined to be the best match with the input image 20, the processor 18 of the second device 14 then obtains the HRTF that is associated with the matching reference image 300 (Item 258).
  • the second device 14 is configured to obtain the HRTF by selecting (e.g., retrieving) the HRTF information that is associated with the best matching reference image 300.
  • the act of obtaining HRTF by the processor 18 of the second device 14 may be considered performed when the processor 18 of the second device 14 has determined (e.g., selected) a best matching reference image 300.
  • the act of obtaining HRTF by the processor 18 of the second device 14 may be considered performed when the processor 18 accesses a non-transitory medium to retrieve the stored corresponding HRTF.
  • the HRTF information that may be obtained by the processor 18 may include one or more sets of filter coefficients for one or more input sources in a hearing instrument for restoring spatial sound cues, or may include any information that is used to represent or derive the one or more sets of filter coefficients.
  • the HRTF information that may be obtained by the processor 18 may include filter configuration and coefficients for different devices to be fitted (depending on form factor, number of microphones, receiver type, etc).
  • the processor 18 when selecting the reference image 300 that best matches the input image 20, the processor 18 also considers other information, such as head size of the subject, head shape of the subject, torso size of the subject, torso shape of the subject, or any combination thereof. For example, in some embodiments, when the first device 12 transmits the input image 20 to the second device 14, the first device 12 also transmits head and torso information for the subject to the second device 14. In such cases, each of the reference images 300 stored at the second device 14 is also associated with corresponding head and torso information.
  • the second device 14 compares the input image 20 with the reference images 300
  • the second device 14 also compares the head and torso information for the subject with the corresponding head and torso information that are associated with the respective reference images 300.
  • the processor 18 of the second device 14 selects the pre-determined HRTF information that is associated a corresponding reference image 300 and corresponding head and torso information providing the best match with the input image 20 and the head and torso information for the subject.
  • the processor 18 of the second device 14 After the processor 18 of the second device 14 has obtained the HRTF, the processor 18 then transmits the HRTF information to the first device 12 (Item 260).
  • the processor 18 of the second device 14 may be configured to generate a signal to cause the second device 14 to transmit the HRTF information through a wireless network (e.g., through the Internet).
  • the processor 18 of the second device 14 may be configured to generate a signal to cause the second device 14 to transmit the HRTF information through a wire or a cable.
  • the first device 12 receives, from the second device 14, data regarding HRTFs that is obtained based on a result of the act of comparing (Item 206).
  • the received data regarding the HRTF may be stored in a non-transitory medium that is associated with the first device 12. Additionally, or alternatively, the received data regarding the HRTF may be displayed on a screen (e.g., a screen at the first device 12) for presentation to the user of the first device 12.
  • the first device 12 may transmit the HRTF information for incorporation into a hearing aid device that is for the subject. For example, if the HRTF information includes filter coefficients for a hearing aid, the HRTF information may be incorporated into such hearing aid (e.g., into a processor of the hearing aid).
  • the selected HRTF for the subject may have an associated reference image 300 that does not provide the best match with the input image 20.
  • the HRTF for the first "matching" reference image 300 may be selected by the second device 14, and the HRTF for the second "matching" reference image 300 may be selected by the second device 14, and the second device 14 then determine HRTF based on an interpolation between the two selected pre-determined HRTFs.
  • the input image 20 may also include an image of the subject's head.
  • the reference images 300 may also include corresponding images of different subjects' heads.
  • the processor 16 of the first device 12, and/or the processor 18 of the second device 14 may process the input image 20 to determine information that is relevant for HRTF determination.
  • the processor 16 and/or the processor 18 may perform image processing to determine head size (e.g., width), head shape, etc.
  • the processor 18 may then select pre-determined HRTF (e.g., filter coefficients) that is stored at the second device 14 based at least in part on the determined head size and/or head shape.
  • pre-determined HRTF e.g., filter coefficients
  • the HRTF that is selected may be the HRTF that is associated with a reference image 300 having head size and/or head shape that best match the head size and/or head shape in the input image 20, and having ear feature(s) that best match the ear feature(s) in the input image 20.
  • the input image 20 may also include an image of the subject's head, as well as at least a part of the subject's torso (e.g., the shoulders of the subject).
  • the reference images 300 may also include corresponding images of different subjects' heads and their torsos.
  • the processor 16 of the first device 12, and/or the processor 18 of the second device 14 may process the input image 20 to determine information that is relevant for HRTF determination.
  • the processor 16 and/or the processor 18 may perform image processing to determine head size (e.g., width), head shape, torso width, torso depth, etc.
  • the processor 18 may then select pre-determined HRTF (e.g., filter coefficients) that is stored at the second device 14 based at least in part on the determined head information and torso information.
  • the HRTF that is selected may be the HRTF that is associated with a reference image 300 having head size and/or head shape that best match the head size and/or head shape in the input image 20, having torso size and/or torso shape that best match the torso size and/or torso shape in the input image 20, and having ear feature(s) that best match the ear feature(s) in the input image 20.
  • pre-determined HRTF information is selected by the processor 18 of the second device 14 based on a comparison between the input image 20 and the reference images 300.
  • pre-determined HRTF information may be selected by the processor 18 of the second device 14 based on a comparison between two input images 20 and two respective sets of reference images 300.
  • the device 12 may be used to obtain two input images 20, with the first input image 20 being a front view of a subject that includes the head, ears, and at least part of the torso, and the second input image 20 being a side view of the subject that includes the head, the ear, and at least part of the torso.
  • two sets of reference images 300 are stored at the second device 14, with the first set of reference images 300 being front images of different persons, and the second set of reference images 300 being side images of the different persons.
  • Each front reference image 300 may include image of a head, images of the ears, and image of at least part of the torso, of a person.
  • Each side reference image 300 may include image of the head, image of the ear, and image of at least part of the torso, of the same corresponding person.
  • HRTF information stored at the second device 14.
  • the processor 18 of the second device 14 selects HRTF information based on a first comparison of the front input image 20 with front reference images 300, and a second comparison of the side input image 20 with side reference images 300. In one implementation, the processor 18 selects the pre-determined HRTF information that is associated with the pair of front reference image 300 and side reference image 300 that best matches the pair of front input image 20 and side input image 20.
  • the processor 18 of the second device 14 may be configured to compare information extracted the input image 20 with information extracted from the reference images 300, and select the pre-determined HRTF information based at least on a result of such comparison.
  • the processor 16 of the first device 12 and/or the processor 18 of the second device 14 may be configured to perform image processing to extract certain information from the two input images 20 (front input image 20 and side input image 20).
  • information that may be extracted from the input images 20 may include head size (width and/or depth), head shape, torso size (width and/or depth), torso shape, cavum concha height, cymba concha height, cavum concha width, fossa height, pinna height, pinna width, intertragal incisure width, cavum concha depth, or any combination of the foregoing.
  • the processor 18 of the second device 12 may be configured to perform image processing to extract certain information from the reference images 300 (front reference images 300 and side reference images 300).
  • information that may be extracted from the reference images 300 may include head size (width and/or depth), head shape, torso size (width and/or depth), torso shape, cavum concha height, cymba concha height, cavum concha width, fossa height, pinna height, pinna width, intertragal incisure width, cavum concha depth, or any combination of the foregoing.
  • the processor 18 of the second device 14 selects HRTF information based on a first comparison of the front input image 20 with front reference images 300, and a second comparison of the side input image 20 with side reference images 300.
  • the processor 18 selects the pre-determined HRTF information that is associated with the reference images 300 (pair of front and side reference images 300) providing information that best matches the information extracted from the pair of front input image 20 and side input image 20.
  • information extracted from the input image 20 may be considered as an example of information regarding the input image 20.
  • information regarding the input image 20 may be the input image 20 itself, in which case, the information may comprise image data for the input image 20.
  • information extracted from the reference image 300 may be considered as an example of information regarding the reference image 300.
  • information regarding the reference image 300 may be the reference image 300 itself, in which case, the information may comprise image data for the reference image 300.
  • the act of comparing input image 20 with reference image(s) 300 may be performed based on classification technique.
  • pre-determined HRTF information may be classified based on the information extracted from the reference images 300, wherein such information may include head size (width and/or depth), head shape, torso size (width and/or depth), torso shape, cavum concha height, cymba concha height, cavum concha width, fossa height, pinna height, pinna width, intertragal incisure width, cavum concha depth, or any combination of the foregoing.
  • the HRTF information stored at the second device 14 may be classified based on information extracted from the pair of corresponding reference images 300.
  • the processor 18 of the second device 14 may select one of the classified HRTF information based on a comparison of the information extracted front the input images 20 (front and side input images 20 for the subject), and information extracted from the reference images 300.
  • the processor 18 selects one of the classified HRTF information that is associated with the reference images 300 (pair of front and side reference images 300) providing information that best matches the information extracted from the pair of front input image 20 and side input image 20.
  • the HRTF for the first "matching" reference image information may be selected by the second device 14, and the HRTF for the second "matching" reference image information may be selected by the second device 14, and the second device 14 then determine HRTF based on an interpolation between the two selected pre-determined HRTFs.
  • the processor 18 of the second device 14 may consider one or more parameters related to an ear canal when selecting HRTF information for the subject. For example, in some embodiments, when the pre-determined HRTF information is being stored at the second device 14, the second device 14 may also store feedback calibration measurements (if available) at the non-transitory medium, and the stored feedback calibration measurements may be linked with the corresponding HRTF information and/or the reference images 300. In one implementation, when different subjects are fitted with a hearing instrument, the feedback path from the receiver in the ear to the microphones on the instrument may be measured, and the measurement may provide a transfer function which may be used to increase the maximum stable gain of the hearing instrument by adaptive filtering.
  • This transfer function may be correlated to the HRTF information in the sense that it embeds information on how sound changes when travelling from the microphone positioned outside the ear canal to the receiver inside the canal.
  • the processor 18 of the second device 14 selects HRTF information based on a comparison between information regarding the input image 20 (or input images 20 if there are front and side input images 20) and information regarding the reference images 300, and a comparison between feedback measurement for the subject and feedback measurements for the different subjects stored at the second device.
  • the processor 18 selects the pre-determined HRTF information that is associated a best matching information from the reference image 300 (or pair of reference images 300 if there are front and side reference images 300), and a best matching corresponding feedback measurement.
  • the selection of the pre-determined HRTF by the second device 14 may be performed without using any images.
  • feedback measurements for different subjects may be stored at the second device 14, and the corresponding HRTF information may also be stored at the second device 14 in a way that are associated with the corresponding feedback measurements.
  • the processor 18 of the second device 14 obtains feedback measurement for the subject (e.g., by receiving such information from the first device 12), and selects HRTF information based on a comparison between the feedback measurement for the subject and feedback measurements for the different subjects stored at the second device 14.
  • the processor 18 selects the pre-determined HRTF information that is associated with one of the pre-determined feedback measurements that best matches with the feedback measurement of the subject.
  • the system 10 has been described with reference to involving the first device 12 and the second device 14.
  • the system 10 may include only one device.
  • the second device 14 may include a camera for capturing the input image 20 of an ear of the subject.
  • the first device 12 and the method 200 may not be required for obtaining HRTF.
  • the act 252 of obtaining the input image 20 is performed by the second device 14 generating an input image 20 of the ear of the subject using a camera.
  • the second device 14 may obtain the input image 20 by receiving it through a port.
  • a user may take a picture of the subject's ear using a digital camera, and the picture is then uploaded to the second device 14 (e.g., through the USB port, or through a media file input port).
  • the second device 14 may not be required to perform the act 260 of transmitting the HRTF information.
  • the obtained HRTF may be displayed at a screen (e.g., a screen at the second device 14, or a screen that is coupled to the second device 14).
  • the methods 200, 250 have been described with reference to comparing a two dimensional input image 20 with two dimensional reference images 300.
  • the input image 20 may be a three dimensional image
  • each of the reference images 300 may be a three dimensional reference image.
  • the first device 12 may be used to obtain two or more two dimensional images of a subject's ear from different respective directions. The two dimensional images may then be processed to reconstruct a three dimensional input image 20 of the subject's ear.
  • the processor 16 of the first device 12 may be configured to perform the image processing to reconstruct the three dimensional input image 20.
  • the processor 16 of the first device 12 may be configured to generate signals to cause the first device 12 to transmit the images to the second device 14.
  • the second device 14 then perform the image processing on the images to reconstruct the three dimensional input image 20.
  • the same image processing technique may be employed to generate each of the three dimensional reference images 300 that are stored at the second device 14 (e.g., at a non-transitory medium that is in the second device 14, or that is coupled to the second device 14).
  • the second device 14 compares the three dimensional input image 20 with the three dimensional reference images 300.
  • the second device 14 determines HRTF based on a result of the act of comparing the three dimensional input image 20 with the three dimensional reference images 300, and transmits the HRTF information to the first device 12.
  • FIG. 5 is a block diagram that illustrates an embodiment of a computer system 1200 upon which one or more embodiments of the invention may be implemented.
  • Computer system 1200 includes a bus 1202 or other communication mechanism for communicating information, and a processor 1204 coupled with the bus 1202 for processing information.
  • the computer system 1200 may be used to implement the first device 12.
  • the computer system 1200 may be used to implement the second device 14.
  • the computer system 1200 also includes a main memory 1206, such as a random access memory (RAM) or other dynamic storage device, coupled to the bus 1202 for storing information and instructions to be executed by the processor 1204.
  • the main memory 1206 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by the processor 1204.
  • the computer system 1200 further includes a read only memory (ROM) 1208 or other static storage device coupled to the bus 1202 for storing static information and instructions for the processor 1204.
  • ROM read only memory
  • a data storage device 1210 such as a magnetic disk or optical disk, is provided and coupled to the bus 1202 for storing information and instructions.
  • the computer system 1200 may be coupled via the bus 1202 to a display 1212, such as a cathode ray tube (CRT), or a flat panel, for displaying information to a user.
  • a display 1212 such as a cathode ray tube (CRT), or a flat panel
  • An input device 1214 is coupled to the bus 1202 for communicating information and command selections to processor 1204.
  • cursor control 1216 is Another type of user input device, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 1204 and for controlling cursor movement on display 1212.
  • This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane.
  • the computer system 1200 may be used for performing various functions (e.g., calculation) in accordance with the embodiments described herein. According to one embodiment, such use is provided by computer system 1200 in response to processor 1204 executing one or more sequences of one or more instructions contained in the main memory 1206. Such instructions may be read into the main memory 1206 from another computer-readable medium, such as storage device 1210. Execution of the sequences of instructions contained in the main memory 1206 causes the processor 1204 to perform the processing act(s) described herein. One or more processors in a multi-processing arrangement may also be employed to execute the sequences of instructions contained in the main memory 1206. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention. Thus, embodiments described herein are not limited to any specific combination of hardware circuitry and software.
  • Non-volatile media includes, for example, optical or magnetic disks, such as the storage device 1210.
  • a non-volatile medium is an example of a non-transitory medium.
  • Volatile media includes dynamic memory, such as the main memory 1206.
  • a volatile medium is another example of a non-transitory medium.
  • Transmission media includes coaxial cables, copper wire and fibre optics, including the wires that comprise the bus 1202. Transmission media can also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.
  • Computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read.
  • Various forms of computer-readable media may be involved in carrying one or more sequences of one or more instructions to the processor 1204 for execution.
  • the instructions may initially be carried on a magnetic disk of a remote computer.
  • the remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem.
  • a modem local to the computer system 1200 can receive the data on the telephone line and use an infrared transmitter to convert the data to an infrared signal.
  • An infrared detector coupled to the bus 1202 can receive the data carried in the infrared signal and place the data on the bus 1202.
  • the bus 1202 carries the data to the main memory 1206, from which the processor 1204 retrieves and executes the instructions.
  • the instructions received by the main memory 1206 may optionally be stored on the storage device 1210 either before or after execution by the processor 1204.
  • the computer system 1200 also includes a communication interface 1218 coupled to the bus 1202.
  • the communication interface 1218 provides a two-way data communication coupling to a network link 1220 that is connected to a local network 1222.
  • the communication interface 1218 may be an integrated services digital network (ISDN) card or a modem to provide a data communication connection to a corresponding type of telephone line.
  • ISDN integrated services digital network
  • the communication interface 1218 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN.
  • LAN local area network
  • Wireless links may also be implemented.
  • the communication interface 1218 sends and receives electrical, electromagnetic or optical signals that carry data streams representing various types of information.
  • the network link 1220 typically provides data communication through one or more networks to other devices.
  • the network link 1220 may provide a connection through local network 1222 to a host computer 1224 or to equipment 1226, which may be any device.
  • the data streams transported over the network link 1220 can comprise electrical, electromagnetic or optical signals.
  • the signals through the various networks and the signals on the network link 1220 and through the communication interface 1218, which carry data to and from the computer system 1200, are exemplary forms of carrier waves transporting the information.
  • the computer system 1200 can send messages and receive data, including program code, through the network(s), the network link 1220, and the communication interface 1218.

Landscapes

  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Neurosurgery (AREA)
  • Otolaryngology (AREA)
  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)

Claims (12)

  1. Verfahren (200) zur Bestimmung von kopfbezogenen Übertragungsfunktionen, HRTF, umfassend:
    Erhalten (252) von Information über ein Eingangsbild (20), umfassend ein Bild eines Ohrs eines Individuums;
    dadurch gekennzeichnet, dass das Verfahren Folgendes umfasst
    Erhalten (254) einer Mehrheit von Referenzbildern (300) von verschiedenen jeweiligen Ohren, wobei ein oder mehrere der Referenzbilder mit einer entsprechenden vorgegebenen HRTF-Information verbunden ist;
    Vergleichen (256) der Information über das Eingangsbild mit Information über die Referenzbilder unter Anwendung eines Prozessors; und
    Auswählen einer der vorgegebenen HRTF-Informationen zumindest teilweise basierend auf einem Ergebnis des Vergleichsvorgangs,
    wobei der Vergleichsvorgang (256) das Vergleichen des Eingangsbilds mit Referenzbildern umfasst, um verschiedene jeweilige Ergebnisse zu erhalten, die verschiedene jeweilige Graden von Übereinstimmung darstellen, und
    der Vorgang des Auswählens der einen der vorgegebenen HRTF-Informationen das Auswählen von derjenigen HRTF-Information umfasst, die dem Referenzbild entspricht, welches das den höchsten Grad an Übereinstimmung darstellende Ergebnis bereitstellt.
  2. Verfahren nach Anspruch 1, wobei mindestens eines der Referenzbilder ein Bild eines Ohrs, ein Bild eines Kopfes und ein Bild von zumindest einem Teil eines Torsos umfasst.
  3. Verfahren nach Anspruch 1 oder 2, wobei die Referenzbilder verschiedene Paare von Vorder- und Seitenreferenzbildern umfassen, wobei jedes Paar mit einer entsprechenden vorgegebenen HRTF-Information verbunden ist.
  4. Verfahren nach einem der vorgehenden Ansprüche, wobei die Information über das Eingangsbild Bilddaten für das Eingangsbild umfasst.
  5. Verfahren nach einem der vorgehenden Ansprüche, wobei die Information über das Eingangsbild vom Eingangsbild entnommene Information umfasst.
  6. Verfahren nach Anspruch 5, wobei die vom Eingangsbild entnommene Information Kopfgröße, Kopfform, Torsogröße, Torsoform, Cavum Concha Höhe, Cymba Concha Höhe, Cavum Concha Breite, Fossa Höhe, Pinna Höhe, Pinna Breite, Intertragal Incisure Breite, Cavum Concha Tiefe oder irgendeine Kombination davon umfasst.
  7. Verfahren nach einem der vorgehenden Ansprüche, wobei die HRTF-Information einen oder mehrere Filterkoeffizienten für ein Hörgerät umfasst.
  8. Verfahren nach einem der vorgehenden Ansprüche, wobei der Vorgang des Erhaltens der Mehrheit von Referenzbildern durch Zugreifen auf eine die Mehrheit von Referenzbildern speichernde Datenbank durchgeführt wird.
  9. Verfahren nach einem der vorgehenden Ansprüche, wobei der Vorgang des Erhaltens der Information über das Eingangsbild durch Empfang der Information über das Eingangsbild über das Internet durchgeführt wird.
  10. Verfahren nach einem der vorgehenden Ansprüche, weiter umfassend Senden von Daten über die ausgewählte vorgegebene HRTF-Information zu einer Einrichtung, die mit dem Prozessor verbunden ist.
  11. Verfahren nach einem der vorgehenden Ansprüche, weiter umfassend Anwendung der ausgewählten vorgegebenen HRTF-Information, um das Hörgerät zu konfigurieren.
  12. Einrichtung zur Bestimmung der HRTF, umfassend einen Prozessor, der zur Ausführung eines Verfahrens nach einem der Ansprüche 1-11 konfiguriert ist.
EP12152430.0A 2011-12-30 2012-01-25 Systeme und Verfahren zur Bestimmung von kopfbezogenen Übertragungsfunktionen Active EP2611216B1 (de)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2012281419A JP5543571B2 (ja) 2011-12-30 2012-12-25 頭部伝達関数を決定するシステムおよび方法
CN201210592852.5A CN103455824B (zh) 2011-12-30 2012-12-31 用于确定头相关传递函数的***和方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DKPA201101008 2011-12-30
US13/342,792 US9030545B2 (en) 2011-12-30 2012-01-03 Systems and methods for determining head related transfer functions

Publications (2)

Publication Number Publication Date
EP2611216A1 EP2611216A1 (de) 2013-07-03
EP2611216B1 true EP2611216B1 (de) 2015-12-16

Family

ID=45540789

Family Applications (1)

Application Number Title Priority Date Filing Date
EP12152430.0A Active EP2611216B1 (de) 2011-12-30 2012-01-25 Systeme und Verfahren zur Bestimmung von kopfbezogenen Übertragungsfunktionen

Country Status (1)

Country Link
EP (1) EP2611216B1 (de)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9426589B2 (en) 2013-07-04 2016-08-23 Gn Resound A/S Determination of individual HRTFs
US9900722B2 (en) 2014-04-29 2018-02-20 Microsoft Technology Licensing, Llc HRTF personalization based on anthropometric features
US9609436B2 (en) 2015-05-22 2017-03-28 Microsoft Technology Licensing, Llc Systems and methods for audio creation and delivery
US10028070B1 (en) 2017-03-06 2018-07-17 Microsoft Technology Licensing, Llc Systems and methods for HRTF personalization
US10278002B2 (en) 2017-03-20 2019-04-30 Microsoft Technology Licensing, Llc Systems and methods for non-parametric processing of head geometry for HRTF personalization
US11205443B2 (en) 2018-07-27 2021-12-21 Microsoft Technology Licensing, Llc Systems, methods, and computer-readable media for improved audio feature discovery using a neural network
JPWO2020075622A1 (ja) 2018-10-10 2021-09-16 ソニーグループ株式会社 情報処理装置、情報処理方法及び情報処理プログラム
GB2600932A (en) * 2020-11-11 2022-05-18 Sony Interactive Entertainment Inc Audio personalisation method and system

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6996244B1 (en) * 1998-08-06 2006-02-07 Vulcan Patents Llc Estimation of head-related transfer functions for spatial sound representative
DE10318191A1 (de) * 2003-04-22 2004-07-29 Siemens Audiologische Technik Gmbh Verfahren zur Erzeugung und Verwendung einer Übertragungsfunktion
JPWO2005025270A1 (ja) * 2003-09-08 2006-11-16 松下電器産業株式会社 音像制御装置の設計ツールおよび音像制御装置
US8787501B2 (en) * 2009-01-14 2014-07-22 Qualcomm Incorporated Distributed sensing of signals linked by sparse filtering

Also Published As

Publication number Publication date
EP2611216A1 (de) 2013-07-03

Similar Documents

Publication Publication Date Title
DK2611216T3 (en) Systems and methods for determining the head-related transfer functions
EP2611216B1 (de) Systeme und Verfahren zur Bestimmung von kopfbezogenen Übertragungsfunktionen
US10939225B2 (en) Calibrating listening devices
US20210211829A1 (en) Calibrating listening devices
US11601775B2 (en) Method for generating a customized/personalized head related transfer function
US11082791B2 (en) Head-related impulse responses for area sound sources located in the near field
CN112425187B (zh) 用于空间音频渲染的装置、方法和存储介质
US9232310B2 (en) Methods, apparatuses and computer program products for facilitating directional audio capture with multiple microphones
KR20060059866A (ko) 음상 제어 장치의 설계 툴 및 음상 제어 장치
CN114998888A (zh) 用于产生头相关传递函数滤波器的布置
US11943587B2 (en) Generating a hearing assistance device shell
CN104469092B (zh) 一种图像采集方法及电子设备
EP4327569A1 (de) Fehlerkorrektur von kopfbezogenen filtern
US11937069B2 (en) Audio system, audio reproduction apparatus, server apparatus, audio reproduction method, and audio reproduction program
Pausch et al. Hybrid multi-harmonic model for the prediction of interaural time differences in individual behind-the-ear hearing-aid-related transfer functions
WO2021129444A1 (zh) 文件聚类方法及装置、存储介质和电子设备
CN112330793A (zh) 耳模三维模型的获取方法、耳机定制方法和计算设备
WO2019174442A1 (zh) 拾音设备、声音输出方法、装置、存储介质及电子装置
CN111246363A (zh) 一种基于听觉匹配的虚拟声定制方法及装置
WO2020075427A1 (ja) 情報処理システム、情報処理方法およびコンピュータプログラム
JP2020198545A (ja) 情報処理装置、情報処理方法及びプログラム
US9794685B2 (en) Video audio recording system, video audio recording device, and video audio recording method
JP2024508899A (ja) オーディオ信号に方向性を適用する電子装置及びその方法
CN111797675A (zh) 一种人耳特征点的提取方法、装置和电子设备

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

17P Request for examination filed

Effective date: 20140103

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

17Q First examination report despatched

Effective date: 20140404

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20150611

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 766058

Country of ref document: AT

Kind code of ref document: T

Effective date: 20160115

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602012013038

Country of ref document: DE

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 5

REG Reference to a national code

Ref country code: DK

Ref legal event code: T3

Effective date: 20160301

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20151216

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160316

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 766058

Country of ref document: AT

Kind code of ref document: T

Effective date: 20151216

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160317

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160131

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160416

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160418

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

Ref country code: LU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160125

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602012013038

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

26N No opposition filed

Effective date: 20160919

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 6

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160125

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20120125

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151216

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230525

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240123

Year of fee payment: 13

Ref country code: GB

Payment date: 20240122

Year of fee payment: 13

Ref country code: CH

Payment date: 20240202

Year of fee payment: 13

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20240117

Year of fee payment: 13

Ref country code: DK

Payment date: 20240117

Year of fee payment: 13