EP3809725A2

EP3809725A2 - Hearing aid system configured to evaluate cognitive load

Info

Publication number: EP3809725A2
Application number: EP21156614.6A
Authority: EP
Inventors: Michael Syskind Pedersen; Jesper Jensen; Adis Bjelosevic; Gary Jones; Claus Nielsen; Niels Henrik Pontoppidan; Louise LYSTRUP; Niels Henrik Meedom; Tanveer BHUIYAN
Original assignee: Oticon AS
Current assignee: Oticon AS
Priority date: 2021-02-11
Filing date: 2021-02-11
Publication date: 2021-04-21

Abstract

The present disclosure relates to a hearing aid system (1) comprising a hearing aid (2) configured to compensate for a hearing loss of a user wearing the hearing aid (2) and a portable auxiliary device (3) configured to be in communicatively contact (4) with the hearing aid (2) and to receive and transmit signals from and to the hearing aid (2) to evaluate a cognitive load experienced by the user during the day.

Description

The present disclosure relates to a hearing aid system comprising a hearing aid and an auxiliary device being in communicatively contact, wherein the hearing aid together with the auxiliary device is configured to evaluate a listening environment that a hearing aid user is exposed to and to evaluate a cognitive load of the hearing aid user in that environment. Further, the disclosure relates to adjustments provided either automatically and/or manually to the hearing aid based on the evaluated cognitive load experienced by a hearing aid user in different sound environments during a day.
It has long been a goal of hearing aid design to work towards improving the user's ability to understand speech under difficult listening conditions. Further is has become increasingly important to reduce and/or assist the user's in understanding the mental effort when in these situations. By improving the user's ability to perform in all listening situations it is possible to preserve cognitive resources, which in turn increases the ability to perform throughout the whole day. However, the reduced energy at the end of the day makes any listening environment troublesome for the hearing impaired, why it is important to make the most of each user's unique abilities and preferences and giving hearing care professionals the support and tools needed to match different user needs. Compared to normal-hearing listeners, hearing-impaired listeners have an additional cognitive load imposed by their hearing loss. In order to assist hearing-impaired listeners in controlling their performance, it can be very useful to give them insight into the actual load they are under, and have been under, so far during the day. By combining this feedback to the user with information about his/her individual abilities, characteristics and preferences, can help him/her to optimize performance during the whole day.
Another reason to estimate the cognitive load of hearing aid users is to avoid stressed situations, which could be caused by an increased cognitive load but also cause other physical reactions experienced by a hearing aid user. Generally, a too high stress load leading to a break down has been appointed one of today's most prominent health issues and could at least be increased by an increased cognitive load. One of the challenges is that the person who is experiencing an increased stress load most often becomes 'speed blind' and do not react before it is too late to avoid a break down. Body responses to stress can have many origins such as memory loss, stomach ache, bad sleep as well as the already mentioned increased cognitive load.
Today several ways of measuring a users cognitive load, listening effort and mental states have been suggested, including among other applications, the use of sensors mounted in connection with a hearing aid. That is, it has been suggested to measure brain activity more directly, e.g. with Electroencephalography (EEG), by tracking eye movements via a small cameras to assist the user in understanding the mental load that the user is experiencing during a day and to user other sensors. In general, information about the specific user may be collected by the hearing aids (sound levels, SNRs, sensor information), a smartphone (time of day, activity), the user him-/herself or the hearing care professional. Several possibilities exist, and the following disclosure aims at suggesting previously non-disclosed solutions for estimating the cognitive load, process the data flow and monitor the environment and mental state that the hearing aid user is exposed to during the day to allow change in hearing aid settings among other applications.
Thus, the following disclosure presents in connection with the provided drawings, several options for measuring and evaluating a cognitive load exposed to a user during a day when wearing a hearing aid. The aspects of the disclosure may be best understood from the detailed description taken in conjunction with the accompanying figures. The figures are schematic and simplified for clarity, and they just show details to improve the understanding of the claims, while other details are left out. Throughout, the same reference numerals are used for identical or corresponding parts. The individual features of each aspect or embodiment may each be combined with any or all features of the other aspects. These and other aspects, features and/or technical effect will be apparent from and elucidated with reference to the illustrations described hereinafter in which:

Figure 1 illustrates a hearing aid system with a hearing aid and an auxiliary device, according to the disclosure;
Figure 2 illustrates a processing configuration according to the hearing aid system of the disclosure;
Figure 3 illustrates a processing configuration according to the hearing aid system of the disclosure;
Figure 4 illustrates a processing configuration according to the hearing aid system of the disclosure; and
Figure 5 illustrates a processing configuration using a portable sensor device in connection with the hearing aid system according to the disclosure.

The description set forth below in connection with the appended drawings is intended as a description of various configurations. The description includes specific details for the purpose of providing a thorough understanding of various concepts. However, it will be apparent to those skilled in the art that these concepts may be practiced without these specific details. Several aspects of the apparatus and methods are described by various blocks, functional units, modules, components, circuits, steps, processes, algorithms, etc. (collectively referred to as "elements"). Depending upon particular application, design constraints or other reasons, these elements may be implemented using electronic hardware, computer program, or any combination thereof.
The electronic hardware may include micro-electronic-mechanical systems (MEMS), integrated circuits (e.g. application specific), microprocessors, microcontrollers, digital signal processors (DSPs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), gated logic, discrete hardware circuits, printed circuit boards (PCB) (e.g. flexible PCBs), and other suitable hardware configured to perform the various functionality described throughout this disclosure, e.g. sensors, e.g. for sensing and/or registering physical properties of the environment, the device, the user, etc. Computer program shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.
A hearing device (or hearing instrument, hearing assistance device) may be or include a hearing aid that is adapted to improve or augment the hearing capability of a user by receiving an acoustic signal from a user's surroundings, generating a corresponding audio signal, possibly modifying the audio signal and providing the possibly modified audio signal as an audible signal to at least one of the user's ears. 'Improving or augmenting the hearing capability of a user' may include compensating for an individual user's specific hearing loss.
The "hearing device" may further refer to a device such as a hearable, an earphone or a headset adapted to receive an audio signal electronically, possibly modifying the audio signal and providing the possibly modified audio signals as an audible signal to at least one of the user's ears. Such audible signals may be provided in the form of an acoustic signal radiated into the user's outer ear, or an acoustic signal transferred as mechanical vibrations to the user's inner ears through bone structure of the user's head and/or through parts of the middle ear of the user or electric signals transferred directly or indirectly to the cochlear nerve and/or to the auditory cortex of the user.
The hearing device is adapted to be worn in any known way. This may include i) arranging a unit of the hearing device behind the ear with a tube leading air-borne acoustic signals into the ear canal or with a receiver/ loudspeaker arranged close to or in the ear canal and connected by conductive wires (or wirelessly) to the unit behind the ear, such as in a Behind-the-Ear type hearing aid, and/ or ii) arranging the hearing device entirely or partly in the pinna and/ or in the ear canal of the user such as in an In-the-Ear type hearing aid or In-the-Canal/ Completely-in-Canal type hearing aid, or iii) arranging a unit of the hearing device attached to a fixture implanted into the skull bone such as in a Bone Anchored Hearing Aid or a Cochlear Implant, or iv) arranging a unit of the hearing device as an entirely or partly implanted unit such as in a Bone Anchored Hearing Aid or a Cochlear Implant. The hearing device may be implemented in one single unit (housing) or in a number of units individually connected to each other.
A "hearing system" refers to a system comprising one or two hearing devices, and a "binaural hearing system" refers to a system comprising two hearing devices where the devices are adapted to cooperatively provide audible signals to both of the user's ears. The hearing system or binaural hearing system may further include one or more auxiliary device(s) that communicates with at least one hearing device, the auxiliary device affecting the operation of the hearing devices and/or benefitting from the functioning of the hearing devices. A wired or wireless communication link between the at least one hearing device and the auxiliary device is established that allows for exchanging information (e.g. control and status signals, possibly audio signals) between the at least one hearing device and the auxiliary device. Such auxiliary devices may include at least one of a remote control, a remote microphone, an audio gateway device, a wireless communication device, e.g. a mobile phone (such as a smartphone) or a tablet or another device, e.g. comprising a graphical interface, a public-address system, a car audio system or a music player, or a combination thereof. The audio gateway may be adapted to receive a multitude of audio signals such as from an entertainment device like a TV or a music player, a telephone apparatus like a mobile telephone or a computer, e.g. a PC. The auxiliary device may further be adapted to (e.g. allow a user to) select and/or combine an appropriate one of the received audio signals (or combination of signals) for transmission to the at least one hearing device. The remote control is adapted to control functionality and/or operation of the at least one hearing device. The function of the remote control may be implemented in a smartphone or other (e.g. portable) electronic device, the smartphone / electronic device possibly running an application (APP) that controls functionality of the at least one hearing device.
In general, a hearing device includes i) an input unit such as a microphone for receiving an acoustic signal from a user's surroundings and providing a corresponding input audio signal, and/or ii) a receiving unit for electronically receiving an input audio signal. The hearing device further includes a signal processing unit for processing the input audio signal and an output unit for providing an audible signal to the user in dependence on the processed audio signal.
The input unit may include multiple input microphones, e.g. for providing direction-dependent audio signal processing. Such directional microphone system is adapted to (relatively) enhance a target acoustic source among a multitude of acoustic sources in the user's environment and/or to attenuate other sources (e.g. noise). In one aspect, the directional system is adapted to detect (such as adaptively detect) from which direction a particular part of the microphone signal originates. This may be achieved by using conventionally known methods. The signal processing unit may include an amplifier that is adapted to apply a frequency dependent gain to the input audio signal. The signal processing unit may further be adapted to provide other relevant functionality such as compression, noise reduction, etc. The output unit may include an output transducer such as a loudspeaker/ receiver for providing an air-borne acoustic signal
Referring now to the Figures different embodiments of the disclosure will be described in more detail.
According to an aspect of the disclosure, the monitoring of an environment which a hearing impaired is exposed to and affecting the mental state of the hearing impaired, can be achieved by a hearing aid configuration configured to continuously monitor the sound that the user is exposed to during the day. The monitored sound can be analyzed by means of different models to estimate the cognitive load. By taking the amount of time into account the amount of listening fatigue (i.e. mental load, mental state etc) can be approximated. This information can be "communicated back" to the user by means of a smartphone (i.e. an auxiliary device) and/or directly communicated to the user via the hearing aid. In this way the user can track his/her cognitive load exposure during the day and can avoid further exposure if demanding tasks are waiting further in the day. Reducing listening effort (i.e. by reducing the cognitive load presented to the user of the hearing aid) enables people to choose when and how active they use their energy throughout the day.
Thus, as illustrated in Figure 1, the cognitive load of a hearing aid user may be estimated by the use of a hearing aid system 1 comprises a hearing aid 2 and an auxiliary device 3, wherein the hearing aid 2 and the auxiliary device 3 is in communicatively contact, i.e. illustrated as a wireless communication 4 (WLC). As illustrated the hearing aid 2 comprises a battery 21, one or more microphones 22, a signal processor (SP), which in Figure 1 is illustrated as forming part of a behind the ear part of the hearing aid. Further, the behind the ear part is connected to an in the ear part via a connection member 23 connecting a speaker unit 24 with the signal processor (SP) and other components of the behind the ear part. Further a dome 25 is configured to be in connection with the speaker in the ear part and configured to be inserted into the ear canal of a user.
The one or more microphones 22 is configured to receive an input audio signal, where the signal processor (SP) internal to the hearing aid is configured to process the input audio signal according to a hearing loos profile of the user. The receiver 24 (also denoted a speaker unit) is configured to transmit the processed input audio signal to the ear of a user.
The system further comprises a portable auxiliary device 3 configured to be in communicatively contact 4 with the hearing aid and to receive and transmit signals from and to the hearing aid 2, respectively.
It should be noted that even though a RITE type hearing aid is illustrated in Figure 1, other types of hearing aids, such as BTE style, CIC, ITE, etc. styles hearing aids could be contemplated as forming part of the hearing aid system.
In an embodiment, one way of measuring a cognitive load during the day is for example by logging the amount of time spent in situations with poor signal-to-noise-ratios (SNRs), high noise levels, poor speech quality and lots of own voice activity, or a combination of these measures. Using this information in a cognitive model it is possible to estimate "how exhausting" the listening day has been so far. An example of this is to estimate the signal-to-noise ratio experienced by the hearing aid user, averaged across the working day of the hearing aid; the lower the estimated SNR, the higher the listening fatigue. When calculating the fatigue the characteristics of the experienced sound environments can be correlated with information about the individual user, such as lifestyle, age, hearing loss, "individualization profile" from fitting and other meaningful individualization characteristics.
Thus, in an embodiment, the auxiliary device and/or the hearing aid may be configured to comprise a cognitive model configured to estimate a ratio of exhaustion experienced by the user during the day. In Figure 1, this cognitive model or at least an output of the cognitive model is schematically illustrated as a "load estimator" 5, which indicates to the user during the day how much energy that the user has spent on listening during the day. That is, in an embodiment, the cognitive model is configured to output a load estimation visually illustrated at the auxiliary device, and indicating to the user during the day how much energy that the user has spent on listening during the day.
In an embodiment, the cognitive model may also be configured such that a user of the hearing aid can actively indicate when he/she is having difficulty hearing, or is getting tired. In this way the cognitive model can be updated and thereby optimized for the individual person.
Furthermore, in embodiments described herein, measuring brain activity more directly, e.g. with Electroencephalography (EEG), or by tracking eye movements via a small camera is considered possible solutions for measuring mental load. This can be done with electrodes connected to the hearing aids. In general information about the specific user can be either, collected by the hearing aids (sound levels, SNRs), the smartphone (time of day, activity), the user him-/herself or the hearing care profession.
Accordingly, as illustrated in Figure 2, the cognitive model 30 may be configured to take as input one or more of a signal to noise ratio (SNR), noise level (NL), speech quality (SQ) and other non-illustrated features. The cognitive model 30 then calculates one or more ratios of these parameters either as a stand alone or in combination to provide an output for each of the parameters or a combined estimation of the cognitive load. This output may be communicated to the user of the hearing aid directly via the hearing aid and or via the auxiliary device, as e.g. illustrated in Figure 1, as the load estimator 5 on the auxiliary device.
In an embodiment, the hearing aid may be configured to detect situations in which no relevant information is present, whereby the detection of "non-relevant information" thereby allows the user to take a listening break.
The detection of "non-relevant information" may in an embodiment result in the hearing aid automatically adjusting for example the gain or muting the hearing aid completely.
The hearing aid continuously analyses the environment that the hearing aid user is in, and when a relevant signal is detected once again, the hearing aid is unmuted and/or the gain is automatically adjusted back to normal. Accordingly, in an embodiment, the cognitive model is configured to output a control signal to the hearing aid, wherein the control signal is configured to automatically adjust the gain of the hearing aid.
In an embodiment, the hearing aid is configured to communicate with an auxiliary device, such as a smartphone, whereby the user may automatically choose between a listening or break mode whenever desired.
This is e.g. illustrated in more detail in Figure 3, where the cognitive model is configured with an information analyzer being configured to analyze the input audio signal in more detail. That is, the input audio is analyzed via the cognitive model to estimate the information content in the input signal. In this way, if the information content contains irrelevant audio, such as noise and/or non-relevant speech, that the user is not interested in listening to, the information analyzer may send an output control signal (GA) indicating a decrease in gain (illustrated as "lower gain" in case of non-relevant information, and a normal gain in case of relevant information to the hearing aid signal processor.
The auxiliary device may be configured to communicate with the hearing aid to be able to assist the user in changing modes of the hearing aid so as to provide the user with a decreased cognitive load. This may be done automatically by a detection of the environment in which the smartphone is situated in or manually adjustments provided by the user.
That is, in an embodiment, for example a light sensor of the smartphone may be configured to evaluate if the smartphone is arranged in a dark environmental condition or a light condition. Evaluated against the time of day, the smartphone may register if for example, during daytime, the smartphone is in a dark environment, the smartphone is likely in a pocket or similar and therefor not in use. Such automatic detection of the condition that the smart phone is in, may instruct the hearing aid to focus on the hearing aid signal processing rather on external inputs.
Further, in an embodiment, a smartphone may be used to set specific use setting of the hearing aid during the day. That is, the smartphone may in one embodiment be equipped with an application which allows the hearing aid user to program breaks communicated to the hearing aid during the day. As an example, the application may have a programmed break at 10 am each day, which break when the time arises automatically is communicated to the hearing aid. In such "breaks" the gain may for example be tuned down in the hearing aid or a sound may be played to instruct the user to take a break. Further installations in the time of day "break" situation may include to set the exact duration of a break etc.
Further, the smartphone GPS may in an embodiment be used to detect the environment a hearing aid used is currently in. If a user is present in environments where his/her attention to sounds is needed based on the GPS signals from the smartphone, the hearing aid settings are kept normal, whereas if the user is in non hearing relevant environments the hearing aid settings may automatically be adjusted.
In embodiments described herein, the hearing aid and/or the smartphone may be equipped with a learning processor configured to learn and recognize several situations, such as:

Recognize patterns regarding acoustics vs. listener responses;
Get better at suggesting breaks when the particular listener wants them;
Get input from the hearing aid user about how she's/he's doing at different times during the day.

A training of the different situations that the hearing aid user is in results in the hearing aid assisting the user in getting better at recognizing when the hearing aid user is:

Getting tired, but still OK;
Feeling tired and will need a break soon;
Needs a break now, and to potentially alarm the user about the different situations that the hearing aid or smartphone is learning about the users environment and mental state.

As already mentioned, the increased cognitive load may cause stressed situations for a user wearing the hearing aid. Thus, early warning that a hearing aid user are in a too high stress load would be advantageous and could help avoid a break down. If a stress load is detected in a hearing aid user this may indicate that the person is in a difficult to listen situation and then be utilized to change the signal processing of the hearing aid to support ease of listening. Stress monitoring could also be used to give feedback to the user and support a more optimal hearing aid use e.g. best program choice for the occasion.
Accordingly, in an embodiment the current cognitive load situation or general mental being of the hearing aid user is evaluated by measuring a physical body response to stress. That is, in an embodiment, the hearing aid may be equipped with one or more sensors, such as accelerometer, heart rate sensor, electrodermal activity sensor and/or EEG sensors. Each of these sensors may be used to measure the situation that the user is in.
For example, providing the hearing aid with an accelerometer may be used to measure a motion of the user, for example during sleep.
Another possibility is to provide the hearing aid with a heart rate sensor, whereby the hearing aid may be configured to measure a state of stress. Heart rate variability, HRV, which can be measured by a heart rate sensor show that a decreasing HRV in situations with high listening effort indicating a higher stress load in these situations.
Further an electrodermal activity sensor may be provided together with a hearing aid, wherein an increase in electrodermal activity, EA, show that EA is associated with increased listening effort thus indicating a higher stress load in difficult listening situations.
As previously mentioned, and as illustrated schematically in Figure 4, an application on a smartphone may be used to evaluate the listening situation, and thereby stress situation, that a user is in. That is the smartphone and/or the hearing aid may be equipped with a stress processor 50 (similar to the load estimator previously described), receiving inputs from one or more of the mentioned sensors 51, 52, 53. In this way it may be possible for the hearing aid or smartphone to combine measures of motion, heart rate variability, electrodermal activity, EEG and for example also signal to noise ratio in the stress calculation processor. This may provide a robust measure of the stress level that the user is experiencing. If all data are available sensor fusion can be used to factorize different kinds of stress. Depending on where the sensors are placed, for example on or in a hearing aid, these signals are sent to the wearers smart phone and can be paired with subjective assessment of listening effort. After a 'learning period' (supported through user input - human and machine trained) the system will be able to warn the wearer of a too high stress load.
Accordingly, in an embodiment, the hearing aid is further configured to learn from sensor inputs when a stressed situation is present for the user of the hearing aid. That is, in an example, upon fitting, a learning period of the hearing aid will begin. When stress is detected by one or more sensors (EA, HRV, EEG) an app will be evoked on the user's smartphone. Sensor data can be combined with information on SNR data from the hearing aid. The user will then be asked to the current situation e.g. "how effort full is the listening situation?" and/or "How much do you think your hearing aid is supporting you right now?". Information on how the user interact with the hearing aid will be logged and used with sensor and subjective data. The learning period will last until stabile measures is reached. The training situation and the questions asked to the user is illustrated as a text on the smartphone 55 in Figure 4.
After the learning period, the system will be able to predict when the user is in a stressful situation and send a command 54 to the hearing aid processor so that the signal processing can be altered, as illustrated in Figure 4.
When the system has been trained during the learning period, it can also be used to monitor the use of the hearing aid and help giving the user feedback (e.g. if the user change to another program in a given situation and the stress level increases the system will be able to give feedback: 'let me help you here - change to program xx to get more help'...). The feedback can be given in e.g. an app of the smartphone that the user typically use to operate the hearing aid, as illustrated in Figure 4 with the text "user interaction", "advice given".
In another embodiment, from which the cognitive load in the form of listening effort and fatigue can be measured is by applying a hearing aid comprising at least one, preferably two near-infrared emitters coupled to a phased array detector that can be electronically oriented to scan a fixed position on the brain or to scan through a region of interest at discrete time-points.
The placement of the near-infrared emitters can both be in the earmold and/or in the behind the ear part of a hearing aid. The sensor may be configured with a planar configuration with grid-spaced sensors allowing for beam-steering in a half sphere. A planar configuration simplifies usage of the of-the-shelf steering algorithms such as, MVDR, MUSIC, etc. With multiple near-infrared emitters it is also possible to combine the angles from sub-arrays to compute the 3D location of the signal origin and not only its direction.
Other applications where the above mentioned setup could be used to e.g. activation of the auditory cortex as a signal of sound detection in the brain; evaluation of hearing thresholds based on activity of the auditory cortex; brain plasticity do to audiological rehabilitation treatment; evaluation of brain processes in the hearing aid that generate electrical artefacts on other modalities or are not compatible with MRI methods (e.g. cochlear implants).
For spatial decoding two options are possible which will be mentioned in the foloowing. That is, then using custom-made earmolds a well defined mechanical placing of the earmold and component, including the one or more near-infrared emitters will detect signals in a well-defined region of the brain. A mechanical adjustment of the arrangement of the near-infrared emitters may be performed during fitting of the hearing aid. That is, the fitting should be made such that the near-infrared emitters is arranged such that the near-infrared emitters comprises a plane being normal points to the most important region in the brain to be measured.
In another alternative, when using generic earmolds, whihc are not custom made it may not possible to predefine the relative near-infrared emitter position. Therefore, the near-infrared emitter may be configured to work in conjunction with an AI based pattern recognition, which detects the scanned brain region by its characteristic signature. This signature recognition can also be used in conjunction with the custom earmold to verify the correct placement of the earmold in the ear canal.
In an embodiment the detection of eye blinking by a hearing aid and/or by an auxiliary device in communication with the hearing aid is also considered when measuring cognition and mental fatigue. That is, in an embodiment illustrated schematically in Figure 5, the use of Electroencephalographic Activity (EEG), Electrooculographic (EOG) techniques, eye activity measures using eye-tracking cameras is proposed as methods for objective alertness and fatigue monitoring.
Accordingly, in this embodiment, a portable system 61 that can detect eye-blinks (EOG, EEG, eye-tracking camera, video camera) and which can be coupled with a hearing device is considered. The portable system 61 is configured to measure the eye blink frequency individually during every day live (e.g. when working at the monitor, being at work or when driving a car). Since a person generally has an individual eye blinking frequency with individual variability of varying degree, a baseline blink frequency is established for the individually person. That is, the system is configured with a post-processing step (62a, 62b), wherein the raw signal for eyeblink is detected. Then, an eye-blink detector 63 (as the one mentioned), is configured to utilize the raw data form the portable device (either earEEG, scalp EEG, or an eye-tracker), to detect changes in the blink frequency. As soon as the frequency changes and reach a critical threshold in comparison to the baseline, this shall be monitored with a detector. From the changes in the blink rate (i.e. the frequency change) it may be evaluated as an indicator of either a change in mental load or fatigue, such as loss of alertness.
The system may be configured to provide a feedback to the user, for example via the hearing aid or a smartphone, as previously described in a plurality of embodiments of the disclosure. The feedback provided to the user, may include the fatigue level during the day, or at a certain point in time and other already mentioned feedbacks, such as listening break indicator etc. Another feedback may be such that the when the blink rate reaches a certain threshold, a control signal is transmitted to the hearing aid, which as a consequence automatically adapts and change its processing in order to adapt to the fatigue level /mental load of the user.
In more detail, the system may comprise a sensor from which eye-blinks are detectable. This sensor can be placed in or around the ear such as earEEG electrodes or EOG electrodes. That is, the sensor may form part of a hearing aid. Further, the sensor can also be worn around or above the eyes such as an infrared camera, or video camera mounted in eye-frames.
An eyeblink detection module for postprocessing 62a of Figure 5 (previously described) of the sensor signal is configured to output the eye-blink rate (i.e. number of blinks in a window of time).
An eyeblink profile module 62b is configured to assess the individual eye-blink baseline at idle times and evaluate the current eye-blink rate against one or more predefined rules for categorization of mental load or fatigue.
The portable system is configured to communicate with a hearing aid and/or a smartphone to provide feedback on the final state decision to the user and /or the hearing aid to automatically change the processing settings of the hearing aid.
As used, the singular forms "a," "an," and "the" are intended to include the plural forms as well (i.e. to have the meaning "at least one"), unless expressly stated otherwise. It will be further understood that the terms "includes," "comprises," "including," and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will also be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, but an intervening element may also be present, unless expressly stated otherwise. Furthermore, "connected" or "coupled" as used herein may include wirelessly connected or coupled. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items. The steps of any disclosed method are not limited to the exact order stated herein, unless expressly stated otherwise.
It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" or "an aspect" or features included as "may" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosure. Furthermore, the particular features, structures or characteristics may be combined as suitable in one or more embodiments of the disclosure. The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects. Reference to an element in the singular is not intended to mean "one and only one" unless specifically so stated, but rather "one or more." Unless specifically stated otherwise, the term "some" refers to one or more.

Claims

A hearing aid system comprising a hearing aid configured to compensate for a hearing loss of a user wearing the hearing aid, the hearing aid comprising
one or more microphones configured to receive an input audio signal; a signal processor internal to the hearing aid configured to process the input audio signal according to a hearing loos profile of the user; and
a receiver configured to transmit said processed input audio signal to the ear of a user;
the system further comprising a portable auxiliary device configured to be in communicatively contact with the hearing aid and to receive and transmit signals from and to the hearing aid, wherein the auxiliary device and/or the hearing aid may be configured to comprise a cognitive model configured to estimate a ratio of exhaustion experienced by the user during the day.
Hearing system according to claim 1, wherein the cognitive model is configured to output a load estimation visually illustrated at the auxiliary device, and indicating to the user during the day how much energy that the user has spent on listening during the day.
Hearing system according to claim 1 and 2, wherein the cognitive model is configured such that a user of the hearing aid can actively indicate as an input to the model if he/she is having difficulty hearing, or is getting tired, whereby the cognitive model is configured to be updated based on the user input.
Hearing system according to claim 1 to 3, wherein the cognitive model is configured to calculate one or more ratios input parameters related to a signal to noise ratio (SNR), noise level (NL), speech quality (SQ) either as a stand alone or in combination to provide an output for each of the parameters or a combined estimation of the cognitive load, wherein the output may be communicated to the user of the hearing aid directly via the hearing aid and or via the auxiliary device.
Hearing system according to any of the previous claims, wherein the cognitive model is configured to output a control signal to the hearing aid, wherein the control signal is configured to automatically adjust the gain of the hearing aid.
Hearing system according to claim 5, wherein the cognitive model is configured with an information analyzer being configured to analyze the input audio signal and to estimate the information content in the input signal, wherein the information content is classified as irrelevant audio, such as noise and/or non-relevant speech or relevant audio, such as relevant speech, wherein the information analyzer is configured to send an output control signal indicating a decrease in gain in case of non-relevant information, and a normal gain in case of relevant information to the hearing aid signal processor.
Hearing system according to any of the previous claims, where the hearing aid is equipped with one or more sensors, such as accelerometer, heart rate sensor, electrodermal activity sensor and/or EEG sensors, wherein each of these sensors is configured to measure a physical situation that a user of the hearing aid is in.
Hearing system according to claim 7, where the smartphone and/or the hearing aid may be equipped with a stress processor, receiving inputs from one or more the sensors, wherein the stress processor is configured to combine measures of motion, heart rate variability, electrodermal activity to asses a level of stress experienced by a hearing aid user.
Hearing system according to claims 7 and 8, wherein the hearing aid and/or the smartphone comprises a learning processor configured to learn from sensor inputs when a stressed situation is present for the user of the hearing aid, wherein the when the stress is detected by one or more sensors (EA, HRV, EEG) an app will be evoked on the user's smartphone.