GB2563892A - Sound monitoring system and method - Google Patents
Sound monitoring system and method Download PDFInfo
- Publication number
- GB2563892A GB2563892A GB1710365.6A GB201710365A GB2563892A GB 2563892 A GB2563892 A GB 2563892A GB 201710365 A GB201710365 A GB 201710365A GB 2563892 A GB2563892 A GB 2563892A
- Authority
- GB
- United Kingdom
- Prior art keywords
- sound
- sound data
- environment
- detected
- abnormal activity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 31
- 238000012544 monitoring process Methods 0.000 title claims abstract description 17
- 230000002159 abnormal effect Effects 0.000 claims abstract description 57
- 230000005236 sound signal Effects 0.000 claims abstract description 32
- 230000008859 change Effects 0.000 claims abstract description 10
- 238000012545 processing Methods 0.000 claims abstract description 7
- 238000012806 monitoring device Methods 0.000 claims description 21
- 230000009471 action Effects 0.000 claims description 20
- 230000004044 response Effects 0.000 claims description 13
- 238000012549 training Methods 0.000 claims description 8
- 238000004891 communication Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 3
- 230000006866 deterioration Effects 0.000 abstract description 2
- 206010011224 Cough Diseases 0.000 description 9
- 238000004458 analytical method Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 5
- 238000001514 detection method Methods 0.000 description 4
- 238000010801 machine learning Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000000750 progressive effect Effects 0.000 description 3
- 206010011469 Crying Diseases 0.000 description 2
- 208000037656 Respiratory Sounds Diseases 0.000 description 2
- 206010047700 Vomiting Diseases 0.000 description 2
- 206010047924 Wheezing Diseases 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 2
- 238000004220 aggregation Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000011521 glass Substances 0.000 description 2
- 230000002035 prolonged effect Effects 0.000 description 2
- 206010041232 sneezing Diseases 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000008673 vomiting Effects 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 208000006545 Chronic Obstructive Pulmonary Disease Diseases 0.000 description 1
- 206010012289 Dementia Diseases 0.000 description 1
- 206010014561 Emphysema Diseases 0.000 description 1
- 206010041235 Snoring Diseases 0.000 description 1
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 239000008186 active pharmaceutical agent Substances 0.000 description 1
- 230000009118 appropriate response Effects 0.000 description 1
- 238000007664 blowing Methods 0.000 description 1
- 238000009835 boiling Methods 0.000 description 1
- 210000000748 cardiovascular system Anatomy 0.000 description 1
- 229940000032 cardiovascular system drug Drugs 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 230000030808 detection of mechanical stimulus involved in sensory perception of sound Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 208000015181 infectious disease Diseases 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 210000004072 lung Anatomy 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000010255 response to auditory stimulus Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 230000002618 waking effect Effects 0.000 description 1
- 230000036642 wellbeing Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/04—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons
- G08B21/0407—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons based on behaviour analysis
- G08B21/0423—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons based on behaviour analysis detecting deviation from an expected pattern of behaviour or schedule
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/04—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons
- G08B21/0438—Sensor means for detecting
- G08B21/0469—Presence detectors to detect unsafe condition, e.g. infrared sensor, microphone
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- General Health & Medical Sciences (AREA)
- Business, Economics & Management (AREA)
- Emergency Management (AREA)
- Gerontology & Geriatric Medicine (AREA)
- General Physics & Mathematics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Multimedia (AREA)
- Acoustics & Sound (AREA)
- Computational Linguistics (AREA)
- Signal Processing (AREA)
- Social Psychology (AREA)
- Psychiatry (AREA)
- Psychology (AREA)
- Emergency Alarm Devices (AREA)
- Testing And Monitoring For Control Systems (AREA)
Abstract
A method of monitoring sounds in an environment in which sound recognition is performed on audio signals produced by microphones to produce detected sound data representing recognised sound events. If the detected sound events indicate that there is abnormal activity in the monitored environment, an alert message may be sent to, for example, a carer of a patient. Audio signal processing technology is used to recognise, classify, and identify patterns in household noises such that the occurrence of noises consistent with illness, or increases/changes in such noises consistent with deterioration in an existing condition, may be identified and made known to the carer. The method may also use the detected sound data to determine abnormal activity by the absence of a sound event or a change in frequency or duration of a sound event, deviation from a reference sound event. The method may also include recognizing human speech and /or utterances.
Description
Sound Monitoring System and Method
Field ofthe Invention
The present invention relates to sound monitoring systems. The invention relates particularly to systems for monitoring activity in domestic environments.
Background to the Invention
In the western world, approximately 50% of people over the age of 75 live alone (2 million people in the UK in 2016) and they are vulnerable to injury through falls, life-threatening infections such as ‘flu which much more greatly impact the elderly, and many suffer from emphysema, COPD and other diseases ofthe lungs and cardio-vascularsystem.
Advances in low-cost computing and sensor technology, along with developments in machinelearning are making it possible to monitor the home environment of vulnerable people at low cost. Speech recognition has advanced greatly in the last few years giving rise to devices such as Google Home and Amazon Echo, and services such as Shazam.
It would be desirable to provide a system for monitoring the well-being of vulnerable people in a domestic environment.
Summary ofthe Invention A first aspect ofthe invention provides a method of monitoring sounds in an environment using at least one microphone, the method comprising: detecting at least one sound using said at least one microphone; producing a corresponding audio signal from the or each microphone; performing sound recognition on the corresponding audio signal from the or each microphone to produce detected sound data representing one or more recognised sound events; determining if said detected sound data is indicative of abnormal activity in said environment; and performing at least one action in response to determining that there is abnormal activity. A second aspect ofthe invention provides a sound monitoring device for monitoring sounds in an environment, the device comprising: at least one microphone, audio signal processing means configured to perform sound recognition on audio signals from the or each microphone to produce detected sound data representing one or more recognised sound events, and to determine if said detected sound data is indicative of abnormal activity in said environment; and means for performing at least one action in response to determining that there is abnormal activity. A third aspect ofthe invention provides a sound monitoring system comprising: at least one sound monitoring device comprising at least one microphone for detecting sounds in said environment; means for performing sound recognition on audio signals produced by the or each microphone to produce detected sound data representing one or more recognised sound events; means for determining if said detected sound data is indicative of abnormal activity in said environment; and means for performing at least one action in response to determining that there is abnormal activity.
Determining if said detected sound data is indicative of abnormal activity in said environment may comprise comparing said detected sound data against reference sound data.
Preferred embodiments further include creating said reference sound data by: detecting sounds in said environment using said at least one microphone; producing corresponding audio signals from the or each microphone; and performing sound recognition on the corresponding audio signals to produce said reference sound data representing one or more recognised sound events.
Said reference sound data may created during a training period.
Preferred embodiments include revising said reference sound data outside of said training period using said detected sound data.
Preferred embodiments include determining that said detected sound data is indicative of abnormal activity in said environment by identifying one or more designated deviation between said detected sound data and said reference sound data.
Determining that said detected sound data is indicative of abnormal activity in said environment may comprise identifying from said detected sound data a change in frequency of occurrence of one or more sound events.
Determining that said detected sound data is indicative of abnormal activity in said environment may comprise identifying from said detected sound data an absence of one or more sound event.
Determining that said detected sound data is indicative of abnormal activity in said environment may comprise identifying from said detected sound data, a change in duration of one or more sound event.
Determining that said detected sound data is indicative of abnormal activity in said environment may comprise identifying from said detected sound data a change in time of occurrence of one or more sound event.
Determining that said detected sound data is indicative of abnormal activity in said environment may comprise identifying from said detected sound data one or more instances of a sound of a type that is designated as being abnormal.
Determining that said detected sound data is indicative of abnormal activity in said environment may comprise identifying from said detected sound data sounds of a combination of types that is designated as being abnormal.
Performing at least one action in response to determining that there is abnormal activity in said environment may comprise sending a message to one or more electronic message-receiving device.
Performing sound recognition may comprise determining if said audio signals correspond to any one or more of said recognisable sound events.
Performing sound recognition may comprise comparing one or more audio samples from said audio signals to audio sample data representing each recognisable sound event.
Said recognisable sound events comprise human utterances, including speech and non-speech utterances (e.g. coughs, sneezes, nose blowing, yells, laughter) and/or non-human sounds, e.g. of types that commonly occur in a domestic environment.
Preferred systems include at least one message-receiving device in communication with said at least one sound monitoring device across a telecommunications network, wherein said at least one action comprises sending a message to at least one of said at least one message-receiving device in response to determining that there is abnormal activity.
In preferred embodiments, audio signal processing technology is used to recognise, classify, and identify patterns in household noises such that the occurrence of noises consistent with illness, or increases/changes in such noises consistent with deterioration in an existing condition, may be identified and made known to a care-giver.
Further advantageous aspects ofthe invention will be apparent to those ordinarily skilled in the art upon review ofthe following specific embodiment and with reference to the accompanying drawings.
Brief Description ofthe Drawings A embodiment of the invention is now described by way of example and with reference to the accompanying drawings in which:
Figure 1 is a schematic diagram of a room in which a sound monitoring device embodying one aspect ofthe invention is installed;
Figure 2 is a block diagram ofthe sound monitoring device of Figure 1;
Figure 3 is a schematic diagram of a sound monitoring system embodying another aspect of the invention; and
Figure 4 is a flow diagram illustrating a preferred operation ofthe device of Figure 1
Detailed Description ofthe Drawings
Referring now to the drawings there is shown a sound monitoring device 10 embodying one aspect ofthe invention. The device 10 is shown installed in a room 12. In the illustrated example the room 12 is a typical living room but this is not limiting to the invention.
In preferred embodiments, the sound monitoring device 10 is a device for monitoring activity in its environment using detected sounds. Typically, the environment in which the device 10 is located in use is a domestic environment, e.g. in a room of a person’s home, although this need not necessary be the case. The device 10 is part of a sound monitoring system 15 embodying another aspect ofthe invention, the system 15 including at least one ofthe devices 10 in communication with a central monitoring system 14 and/or one or more carer devices 16 across a telecommunications network 17, which may for example comprise a computer network (e.g. the internet) and or a telephone network. The central monitoring device 14 (if present) may comprise one or more computers and may provide server functionality to the monitoring devices 10 and/or the carer devices 16. The carer devices 16 may comprise any electronic device capable of receiving messages from the devices 10 and/or the central monitoring device 14, for example a computer or computing device, or a telephone, especially a smartphone. Where there are multiple devices 10, a respective one may be located in a respective room ofthe same building, and/or in other buildings as required.
The sound monitoring device 10 is an electronic apparatus that detects sounds in its environment (the room 12 in the illustrated example) and supports sound recognition ofthe detected sounds. This may be achieved using any conventional sound recognition techniques. As part ofthe sound recognition, the device 10 preferably configured to perform classification of detected sounds.
The device 10 is configured to monitor the environment and, using the detected and recognised sounds, determine whether or not there is any abnormal activity in the environment. If any abnormal activity is detected, then the device 10 may take one or more actions. The or each action may depend on the type of abnormal activity that is detected. Typically, the actions include notifying one or more carers ofthe abnormal activity. This may involve sending a message to one or more ofthe carer devices 16. Such messages may be sent directly to the carer device(s) 16 by the monitoring device 10 across the network 17, or may be sent by the central monitoring system 14 to the carer device(s) 16 in response to notification by the monitoring device 10 of abnormal activity. The message may comprise an email, SMS message, MMS message, pager notification, phone call and/or any other messaging means supported by the relevant components ofthe system 15.
The device 10 may be configured to take one or more action in response to sounds that it recognises as being of a known type as determined by the classification process. Other actions that may be taken as well as or instead of sending a message to a carer include, but are not limited to: issuing an alarm (e.g. an audio and/or visual alarm generated by a suitable audio and/or visual output device with which the device 10 may be provided); operating some other equipment to which the device 10 is connected; and/or reporting the detected sound(s) and/or an analysis thereof to the central monitoring device 14.
The device 10 typically includes a controller 11 for controlling the overall operation of the device 10. The controller 11 may comprise any suitably configured or programmed processor(s), for example a microprocessor, microcontroller or multi-core processor. Typically the controller 11 causes the device 10 to take whichever action(s) are required in response to detection of recognised sounds and the analysis of detected sounds. Typically the device 10 is implemented using a multi-core processor running a plurality of processes, one of which may be designated as the controller and the others performing the other tasks described herein as required. Each process may be performed in software, hardware ora combination of software as is convenient. One or more hardware digital signal processors may be provided to perform one or more ofthe processes as is convenient and as applicable.
Figure 2 is a block diagram of a typical embodiment ofthe sound monitoring device 10. The device 10 comprises at least one microphone 18. Typical embodiments include two or more (four or more is preferred) microphones 18 to facilitate determining the location of sound sources and for isolating sounds of interest from background noise. Preferably one or more sensitive microphone, for example a MEMS microphone is provided.
The device 10 comprises an audio signal processor 20 for receiving and processing audio signals produced by the microphones 18 in response to detecting sounds in the room 12 or other environment. The audio signal processor 20 may take any convenient conventional form, being implemented in hardware, software or a combination of hardware and software. Accordingly, the audio signal processor 20 may be implemented by one or more suitably configured ASIC, FPGA or other integrated circuit, and/or a computing device with suitably programmed microprocessor(s). In preferred embodiments the audio signal processor 20 is configured to perform sound recognition (which typically includes sound classification) on the microphone output signals. After sound recognition, the device 10 may analyse the detected recognised sounds to determine whether or not there is any abnormal activity in the environment. The analysis may involve applying a set of rules to the detected recognised sounds and to this end the device 10 may include a rules engine 24. The analysis, including implementation ofthe rules engine 24, may be performed by the audio signal processor 20, or by any other suitable component ofthe sound monitoring device 10 as is convenient. The audio signal processor 20 includes components or other means for performing the relevant audio signal processing and/or analysis functions, as indicated in the example of Figure 2 as 22 and 24. Sound recognition may be performed using any convention sound recognition techniques. For example, sound recognition may be performed by comparing the audio signals captured by the microphones 18 against machine-learned audio templates for each sound to be recognised. For example the system may employ machine-learning training based on a large database of audio-samples, combined, optionally, with specific training on the household environment.
In alternative embodiments, some or all ofthe audio signal processing and/or the analysis ofthe detected sounds may be performed by other component(s) ofthe system 15, for example by the central monitoring device 14. For example the sound recognition 22 and/or the rules engine 24 may be implemented by the central monitoring device 14 (instead of or as well as being performed by the device 10). This allows the analysis of detected sounds to be performed not only in relation to sounds detected by a single device 10, but to sounds detected by multiple devices 10, for example multiple devices 10 located in different rooms ofthe same building.
The analysis of detected recognised sounds is intended to determine if any abnormal activity is occurring in the environment being monitored (which may be a single room 12 monitored by one or more device 10, or multiple rooms each monitored by one or more device 10) in respect of which one or more actions needs to be taken by the relevant device 10 or system 15.
In preferred embodiments the system 15 and/or the device 10 as applicable may be configured to identify abnormal activity by any one or more ofthe following: 1. Detection of one or more instances of a sound of a type that is designated as being abnormal (for example a scream or a gun shot), where one or more types may be designated as abnormal; 2. Detection of sounds of a combination of types that is designated as being abnormal (for example a cough and a sneeze), where one or more combination of types may be designated as abnormal; 3. Detection of one or more sound events that deviates from sound reference data; 4. Detection of changes in one or more sounds, or in one or more patterns of one or more sounds (for example a cough becoming louder and/or more frequent, or occurrence of a sound at an unusual time).
Any ofthe foregoing may take into account not only detected sounds but also detected silences and/or the absence of one or more expected sounds.
Any ofthe foregoing may take into account sounds of types that emanate from a person (e.g. speech, shouts, coughs, sneezes or snores) or from the environment (e.g. a door bell, telephone ring, running tap and so on), or a combination of both.
Any ofthe foregoing may be performed in respect of a specified time period which may be long or short depending on whether it is desired to monitor sudden or progressive activities.
The sound reference data may be obtained by any convenient means. For example the or each device 10 may monitor sounds in its environment during a training period and generate sound reference data from the sounds detected during that period. The sound reference data may be periodically or continually updated by the system 15 or device 10 as applicable depending on the sounds that it detects during use (for example using any conventional machine learning techniques). The sound reference data represents a profile of sound events that is deemed to be normal for the relevant environment. The reference sound data may identify any one or more of: types of sounds; combinations of types of sounds; patterns of one or more sounds; time of occurrence of one or more sounds; frequency of occurrence of one or more sounds; volume of sounds; absence(s) of one or more sounds; silence(s), and/or any other sound characteristics that may be used to define a normal sound environment. The reference sound data may be defined over a specified period of time, for example a day, week or month.
During use, the or each device 10 detects sound data from the environment. The detected sound data therefore represents one or more recognised sound events detected by the device 10. The detected sound data may identify any one or more of: types of detected sounds; combinations of types of detected sounds; patterns of one or more detected sounds; time of occurrence of one or more detected sounds; frequency of occurrence of one or more detected sounds; volume of detected sounds; absence(s) of one or more sounds; silence(s), and/or any other sound characteristics that may be used to define the detected sound events. The detected sound data may be defined over a specified period of time, for example a day, week or month. The detected sound data is analysed, by the rules engine 24 in the present example, to determine if any one or more ofthe recognised sound events indicates that there is abnormal activity in the environment. This may involve comparing the detected sound data against the reference sound data. Any deviations between the detected sound data and the reference sound data may be indicative of abnormal activity in the environment. Typically any such deviations are analysed (conveniently by the rules engine 24) to determine if they are indicative of abnormal activity of a type that warrants further action by the system 15, e.g. notification of a carer or generation of an alarm. Such deviations may be referred to as designated deviations and may include for example a change in frequency of one or more particular sound events (e.g. increased coughing or reduced conversation), the absence of one or more sound event that was previously present, a change in duration of one or more particular sound event (e.g. increased periods of silence), a change in time of occurrence of one or more particular sound events (e.g. door slams at unusual times).
Figure 4 shows a typical operation ofthe device 10 or system 15. At 301 the or each device 10 captures sound samples from the audio signals produced by the microphones 18. The sound samples are subjected to sound recognition to identify the type of sound(s) detected (302). Recognised sounds are sent to the rules engine 24 which applies a set of rules to the detected recognised sounds to determine if one or more abnormal activity is detected (303). If abnormal activity is detected, then appropriate action is taken, typically involving notification of a carer (304).
In typical embodiments, household sounds that may be detected and analysed by the device 10/system 15 may include any one or more ofthe following (without limitation): telephone; doorbell; door slams; footsteps; boiling kettle; coughs; sneezes; wheezing; conversation; laughter; crying; cries of pain or for help; breaking glass, or other household items; TV or radio; running water (e.g. indicative of a shower or bath).
During a training period (e.g. of 1 to 2 weeks) the device 10/system 15 can learn the frequency, and time of occurrence of such sounds to create or adapt the reference data for the environment. Using the reference data as a baseline, which may be allowed to evolve overtime, the system/device can then use rules to identify both sudden and progressive changes in the environment, particularly those that are deemed to be abnormal and that warrant further action being taken. Examples of abnormal activities may include any one or more ofthe following (without limitation): o Sudden Changes:
Silence
Cries of pain or for help
Door slams at unusual hours
Footsteps at unusual hours
Crying
Sneezing
Coughing
Vomiting
Breaking of glass, or smashing of other household items o Progressive Changes
Increased coughing
Increased wheezing
Reduction in conversation
Decrease in doorbell and/or telephone rings indicating growing social isolation
Lengthening periods of silence during normal waking hours potentially indicating depression, dementia or other illness
Decrease in frequency of incidence of running water indicating reduced self care indicating depression or other illness.
Advantageously knowledge of time of day, and weather conditions (which may readily be obtained from web APIs for example) may be used to eliminate false alarms, e.g. prolonged silence in the middle of a sunny day may mean that the monitored individual has just gone out, whereas prolonged silence on a cold, wet, afternoon is more concerning.
Advantageously aggregation of machine-learning across multiple devices 10 improves the accuracy of recognition of sounds.
Advantageously aggregation of learning across localities can give evidence of increasing levels of coughing/sneezing/vomiting consistent with possible ‘flu epidemics.
Overtime it is possible to assign a “health score” based on a large corpus of data from multiple monitored residences and trigger an appropriate response from care givers using this score (or changes to it).
The system 15/device10 may be trained to recognise a noise specific to its environment, for example to enable a learning mode, then trigger the door bell (or other sound) one or more times, and then label that sound appropriately within the system (as a door bell in this case).
The invention is not limited to the embodiment(s) described herein but can be amended or modified without departing from the scope ofthe present invention.
Claims (19)
1. A method of monitoring sounds in an environment using at least one microphone, the method comprising: detecting at least one sound using said at least one microphone; producing a corresponding audio signal from the or each microphone; performing sound recognition on the corresponding audio signal from the or each microphone to produce detected sound data representing one or more recognised sound events; determining if said detected sound data is indicative of abnormal activity in said environment; and performing at least one action in response to determining that there is abnormal activity.
2. The method of claim 1 wherein determining if said detected sound data is indicative of abnormal activity in said environment comprises comparing said detected sound data against reference sound data.
3. The method of claim 2 further including creating said reference sound data by: detecting sounds in said environment using said at least one microphone; producing corresponding audio signals from the or each microphone; and performing sound recognition on the corresponding audio signals to produce said reference sound data representing one or more recognised sound events.
4. The method of claim 3, wherein said reference sound data is created during a training period.
5. The method of claim 3 or 4, including revising said reference sound data outside of said training period using said detected sound data.
6. The method of any one of claims 2 to 5 including determining that said detected sound data is indicative of abnormal activity in said environment by identifying one or more designated deviation between said detected sound data and said reference sound data.
7. The method of any preceding claim including determining that said detected sound data is indicative of abnormal activity in said environment by identifying from said detected sound data a change in frequency of occurrence of one or more sound events.
8. The method of any preceding claim including determining that said detected sound data is indicative of abnormal activity in said environment by identifying from said detected sound data an absence of one or more sound event.
9. The method of any preceding claim including determining that said detected sound data is indicative of abnormal activity in said environment by identifying from said detected sound data, a change in duration of one or more sound event.
10. The method of any preceding claim including determining that said detected sound data is indicative of abnormal activity in said environment by identifying from said detected sound data a change in time of occurrence of one or more sound event.
11. The method of any preceding claim including determining that said detected sound data is indicative of abnormal activity in said environment by identifying from said detected sound data one or more instances of a sound of a type that is designated as being abnormal.
12. The method of any preceding claim including determining that said detected sound data is indicative of abnormal activity in said environment by identifying from said detected sound data sounds of a combination of types that is designated as being abnormal.
13. The method of any preceding claim wherein performing at least one action in response to determining that there is abnormal activity in said environment comprises sending a message to one or more electronic message-receiving device.
14. The method of any preceding claim wherein performing sound recognition comprises determining if said audio signals correspond to any one or more of said recognisable sound events.
15. The method of any preceding claim wherein performing sound recognition comprises comparing one or more audio samples from said audio signals to audio sample data representing each recognisable sound event.
16. The method of any preceding claim wherein said recognisable sound events comprise human utterances.
17. A sound monitoring device for monitoring sounds in an environment, the device comprising: at least one microphone, audio signal processing means configured to perform sound recognition on audio signals from the or each microphone to produce detected sound data representing one or more recognised sound events, and to determine if said detected sound data is indicative of abnormal activity in said environment; and means for performing at least one action in response to determining that there is abnormal activity.
18. A sound monitoring system comprising: at least one sound monitoring device comprising at least one microphone for detecting sounds in said environment; means for performing sound recognition on audio signals produced by the or each microphone to produce detected sound data representing one or more recognised sound events; means for determining if said detected sound data is indicative of abnormal activity in said environment; and means for performing at least one action in response to determining that there is abnormal activity.
19. The system of claim 18, further including at least one message-receiving device in communication with said at least one sound monitoring device across a telecommunications network, wherein said at least one action comprises sending a message to at least one of said at least one message-receiving device in response to determining that there is abnormal activity.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GB1710365.6A GB2563892B (en) | 2017-06-28 | 2017-06-28 | Sound monitoring system and method |
GB2019008.8A GB2588036B (en) | 2017-06-28 | 2017-06-28 | Sound monitoring system and method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GB1710365.6A GB2563892B (en) | 2017-06-28 | 2017-06-28 | Sound monitoring system and method |
Publications (3)
Publication Number | Publication Date |
---|---|
GB201710365D0 GB201710365D0 (en) | 2017-08-09 |
GB2563892A true GB2563892A (en) | 2019-01-02 |
GB2563892B GB2563892B (en) | 2021-01-20 |
Family
ID=59523575
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
GB1710365.6A Active GB2563892B (en) | 2017-06-28 | 2017-06-28 | Sound monitoring system and method |
Country Status (1)
Country | Link |
---|---|
GB (1) | GB2563892B (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102020104855A1 (en) | 2020-02-25 | 2021-08-26 | Olympus Winter & Ibe Gmbh | Method and system for monitoring an operating room |
EP4057625A1 (en) * | 2021-03-10 | 2022-09-14 | Honeywell International Inc. | Video surveillance system with audio analytics adapted to a particular environment to aid in identifying abnormal events in the particular environment |
EP4403830A1 (en) | 2023-01-23 | 2024-07-24 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Cooking appliance and method |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150077567A1 (en) * | 2013-07-26 | 2015-03-19 | Joseph Frank Scalisi | Light socket cameras |
US20150379836A1 (en) * | 2014-06-26 | 2015-12-31 | Vivint, Inc. | Verifying occupancy of a building |
US20160140986A1 (en) * | 2014-11-17 | 2016-05-19 | Elwha Llc | Monitoring treatment compliance using combined performance indicators |
US20160163168A1 (en) * | 2014-12-05 | 2016-06-09 | Elwha Llc | Detection and classification of abnormal sounds |
US20160328949A1 (en) * | 2015-05-06 | 2016-11-10 | Victoria Zhong | Method for an Automated Distress Alert System with Speech Recognition |
US20170178667A1 (en) * | 2015-12-22 | 2017-06-22 | Joachim Hofer | Technologies for robust crying detection using temporal characteristics of acoustic features |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
FI122787B (en) * | 2010-12-28 | 2012-07-13 | Lano Group Oy | Remote Monitoring System |
-
2017
- 2017-06-28 GB GB1710365.6A patent/GB2563892B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150077567A1 (en) * | 2013-07-26 | 2015-03-19 | Joseph Frank Scalisi | Light socket cameras |
US20150379836A1 (en) * | 2014-06-26 | 2015-12-31 | Vivint, Inc. | Verifying occupancy of a building |
US20160140986A1 (en) * | 2014-11-17 | 2016-05-19 | Elwha Llc | Monitoring treatment compliance using combined performance indicators |
US20160163168A1 (en) * | 2014-12-05 | 2016-06-09 | Elwha Llc | Detection and classification of abnormal sounds |
US20160328949A1 (en) * | 2015-05-06 | 2016-11-10 | Victoria Zhong | Method for an Automated Distress Alert System with Speech Recognition |
US20170178667A1 (en) * | 2015-12-22 | 2017-06-22 | Joachim Hofer | Technologies for robust crying detection using temporal characteristics of acoustic features |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102020104855A1 (en) | 2020-02-25 | 2021-08-26 | Olympus Winter & Ibe Gmbh | Method and system for monitoring an operating room |
EP4057625A1 (en) * | 2021-03-10 | 2022-09-14 | Honeywell International Inc. | Video surveillance system with audio analytics adapted to a particular environment to aid in identifying abnormal events in the particular environment |
US11765501B2 (en) | 2021-03-10 | 2023-09-19 | Honeywell International Inc. | Video surveillance system with audio analytics adapted to a particular environment to aid in identifying abnormal events in the particular environment |
EP4403830A1 (en) | 2023-01-23 | 2024-07-24 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Cooking appliance and method |
Also Published As
Publication number | Publication date |
---|---|
GB2563892B (en) | 2021-01-20 |
GB201710365D0 (en) | 2017-08-09 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11276299B2 (en) | DIT monitoring apparatus and method | |
US11887462B2 (en) | System and method for identifying vaping and bullying | |
US10176705B1 (en) | Audio monitoring and sound identification process for remote alarms | |
US10152877B2 (en) | Systems and methods for adaptive detection of audio alarms | |
EP3360138B1 (en) | System and method for audio scene understanding of physical object sound sources | |
US20130100268A1 (en) | Emergency detection and response system and method | |
US20100302042A1 (en) | Sensor-based independent living assistant | |
US8063764B1 (en) | Automated emergency detection and response | |
US11615893B2 (en) | Method for evaluating infection risk, infection risk evaluation system, and medium | |
US20200035261A1 (en) | Sound detection | |
GB2563892A (en) | Sound monitoring system and method | |
US20200258363A1 (en) | System and method for notifying detection of vaping, smoking, or potential bullying | |
US20220008030A1 (en) | Detection of agonal breathing using a smart device | |
KR20230015980A (en) | Simultaneous acoustic event detection on multiple assistant devices | |
GB2588036A (en) | Sound monitoring system and method | |
US11450327B2 (en) | Systems and methods for improved accuracy of bullying or altercation detection or identification of excessive machine noise | |
Devos et al. | Towards understanding healthy and supportive acoustic environments: The case of a nursing home | |
CN112669837B (en) | Awakening method and device of intelligent terminal and electronic equipment | |
US20210241927A1 (en) | System and method for monitoring via smart devices | |
CN112700765A (en) | Assistance techniques | |
CN115605859A (en) | Inferring semantic tags for an assistant device based on device-specific signals | |
WO2020167322A1 (en) | System and method for notifying detection of vaping, smoking, or bullying | |
US10325470B2 (en) | System for monitoring health status of a person within an enclosed area | |
TWI689899B (en) | Warning system for person consciousness and method thereof | |
US10728383B1 (en) | Controlled-environment facility resident pattern electronic communication detection and controlled-environment facility action |