EP3493173A1 - Procédé de génération d'alertes et dispositif électronique correspondant, système de communication, produits- programme lisibles par ordinateur et support d'informations lisible par ordinateur - Google Patents

Procédé de génération d'alertes et dispositif électronique correspondant, système de communication, produits- programme lisibles par ordinateur et support d'informations lisible par ordinateur Download PDF

Info

Publication number
EP3493173A1
EP3493173A1 EP17306659.8A EP17306659A EP3493173A1 EP 3493173 A1 EP3493173 A1 EP 3493173A1 EP 17306659 A EP17306659 A EP 17306659A EP 3493173 A1 EP3493173 A1 EP 3493173A1
Authority
EP
European Patent Office
Prior art keywords
events
monitoring
alert
time
electronic device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP17306659.8A
Other languages
German (de)
English (en)
Inventor
Claire-Hélène Demarty
Quang Khanh Ngoc DUONG
Frédéric Lefebvre
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Thomson Licensing SAS
Original Assignee
Thomson Licensing SAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Thomson Licensing SAS filed Critical Thomson Licensing SAS
Priority to EP17306659.8A priority Critical patent/EP3493173A1/fr
Publication of EP3493173A1 publication Critical patent/EP3493173A1/fr
Withdrawn legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/18—Status alarms
    • G08B21/24—Reminder alarms, e.g. anti-loss alarms

Definitions

  • the present disclosure relates to the technical domain of automatic, or semi-automatic user alerts.
  • a method for generating an alert and corresponding electronic device, computer readable program products and computer readable storage medium are described.
  • the present principles propose a method comprising:
  • said method comprises obtaining said reference time of day upon receiving an alert request from a user interface of a second electronic device of said communication system.
  • generating an alert comprises outputting an alerting element obtained upon receiving said alert request.
  • said method comprises rendering said alerting element on a user interface of at least one third electronic device of said communication system.
  • said alert is generated upon determining a matching between said monitored events and said reference pattern of events.
  • determining a matching takes into account a probability of presence, beyond said monitored events, of at least one event of said pattern of events, said probability of presence being determined based on at least one previous monitoring determined based on at least one previous monitoring.
  • determining a matching takes into account a first value of said probability of presence that decreases when a time difference between a current time and said reference time of day decreases.
  • generating an alert takes into account a contextual information.
  • said alert is generated upon detecting a deviating event, between said monitored events and said reference pattern of events.
  • generating said alert comprises outputting an alerting element associated to said deviating event.
  • said method comprises obtaining at least one initial pattern of events during at least one initial monitoring of events captured by said sensors and obtaining said pattern of events from at least one said initial pattern of events.
  • a constant number of initial monitorings is performed for obtaining said reference pattern of events.
  • said method comprises obtaining said reference pattern of events by fine tuning said at least one initial pattern of events.
  • said method comprises getting a feedback (from instance from a user interface) about said generated alert.
  • said method comprises updating said pattern of events according to said feedback and to said monitored events.
  • the present disclosure relates to an electronic device comprising at least one memory and one or several processors configured for collectively:
  • said one or several processors are configured for collectively obtaining said reference time of day upon receiving an alert request on a communication interface of said first device.
  • said one or several processors are configured for collectively transmitting said alert to at least one third electronic device of said communication system.
  • the electronic device of the present disclosure can be adapted to perform the method of the present disclosure in any of its embodiments.
  • the present disclosure relates to an electronic device comprising at least one memory and at least one processing circuitry configured for:
  • said at least one processing circuitry are configured for obtaining said reference time of day upon receiving an alert request on a communication interface of said first device.
  • said at least one processing circuitry are configured for transmitting said alert to at least one third electronic device of said communication system.
  • the electronic device of the present disclosure can be adapted to perform the method of the present disclosure in any of its embodiments.
  • the present disclosure relates to a communication system comprising a first electronic device connected with at least one sensor, said first electronic device comprising at least one memory and one or several processors configured for collectively:
  • said one or several processors are configured for obtaining said reference time of day upon receiving an alert request on a communication interface of said first device.
  • said one or several processors are configured for configured for transmitting said alert to at least one third electronic device of said communication system.
  • the communication system of the present disclosure can be adapted to perform the method of the present disclosure in any of its embodiments.
  • the present disclosure relates to a non-transitory computer readable program product comprising program code instructions for performing the method of the present disclosure, in any of its embodiments, when said program product is executed by a computer.
  • said non-transitory computer readable program product comprises program code instructions for performing, when said program product is executed by a computer, a method comprising:
  • the present disclosure relates to a computer readable storage medium carrying a software program.
  • said software program comprises program code instructions for performing the method of the present disclosure, in any of its embodiments, when said software program is executed by a computer.
  • said software program comprises program code instructions for performing, when said non-transitory software program is executed by a computer, a method comprising:
  • At least some principles of the present disclosure propose to provide a reminder to a user (or a group of users) linked to a recurrent activity thanks to an intelligent, self-learning application.
  • a reminder can be provided to the user.
  • the reminder can be generated in many ways, including for instance by rendering an alert element, like a textual message, an image, a sound, a smell or perfume, and/or a combination thereof, in a user interface of a device.
  • the reminder can be provided systematically or optionally, when a deviation is detected inside the activity, and/or conditionally, depending on at least one further condition, like a condition external or independent to the activity.
  • At least some principles of the present disclosure relate to a way of collecting events linked to the activity, notably events occurring during the activity, and/or in a time window before the activity, in order to get, at least partially automatically, a signature, in terms of events of the activity, and thus to be able to generate a reminder to the user when at least some of the events constituting the signature of the activity occurs or have occurred.
  • the activity can be performed by the user to be reminded him/herself, by at least one user of the group of users to which the user to be reminded belongs to, by a different user, or group of users, by at least one animal, and/or by at least one device.
  • Term "activity" is also used hereinafter for encompassing a sequence of events linked by a same semantic concept.
  • pattern of events The signature of the activity, in terms of events, is also called herein “pattern of events", or “reference pattern of events”.
  • pattern of events is learned by the application and used as a reference that is compared with monitored events in order to determine if the activity is being performed.
  • Figure 1 illustrates a communication system adapted to implement at least some embodiments of the present disclosure.
  • the communication system comprises logical blocks (or modules) that can be part of a single electronic device connected to sensors, or by several distinct (and separate) physical devices.
  • the communication system which will be further described hereinafter, can comprise at least one electronic device like the one described hereinafter in link with figure 2 .
  • Figure 2 describes the structure of an electronic device 20 that can be configured notably to perform one or several aspects of the present disclosure.
  • the electronic device can be any electronic device with input and/or output means and processing means, like a smart phone, a personal computer, a tablet, a TV, a STB, and /or a wearable communication device (also known sometimes in today life as a "connected” or “smart” object or device).
  • the electronic device 20 can include different devices, or apparatus, linked together via a data and address bus 200, which can also carry a timer signal.
  • the electronic device 20 can include at least one micro-processor 21 (or CPU), a graphic card 22 (depending on embodiments, such a card may be optional), a ROM (or « Read Only Memory ») 25, a RAM (or « Random Access Memory ») 26, at least one Input/Output (I/O) module 24, for instance at least one Input/Output audio module (like a microphone, a loudspeaker, and so on) or at least one other Input/ Output module (like a keyboard, a mouse, a led, and so on).
  • I/O Input/Output
  • the input modules can notably comprise sensing modules (for instance one or more webcams, microphones, temperature sensors, ).
  • the sensing modules are optional.
  • the electronic device can be connected either directly or indirectly (via a monitoring unit), through at least one communication interface, to at least one sensing module.
  • the electronic device can also comprise at least one communication interface configured for the reception and/or transmission of data, notably audio and/or video data, or information computed remotely and linked to the alert request (for instance weather forecast from a distant weather forecast server).
  • the electronic device can include at least one communication interface 27 adapted to receive and/or transmit data via a wireless connection (notably of type WIFI® or Bluetooth®), and/or at least one wired communication interface 28.
  • a wireless connection notably of type WIFI® or Bluetooth®
  • the electronic device can include at least one sensor and acquire itself information regarding the monitored environment.
  • the electronic device 20 can include a power supply 29.
  • the power supply 29 is external to the electronic device 20.
  • the electronic device 20 can also include, or be connected to, a display module 23, for instance a screen, directly connected to the graphic card 22 by a dedicated bus 220.
  • a display module 23 for instance a screen, directly connected to the graphic card 22 by a dedicated bus 220.
  • the Input/ Output module 24 for instance an audio module, and/or a display module, can be used for instance in order to output information, as described with reference to the rendering steps of the method of the present disclosure described hereinafter.
  • Each of the mentioned memories can include at least one register.
  • a first example of register is a memory zone of low capacity (a few binary data).
  • a second example of register can be a memory of a higher capacity (with a capability of storage of an entire audio and/or video file notably).
  • the microprocessor 21 loads the program instructions 260 in a register of the RAM 26, notably the program instructions needed for performing at least one embodiment of the method described herein, and executes the program instructions.
  • the electronic device 20 includes several microprocessors.
  • microprocessor 21 can be configured for:
  • the monitoring can be performed only when the monitoring unit 110 of the communication system 100, or in a variant an alert rendering unit 150 (like a mobile and/or wearable device) on which an alert is to be rendered and/or a mobile device assumed to be in the vicinity of a user implied in the alert, is in a given location (called hereinafter “reference location” or “monitoring location or in a vicinity of this reference location).
  • reference location or “monitoring location or in a vicinity of this reference location
  • the term "recurrent” is herein to be understood as synonymous of the term “repeated”, the repetition being either periodic or not.
  • the period can vary between embodiments. For instance, monitoring can be performed on a daily and/or weekly base. The period can notably to configured by a user input.
  • the time interval of a monitoring can be of diverse duration.
  • the time interval can correspond to an entire day and thus the monitoring can be performed continuously or at least during some days of the week (like weekdays or weekends).
  • the monitoring can be performed after a user input, indicative of a reminder (or alert) request.
  • the alert request can comprise at least one indication of a time of day (simply called hereinafter time indication) explicitly included in the user input.
  • the alert request can also be assigned at least one time indication being a default value, like the current time of the alert request.
  • the time interval (or time period) during which a monitoring is performed can notably take into account the time indication.
  • a time interval of monitoring can begin before a time indication and/or end after a time indication (for instance two hours, one hour, half an hour, a quarter or five minutes before and/or after a time indication).
  • a time interval of monitoring can notably comprise one or several of the time indications.
  • the time indication can include a given hour, a given number of minutes and/or a given number of seconds (like 08:30:00 am).
  • the time interval can be centered around the time indication, like the time interval from 8 o'clock to 9 o'clock in the morning.
  • a first time indication and a second time indication can be defined (like 08:00:00 and 09:00:00).
  • the time interval can begin before the first time indication and can end after the second time indication (like a time interval from 7 o'clock to 10 o'clock or a time interval from 7.30 to 9.15 in the morning).
  • the time period can be a time interval beginning at a time close to the first time indication and ending at a time close to the second time indication (like the time interval from 8 o'clock to 9 o'clock in the morning).
  • the monitoring can occur during a first and a second time interval (like from 7.30 to 8.05 in the morning and from 8.30 am to 9.05 in the morning), each comprising one of the first and second time indication.
  • the beginning, the end and/or the duration of the time interval of monitoring can be configured by a user input, for instance by reference to the time indication.
  • the time interval for the monitoring can vary upon the time. For instance, it can depend of the variation of a time of detecting a match with the pattern of events. For instance, if a match with the pattern of events is always detected at the same time, the size of the time interval of the monitoring can be reduced over the time. Conversely, if the time of detecting a matching with the pattern of events vary over the days, the size of the time interval of monitoring can be increased.
  • Figures 3A and 3B illustrates embodiments of the method of the present disclosure.
  • the embodiments can be implemented by logical blocks (or modules) of the communication system 100 illustrated by figure 1 .
  • the illustrated logical blocks are introduced hereinafter in connection with the embodiments illustrated of figure 3A and 3B .
  • Figure 3A describes a first exemplary embodiment of the method of the present disclosure where a preconfigured reminder (or in other words an alert) is generated in case of a match between a known pattern of events and the events currently detected by monitoring while a device of the communication system is in a given location.
  • the reminder is generated even if the time of the match is different from the time indication associated with the alarm.
  • a first use case of this exemplary embodiment is a user wanting to be reminded of taking the key of his/her car, or his/her sun glasses before leaving home.
  • the user can set an alarm (or make an alert request) at the time scheduled for leaving home before driving the children to school (e.g. at 8.00 am).
  • opening and closing of doors, at home, shortly before the alarm time can be a good indicator that the activity "leaving home" is happening.
  • the opening/closing of doors can also happen earlier, e.g. 7.30 in the morning. In this case, it can be a good indicator that the activity is performed in advance and thus that an alarm needs to be generated in advance.
  • an alarm is requested with a generation delay, in order to be generated after a time interval following the end of the activity.
  • the alarm "take your keys” can be generated 10 minutes after a sensor has detected an opening and closing of a dish washing machine.
  • a second use case is a policeman who wants to regulate traffic in front of a football pitch where weekly training sessions are organized for kids, in view of protecting kids when crossing the street in front of the football pitch.
  • the policeman When the policeman is working (and thus is located in the area of the football pitch), he wants to be alerted of the effective ending time of the training session, which can be associated to a pattern of events including the sound of a final whistle, of kids screaming, a victory song, ....
  • the reminder can for instance take the form of a textual message ("Match finished”) and/or sound messages (like beeps) rendered on his/her connected watch.
  • the method 300 can comprise registering 310 an alert upon receiving an alert request.
  • the alert request can notably be entered through a user interface 140, or received through a communication interface (for instance from a remote device (like a user's smartphone) using a Wi-Fi communication interface (as introduced in link with the device 20 of figure 2 ).
  • the registering 310 can notably include obtaining and/or storing at least one monitoring condition, that is to be fulfilled for a monitoring to occur, (like at least one indication of a time of day and/or one time interval of monitoring and/or at least one reference location).
  • a location used as a monitoring condition can be the current location of the monitoring unit 110 and/or of a user at the time of the registering of the alert (when the device is connected with a camera and/or a GPS for instance and/or is using some wireless transmission techniques that permit to localize a device).
  • the reference location to be used as a monitoring condition can be acquired explicitly from a user interface.
  • the reference location can be expressed as GPS coordinates, by using Wi-Fi-based localization technique where a position of mobile devices inside a house can be localized by Wi-Fi signal in the house, as a position on a map (for instance a position selected by a user on an interactive map).
  • the registering can also comprise obtaining (and/or storing) rendering information related to the way the alert being registered will be output (like a designation of a device being an intended receiver of an alert message, an alert element to be rendered (for instance a content of an alert message), and other information that would be obvious to the one of skill in the art.)
  • a user can set (or provide) via a user's interface 140 of the communication system 100 of figure 1 , rendering information (that will be used for rendering an alarm on a user interface 152 of a rendering unit 150.
  • the user interfaces used for registering an alarm and for generating an alarm can be different (and notably can belong to two distinct devices).
  • the way the alert will be rendered and the identification of at least one rendering unit 150 that will render the alert can be acquired from the user interface 140.
  • an alert element like a sound and/or a vibration to be rendered for alerting a user, an identification of the device 150 that will render the sound or vibration, can be obtained from the user interface 140.
  • the communication system 100 can also obtain a textual label to be rendered for alerting a user (like 'Do not forget your keys'), the label being entered using a keypad or a touchscreen of the user interface 140, or a voice message acquired via a microphone connected to the user interface 140.
  • a user can provide an audiovisual content (like a still image or a video) or a link to such an audiovisual content, to be played on a display of the device 150 during the alert.
  • a user can provide a keyword to the communication system, that will be used by the communication system for obtaining an alerting element (for instance by accessing a storage unit 160 of the communication system for retrieving an image, to be played when rendering the alert on a user interface (Ul) 152 of the rendering unit 150.
  • a speech can be acquired from at least one microphone of the user interface 140 and can then be processed by a computation machine of the communication system, using a speech-to-text algorithm (like a keyword spotting algorithm) for obtaining at least one keyword (like "Sunglasses").
  • One or more visual contents, associated with the keywords, can then be retrieved from a storage unit (like a database) at the time of the alert request or upon generating the alert.
  • a storage unit like a database
  • the several visual contents can be, at least partially, rendered on a user interface during the alert request step, for at least one of the rendered contents to be selected by a user (via a touchscreen or a keypad or via a microphone) and thus be associated with the alert for being rendered later at the time of the alarm.
  • an alert request can be a request for a conditional alert.
  • a generation of such an alert can be performed conditionally, depending on another event.
  • the generation of the alert can be omitted or modified depending on this other event.
  • a user can request to be alerted about not forgetting his/her umbrella depending on the weather.
  • the alert can be generated only if the weather is rainy (e.g. "Take your keys") or the rendering of the alert can differ depending on the weather (e.g. being either "Take your keys” or "Take your keys and your umbrella”).
  • conditional, adaptive, alert can be considered as useful by a user. Indeed, as some events (like rain) are only happening from time to time, it can be considered as helpful for a user of being alerted when such a conditional event occurs, in order to behave accordingly.
  • the method can comprise checking 320 if monitoring conditions related to at least one alert are fulfilled.
  • the checking can notably include checking if the current time belongs to a time interval of monitoring and if the location tracked device is in the reference location).
  • the checking can also include in some embodiments checking if at least one condition for the conditional alert is fulfilled (e.g. checking the weather forecast to see if it is actually rainy at a time interval of monitoring).
  • the checking can notably be performed periodically, or by setting a timer ending at the beginning of a time interval of monitoring.
  • the method can thus comprise monitoring 330 events.
  • the monitoring 330 can be performed for learning a pattern of events to be tracked (as being characteristic of the activity to monitor) and/or for tracking the learned pattern of events in the events collected during the monitoring.
  • a learning module 122 of the machine learning unit 120 of the communication system can start to record and learn the user or user's family or user's house events that happen during the time interval of monitoring, so that it is able to learn a model of what characterizes the user's activity during this time interval and at the given location.
  • the monitoring 330 can notably comprise obtaining 332 events from at least one sensor 101, 102 and/or from a communication interface 105 (like a weather or/and traffic forecast for instance).
  • a sensor is to be understood as a module adapted to detect at least one kind of event or information in its physical environment, and encompasses many types of sensor, including electronic or analog sensors.
  • sensors include an audio sensor like a microphone, a visual sensor like a webcam, a tactile sensor, a temperature sensor, a pressure sensor, a force-sensing resistor, potentiometers, a light sensor, a motion sensor, a magnetic and/or electric fields sensor, an accelerometer, a gravity sensor, a humidity sensor, a moisture sensor, a vibration sensor, a positioning sensor.
  • a positioning sensor (like a GPS module) can notably be used for providing a current location of a user and/or of location tracked device.
  • a sensor can be located in a vicinity of the reference location specified by the alarm request, or in the environment of a mobile device (that is likely to be sometimes located at the reference location specified by the alarm request). For instance, sensors can be located in different places around the reference location specified at the alarm request (e.g., when the reference location specified by the alarm request is the house of a user, it can be anywhere in the house, it can even be anywhere outside the house (like in case of humidity sensor for rain detection).
  • a "sensor” is considered herein, for simplicity purposes, as adapted to detect events that can then be used directly by the machine learning unit 120 for learning and/or tracking a pattern of events.
  • the detected events can be used by a learning module 122 of the machine learning unit 120 for learning a pattern of events, as detailed hereinafter, and/or by an inference module 124 of the machine learning unit 120 for tracking a pattern of events in a sequence of detected events.
  • a sensor can generate signals that can be used (alone or in combination with other signals from other sensors for instance) to create an event representative information usable by the learning module 122 and/or the inference module 124 of the machine learning unit 120 for learning and/or tracking a pattern of events (or a pattern of event representative information).
  • the recording of events for the above exemplary alarm can be performed notably thanks to sensors attached, for example, to the different doors in the house, and capable of detecting their opening and closing.
  • the monitoring unit 110 of the communication system 100 can also obtain events from audio sensors, like sounds of doors closing/opening.
  • the monitoring unit 110 can also obtain voices and their spatial origin in the house. Such audio events can permit to determine the location of a person speaking (notably, in the first exemplary use case, whether or not the different people are situated close to the main entrance).
  • processing of voice by a speech-to-text algorithm can permit to obtain key words (such as 'let's go', 'time to go', etc.) that can be used for the pattern of events.
  • Voice recognition techniques can also permit to identify speakers.
  • the monitoring unit 110 can also obtain visual events from a camera, or a webcam (like a person picking up his/her handbag and/or the children picking up their schoolbags).
  • the monitoring 330 can also comprise transmitting obtained events to a module, like the machine learning unit120 of figure 1 , in charge of learning a pattern of events and/or of checking a matching with a pattern of events associated to the alert.
  • the machine learning unit 120 can be part of the same device as the monitoring unit 110 or can be part of another device, like a remote server comprising large computing capabilities.
  • the monitoring 330 can be performed with the help of a monitoring unit 110, which obtains events from sensors, and transmits the obtained events (or information representative of the obtained events) to an inference module 124 of a machine learning unit 120, for checking about a matching with a learned model (also called herein "pattern of events"), dedicated of the registered alert, constructed (and update) at least partially by a learning module 122 of the machine learning unit.
  • the machine learning unit 120 can notably make use of machine learning techniques, such as for instance deep neural network, regression techniques, support vector machine, random forest, etc ... in its learning of inference modules...
  • the machine learning unit 120 can comprise or be connected to a Deep Neural Network.
  • the machine learning unit can also have been trained, in advance, in a preliminary step (for instance prior of being used for implementing the method of the present disclosure)
  • the training can notably depend of the type of sensors present in the communication system (on thus of the type of events that are detectable). For instance, when the communication system comprises a visual sensor, the machine learning unit can be trained on annotated images. Similarly, when the device is connected to an audio (or sound) sensor, the machine learning unit can be trained on annotated audio samples. When more than one kind of sensors (like audio and visual sensors) are used, the machine learning unit can be trained on annotated multimodal samples.
  • the preliminary training of the machine learning unit can be optional in some embodiments.
  • the machine learning unit can further be trained specifically on an activity to be monitored.
  • the method can comprise generating 340 an alert. (For ease of understanding, the way a pattern of events is learned by the communication system is detailed later).
  • the monitoring is performed during a time interval, it can happen that a match is detected before the alarm time indication (or, in other words, in advance compared to the alarm time indication), or after, or later than, the alarm time indication.
  • the alert can be generated as soon as the match is detected (taking into account a delay introduced by a processing time).
  • detecting activities similar to a learned pattern of events at a time close to the alarm time indication may mean that the associated activity is performed in advance of or later than its scheduled time (that is, the time indication of the alarm). For instance, in the first exemplary use case, the user is leaving home in advance or later than usual. Consequently, the alarm has to be raised at a time different from the time indication registered for the alarm and used as a reference time.
  • similarities between at least a part of the monitored sequence of events and a learned pattern of events may be computed for determining a probability for the current monitored sequence of events to correspond to the learned pattern of events.
  • the alert can be generated (or raised) when a probability of matching, between one or several detected events and the pattern of events, reaches a first value (for instance a first value used as a threshold).
  • a first value for instance a first value used as a threshold.
  • an alert can be generated, notably, before the ending of an activity, if the already occurred events permit to reach this first value.
  • determining the probability can also take into account a time difference between the (scheduled) time indication of the alert and the time where similarities are detected. Such an embodiment can permit to put less weight to similarities, when the time difference is large than when the time difference is low (and thus, by decreasing or increasing a probability of matching, to eventually delay (or even disable) or advance a generation of an alert upon detecting similarities if the time difference is important).
  • the time difference between the (scheduled) time indication of the alert and the time where similarities are detected can be ignored in the determining of a probability of a match.
  • the impact of the time difference between the time indication of the alert and the time where similarities are detected can vary over the time, so as to decrease when a confidence of the communication system in the reliability of the learned pattern of events increases and vice-versa.
  • a probability of presence can be assigned to at least one event of the pattern of events.
  • the assigning of the probability can notably take into account a detecting of a same event during at least one previous monitoring. Determining a match can then be performed by taking into account a probability of presence, in the detected events, of at least one of the events of the pattern of events. Indeed, if an event of the pattern of events has a low probability of presence, a match can be assumed even if the event having the low probability is not captured during the monitoring. Conversely, in some embodiments, the system will not conclude to a match while an event having a very high probability of presence in the pattern is not captured during the monitoring,
  • the generating can comprise a rendering of the alert on a user interface (like the user interface 152 of the alert rendering unit 150 of the communication system 100) and/or a sending of information related to the alert to at least another device.
  • the other device can be located for instance in the vicinity of the monitoring unit, like a tablet, a TV screen, and/or a loudspeaker located in a same house. Notably, the other device can a loudspeaker located at the house entrance).
  • the other device can also be a known device like a user's smartphone, tablet, and/or personal computer. It can also be located remotely.
  • the method includes getting 342 at least one feedback regarding at least one generated alert.
  • a generated alert can be assessed as being a "true” or "false” alert by means of a user interface.
  • the feedback can be used as illustrated by figure 1 for transmitting an indication representative of a relevance of the generated alert from the user interface 152 of the alert rendering unit 150 to the machine learning unit 120 in charge of the learning of the reference pattern of events.
  • the method can thus comprise updating 344 the reference pattern of events, if needed.
  • getting feedback can be optional and/or can be performed only upon a user request (for instance because of a false alarm).
  • a feedback about an alarm is provided (or transmitted) to the machine learning unit, at least some of the events that have led to the generation of the alarm can be used by the machine learning unit for fine-tuning (and thus, if needed, updating 344) the pattern of events (by being considered as positive or negative examples for the learning, for instance).
  • a generation and/or a rendering of an alert can be altered depending upon an additional contextual information.
  • the alarm generation unit 130 of the communication system can take into account additional contextual information when a match is assumed by the machine learning unit 120. For instance, when a match with a pattern of events associated with the user leaving home is assumed, if the alarm generation unit 130 obtains information representative of the weather forecast, predicting a sunny day, the alarm generation unit can disable the alarm 'Take your umbrella'.
  • the alarm generating unit 130 can modify a rendering of an alert regarding the objects (keys etc..) to be taken when leaving home, in order to exclude a reminder related to rain. For instance, the alert can be rendered with an image of a key and without an image of an umbrella.
  • an additional contextual information can be taken into account by the machine learning unit 120.
  • the machine learning unit can obtain a weather forecast predicting a sunny day and can infer that a user will not need his umbrella and consequently will not conclude to a matching with a sequence of events including an event of weather forecast type predicting a rainy day.
  • the way a pattern of events, associated with a newly registered alert, is learned by the machine learning unit can vary upon embodiments.
  • an initial pattern of events can be learned from one or several "initial" monitorings.
  • the method upon registering an alert, the method can comprise requesting a user to perform the activity to which the alert is associated.
  • one of the objectives of the pattern of events is to be representative of this activity.
  • the user can simulate his/her leaving home (by playing his/her own role when leaving home).
  • Monitoring the events that occurs while the user behaves as he/she usually does during the activity can permit to acquire a first, rough, "initial" pattern of events, that can later be refined by successive monitoring (and, optionally, by taking into account a user feedback regarding the relevance of the generated alerts).
  • the machine learning unit may then need several monitorings (and thus several days) to learn a more accurate pattern of events representative of an activity associated with an alarm using a user's feedback, for instance, and/or by taking account of the repeatability of events.
  • the training of the machine learning unit can be done by monitoring an activity when the current time is in the monitoring time interval for the first time after registering the alarm, the learned model of the activity being refined over the next monitoring of the activity, each time the current time is in the monitoring time interval.
  • the communication system can generate the alarm systematically when the current time reaches the time indication associated to the alarm (for instance each day, each weekday, and/or each Monday at 8:00 am), while monitoring events during the time interval of monitoring associated with the alarm. This can be performed similarly only for the first monitoring or during the monitoring performed for n first days.
  • the monitoring(s), being the first, or the n first that occur after the registering of a new alert, and being used for training the learning module of the machine unit can be qualified as "initial" monitoring(s). Once the pattern of events is assumed to be learned, the alarm can then be generating according to a matching with the pattern of events.
  • the model, or pattern, of events can be assumed to be learned after a certain number of monitorings. Notably, this number of monitorings can be a constant number, being the same for all new alarm requests. In some embodiments, the model, once learned, can be kept unchanged. Such an embodiment can be adapted to a device with low storage and processing capabilities as no update of the model is needed.
  • the pattern of events can be refined automatically at each monitoring, periodically, or depending on the result of a monitoring, so that it can become more reliable (and for instance take into account small changes occurring in the user's activity and/or environment).
  • the refinement can be done by fine-tuning the model parameters with the new collected data.
  • a confidence factor representative of a confidence of the communication system of the reliability the learned pattern of events.
  • the confidence factor can notably be function of a number of monitorings (or learning) performed since the alarm request, or be a function of an amount of data, related to event representative information, available to the machine learning unit (or of their quality) or be a function of the model accuracy based on some validation data set (for instance a user's feedback regarding the relevance of the alerts generated because of detected matching).
  • Validation data can comprise at least one sequence of events that correspond to the activity to detect and that therefore should generate an alarm, and/or at least one sequence of events that does not correspond to the activity to detect and therefore that should not generate an alarm.
  • applying such validation data on the model used by the machine learning unit for obtaining a pattern of events representative of the activity and checking the model behavior with the expected answer can allow to check the performance of the learned pattern.
  • the confidence factor can notably be used for determining a probability of matching (notably as a complement of the weight attributed to the time difference between the time indication and the effective time of matching as explained above).
  • the machine learning unit does not need to identify individually, in the detected events, each event comprised in a learned pattern.
  • the detected events can be compared globally to the learned pattern.
  • a match can be found even if some events comprised in the pattern of events are not part of a sequence of monitored events.
  • a match can be detected (and thus an alarm being generated) before the user has finished performing an activity entirely.
  • a reminder about taking his//her key can be rendered after the user puts his/her coat but before the user has taken his/her handbag.
  • Figure 3B describes a second exemplary embodiment of the method of the present disclosure that proposes to generate an alert when a difference (or deviation) between the learned pattern and the current monitored events is detected.
  • a user that usually decreases the heating temperature before going out can be reminded of decreasing the heat when he/she leaves the home after having forgot to decrease the heating.
  • the method 350 can include, similarly to the method illustrated by figure 3A , receiving an alert request, for instance from the alert registering user interface 140, registering 310 the alert request (and the corresponding monitoring conditions), and checking 320 the monitoring conditions. If a monitoring is needed 422, the method can comprise monitoring 360 events.
  • the monitoring 360 can differ from the monitoring 330 performed in the embodiment of figure 3A . Indeed, different from some of the embodiments illustrated by figure 3A , in the embodiments illustrated by figure 3B , the machine learning unit may need to identify individual events comprised in a learned pattern in order to be able to detect a deviation between at least one "deviating" event of a detected sequence of events and/or at least one event of a reference pattern of events.
  • a deviating event can be an event of the pattern of events being absent (or omitted) or modified in the detected sequence.
  • the detecting of a deviating event can be performed at a per group of events bases, events from a same group being processed separately (via a specific pattern of events), or more globally via a global pattern of events.
  • the monitoring 360 can include obtaining at least one event (or signal) 361 and comparing 362 the obtained event to a learned global/and/or specific model (or pattern). For instance, detected audio signals can be compared to an acoustic event model, and/or detected audio and/or video signals can be compared by a human activity recognition model.
  • the communication system 100 can comprise a single machine learning unit 120 in charge of the learning and/or tracking of a global and/or to all specific reference pattern(s) of events, or comprise different machine learning units, each being in charge of learning and/or tracking of a specific reference pattern of events.
  • the global or specific pattern(s) of events used in the comparing can have been learned in advance in a training phase (for instance similarly to what have been described in connection with figure 3A ).
  • the method can include getting feedback 3652 from a user, and updating 3654 the reference global and/or specific pattern of events based on actual data collected during the current monitoring.
  • the feedback of a user, and the collecting of the current or more accurate data can permit to refine the reference global and/or specific pattern of events. For instance, it can permit to include some alternative events or to ignore some non-significant change in the pattern of events.
  • the method 350 can include checking a matching 363 of the detected sequence of events.
  • the checking can notably be based on the output of the comparing 362.
  • matching 363 can be assumed when a first sequence (or first plurality) of events matches the corresponding first specific model pattern of events (i.e. when a positive result is output by comparing the first sequence and the first pattern of events), even if a second sequence of events obtained leads to a negative result when compared to the second specific model.
  • a match can be assessed by fusing the outputs of the comparing performed for several (for instance all) specific models.
  • the matching 363 can be assessed because the monitored events include either identical or almost identical events to those of the global and/or specific learned pattern.
  • the method can comprise generating an alert (or in other words alerting) 365 about the event (omitted or modified) that causes the detected deviation.
  • the chronological order of events inside a sequence and/or the pattern can be taken into account for determining a matching between the sequence and the pattern, while in other embodiments, the chronological order can be ignored for determining the matching.
  • the generating 365 of an alert can be performed similarly to what have been described in connection with figure 3A .
  • a single and/or a plurality of feedbacks can be obtained for one or several generated alarm(s).
  • a user can be requested to enter feedback for each generated alarm, or can provide feedback only on his own initiative, for only one, some of, or all generated alarms.
  • the got (or received) feedback can be used for updating 3654 the global and/or specific pattern of events used for detecting a missing and/or modified event and/or for updating 3654 the global and/or specific pattern of events. It can also be used for instance for modifying a weight associated to a result of a comparing with a specific pattern and used during the checking of the matching.
  • aspects of the present principles can be embodied as a system, method, or computer readable medium. Accordingly, aspects of the present disclosure can take the form of a hardware embodiment, a software embodiment (including firmware, resident software, micro-code, and so forth), or an embodiment combining software and hardware aspects that can all generally be referred to herein as a "circuit", module" or "system”. Furthermore, aspects of the present principles can take the form of a computer readable storage medium. Any combination of one or more computer readable storage medium(s) may be utilized.
  • a computer readable storage medium can take the form of a computer readable program product embodied in one or more computer readable medium(s) and having computer readable program code embodied thereon that is executable by a computer.
  • a computer readable storage medium as used herein is considered a non-transitory storage medium given the inherent capability to store the information therein as well as the inherent capability to provide retrieval of the information therefrom.
  • a computer readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing.

Landscapes

  • Business, Economics & Management (AREA)
  • Emergency Management (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Alarm Systems (AREA)
EP17306659.8A 2017-11-30 2017-11-30 Procédé de génération d'alertes et dispositif électronique correspondant, système de communication, produits- programme lisibles par ordinateur et support d'informations lisible par ordinateur Withdrawn EP3493173A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP17306659.8A EP3493173A1 (fr) 2017-11-30 2017-11-30 Procédé de génération d'alertes et dispositif électronique correspondant, système de communication, produits- programme lisibles par ordinateur et support d'informations lisible par ordinateur

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP17306659.8A EP3493173A1 (fr) 2017-11-30 2017-11-30 Procédé de génération d'alertes et dispositif électronique correspondant, système de communication, produits- programme lisibles par ordinateur et support d'informations lisible par ordinateur

Publications (1)

Publication Number Publication Date
EP3493173A1 true EP3493173A1 (fr) 2019-06-05

Family

ID=60661878

Family Applications (1)

Application Number Title Priority Date Filing Date
EP17306659.8A Withdrawn EP3493173A1 (fr) 2017-11-30 2017-11-30 Procédé de génération d'alertes et dispositif électronique correspondant, système de communication, produits- programme lisibles par ordinateur et support d'informations lisible par ordinateur

Country Status (1)

Country Link
EP (1) EP3493173A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6989753B1 (en) * 2003-12-12 2006-01-24 Hewlett-Packard Development Company, L.P. Method of and computer for identifying reminder event
WO2013072774A2 (fr) * 2011-11-14 2013-05-23 Yougetitback Limited Systèmes et procédés de récupération de dispositifs de faible puissance
US20160063837A1 (en) * 2014-09-03 2016-03-03 Persia, Inc. Verifying and monitoring stove operation

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6989753B1 (en) * 2003-12-12 2006-01-24 Hewlett-Packard Development Company, L.P. Method of and computer for identifying reminder event
WO2013072774A2 (fr) * 2011-11-14 2013-05-23 Yougetitback Limited Systèmes et procédés de récupération de dispositifs de faible puissance
US20160063837A1 (en) * 2014-09-03 2016-03-03 Persia, Inc. Verifying and monitoring stove operation

Similar Documents

Publication Publication Date Title
US10467509B2 (en) Computationally-efficient human-identifying smart assistant computer
US11687319B2 (en) Speech recognition method and apparatus with activation word based on operating environment of the apparatus
WO2021180062A1 (fr) Procédé d'identification d'intention et dispositif électronique
US11822859B2 (en) Self-learning digital assistant
CN107784357B (zh) 基于多模态深度神经网络的个性化智能唤醒***及方法
US10031491B2 (en) Adaptive sensor data selection and sampling based on current and future context
CN109643158B (zh) 使用多模态信号分析进行命令处理
US20210234723A1 (en) Machine learning for home understanding and notification
US20200118010A1 (en) System and method for providing content based on knowledge graph
US11217246B2 (en) Communication robot and method for operating the same
US11710481B2 (en) Electronic device and method for providing conversational service
KR20230066300A (ko) 전자 장치 및 그 동작 방법
US20190297381A1 (en) Artificial intelligence device and operating method thereof
EP3756185B1 (fr) Dispositif électronique permettant de générer une réponse à une entrée vocale à l'aide d'une application, et procédé de fonctionnement associé
CN108351870A (zh) 根据活动模式的计算机语音识别和语义理解
KR20200052448A (ko) 지식 그래프에 기초하여 데이터베이스들을 통합하는 시스템 및 방법
JP6254726B1 (ja) 生成装置、生成方法、及び生成プログラム
US10614693B2 (en) Dangerous situation notification apparatus and method
KR102420567B1 (ko) 음성 인식 장치 및 방법
US11755924B2 (en) Machine learning for home understanding and notification
KR20190105182A (ko) 전자 장치 및 그 제어 방법
KR20200085143A (ko) 외부 장치를 등록하는 대화형 제어 시스템 및 방법
US20180349962A1 (en) System and method for using electromagnetic noise signal-based predictive analytics for digital advertising
KR20190127513A (ko) 음성 입력 인증 디바이스 및 그 방법
KR20180072534A (ko) 텍스트와 연관된 이미지 제공 방법 및 이를 위한 전자 장치

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20191206