WO2012008961A1 - A user analytics engine for detecting and mitigating stress and/or concentration lapses for an associated user of an electronic device - Google Patents

A user analytics engine for detecting and mitigating stress and/or concentration lapses for an associated user of an electronic device Download PDF

Info

Publication number
WO2012008961A1
WO2012008961A1 PCT/US2010/042088 US2010042088W WO2012008961A1 WO 2012008961 A1 WO2012008961 A1 WO 2012008961A1 US 2010042088 W US2010042088 W US 2010042088W WO 2012008961 A1 WO2012008961 A1 WO 2012008961A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
electronic device
concentration
stress
lapse
Prior art date
Application number
PCT/US2010/042088
Other languages
French (fr)
Inventor
Srinivas Annambhotla
Siva Mupparaju
Original Assignee
Sony Ericsson Mobile Communications Ab
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Ericsson Mobile Communications Ab filed Critical Sony Ericsson Mobile Communications Ab
Priority to PCT/US2010/042088 priority Critical patent/WO2012008961A1/en
Publication of WO2012008961A1 publication Critical patent/WO2012008961A1/en

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/48—Other medical applications
    • A61B5/4803—Speech analysis specially adapted for diagnostic purposes
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M21/00—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis
    • A61M21/02—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis for inducing sleep or relaxation, e.g. by direct nerve stimulation, hypnosis, analgesia
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0002—Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network
    • A61B5/0015—Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network characterised by features of the telemetry system
    • A61B5/0022—Monitoring a patient using a global network, e.g. telephone networks, internet
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6887—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient mounted on external non-worn devices, e.g. non-medical devices
    • A61B5/6897—Computer input devices, e.g. mice or keyboards
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6887—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient mounted on external non-worn devices, e.g. non-medical devices
    • A61B5/6898—Portable consumer electronic devices, e.g. music players, telephones, tablet computers
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M21/00—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis
    • A61M2021/0005—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis by the use of a particular sense, or stimulus
    • A61M2021/0027—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis by the use of a particular sense, or stimulus by the hearing sense
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M21/00—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis
    • A61M2021/0005—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis by the use of a particular sense, or stimulus
    • A61M2021/0044—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis by the use of a particular sense, or stimulus by the sight sense
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M2205/00—General characteristics of the apparatus
    • A61M2205/35—Communication
    • A61M2205/3546—Range
    • A61M2205/3553—Range remote, e.g. between patient's home and doctor's office
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M2205/00—General characteristics of the apparatus
    • A61M2205/35—Communication
    • A61M2205/3576—Communication with non implanted data transmission devices, e.g. using external transmitter or receiver
    • A61M2205/3584—Communication with non implanted data transmission devices, e.g. using external transmitter or receiver using modem, internet or bluetooth
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M2205/00—General characteristics of the apparatus
    • A61M2205/50—General characteristics of the apparatus with microprocessors or computers
    • A61M2205/502—User interfaces, e.g. screens or keyboards
    • A61M2205/505—Touch-screens; Virtual keyboard or keypads; Virtual buttons; Soft keys; Mouse touches
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M2205/00—General characteristics of the apparatus
    • A61M2205/50—General characteristics of the apparatus with microprocessors or computers
    • A61M2205/52—General characteristics of the apparatus with microprocessors or computers with memories providing a history of measured variating parameters of apparatus or patient
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M2230/00—Measuring parameters of the user
    • A61M2230/04—Heartbeat characteristics, e.g. ECG, blood pressure modulation
    • A61M2230/06—Heartbeat rate only
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M2230/00—Measuring parameters of the user
    • A61M2230/08—Other bio-electrical signals
    • A61M2230/10—Electroencephalographic signals
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M2230/00—Measuring parameters of the user
    • A61M2230/30—Blood pressure
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M2230/00—Measuring parameters of the user
    • A61M2230/40—Respiratory characteristics
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M2230/00—Measuring parameters of the user
    • A61M2230/63—Motion, e.g. physical activity

Definitions

  • TITLE A USER ANALYTICS ENGINE FOR DETECTING AND
  • the technology of the present disclosure relates generally to utilizing user analytics for detecting and mitigating user stress and/or concentration lapses and, more particularly, to a system and method for taking a responsive action to mitigate detected user stress and/or concentration lapse by correlating current user performance with past user performance.
  • the present invention overcomes this problem by providing an in-feature/in-application mechanism to infer stress or concentration lapses and provide multimedia content to assist the user in mitigating the detected stress and/or concentration lapse.
  • the present disclosure describes an improved system and method of taking responsive action to a detected level of stress and/or concentration lapse.
  • a number of specific exemplary situations in which the disclosed systems and methods may be used are described. It will be appreciated that the disclosed systems and methods may be used in a wide variety of situations other than these specific examples.
  • One aspect of the invention relates to a method for mitigating stress and/or concentration lapses of a user of an electronic device, the method including: a) collecting information related an associated user's use of an electronic device; b) processing the information; and c) determining if the user is under stress and/or experiencing a concentration lapse based upon the collected information; and d) rendering multimedia content to mitigate the determined stress and/or concentration lapse.
  • Another aspect of the invention relates to the information collected relating to at least one user characteristic associated with the user's use of the electronic device to perform one or more tasks with the electronic device.
  • Another aspect of the invention relates to the information collected relating to user input from a user input device coupled to the electronic device.
  • Another aspect of the invention relates to the information being processed to determine if a user input error was received by the electronic device.
  • Another aspect of the invention relates to the user input error being identified by receiving user input consisting of at least one selected from a group consisting of a delete key and/or a backspace key.
  • Another aspect of the invention relates to information collected relating to user performance in games executed by the electronic device.
  • Another aspect of the invention relates to information collected relating to a volume in which the associated user speaks while using the electronic device.
  • Another aspect of the invention relates to the step of determining if the user is under stress and/or experiencing a concentration lapse including comparing the collected information with baseline information associated with the user.
  • Another aspect of the invention relates to the baseline information being determined by taking an average of the collected information during one or more training periods.
  • Another aspect of the invention relates to the baseline information being determined by taking an average of the collected information during normal operation of the electronic device.
  • Another aspect of the invention relates to prior to rendering multimedia content to mitigate the detected stress and/or the lapse of concentration, the user is prompted by a message presented on a display of the electronic device to determine if the user would like to view multimedia content to mitigate stress and/or lapse of concentration.
  • Another aspect of the invention relates to the plurality of multimedia content selections that may appeal to the associated user are stored in a storage device of the electronic device.
  • One aspect of the invention relates to an electronic device including: a memory for storing information about an associated user's use of the electronic device; a controller coupled to the memory, wherein the controller processes the information in a
  • a rendering device coupled to the processor for rendering multimedia content to the associated user in order to mitigate the determined stress and/or concentration lapse of the associated user.
  • Another aspect of the invention relates to the electronic device being a mobile telephone.
  • Another aspect of the invention relates to the memory including one or more baseline metric associated with the user's use of the electronic equipment, wherein the baseline metric includes at least one user characteristic associated with the user's use of the electronic device to perform one or more tasks with the electronic device.
  • Another aspect of the invention relates to the memory including a plurality of multimedia content sources that appeals to the user.
  • Another aspect of the invention relates to the rendering device being at least one selected from the group consisting of a display and/or a speaker.
  • One aspect of the invention relates to a program stored on a machine usable medium for taking action in response to detected stress and/or lapse of concentration of an associated user of an electronic device, the program comprising executable logic to: establish at least one baseline metric for an associated user, wherein the baseline metric corresponds to the associated user's use of the electronic device to perform one or more tasks with an electronic device; monitoring at least one user characteristic related to the associated user's use of the electronic device to perform the one or more tasks, wherein the monitored user characteristic is related to the at least one baseline metric; and comparing the monitored at least one user characteristic with the baseline metric to determine at least one of a detected stress and/or a lapse of concentration; and rendering multimedia content on a display of the electronic device to mitigate the detected stress and/or the lapse of concentration when the detected stress and/or the lapse of
  • concentration is above a predetermined threshold.
  • Another aspect of the invention relates to logic to drive the display to display one or more alternative sources of multimedia content when the detected stress and/or the lapse of concentration is above a predetermined threshold.
  • FIG. 1 is a schematic view of an exemplary system for taking responsive action to detected stress and/or lapse of concentration.
  • FIGs. 2-3 are flow charts representing exemplary methods for implementing an analytics engine in accordance with aspects of the present invention.
  • a portable radio communications device such as the illustrated mobile telephone. It will be appreciated, however, that the exemplary context of a mobile telephone is not the only operational environment in which aspects of the disclosed systems and methods may be used. Therefore, the techniques described in this document may be applied to any type of appropriate electronic device, primary examples of which are a mobile telephone and a computer, such as a laptop computer or a desktop computer. But other examples include, without limitation, a media player, a gaming device, an electronic organizer, a personal digital assistant (PDA), etc.
  • PDA personal digital assistant
  • a system for taking responsive action to detected stress and/or concentration lapse includes an electronic device 10.
  • the electronic device 10 includes an analytics engine 12 that is operable to infer stress and/or concentration lapse by correlating current user performance with past user performance.
  • the analytics engine is configured to analyze user performance while the user is using various features and/or applications on the electronic device 10 and correlate the current performance with past performance. Based on this correlation, inferences may be made about the stress and/or concentration levels of the user and one or more mitigation techniques (e.g., rendering multimedia content) may be suggested to the user of the electronic device 10 to mitigate the inferred stress and/or concentration lapse. Additional details and operation of the analytics engine 12 will be described in greater detail below.
  • the analytics engine 12 may be embodied as executable code that is resident in and executed by the electronic device 10.
  • the analytics engine 12 may be one or more programs that are stored on a computer or machine readable medium.
  • the analytics engine 12 may be a stand-alone software application or form a part of a software application that carries out additional tasks related to the electronic device 10.
  • exemplary techniques for taking action in response to detected stress and/or concentration lapse are described. It will be appreciated that through the description of the exemplary techniques, a description of steps that may be carried out in part by executing software is described. The described steps are the foundation from which a programmer of ordinary skill in the art may write code to implement the described functionality. As such, a computer program listing is omitted for the sake of brevity. However, the described steps may be considered a method that the corresponding device is configured to carry out. Also, while the analytics engine 12 is implemented in software in accordance with an embodiment, such functionality could also be carried out via dedicated hardware or firmware, or some combination of hardware, firmware and/or software.
  • the electronic device of the illustrated embodiment is a mobile telephone, but will be referred to as the electronic device 10.
  • the electronic device 10 may be a device other than a mobile telephone.
  • the electronic device 10 may include a display 14.
  • the display 14 (also referred to herein as a rendering device) presents information to a user such as operating state, time, telephone numbers, contact information, various menus, etc., that enable the user to utilize the various features of the electronic device 10.
  • the display 14 also may be used to visually display content received by the electronic device 10 and/or retrieved from a memory 16 of the electronic device 10.
  • the display 14 may be used to present images, video and other graphics to the user, such as photographs, mobile television content, Internet pages, and video associated with games. As described below, upon detection of stress and/or concentration lapse, the display may present multimedia content to alleviate the detected stress and/or concentration lapse.
  • the multimedia content may be stored at a remote server (e.g., server 26) and/or stored locally in memory 16, for example.
  • a keypad 18 provides for a variety of user input operations.
  • the keypad 18 may include alphanumeric keys for allowing entry of alphanumeric information (e.g., telephone numbers, phone lists, contact information, notes, text, etc.), special function keys (e.g., a call send and answer key, multimedia playback control keys, a camera shutter button, etc.), navigation and select keys or a pointing device, and so forth. Keys or key-like functionality also may be embodied as a touch screen associated with the display 18. Also, the display 14 and keypad 18 may be used in conjunction with one another to implement soft key functionality.
  • the electronic device 10 includes communications circuitry that enables the electronic device 10 to establish communications with another device. Communications may include calls, data transfers, and the like. Calls may take any suitable form such as, but not limited to, voice calls and video calls. The calls may be carried out over a cellular circuit-switched network or may be in the form of a voice over Internet Protocol (VoIP) call that is established over a packet-switched capability of a cellular network or over an alternative packet-switched network (e.g., a network compatible with IEEE 802.11, which is commonly referred to as WiFi, or a network compatible with IEEE 802.16, which is commonly referred to as WiMAX), for example.
  • VoIP voice over Internet Protocol
  • Data transfers may include, but are not limited to, receiving streaming multimedia content (e.g., streaming audio, streaming video, etc.), receiving data feeds (e.g., pushed data, podcasts, really simple syndication (RSS) data feeds data feeds), downloading and/or uploading data (e.g., image files, video files, audio files, ring tones, Internet content, etc.), receiving or sending messages (e.g., text messages, instant messages, electronic mail messages, multimedia messages), and so forth.
  • This data may be processed by the electronic device 10, including storing the data in the memory 16, executing applications to allow user interaction with the data, displaying video and/or image content associated with the data, outputting audio sounds associated with the data, and so forth.
  • the communications circuitry may include an antenna 20 coupled to a radio circuit 22.
  • the radio circuit 22 includes a radio frequency transmitter and receiver for transmitting and receiving signals via the antenna 20.
  • the radio circuit 22 may be configured to operate in a mobile communications system.
  • Radio circuit 22 types for interaction with a mobile radio network and/or broadcasting network include, but are not limited to, global system for mobile communications (GSM), code division multiple access (CDMA), wideband CDMA (WCDMA), general packet radio service (GPRS), WiFi, WiMAX, digital video broadcasting-handheld (DVB-H), integrated services digital broadcasting (ISDB), high speed packet access (HSPA), etc., as well as advanced versions of these standards or any other appropriate standard.
  • GSM global system for mobile communications
  • CDMA code division multiple access
  • WCDMA wideband CDMA
  • GPRS general packet radio service
  • WiFi Wireless Fidelity
  • WiMAX wireless personal area network
  • DVB-H digital video broadcasting-handheld
  • ISDB integrated services digital broadcasting
  • HSPA high
  • the communications system may include a communications network 24 having a server 26 (or servers) for managing calls placed by and destined to the electronic device 10, transmitting data to and receiving data from the electronic device 10 and carrying out any other support functions.
  • the server 26 communicates with the electronic device 10 via a transmission medium.
  • the transmission medium may be any appropriate device or assembly, including, for example, a communications base station (e.g., a cellular service tower, or "cell" tower), a wireless access point, a satellite, etc.
  • the network 24 may support the communications activity of multiple electronic devices 10 and other types of end user devices.
  • the server 26 may be configured as a typical computer system used to carry out server functions and may include a processor configured to execute software containing logical instructions that embody the functions of the server 26 and a memory to store such software and any related databases.
  • the electronic device 10 may wirelessly communicate directly with another electronic device 10 (e.g., another mobile telephone or a computer) and without an intervening network.
  • the electronic device 10 may include a primary control circuit 28 that is configured to carry out overall control of the functions and operations of the electronic device 10.
  • the control circuit 28 may include a processing device 30, such as a central processing unit (CPU), microcontroller or microprocessor.
  • the processing device 30 executes code stored in a memory (not shown) within the control circuit 28 and/or in a separate memory, such as the memory 16, in order to carry out operation of the electronic device 10. For instance, the processing device 30 may execute and the memory 16 may store code that implements the analytics engine 12 to detect stress and/or concentration lapse of the user of the electronic device 10.
  • the memory 16 may be, for example, one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, a random access memory (RAM), or other suitable device.
  • the memory 16 may include a non-volatile memory for long term data storage and a volatile memory that functions as system memory for the control circuit 28.
  • the memory 16 may exchange data with the control circuit 28 over a data bus.
  • the electronic device 10 further includes a sound signal processing circuit 32 for processing audio signals transmitted by and received from the radio circuit 22. Coupled to the sound processing circuit 32 are a speaker 34 (also referred to as rendering device herein) and a microphone 36 that enable a user to listen and speak via the electronic device 10.
  • the radio circuit 22 and sound processing circuit 32 are each coupled to the control circuit 28 so as to carry out overall operation. Audio data may be passed from the control circuit 28 to the sound signal processing circuit 32 for playback to the user.
  • the audio data may include, for example, audio data from an audio file stored by the memory 16 and retrieved by the control circuit 28, or received audio data such as in the form of voice communications or streaming audio data from a mobile radio service.
  • the sound processing circuit 32 may include any appropriate buffers, decoders, amplifiers and so forth.
  • the display 14 may be coupled to the control circuit 28 by a video processing circuit 38 that converts video data to a video signal used to drive the display 14.
  • the video processing circuit 38 may include any appropriate buffers, decoders, video data processors and so forth.
  • the video data may be generated by the control circuit 28, retrieved from a video file that is stored in the memory 16, derived from an incoming video data stream that is received by the radio circuit 22 or obtained by any other suitable method.
  • the electronic device 10 may further include one or more input/output (I/O) interface(s) 40.
  • the I O interface(s) 40 may be in the form of typical mobile telephone I/O interfaces and may include one or more electrical connectors.
  • the I/O interfaces 40 may form one or more data ports for connecting the electronic device 10 to another device (e.g., a computer) or an accessory (e.g., a personal handsfree (PHF) device) via a cable.
  • operating power may be received over the I/O interface(s) 40 and power to charge a battery of a power supply unit (PSU) 42 within the electronic device 10 may be received over the I/O interface(s) 40.
  • the PSU 42 may supply power to operate the electronic device 10 in the absence of an external power source.
  • the electronic device 10 also may include various other components.
  • a system clock 44 may clock components such as the control circuit 28 and the memory 16.
  • a camera 46 may be present for taking digital pictures and/or movies. Image and/or video files corresponding to the pictures and/or movies may be stored in the memory 16.
  • a position data receiver 48 such as a global positioning system (GPS) receiver, Galileo satellite system receiver or the like, may be involved in determining the location of the electronic device 10.
  • GPS global positioning system
  • Galileo satellite system receiver Galileo satellite system receiver or the like
  • a local wireless transceiver 50 such as an infrared transceiver and/or an RF transceiver (e.g., a Bluetooth chipset) may be used to establish
  • a nearby device such as an accessory (e.g., a PHF device), another mobile radio terminal, a computer or another device.
  • an accessory e.g., a PHF device
  • another mobile radio terminal e.g., a computer or another device.
  • Various exemplary functions involving detecting stress and/or concentration lapses in a user of the electronic device 10 will now be described. Many of the functions may have particular relevance to the users of portable devices, such as the exemplary mobile telephone. However, some of the functions also may be used in connection with more stationary electronic devices. Each of the described functions involves analyzing user performance and correlating the performance to draw an inference about the level of stress and/or concentration lapse that the user is currently experiencing.
  • Example 1 User A is a heavy user of the messaging features of an electronic device. For example, User A most frequently uses short message service (SMS) and electronic mail (E-mail) functionality. User A is a seasoned typist on the user input (e.g., keypad, keyboard, etc.) of the electronic device. However, there are days when User A is under enormous stress either due to home, work, and/or academic pressure and makes typing errors more often than usual.
  • SMS short message service
  • E-mail electronic mail
  • User A is a seasoned typist on the handset keyboard and usually types 120 letters in a minute with an accuracy of 90%.
  • the analytics engine 12 in the electronic device 10 may detect that User A is using the back space/delete key more frequently than usual.
  • the analytics engine 12 compares User A's current performance with past performance. When User A's accuracy falls below a predetermined threshold (e.g., performance (typing accuracy) drops by more than 30%), the analytics engine 12 determines that User A is under increased stress and/or experiencing a concentration lapse. By correlating the current performance with her past average performance, the analytics engine 12 may make the inference that User A is either under stress or is experiencing a significant lapse in concentration.
  • a predetermined threshold e.g., performance (typing accuracy) drops by more than 30%
  • User B plays games on an electronic device 10 every day while travelling to work on a train. User B especially likes a particular game (e.g., the Bubble Breaker game) and plays the game very often. While user B plays the game constantly, there are days when User B is subjected to enormous stress and scores more poorly than previously on the game. This generally occurs due to high stress and lapses of concentration.
  • a particular game e.g., the Bubble Breaker game
  • User B plays the Bubble Breaker game on his electronic device 10 every day while travelling to work on the train. Due to his constant practice, he generally scores in the range of 800-900. However on a particular day, due to high stress at work, he ends up constantly scoring in the range of 500-600.
  • the analytics engine 12 in the electronic device 10 may correlate the current score of User B to earlier scores and detect anomalous behavior (e.g., poor performance based on the final score of the Bubble Breaker game). The anomalous behavior prompts the analytics engine 12 to infer that User B is probably under higher stress levels or User B is experiencing a significant concentration lapse.
  • the analytics engine 12 may utilize any physical characteristic and/or performance characteristic to determine whether a user is under more stress than usual is experiencing a concentration lapse.
  • the volume with which a user speaks through the microphone 36 of the electronic device 10 may be used to determine whether the user is experiencing increased stress.
  • the volume with which a user speaks on the electronic device 10 in normal circumstances is usually different from the volume with which the user speaks when under stress.
  • pressure measured in the form of grip pressure and/or pressure applied to a touchscreen and/or keypad or keyboard may also be monitored by the analytics engine 12 to determine stress and/or concentration lapse.
  • the analytics engine 12 may also utilize other information stored in the electronic device 10 to assist in estimating whether a user is under stress.
  • the analytics engine 12 may use other data stored on the electronic device to further estimate stress and/or concentration conditions.
  • calendar information may be used to further accurately estimate stress and/or user concentration conditions.
  • the analytics engine may take those into account by applying a weighting factor or other mechanism to more accurately predict the stress and/or concentration lapse that the user may be currently experiencing. For example, the more business appointments scheduled, the more easily increased stress and/or a concentration lapse may be inferred.
  • the analytics engine 12 may suggest the user render multimedia content that may mitigate the detected stress and/or concentration lapse.
  • the analytics engine 12 may request multimedia content from the Internet (through remote server 26, for example) or from contents stored in the memory 16.
  • the analytics engine 12 may request multimedia content that the user would likely deem funny, enlightening, entertaining, or otherwise interesting, in order to reduce the detected stress and/or concentration lapse.
  • the analytics may request content from www.YouTube.com or other remote content source that the user has previously indicated may alleviate stress and/or concentration lapses.
  • keyword searching may be performed from keywords chosen by the user and/or selected by the analytics engine 12 based on the user's monitored interests, as collected by the analytics engine.
  • the analytics engine 12 may automatically select multimedia content when stress and/or a lapse of concentration is inferred. This way, users could be helped by the electronic device 10 to feel better automatically (without additional user interaction). Prior to rendering the content, the user may be provided the ability to override suggestions and continue normal usage of the electronic device 10.
  • the analytics engine 12 may receive input from a sensor 52, which may be utilized to detect one or more physical and/or physiological characteristic associated with the user while the user is using the electronic device.
  • Exemplary sensors include heart rate monitors, grip pressure sensors, keypad pressure sensors, touchscreen pressure sensors, cameras, accelerometers, blood pressure monitors, moisture sensors, breathing sensors, brain sensors, etc.
  • the senor 52 is connected to the electronic device 10 through a wired connection with one of the I/O interfaces 40 of the electronic device 10. In other embodiments, the sensor 52 may be incorporated within the electronic device 10. The sensor 52 may also include a wireless transceiver for communicating with the electronic device 10 through the local wireless transceiver 50 using a wireless interface. Also, while the processing to carry out the described functions is conducted by the electronic device 10 in the illustrated examples, at least some of the processing may be carried out by the server 26.
  • An exemplary method 100 may be carried out by executing an embodiment of the analytics engine 12, for example.
  • the flow charts of FIGs. 2 and 3 may be thought of as depicting steps of a method carried out by the electronic device 10.
  • FIGs. 2 and 3 show a specific order of executing functional logic blocks, the order of executing the blocks may be changed relative to the order shown. Also, two or more blocks shown in succession may be executed concurrently or with partial concurrence. Certain blocks also may be omitted.
  • the method 100 includes, at block 102, collecting information related an associated user's use of an electronic device.
  • the information collected may relate to at least one user characteristic associated with the user's use of the electronic device and/or to the user's performance of one or more tasks with the electronic device 10.
  • the collection of information may be any user input entered into the electronic device 10 by hand and/or voice.
  • user input associated with the keypad 18 may be monitored, an audible characteristic of the user's speech (e.g. volume of voice spoken, inflection of voice, etc) received by the microphone 36 may be monitored, pressure asserted on the electronic device (e.g., an input device (e.g., a touchscreen, keypad, keyboard), grip pressure, etc).
  • the information collected may relate to the type of application and/or feature used by the associated user.
  • a messaging application e.g., SMS, E-mail, word processing application, etc.
  • the analytics engine 12 may monitor characters received from the keypad, keyboard, pressure exerted on the keypad, keyboard, etc.
  • the analytics engine 12 may monitor the user's voice and/or inflection associated with the user's voice, for example.
  • the information is processed by the analytics engine 12. Processing of the information may take a variety of forms depending on the type of user input monitored and/or the application/feature currently being used. One or more keys may be monitored to determine how accurately the user is performing a task (e.g., typing a message). Likewise, user performance of a game may also be monitored to determine a level of stress and/or concentration lapse. Furthermore, voice signals received by the electronic device 10 may also be monitored to determine, for example, at least one characteristic associated with the voice. For example, processing of the information may include determining if a user input error was received (e.g., a backspace input or delete input for a messaging application).
  • a user input error e.g., a backspace input or delete input for a messaging application.
  • exemplary processing steps may include determining a user's score from a game and/or processing sound information received through the microphone 36, for example.
  • the step of processing 104 may also include storing information collected in memory 16 for example.
  • a determination is made if the user is under stress and/or experiencing a concentration lapse based upon the collected information. In general, this step involves comparing one or more user characteristic that was recently received with baseline information associated with the user.
  • the baseline information may be determined in a number of ways. For example, the baseline information may be determined by taking an average of the collected information during normal operation of the electronic device. In another embodiment, the baseline information may determined by taking an average of the collected information during one or more training periods.
  • the determination made at block 106 may further include a predetermined performance threshold. For example, if the user's current performance was less than 20% of the user's average performance, then it may be determined that the user is experiencing stress and/or a concentration lapse.
  • the performance threshold may be any desired performance metric and may depend on the application/feature. For example, some application, especially games, may have a wide variation. Therefore, a more suitable performance threshold may be more than 40%.
  • the analytics engine 12 may infer that the user has increased stress and/or concentration lapses.
  • the baseline information may be combined with other information stored in the electronic device 10 to assist in estimating whether a user is under stress.
  • the analytics engine 12 may use other data stored on the electronic device to further estimate stress and/or concentration conditions.
  • calendar information may be used to further accurately estimate stress and/or user concentration conditions.
  • a user has more appointments than usual scheduled in the user's electronic calendar, the analytics engine 12 may take those into account by applying a weighting factor or other mechanism to more accurately predict the stress and/or concentration lapse that the user may be currently experiencing. For example, the more business
  • appointments scheduled the predetermined threshold which would normally be 40% may be lowered to 30%, for example, in order that increased stress and/or a concentration lapse may be more easily inferred.
  • the analytics engine 12 may provide one or more multimedia content selections to be rendered on the electronic device 10 to mitigate the determined stress and/or concentration lapse. Any multimedia content that may appeal to the associated user may be presented to the user.
  • the multimedia content may be selected by the user, chosen randomly from a predefined group of content that was previously selected by the user. Alternatively, one or more keyword search may be performed based on keywords and/or topics of interest to the user, as discussed above.
  • the exemplary method 120 is identical to exemplary method 100, except that exemplary method 120 includes block 122.
  • block 122 prior to rendering multimedia content to mitigate the detected stress and/or the lapse of concentration, the user is prompted by a message presented on a display of the electronic device to determine if the user would like to view multimedia content to mitigate stress and/or lapse of concentration. If the user would like to continue with performing the task and/or using the feature, program flow returns to block 102. If the user would like to take a break from current activities, program flow moves to block 108, as discussed above.
  • the memory 16 may store a database in association with the analytics engine 12 or the database may be stored within the analytics engine 12.
  • the database may be used to store information used by the analytics engine 12, including predetermined thresholds, baseline information relating to one or more applications and/or features of the electronic device 10, multimedia content of interest to the user, key words selected by the user, etc.
  • the user may be prompted to enter key words or otherwise identify multimedia content of interest to the user.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Public Health (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Psychiatry (AREA)
  • Veterinary Medicine (AREA)
  • Physics & Mathematics (AREA)
  • Pathology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Surgery (AREA)
  • Biophysics (AREA)
  • Psychology (AREA)
  • Molecular Biology (AREA)
  • Social Psychology (AREA)
  • Hospice & Palliative Care (AREA)
  • Educational Technology (AREA)
  • Developmental Disabilities (AREA)
  • Child & Adolescent Psychology (AREA)
  • Anesthesiology (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Pain & Pain Management (AREA)
  • Acoustics & Sound (AREA)
  • Hematology (AREA)
  • Telephone Function (AREA)

Abstract

To enhance interaction and use with electronic devices, the present disclosure describes an improved method and system of taking responsive action to an inferred stress and/or lapse of concentration of an associated user. One aspect of the invention relates to a method for mitigating stress and/or concentration lapses of a user of an electronic device (10). The method includes: a) collecting information related an associated user's use of an electronic device; b) processing the information; and c) determining if the user is under stress and/or experiencing a concentration lapse based upon the collected information; and d) rendering multimedia content to mitigate the determined stress and/or concentration lapse.

Description

TITLE : A USER ANALYTICS ENGINE FOR DETECTING AND
MITIGATING STRESS AND/OR CONCENTRATION LAPSES FOR AN ASSOCIATED USER OF AN ELECTRONIC DEVICE
TECHNICAL FIELD OF THE INVENTION
The technology of the present disclosure relates generally to utilizing user analytics for detecting and mitigating user stress and/or concentration lapses and, more particularly, to a system and method for taking a responsive action to mitigate detected user stress and/or concentration lapse by correlating current user performance with past user performance.
BACKGROUND
In today's ultra-competitive world, users of mobile devices are subjected to constant stress both in personal and professional life. The use of such devices has become ubiquitous both in personal and professional life. Many electronic devices include a variety of features and/or applications to facilitate the exchange of information between the user and other parties.
SUMMARY
One problem with electronic devices is the lack of ability to determine whether a user of the electronic device is experiencing increased stress and/or concentration loss. The present invention overcomes this problem by providing an in-feature/in-application mechanism to infer stress or concentration lapses and provide multimedia content to assist the user in mitigating the detected stress and/or concentration lapse.
To enhance interaction and use with electronic devices, the present disclosure describes an improved system and method of taking responsive action to a detected level of stress and/or concentration lapse. Through the descriptions herein, a number of specific exemplary situations in which the disclosed systems and methods may be used are described. It will be appreciated that the disclosed systems and methods may be used in a wide variety of situations other than these specific examples.
One aspect of the invention relates to a method for mitigating stress and/or concentration lapses of a user of an electronic device, the method including: a) collecting information related an associated user's use of an electronic device; b) processing the information; and c) determining if the user is under stress and/or experiencing a concentration lapse based upon the collected information; and d) rendering multimedia content to mitigate the determined stress and/or concentration lapse.
Another aspect of the invention relates to the information collected relating to at least one user characteristic associated with the user's use of the electronic device to perform one or more tasks with the electronic device.
Another aspect of the invention relates to the information collected relating to user input from a user input device coupled to the electronic device.
Another aspect of the invention relates to the information being processed to determine if a user input error was received by the electronic device.
Another aspect of the invention relates to the user input error being identified by receiving user input consisting of at least one selected from a group consisting of a delete key and/or a backspace key.
Another aspect of the invention relates to information collected relating to user performance in games executed by the electronic device.
Another aspect of the invention relates to information collected relating to a volume in which the associated user speaks while using the electronic device.
Another aspect of the invention relates to the step of determining if the user is under stress and/or experiencing a concentration lapse including comparing the collected information with baseline information associated with the user. Another aspect of the invention relates to the baseline information being determined by taking an average of the collected information during one or more training periods.
Another aspect of the invention relates to the baseline information being determined by taking an average of the collected information during normal operation of the electronic device.
Another aspect of the invention relates to prior to rendering multimedia content to mitigate the detected stress and/or the lapse of concentration, the user is prompted by a message presented on a display of the electronic device to determine if the user would like to view multimedia content to mitigate stress and/or lapse of concentration.
Another aspect of the invention relates to the plurality of multimedia content selections that may appeal to the associated user are stored in a storage device of the electronic device.
One aspect of the invention relates to an electronic device including: a memory for storing information about an associated user's use of the electronic device; a controller coupled to the memory, wherein the controller processes the information in a
predetermined manner to determine if the user is under stress and/or experiencing a concentration lapse; and a rendering device coupled to the processor for rendering multimedia content to the associated user in order to mitigate the determined stress and/or concentration lapse of the associated user.
Another aspect of the invention relates to the electronic device being a mobile telephone.
Another aspect of the invention relates to the memory including one or more baseline metric associated with the user's use of the electronic equipment, wherein the baseline metric includes at least one user characteristic associated with the user's use of the electronic device to perform one or more tasks with the electronic device.
Another aspect of the invention relates to the memory including a plurality of multimedia content sources that appeals to the user. Another aspect of the invention relates to the rendering device being at least one selected from the group consisting of a display and/or a speaker.
One aspect of the invention relates to a program stored on a machine usable medium for taking action in response to detected stress and/or lapse of concentration of an associated user of an electronic device, the program comprising executable logic to: establish at least one baseline metric for an associated user, wherein the baseline metric corresponds to the associated user's use of the electronic device to perform one or more tasks with an electronic device; monitoring at least one user characteristic related to the associated user's use of the electronic device to perform the one or more tasks, wherein the monitored user characteristic is related to the at least one baseline metric; and comparing the monitored at least one user characteristic with the baseline metric to determine at least one of a detected stress and/or a lapse of concentration; and rendering multimedia content on a display of the electronic device to mitigate the detected stress and/or the lapse of concentration when the detected stress and/or the lapse of
concentration is above a predetermined threshold.
Another aspect of the invention relates to logic to drive the display to display one or more alternative sources of multimedia content when the detected stress and/or the lapse of concentration is above a predetermined threshold.
These and further features will be apparent with reference to the following description and attached drawings. In the description and drawings, particular embodiments of the invention have been disclosed in detail as being indicative of some of the ways in which the principles of the invention may be employed, but it is understood that the invention is not limited correspondingly in scope. Rather, the invention includes all changes, modifications and equivalents coming within the scope of the claims appended hereto.
Features that are described and/or illustrated with respect to one embodiment may be used in the same way or in a similar way in one or more other embodiments and/or in combination with or instead of the features of the other embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a schematic view of an exemplary system for taking responsive action to detected stress and/or lapse of concentration.
FIGs. 2-3 are flow charts representing exemplary methods for implementing an analytics engine in accordance with aspects of the present invention.
DETAILED DESCRIPTION OF EMBODIMENTS
Embodiments will now be described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. It will be understood that the figures are not necessarily to scale.
In the present document, embodiments are described primarily in the context of a portable radio communications device, such as the illustrated mobile telephone. It will be appreciated, however, that the exemplary context of a mobile telephone is not the only operational environment in which aspects of the disclosed systems and methods may be used. Therefore, the techniques described in this document may be applied to any type of appropriate electronic device, primary examples of which are a mobile telephone and a computer, such as a laptop computer or a desktop computer. But other examples include, without limitation, a media player, a gaming device, an electronic organizer, a personal digital assistant (PDA), etc.
Referring initially to FIG. 1, a system for taking responsive action to detected stress and/or concentration lapse includes an electronic device 10. The electronic device 10 includes an analytics engine 12 that is operable to infer stress and/or concentration lapse by correlating current user performance with past user performance. For example, the analytics engine is configured to analyze user performance while the user is using various features and/or applications on the electronic device 10 and correlate the current performance with past performance. Based on this correlation, inferences may be made about the stress and/or concentration levels of the user and one or more mitigation techniques (e.g., rendering multimedia content) may be suggested to the user of the electronic device 10 to mitigate the inferred stress and/or concentration lapse. Additional details and operation of the analytics engine 12 will be described in greater detail below. The analytics engine 12 may be embodied as executable code that is resident in and executed by the electronic device 10. In one embodiment, the analytics engine 12 may be one or more programs that are stored on a computer or machine readable medium. The analytics engine 12 may be a stand-alone software application or form a part of a software application that carries out additional tasks related to the electronic device 10.
Also, through the following description, exemplary techniques for taking action in response to detected stress and/or concentration lapse are described. It will be appreciated that through the description of the exemplary techniques, a description of steps that may be carried out in part by executing software is described. The described steps are the foundation from which a programmer of ordinary skill in the art may write code to implement the described functionality. As such, a computer program listing is omitted for the sake of brevity. However, the described steps may be considered a method that the corresponding device is configured to carry out. Also, while the analytics engine 12 is implemented in software in accordance with an embodiment, such functionality could also be carried out via dedicated hardware or firmware, or some combination of hardware, firmware and/or software.
The electronic device of the illustrated embodiment is a mobile telephone, but will be referred to as the electronic device 10. As indicated, the electronic device 10 may be a device other than a mobile telephone.
The electronic device 10 may include a display 14. The display 14 (also referred to herein as a rendering device) presents information to a user such as operating state, time, telephone numbers, contact information, various menus, etc., that enable the user to utilize the various features of the electronic device 10. The display 14 also may be used to visually display content received by the electronic device 10 and/or retrieved from a memory 16 of the electronic device 10. The display 14 may be used to present images, video and other graphics to the user, such as photographs, mobile television content, Internet pages, and video associated with games. As described below, upon detection of stress and/or concentration lapse, the display may present multimedia content to alleviate the detected stress and/or concentration lapse. The multimedia content may be stored at a remote server (e.g., server 26) and/or stored locally in memory 16, for example. A keypad 18 provides for a variety of user input operations. For example, the keypad 18 may include alphanumeric keys for allowing entry of alphanumeric information (e.g., telephone numbers, phone lists, contact information, notes, text, etc.), special function keys (e.g., a call send and answer key, multimedia playback control keys, a camera shutter button, etc.), navigation and select keys or a pointing device, and so forth. Keys or key-like functionality also may be embodied as a touch screen associated with the display 18. Also, the display 14 and keypad 18 may be used in conjunction with one another to implement soft key functionality.
The electronic device 10 includes communications circuitry that enables the electronic device 10 to establish communications with another device. Communications may include calls, data transfers, and the like. Calls may take any suitable form such as, but not limited to, voice calls and video calls. The calls may be carried out over a cellular circuit-switched network or may be in the form of a voice over Internet Protocol (VoIP) call that is established over a packet-switched capability of a cellular network or over an alternative packet-switched network (e.g., a network compatible with IEEE 802.11, which is commonly referred to as WiFi, or a network compatible with IEEE 802.16, which is commonly referred to as WiMAX), for example. Data transfers may include, but are not limited to, receiving streaming multimedia content (e.g., streaming audio, streaming video, etc.), receiving data feeds (e.g., pushed data, podcasts, really simple syndication (RSS) data feeds data feeds), downloading and/or uploading data (e.g., image files, video files, audio files, ring tones, Internet content, etc.), receiving or sending messages (e.g., text messages, instant messages, electronic mail messages, multimedia messages), and so forth. This data may be processed by the electronic device 10, including storing the data in the memory 16, executing applications to allow user interaction with the data, displaying video and/or image content associated with the data, outputting audio sounds associated with the data, and so forth.
In the exemplary embodiment, the communications circuitry may include an antenna 20 coupled to a radio circuit 22. The radio circuit 22 includes a radio frequency transmitter and receiver for transmitting and receiving signals via the antenna 20. The radio circuit 22 may be configured to operate in a mobile communications system. Radio circuit 22 types for interaction with a mobile radio network and/or broadcasting network include, but are not limited to, global system for mobile communications (GSM), code division multiple access (CDMA), wideband CDMA (WCDMA), general packet radio service (GPRS), WiFi, WiMAX, digital video broadcasting-handheld (DVB-H), integrated services digital broadcasting (ISDB), high speed packet access (HSPA), etc., as well as advanced versions of these standards or any other appropriate standard. It will be appreciated that the electronic device 10 may be capable of communicating using more than one standard. Therefore, the antenna 20 and the radio circuit 22 may represent one or more than one radio transceiver.
The communications system may include a communications network 24 having a server 26 (or servers) for managing calls placed by and destined to the electronic device 10, transmitting data to and receiving data from the electronic device 10 and carrying out any other support functions. The server 26 communicates with the electronic device 10 via a transmission medium. The transmission medium may be any appropriate device or assembly, including, for example, a communications base station (e.g., a cellular service tower, or "cell" tower), a wireless access point, a satellite, etc. The network 24 may support the communications activity of multiple electronic devices 10 and other types of end user devices. As will be appreciated, the server 26 may be configured as a typical computer system used to carry out server functions and may include a processor configured to execute software containing logical instructions that embody the functions of the server 26 and a memory to store such software and any related databases. In alternative arrangements, the electronic device 10 may wirelessly communicate directly with another electronic device 10 (e.g., another mobile telephone or a computer) and without an intervening network.
The electronic device 10 may include a primary control circuit 28 that is configured to carry out overall control of the functions and operations of the electronic device 10. The control circuit 28 may include a processing device 30, such as a central processing unit (CPU), microcontroller or microprocessor. The processing device 30 executes code stored in a memory (not shown) within the control circuit 28 and/or in a separate memory, such as the memory 16, in order to carry out operation of the electronic device 10. For instance, the processing device 30 may execute and the memory 16 may store code that implements the analytics engine 12 to detect stress and/or concentration lapse of the user of the electronic device 10. The memory 16 may be, for example, one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, a random access memory (RAM), or other suitable device. In a typical arrangement, the memory 16 may include a non-volatile memory for long term data storage and a volatile memory that functions as system memory for the control circuit 28. The memory 16 may exchange data with the control circuit 28 over a data bus.
Accompanying control lines and an address bus between the memory 16 and the control circuit 28 also may be present.
The electronic device 10 further includes a sound signal processing circuit 32 for processing audio signals transmitted by and received from the radio circuit 22. Coupled to the sound processing circuit 32 are a speaker 34 (also referred to as rendering device herein) and a microphone 36 that enable a user to listen and speak via the electronic device 10. The radio circuit 22 and sound processing circuit 32 are each coupled to the control circuit 28 so as to carry out overall operation. Audio data may be passed from the control circuit 28 to the sound signal processing circuit 32 for playback to the user. The audio data may include, for example, audio data from an audio file stored by the memory 16 and retrieved by the control circuit 28, or received audio data such as in the form of voice communications or streaming audio data from a mobile radio service. The sound processing circuit 32 may include any appropriate buffers, decoders, amplifiers and so forth.
The display 14 may be coupled to the control circuit 28 by a video processing circuit 38 that converts video data to a video signal used to drive the display 14. The video processing circuit 38 may include any appropriate buffers, decoders, video data processors and so forth. The video data may be generated by the control circuit 28, retrieved from a video file that is stored in the memory 16, derived from an incoming video data stream that is received by the radio circuit 22 or obtained by any other suitable method.
The electronic device 10 may further include one or more input/output (I/O) interface(s) 40. The I O interface(s) 40 may be in the form of typical mobile telephone I/O interfaces and may include one or more electrical connectors. The I/O interfaces 40 may form one or more data ports for connecting the electronic device 10 to another device (e.g., a computer) or an accessory (e.g., a personal handsfree (PHF) device) via a cable. Further, operating power may be received over the I/O interface(s) 40 and power to charge a battery of a power supply unit (PSU) 42 within the electronic device 10 may be received over the I/O interface(s) 40. The PSU 42 may supply power to operate the electronic device 10 in the absence of an external power source.
The electronic device 10 also may include various other components. For instance, a system clock 44 may clock components such as the control circuit 28 and the memory 16. A camera 46 may be present for taking digital pictures and/or movies. Image and/or video files corresponding to the pictures and/or movies may be stored in the memory 16. A position data receiver 48, such as a global positioning system (GPS) receiver, Galileo satellite system receiver or the like, may be involved in determining the location of the electronic device 10. A local wireless transceiver 50, such as an infrared transceiver and/or an RF transceiver (e.g., a Bluetooth chipset) may be used to establish
communication with a nearby device, such as an accessory (e.g., a PHF device), another mobile radio terminal, a computer or another device.
Various exemplary functions involving detecting stress and/or concentration lapses in a user of the electronic device 10 will now be described. Many of the functions may have particular relevance to the users of portable devices, such as the exemplary mobile telephone. However, some of the functions also may be used in connection with more stationary electronic devices. Each of the described functions involves analyzing user performance and correlating the performance to draw an inference about the level of stress and/or concentration lapse that the user is currently experiencing.
There are a variety of ways to detect user performance. The manner of detection may depend on the application and/or feature of the electronic device 10 that is currently being used. The following examples illustrate various aspects of the invention. One of ordinary skill in the art will appreciate that these examples are illustrative in nature and not intended to limit the scope of the present invention.
Example 1: User A is a heavy user of the messaging features of an electronic device. For example, User A most frequently uses short message service (SMS) and electronic mail (E-mail) functionality. User A is a seasoned typist on the user input (e.g., keypad, keyboard, etc.) of the electronic device. However, there are days when User A is under enormous stress either due to home, work, and/or academic pressure and makes typing errors more often than usual.
For example, User A is a seasoned typist on the handset keyboard and usually types 120 letters in a minute with an accuracy of 90%. However on a particular day of high stress User A makes typographical errors more often than usual. The analytics engine 12 in the electronic device 10 may detect that User A is using the back space/delete key more frequently than usual. The analytics engine 12 compares User A's current performance with past performance. When User A's accuracy falls below a predetermined threshold (e.g., performance (typing accuracy) drops by more than 30%), the analytics engine 12 determines that User A is under increased stress and/or experiencing a concentration lapse. By correlating the current performance with her past average performance, the analytics engine 12 may make the inference that User A is either under stress or is experiencing a significant lapse in concentration.
Example 2:
User B plays games on an electronic device 10 every day while travelling to work on a train. User B especially likes a particular game (e.g., the Bubble Breaker game) and plays the game very often. While user B plays the game constantly, there are days when User B is subjected to enormous stress and scores more poorly than previously on the game. This generally occurs due to high stress and lapses of concentration.
For example, User B plays the Bubble Breaker game on his electronic device 10 every day while travelling to work on the train. Due to his constant practice, he generally scores in the range of 800-900. However on a particular day, due to high stress at work, he ends up constantly scoring in the range of 500-600. The analytics engine 12 in the electronic device 10 may correlate the current score of User B to earlier scores and detect anomalous behavior (e.g., poor performance based on the final score of the Bubble Breaker game). The anomalous behavior prompts the analytics engine 12 to infer that User B is probably under higher stress levels or User B is experiencing a significant concentration lapse.
The analytics engine 12 may utilize any physical characteristic and/or performance characteristic to determine whether a user is under more stress than usual is experiencing a concentration lapse. For example, the volume with which a user speaks through the microphone 36 of the electronic device 10 may be used to determine whether the user is experiencing increased stress. The volume with which a user speaks on the electronic device 10 in normal circumstances is usually different from the volume with which the user speaks when under stress. In addition, pressure measured in the form of grip pressure and/or pressure applied to a touchscreen and/or keypad or keyboard may also be monitored by the analytics engine 12 to determine stress and/or concentration lapse.
The analytics engine 12 may also utilize other information stored in the electronic device 10 to assist in estimating whether a user is under stress. For example, the analytics engine 12 may use other data stored on the electronic device to further estimate stress and/or concentration conditions. For example, calendar information may be used to further accurately estimate stress and/or user concentration conditions. For example, in Example 2, if User B has more appointments than usual scheduled in the user's electronic calendar, the analytics engine may take those into account by applying a weighting factor or other mechanism to more accurately predict the stress and/or concentration lapse that the user may be currently experiencing. For example, the more business appointments scheduled, the more easily increased stress and/or a concentration lapse may be inferred.
In response to determining an increased amount of stress and/or concentration lapse, the analytics engine 12 may suggest the user render multimedia content that may mitigate the detected stress and/or concentration lapse. The analytics engine 12 may request multimedia content from the Internet (through remote server 26, for example) or from contents stored in the memory 16. For example, the analytics engine 12 may request multimedia content that the user would likely deem funny, enlightening, entertaining, or otherwise interesting, in order to reduce the detected stress and/or concentration lapse. In one embodiment, the analytics may request content from www.YouTube.com or other remote content source that the user has previously indicated may alleviate stress and/or concentration lapses. Depending on the source, keyword searching may be performed from keywords chosen by the user and/or selected by the analytics engine 12 based on the user's monitored interests, as collected by the analytics engine. The analytics engine 12 may automatically select multimedia content when stress and/or a lapse of concentration is inferred. This way, users could be helped by the electronic device 10 to feel better automatically (without additional user interaction). Prior to rendering the content, the user may be provided the ability to override suggestions and continue normal usage of the electronic device 10.
In another embodiment, the analytics engine 12 may receive input from a sensor 52, which may be utilized to detect one or more physical and/or physiological characteristic associated with the user while the user is using the electronic device.
Exemplary sensors include heart rate monitors, grip pressure sensors, keypad pressure sensors, touchscreen pressure sensors, cameras, accelerometers, blood pressure monitors, moisture sensors, breathing sensors, brain sensors, etc.
In the illustrated embodiment of FIG. 1 , the sensor 52 is connected to the electronic device 10 through a wired connection with one of the I/O interfaces 40 of the electronic device 10. In other embodiments, the sensor 52 may be incorporated within the electronic device 10. The sensor 52 may also include a wireless transceiver for communicating with the electronic device 10 through the local wireless transceiver 50 using a wireless interface. Also, while the processing to carry out the described functions is conducted by the electronic device 10 in the illustrated examples, at least some of the processing may be carried out by the server 26.
An exemplary method 100 may be carried out by executing an embodiment of the analytics engine 12, for example. Thus, the flow charts of FIGs. 2 and 3 may be thought of as depicting steps of a method carried out by the electronic device 10. Although FIGs. 2 and 3 show a specific order of executing functional logic blocks, the order of executing the blocks may be changed relative to the order shown. Also, two or more blocks shown in succession may be executed concurrently or with partial concurrence. Certain blocks also may be omitted.
With reference to FIG. 2, illustrated are logical operations to implement an exemplary method 100 of mitigating stress and/or concentration lapses of an associated user of an electronic device 10. The method 100 includes, at block 102, collecting information related an associated user's use of an electronic device. The information collected may relate to at least one user characteristic associated with the user's use of the electronic device and/or to the user's performance of one or more tasks with the electronic device 10. As stated above, the collection of information may be any user input entered into the electronic device 10 by hand and/or voice. For example, user input associated with the keypad 18 may be monitored, an audible characteristic of the user's speech (e.g. volume of voice spoken, inflection of voice, etc) received by the microphone 36 may be monitored, pressure asserted on the electronic device (e.g., an input device (e.g., a touchscreen, keypad, keyboard), grip pressure, etc).
In addition, the information collected may relate to the type of application and/or feature used by the associated user. For example, when a messaging application (e.g., SMS, E-mail, word processing application, etc.) is used the analytics engine 12 may monitor characters received from the keypad, keyboard, pressure exerted on the keypad, keyboard, etc. In addition, for voice communications, the analytics engine 12 may monitor the user's voice and/or inflection associated with the user's voice, for example.
At block 104, the information is processed by the analytics engine 12. Processing of the information may take a variety of forms depending on the type of user input monitored and/or the application/feature currently being used. One or more keys may be monitored to determine how accurately the user is performing a task (e.g., typing a message). Likewise, user performance of a game may also be monitored to determine a level of stress and/or concentration lapse. Furthermore, voice signals received by the electronic device 10 may also be monitored to determine, for example, at least one characteristic associated with the voice. For example, processing of the information may include determining if a user input error was received (e.g., a backspace input or delete input for a messaging application). Other exemplary processing steps may include determining a user's score from a game and/or processing sound information received through the microphone 36, for example. The step of processing 104 may also include storing information collected in memory 16 for example. At block 106, a determination is made if the user is under stress and/or experiencing a concentration lapse based upon the collected information. In general, this step involves comparing one or more user characteristic that was recently received with baseline information associated with the user. The baseline information may be determined in a number of ways. For example, the baseline information may be determined by taking an average of the collected information during normal operation of the electronic device. In another embodiment, the baseline information may determined by taking an average of the collected information during one or more training periods.
The determination made at block 106 may further include a predetermined performance threshold. For example, if the user's current performance was less than 20% of the user's average performance, then it may be determined that the user is experiencing stress and/or a concentration lapse. One of ordinary will readily appreciate that the performance threshold may be any desired performance metric and may depend on the application/feature. For example, some application, especially games, may have a wide variation. Therefore, a more suitable performance threshold may be more than 40%. Upon, exceeding the performance threshold, the analytics engine 12 may infer that the user has increased stress and/or concentration lapses.
The baseline information may be combined with other information stored in the electronic device 10 to assist in estimating whether a user is under stress. For example, the analytics engine 12 may use other data stored on the electronic device to further estimate stress and/or concentration conditions. For example, calendar information may be used to further accurately estimate stress and/or user concentration conditions. For example, a user has more appointments than usual scheduled in the user's electronic calendar, the analytics engine 12 may take those into account by applying a weighting factor or other mechanism to more accurately predict the stress and/or concentration lapse that the user may be currently experiencing. For example, the more business
appointments scheduled the predetermined threshold which would normally be 40% may be lowered to 30%, for example, in order that increased stress and/or a concentration lapse may be more easily inferred.
If it is determined that the user's current performance is less than the baseline information, stress and/or concentration lapse of a user may be inferred. In such cases, program flow moves to block 108. If the current performance is greater than or substantially the same as the baseline information, it may be inferred that the user is not currently in a stressful situation and/or not experiencing a concentration lapse. In such cases, program flow returns to block 102 and continues as discussed above. If a determination is made the user has increased stress and/or concentration lapses, program flow moves to block 108. At block 108, the analytics engine 12 may provide one or more multimedia content selections to be rendered on the electronic device 10 to mitigate the determined stress and/or concentration lapse. Any multimedia content that may appeal to the associated user may be presented to the user. The multimedia content may be selected by the user, chosen randomly from a predefined group of content that was previously selected by the user. Alternatively, one or more keyword search may be performed based on keywords and/or topics of interest to the user, as discussed above.
Referring to Figure 3, the exemplary method 120 is identical to exemplary method 100, except that exemplary method 120 includes block 122. At block 122, prior to rendering multimedia content to mitigate the detected stress and/or the lapse of concentration, the user is prompted by a message presented on a display of the electronic device to determine if the user would like to view multimedia content to mitigate stress and/or lapse of concentration. If the user would like to continue with performing the task and/or using the feature, program flow returns to block 102. If the user would like to take a break from current activities, program flow moves to block 108, as discussed above.
The memory 16 may store a database in association with the analytics engine 12 or the database may be stored within the analytics engine 12. The database may be used to store information used by the analytics engine 12, including predetermined thresholds, baseline information relating to one or more applications and/or features of the electronic device 10, multimedia content of interest to the user, key words selected by the user, etc. In one embodiment, the user may be prompted to enter key words or otherwise identify multimedia content of interest to the user.
Although certain embodiments have been shown and described, it is understood that equivalents and modifications falling within the scope of the appended claims will occur to others who are skilled in the art upon the reading and understanding of this specification.

Claims

CLAIMS What is claimed is:
1. A method for mitigating stress and/or concentration lapses of a user of an electronic device (10), the method comprising:
a) collecting information related an associated user's use of an electronic device;
b) processing the information; and
c) determining if the user is under stress and/or experiencing a concentration lapse based upon the collected information; and
d) rendering multimedia content to mitigate the determined stress and/or concentration lapse.
2. The method of claim 1, wherein the information collected relates to at least one user characteristic associated with the user's use of the electronic device to perform one or more tasks with the electronic device.
3. The method of any of one of claims 1-2, wherein the information collected relates to user input from a user input device (14, 18, 36, 46, 52) coupled to the electronic device.
4. The method of any one of claims 1-3, wherein the information is processed to determine if a user input error was received by the electronic device.
5. The method of claim 4, wherein the user input error is identified by receiving user input consisting of at least one selected from a delete key and/or a backspace key.
6. The method of any one of claims 1-5, wherein the information collected relates to user performance in games executed by the electronic device.
7. The method of any of claims 1-6, wherein the information collected relates to a volume in which the associated user speaks while using the electronic device.
8. The method of any one of claims 1-7, wherein the step of determining if the user is under stress and/or experiencing a concentration lapse includes comparing the collected information with baseline information associated with the user.
9. The method of claim 8, wherein the baseline information is determined by taking an average of the collected information during one or more training periods.
10. The method of any one of claims 8-9, wherein the baseline information is determined by taking an average of the collected information during normal operation of the electronic device.
1 1. The method of any one of claims 1-10, wherein prior to rendering multimedia content to mitigate the detected stress and/or the lapse of concentration, the user is prompted by a message presented on a display (14) of the electronic device to determine if the user would like to view multimedia content to mitigate stress and/or lapse of concentration.
12. The method of any one of claims 1-1 1, wherein a plurality of multimedia content selections that may appeal to the associated user are stored in a storage device of the electronic device.
13. An electronic device (10) comprising:
a memory (16) for storing information about an associated user's use of the electronic device;
a controller (28) coupled to the memory, wherein the controller processes the information in a predetermined manner to determine if the user is under stress and/or experiencing a concentration lapse; and a rendering device (14, 34) coupled to the processor for rendering multimedia content to the associated user in order to mitigate the determined stress and/or
concentration lapse of the associated user.
14. The device of claim 13, wherein the electronic device is a mobile telephone.
15. The device of any one of claims 13-14, wherein the memory includes one or more baseline metric associated with the user's use of the electronic equipment, wherein the baseline metric includes at least one user characteristic associated with the user's use of the electronic device to perform one or more tasks with the electronic device.
16. The device of any one of claims 13-15, wherein the memory includes a plurality of multimedia content sources that appeals to the user.
17. The device of any one of claims 13-16, wherein the rendering device is at least one selected from the group consisting of a display (14) and/or a speaker (34).
18. A program (12) stored on a machine usable medium for taking action in response to detected stress and/or lapse of concentration of an associated user of an electronic device (10), the program comprising executable logic to:
establish at least one baseline metric for an associated user, wherein the baseline metric corresponds to the associated user's use of the electronic device to perform one or more tasks with an electronic device;
monitoring at least one user characteristic related to the associated user's use of the electronic device to perform the one or more tasks, wherein the monitored user characteristic is related to the at least one baseline metric; and
comparing the monitored at least one user characteristic with the baseline metric to determine at least one of a detected stress and/or a lapse of concentration; and
rendering multimedia content on a display (14) of the electronic device to mitigate the detected stress and/or the lapse of concentration when the detected stress and/or the lapse of concentration is above a predetermined threshold.
19. The program of claim 18, further comprising logic to drive the display to display one or more alternative sources of multimedia content when the detected stress and/or the lapse of concentration is above a predetermined threshold.
PCT/US2010/042088 2010-07-15 2010-07-15 A user analytics engine for detecting and mitigating stress and/or concentration lapses for an associated user of an electronic device WO2012008961A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/US2010/042088 WO2012008961A1 (en) 2010-07-15 2010-07-15 A user analytics engine for detecting and mitigating stress and/or concentration lapses for an associated user of an electronic device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2010/042088 WO2012008961A1 (en) 2010-07-15 2010-07-15 A user analytics engine for detecting and mitigating stress and/or concentration lapses for an associated user of an electronic device

Publications (1)

Publication Number Publication Date
WO2012008961A1 true WO2012008961A1 (en) 2012-01-19

Family

ID=43827847

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2010/042088 WO2012008961A1 (en) 2010-07-15 2010-07-15 A user analytics engine for detecting and mitigating stress and/or concentration lapses for an associated user of an electronic device

Country Status (1)

Country Link
WO (1) WO2012008961A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102012214697A1 (en) * 2012-08-01 2014-02-06 Soma Analytics Ug (Haftungsbeschränkt) Device, method and application for determining a current load level
DE102014223366A1 (en) 2014-11-17 2016-05-19 BSH Hausgeräte GmbH Domestic appliance with a touch-sensitive operating device and method for its operation

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1997001984A1 (en) * 1995-06-30 1997-01-23 Samuel Ron Speech-based biofeedback method and system
WO2008008729A2 (en) * 2006-07-10 2008-01-17 Accenture Global Services Gmbh Mobile personal services platform for providing feedback
WO2008072137A1 (en) * 2006-12-12 2008-06-19 Koninklijke Philips Electronics N.V. Stress reduction
US20100076274A1 (en) * 2008-09-23 2010-03-25 Joan Severson Human-Digital Media Interaction Tracking

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1997001984A1 (en) * 1995-06-30 1997-01-23 Samuel Ron Speech-based biofeedback method and system
WO2008008729A2 (en) * 2006-07-10 2008-01-17 Accenture Global Services Gmbh Mobile personal services platform for providing feedback
WO2008072137A1 (en) * 2006-12-12 2008-06-19 Koninklijke Philips Electronics N.V. Stress reduction
US20100076274A1 (en) * 2008-09-23 2010-03-25 Joan Severson Human-Digital Media Interaction Tracking

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102012214697A1 (en) * 2012-08-01 2014-02-06 Soma Analytics Ug (Haftungsbeschränkt) Device, method and application for determining a current load level
US11468984B2 (en) 2012-08-01 2022-10-11 Soma Analytics Ug (Haftungsbeschränkt) Device, method and application for establishing a current load level
DE102014223366A1 (en) 2014-11-17 2016-05-19 BSH Hausgeräte GmbH Domestic appliance with a touch-sensitive operating device and method for its operation
WO2016078982A1 (en) 2014-11-17 2016-05-26 BSH Hausgeräte GmbH Domestic appliance having a touch-senstive operator control device, and method for operating the same

Similar Documents

Publication Publication Date Title
US11477317B2 (en) Notification of mobile device events
US10089986B2 (en) Systems and methods to present voice message information to a user of a computing device
US20080281940A1 (en) Advertising on a portable communication device
US10080111B2 (en) Techniques for communication using audio stickers
CN108319657B (en) Method for detecting strong rhythm point, storage medium and terminal
US7960945B1 (en) Estimating remaining use time of a mobile device
US20100199176A1 (en) Electronic device with text prediction function and method
US20190188275A1 (en) Method, device, storage medium, terminal for serching and retrieving applications
EP2431900A1 (en) Audiovisual content tagging using biometric sensor
CN106506321B (en) Group message processing method and terminal device
EP2057826B1 (en) System and method for coordinating audiovisual content with contact list information
US20150186612A1 (en) Method and apparatus for distinguishing user health-related states based on user interaction information
US8825783B1 (en) Recording events for social media
JP2023511185A (en) CHAT INFORMATION SEARCH METHOD AND ELECTRONIC DEVICE
US20100067708A1 (en) System and method for automatically updating presence information based on sound detection
CN107291772B (en) Search access method and device and electronic equipment
US20150119004A1 (en) Methods for Voice Management, and Related Devices
EP2827252A1 (en) Device and method for providing user activity information in portable terminal
WO2015106586A1 (en) Method and apparatus for processing reminder notification messages
EP2619967B1 (en) System and method of enhancing messages
AU2014202423B2 (en) Notification of mobile device events
CN112189344A (en) Method and device for selecting audio track from audio/video file
JP6915074B2 (en) Message notification method and terminal
US20110257972A1 (en) System and method for location tracking using audio input
WO2018145539A1 (en) Streaming media data processing method and mobile terminal

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10744765

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 10744765

Country of ref document: EP

Kind code of ref document: A1