US11983305B2 - Content presentation system, content presentation device, and content presentation method - Google Patents

Content presentation system, content presentation device, and content presentation method Download PDF

Info

Publication number
US11983305B2
US11983305B2 US18/001,760 US202118001760A US11983305B2 US 11983305 B2 US11983305 B2 US 11983305B2 US 202118001760 A US202118001760 A US 202118001760A US 11983305 B2 US11983305 B2 US 11983305B2
Authority
US
United States
Prior art keywords
information
emotion
user
content
content information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
US18/001,760
Other versions
US20230244297A1 (en
Inventor
Takanori Ishikawa
Ryo Sasaki
Yuta Aoki
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Group Corp
Original Assignee
Sony Group Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Group Corp filed Critical Sony Group Corp
Assigned to Sony Group Corporation reassignment Sony Group Corporation ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: AOKI, YUTA, ISHIKAWA, TAKANORI, SASAKI, RYO
Publication of US20230244297A1 publication Critical patent/US20230244297A1/en
Application granted granted Critical
Publication of US11983305B2 publication Critical patent/US11983305B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/0205—Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
    • A61B5/02055—Simultaneously evaluating both cardiovascular condition and temperature
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/092—Reinforcement learning
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/63—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for local operation
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2560/00—Constructional details of operational features of apparatus; Accessories for medical measuring apparatus
    • A61B2560/02—Operational features
    • A61B2560/0242—Operational features adapted to measure environmental factors, e.g. temperature, pollution
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2562/00—Details of sensors; Constructional details of sensor housings or probes; Accessories for sensors
    • A61B2562/02—Details of sensors specially adapted for in-vivo measurements
    • A61B2562/0219—Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0002—Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network
    • A61B5/0015—Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network characterised by features of the telemetry system
    • A61B5/002—Monitoring the patient using a local or closed circuit, e.g. in a room or building
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/021—Measuring pressure in heart or blood vessels
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/024—Detecting, measuring or recording pulse rate or heart rate
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/08—Detecting, measuring or recording devices for evaluating the respiratory organs
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1112—Global tracking of patients, e.g. by using GPS
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/145—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue
    • A61B5/14532—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue for measuring glucose, e.g. by tissue impedance measurement
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/145—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue
    • A61B5/14542—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue for measuring blood gases
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/318—Heart-related electrical modalities, e.g. electrocardiography [ECG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01—Indexing scheme relating to G06F3/01
    • G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns

Definitions

  • the present technology relates to a content presentation system, a content presentation device, and a content presentation method.
  • Patent Document 1 discloses “a preference determination system including a biometric information measurement unit that measures biometric information of a user, a behavior information measurement unit that measures behavior information of the user, an attribute information input unit that inputs attribute information of the user, a database that stores past biometric information, behavior information, attribute information, and preference items of a user in association with each other, and a preference determination unit that acquires biometric information measured by the biometric information measurement unit, behavior information measured by the behavior information measurement unit, and attribute information input by the attribute information input unit, and determines preference of the user by collating the acquired biometric information, the behavior information, and the attribute information with the database”.
  • Patent Document 1 describes technology of determining user's preference by a machine learning method and presenting recommended contents or the like to the user.
  • Patent Document 2 discloses “a purchasing behavior management device that gives an instruction for purchasing behavior of food, the purchasing behavior management device including a basic information acquisition unit that acquires basic information that is information regarding a user, and a purchasing behavior determination unit that determines an intake state of various nutritional components of a user on the basis of basic information acquired by the basic information acquisition unit described above and determines propriety of purchasing behavior on the basis of a result of the determination, in which the basic information acquisition unit described above continuously acquires, as the basic information described above, at least biometric information of a user and purchasing behavior information indicating a content of purchasing behavior by a user”.
  • Patent Document 2 describes technology of advising a user on food purchase behavior suitable for the user.
  • Patent Document 1 a machine learning model used in Patent Document 1 and the like is required to have high generalization performance.
  • the generalization performance refers to performance in which determination can be performed appropriately and accurately even on the basis of information obtained from an unlearned user. Patent Document 1 does not disclose this generalization performance.
  • Patent Document 2 information such as things a user likes or dislikes, for example, is required to be registered.
  • information changes depending on the context. Therefore, information corresponding to various contexts, and there is an issue that a burden on a user is large.
  • a main object of the present technology is to provide a content presentation system, a content presentation device, and a content presentation method that reduce a burden on a user and present suitable contents to the user with high accuracy.
  • the present technology provides a content presentation system including a computer device that holds content information associated with emotion information indicating an emotion of a user, in which the computer device at least includes a machine learning model that, on the basis of a plurality of pieces of content information presented to the user corresponding to desired emotion information indicating emotion information desired by the user and content information selected by the user from the plurality of pieces of content information, performs machine learning so as to present the content information suitable for the emotion information.
  • the emotion information and the content information may be associated with each other on the basis of a coordinate system having elements of the emotion as coordinate axes.
  • the coordinate system may be a Russell's circumplex model.
  • Current emotion information indicating the emotion that is current of the user may be estimated on the basis of biometric information obtained from sensing information detected by a biometric sensor.
  • the current emotion information may be corrected on the basis of context information related to a context of the user obtained from sensing information detected by a context sensor.
  • An emotion estimation unit may be further included and the emotion estimation unit may perform machine learning so as to estimate the emotion information suitable for the biometric information on the basis of the biometric information.
  • An operation interface unit may be further included and the operation interface unit may encourage input of the desired emotion information.
  • the operation interface unit may present the content information corresponding to the desired emotion information and/or encourage selection of the content information.
  • the operation interface unit may encourage continuous or stepwise input of the desired emotion information, and the machine learning model, on the basis of a plurality of pieces of the content information presented to the user corresponding to the desired emotion information that is predetermined in a route formed by the input and content information selected by the user from the plurality of pieces of content information, may perform machine learning so as to present the content information suitable for the desired emotion information.
  • the operation interface unit may encourage continuous or stepwise input of the desired emotion information, and the machine learning model, on the basis of a plurality of pieces of the content information presented to the user corresponding to a shape of a route formed by the input and the desired emotion information and content information selected by the user from the plurality of pieces of content information, may perform machine learning so as to present the content information suitable for the desired emotion information.
  • the present technology provides a content presentation device that holds content information associated with emotion information indicating an emotion of a user, the content presentation device at least including a machine learning model that, on the basis of a plurality of pieces of content information presented to the user corresponding to desired emotion information indicating emotion information desired by the user and content information selected by the user from the plurality of pieces of content information, performs machine learning so as to present the content information suitable for the emotion information.
  • the present technology provides a content presentation method including a computer device that holds content information associated with emotion information indicating an emotion of a user, the content presentation method at least including, on the basis of a plurality of pieces of content information presented to the user corresponding to desired emotion information indicating emotion information desired by the user and content information selected by the user from the plurality of pieces of content information, performing machine learning so as to present the content information suitable for the emotion information by the computer device.
  • FIG. 1 is a block diagram illustrating a configuration of a content presentation system 100 according to one embodiment of the present technology.
  • FIG. 2 is a conceptual diagram illustrating an example of an emotion model according to the one embodiment of the present technology.
  • FIG. 3 is a flowchart illustrating an example of a procedure of the content presentation system 100 according to the one embodiment of the present technology.
  • FIG. 4 is a block diagram illustrating a hardware configuration of a computer device 1 according to the one embodiment of the present technology.
  • FIG. 5 is a block diagram illustrating a configuration of a content presentation system 100 according to one embodiment of the present technology.
  • FIG. 6 is a flowchart illustrating an example of a procedure of an emotion estimation unit 15 according to the one embodiment of the present technology.
  • FIG. 7 is a flowchart illustrating an example of a procedure of the content presentation system 100 according to the one embodiment of the present technology.
  • FIG. 8 is a block diagram illustrating a configuration of a content presentation system 100 according to one embodiment of the present technology.
  • FIG. 9 is a flowchart illustrating an example of a procedure of the content presentation system 100 according to the one embodiment of the present technology.
  • FIG. 10 is a diagram illustrating an example of a screen displayed on an operation interface unit 11 according to one embodiment of the present technology.
  • FIG. 11 is a diagram illustrating an example of a screen displayed on an operation interface unit 11 according to one embodiment of the present technology.
  • FIG. 12 is a block diagram illustrating a configuration of a content presentation device 200 according to one embodiment of the present technology.
  • FIG. 13 is a flowchart illustrating an example of a procedure of a content presentation method according to one embodiment of the present technology.
  • a content presentation system can be used, for example, for a user to control his/her emotion.
  • An emotion refers to overall mental process, and include affect, mood, sentiment, and the like. Examples of an emotion include happiness, relaxation, lethargy, anxiety, tension, and the like.
  • a user can control his/her own emotion. For example, a user who feels tense can feel relaxed by viewing contents such as a moving image that makes the user feel relaxed.
  • the content presentation system includes a computer device that holds content information associated with emotion information indicating an emotion of a user.
  • the computer device encourages a user to input emotion information desired by the user (desired emotion information).
  • the computer device presents content information associated with the emotion information to the user.
  • the content information includes, for example, information such as a scent, temperature, and lighting in addition to a moving image, a still image, voice, music, text, and the like.
  • the scent includes, in addition to a scent that can be perceived by a human as a scent, a scent that cannot be perceived by a human as a scent but is inhaled to exert some action on a human.
  • a medical sedative to be inhaled, gas that is odorless and acts on a physical condition of a human by being inhaled, such as oxygen or carbon dioxide, or the like is also included in the scent.
  • FIG. 1 is a block diagram illustrating a configuration of a content presentation system 100 according to the one embodiment of the present technology.
  • the content presentation system 100 according to the one embodiment of the present technology includes a computer device 1 .
  • the computer device 1 can include, for example, an operation interface unit 11 , a storage unit 12 , a machine learning model 13 , and a control unit 14 .
  • the operation interface unit 11 encourages a user to input desired emotion information.
  • the operation interface unit 11 can be implemented by using, for example, a touch panel or the like.
  • the operation interface unit 11 can acquire desired emotion information by recognizing a touch operation of a user.
  • the operation interface unit 11 can be implemented by using, for example, a microphone or the like.
  • the operation interface unit 11 can acquire desired emotion information by recognizing spoken voice of a user.
  • the operation interface unit 11 can be implemented by using, for example, a camera or the like.
  • the operation interface unit 11 can acquire desired emotion information by performing image recognition on facial expression, a line-of-sight, and the like of a user.
  • the storage unit 12 holds content information associated with emotion information.
  • the storage unit 12 may hold, for example, parameters of the machine learning model 13 , history information related to user operations, attribute information related to a user, and the like.
  • the machine learning model 13 performs machine learning so as to present content information suitable for emotion information. More specifically, the machine learning model 13 , on the basis of a plurality of pieces of content information presented to a user corresponding to the emotion information and content information selected by the user from the plurality of pieces of content information, performs machine learning so as to present the content information suitable for the emotion information.
  • the control unit 14 controls operation of the operation interface unit 11 , the storage unit 12 , and the machine learning model 13 .
  • the content presentation system 100 encourages the user to select optimal content information from the plurality of pieces of content information.
  • the plurality of pieces of presented content information may be ranked in a suitable order for emotion information of a user.
  • the machine learning model 13 can learn content information suitable for the emotion of the user.
  • the content presentation system 100 can present content information suitable for emotion information of the user.
  • the presented content information is customized for the user.
  • What the machine learning model 13 should figure out is to “present content information suitable for emotion information”. Determination as to whether or not it is suitable can be made, for example, by a difference between content information presented to a user and content information selected by the user. As the difference is smaller, it can be said that contents suitable for the emotion information can be presented.
  • the content information presented to a user may be, for example, content information suitable for the characteristic of the user or content information that tends to be suitable for a large number of users.
  • the former corresponds to, for example, favorite music or the like.
  • the latter corresponds to, for example, popular music or the like.
  • the content information is obtained from history information related to selection of content information or the like.
  • the emotion information may be obtained from a biometric sensor, or may be obtained by the operation interface unit 11 encouraging a user to input. Details will be described below.
  • the machine learning model 13 performs machine learning so as to reduce this difference.
  • the machine learning technique is not particularly specified, but for example, parameters (weighting coefficients) of a neural network that is the machine learning model 13 may be updated by reinforcement learning.
  • the machine learning model 13 can be implemented to obtain a higher reward as the difference is smaller to perform reinforcement learning.
  • the difference may be quantified.
  • the machine learning model 13 may update this difference by machine learning.
  • a user only needs to select content information suitable for his/her own emotion.
  • the user does not need to register attribute information, for example, such as his/her age, sex, and favorite food in advance. Therefore, a burden on a user is small.
  • FIG. 2 is a conceptual diagram illustrating an example of an emotion model according to the one embodiment of the present technology.
  • FIG. 2 illustrates a coordinate system including elements of an emotion as coordinate axes.
  • a two-dimensional coordinate system is illustrated in which the horizontal axis represents valence of “pleasant—unpleasant” and the vertical axis represents arousal.
  • the elements of an emotion are arranged in an annular shape. For example, similar emotions such as “happy” and “contented” are arranged at positions close to each other on the annulus. On the other hand, for example, dissimilar emotions such as “happy” and “sad” are arranged at opposing positions on the annulus.
  • the storage unit 12 includes this coordinate system.
  • This coordinate system is conventionally called Russell's circumplex model.
  • This Russell's circumplex model can be used for associating emotion information with content information.
  • a coordinate system that is not a Russell's circumplex model may be used for associating emotion information with content information.
  • Millenson's three-dimensional model or the like may be used for associating emotion information with content information.
  • Emotion information related to a user can be mapped to a specific coordinate with reference to the two axes.
  • the intensity of an emotion related to a user can be indicated by a distance from an intersection X of the two axes to the specific coordinate.
  • an emotion related to a user can be quantitatively evaluated.
  • Emotion information and content information are associated with coordinate information. As a result, emotion information and content information can be associated. In a case where a coordinate related to emotion information is selected, content information related to the coordinate is selected.
  • FIG. 3 is a flowchart illustrating an example of the procedure of the content presentation system 100 according to the one embodiment of the present technology.
  • the machine learning model 13 selects a plurality of pieces of content information associated with obtained emotion information.
  • the plurality of pieces of content information may be ranked according to the emotion information, for example.
  • step S 12 the operation interface unit 11 presents the selected plurality of pieces of content information to a user.
  • the operation interface unit 11 encourages the user to select specific content information from the plurality of pieces of content information.
  • step S 13 the machine learning model 13 , on the basis of the plurality of pieces of content information presented to the user and the content information selected by the user from the plurality of pieces of content information, performs machine learning so as to present the content information suitable for the emotion information.
  • a coordinate related to emotion information desired by a user is assumed to be (0, ⁇ 10).
  • the machine learning model 13 selects a plurality of pieces of content information associated with the coordinate or a coordinate near the coordinate (step S 11 ).
  • the operation interface unit 11 presents the plurality of pieces of content information to the user and encourages selection (step S 12 ).
  • the machine learning model 13 corrects the coordinate related to the content information to (0, ⁇ 10) (step S 13 ).
  • FIG. 4 is a block diagram illustrating the hardware configuration of the computer device 1 according to the one embodiment of the present technology.
  • the computer device 1 can include a CPU 101 , a storage 102 , a random access memory (RAM) 103 , a communication interface 104 , and a touch panel 105 as components.
  • Each of the components is connected by, for example, a bus as a data transmission path.
  • the CPU 101 is implemented by, for example, a microcomputer, and controls each of the components of the computer device 1 .
  • the CPU 101 can function as, for example, the control unit 14 , the machine learning model 13 , and the like.
  • the machine learning model 13 can be implemented by, for example, a program.
  • the CPU 101 functions by reading this program.
  • the storage 102 stores control data and the like such as a program and operation parameters used by the CPU 101 .
  • the storage 102 can be implemented by using, for example, a hard disk drive (HDD), a solid state drive (SSD), or the like.
  • the storage 102 can function as, for example, the storage unit 12 .
  • the RAM 103 temporarily stores, for example, a program, and the like executed by the CPU 101 .
  • the communication interface 104 has a function of communicating via an information communication network 3 using communication technology such as Wi-Fi, Bluetooth (registered trademark), or long term evolution (LTE) for example.
  • communication technology such as Wi-Fi, Bluetooth (registered trademark), or long term evolution (LTE) for example.
  • the touch panel 105 encourages a user to perform an operation by a touch operation.
  • the touch panel 105 can function as, for example, the operation interface unit 11 .
  • the computer device 1 may be, for example, a smartphone terminal, a tablet terminal, a mobile phone terminal, a personal digital assistant (PDA), a personal computer (PC), a portable music player, a portable game machine, or a wearable terminal (head mounted display: HMD, glasses-type HMD, watch-type terminal, band-type terminal, and the like).
  • PDA personal digital assistant
  • PC personal computer
  • portable music player a portable game machine
  • wearable terminal head mounted display: HMD, glasses-type HMD, watch-type terminal, band-type terminal, and the like.
  • a program for implementing the machine learning model 13 and the like may be stored in another computer device of the content presentation system 100 or a computer system.
  • the content presentation system 100 can use a cloud service that provides the function of this program.
  • the cloud service include software as a service (SaaS), infrastructure as a service (IaaS), platform as a service (PaaS), and the like.
  • the program can be stored using various types of non-transitory computer readable media and supplied to the computer.
  • the non-transitory computer readable media include various types of tangible storage media. Examples of the non-transitory computer readable media include a magnetic recording medium (for example, flexible disk, magnetic tape, or hard disk drive), a magneto-optical recording medium (for example, magneto-optical disk), a compact disc read only memory (CD-ROM), a CD-R, a CD-R/W, and a semiconductor memory (for example, mask ROM, programmable ROM (PROM), erasable PROM (EPROM), flash ROM, or random access memory (RAM)).
  • the program described above may be supplied to the computer by various types of transitory computer readable media. Examples of the transitory computer readable media include electrical signals, optical signals, and electromagnetic waves.
  • the transitory computer readable medium can supply the program described above to the computer via a wired communication path such as an electric wire and an optical fiber or a wireless communication path.
  • a content presentation system 100 according to one embodiment of the present technology can include a biometric sensor or the like to estimate a current emotion of a user. This will be described with reference to FIG. 5 .
  • FIG. 5 is a block diagram illustrating a configuration of the content presentation system 100 according to the one embodiment of the present technology.
  • the content presentation system 100 can further include a biometric sensor 2 and an emotion estimation unit 15 .
  • the emotion estimation unit 15 is included in a computer device 1 .
  • the biometric sensor 2 and the computer device 1 are connected via an information communication network 3 .
  • the biometric sensor can acquire biometric information related to a user.
  • the biometric information includes, for example, a heart rate, body temperature, blood pressure, blood oxygen concentration, respiration, water, blood glucose, an electrocardiogram, brain waves, and the like.
  • the emotion estimation unit 15 estimates current emotion information indicating a current emotion of a user on the basis of biometric information obtained from sensing information detected by the biometric sensor 2 .
  • FIG. 6 is a flowchart illustrating the example of a procedure of the emotion estimation unit 15 according to the one embodiment of the present technology.
  • step S 21 the emotion estimation unit 15 performs preprocessing on sensing information detected by the biometric sensor 2 .
  • the emotion estimation unit 15 decomposes the sensing information or removes unnecessary information such as noise.
  • step S 22 the emotion estimation unit 15 extracts features included in the sensing information to obtain biometric information. These features may be predefined or may be automatically extracted by, for example, deep learning.
  • step S 23 the emotion estimation unit 15 estimates emotion information on the basis of the biometric information.
  • the emotion estimation unit 15 can perform machine learning so as to estimate emotion information suitable for the biometric information on the basis of the biometric information.
  • the emotion estimation unit 15 can be implemented by using, for example, a neural network on which supervised learning is performed. This supervised learning can be performed, for example, by using biometric information as input and emotion information as output.
  • biometric information of a user has a characteristic depending on the user, according to the present technology, suitable emotion information can be estimated by the characteristic being incorporated.
  • the emotion estimation unit 15 may perform machine learning so as to estimate emotion information suitable for the biometric information on the basis of biometric information in a case where a user is viewing contents or the like. As a result, the degree of association between the emotion information and the content information is stronger, and content information more suitable for the emotion information is presented.
  • the emotion estimation unit 15 may use, for example, logistic regression, support vector machine, Bayesian estimation, decision tree, random forest, k-nearest neighbor algorithm, principal component analysis, k-means clustering, or the like.
  • the current emotion information estimated by the emotion estimation unit 15 is presented in the emotion model illustrated in FIG. 2 .
  • the user can objectively grasp his/her current emotion.
  • the user can easily select a desired emotion. For example, in a case where a current emotion is a near-lethargic emotion even through the user is at work, the user may select an emotion that enhances concentration.
  • FIG. 7 is a flowchart illustrating an example of the procedure of the content presentation system 100 according to the one embodiment of the present technology.
  • step S 31 the biometric sensor 2 detects sensing information from a user and acquires biometric information.
  • step S 32 the emotion estimation unit 15 estimates current emotion information related to the user on the basis of the biometric information.
  • step S 33 the operation interface unit 11 presents the current emotion information to the user.
  • step S 34 the operation interface unit 11 encourages input of the emotion information desired by the user.
  • steps S 35 to S 37 Since the procedure of steps S 35 to S 37 is similar to that of steps S 11 to S 13 in FIG. 3 , detailed description thereof is omitted.
  • the biometric information obtained in the second embodiment may have different features depending on a context of a user. For example, in a case where a heart rate in a case where a user is sitting is compared with a heart rate in a case where the user is standing, the heart rate in a case where the user is standing tends to be higher. Therefore, in a case where the emotion estimation unit 15 is generated on the basis of biometric information in a case where the user is sitting, an emotion in a case where the user is standing may be erroneously estimated as “nervous”. Therefore, biometric information is preferably corrected according to the context of a user.
  • a content presentation system 100 can include a context sensor or the like to estimate a current context of a user.
  • the context sensor obtains context information indicating information related to a context such as posture or a location of the user.
  • the context information obtains context information such as, for example, whether the user is indoors or outdoors and whether the user is meeting a friend, shopping, walking, or running.
  • the context sensor can be implemented by using, for example, an angular rate sensor, an acceleration sensor, an inertial measurement unit (IMU), a global positioning system (GPS) positioning unit, a microphone, an ambient light sensor, a schedule management application, or the like.
  • an angular rate sensor, an acceleration sensor, an inertial sensor, or the like context information such as whether the user is walking or walking is obtained.
  • a GPS positioning unit, a microphone, an ambient light sensor, or the like context information such as whether the user is indoors or outdoors is obtained.
  • context information such as whether the user is meeting a friend can be obtained.
  • FIG. 8 is a block diagram illustrating the configuration of the content presentation system 100 according to the one embodiment of the present technology.
  • the content presentation system 100 can further include a context sensor 4 and a context estimation unit 16 .
  • the context estimation unit 16 is included in a computer device 1 .
  • the context sensor 4 and the computer device 1 are connected via an information communication network 3 .
  • the context estimation unit 16 estimates context information on the basis of sensing information detected by the context sensor 4 .
  • the context estimation unit 16 can be implemented by using, for example, a neural network on which supervised learning is performed. This supervised learning can be performed, for example, by using sensing information as input and context information as output.
  • An emotion estimation unit 15 can correct current emotion information on the basis of the context information. Using the example of the heart rate described above, in a case where a heart rate is high because a user is standing, the emotion estimation unit 15 may subtract the heart rate. Alternatively, in a case where the emotion estimation unit 15 is generated, the emotion estimation unit 15 may be generated using the subtracted heart rate as an explanatory variable.
  • the emotion estimation unit 15 may correct, for example, the intersection X of the coordinate system illustrated in FIG. 2 on the basis of the context information. As a result, for example, the intensity of an emotion or the like is corrected.
  • the coordinate system and the context information may be associated with each other.
  • a coordinate system in a case where a user is sitting may be different from a coordinate system in a case where the user is standing.
  • biometric information after the correction may be an m*n-dimensional vector.
  • the emotion estimation unit 15 can perform machine learning so as to estimate emotion information suitable for the biometric information on the basis of content information in addition to the biometric information.
  • the emotion estimation unit 15 can be implemented by using, for example, a neural network on which supervised learning is performed. This supervised learning can be performed, for example, by using biometric information and content information as input and emotion information as output.
  • the emotion estimation unit 15 may perform machine learning on the basis of a data set in which emotion information and content information for every context are associated with each other.
  • FIG. 9 is a flowchart illustrating an example of the procedure of the content presentation system 100 according to the one embodiment of the present technology.
  • a biometric sensor 2 detects sensing information from a user and acquires biometric information.
  • step S 42 the emotion estimation unit 15 estimates current emotion information related to the user on the basis of the biometric information.
  • step S 43 the context estimation unit 16 estimates context information on the basis of sensing information detected by the context sensor 4 .
  • step S 44 the emotion estimation unit 15 corrects the current emotion information on the basis of the context information.
  • steps S 45 to S 49 Since the procedure of steps S 45 to S 49 is similar to that of steps S 33 to S 37 in FIG. 7 , detailed description thereof is omitted.
  • an operation interface unit 11 that encourages a user to input desired emotion information, for example, a microphone or a camera may be used, but here, an operation interface unit 11 in a case where a display is used as one embodiment will be described.
  • FIG. 10 is a diagram illustrating an example of a screen displayed on the operation interface unit 11 according to the one embodiment of the present technology. Similarly to FIG. 2 , a coordinate system including elements of an emotion as coordinate axes is illustrated.
  • current emotion information indicating a current emotion of a user is displayed at a first point P 1 .
  • the user who has objectively recognized the current own emotion can move the first point P 1 to a second point P 2 indicating desired emotion information.
  • This operation can be performed by a touch operation as long as the operation interface unit 11 includes a touch panel. Alternatively, this operation may be performed by using a mouse. Alternatively, in a case where a head mounted display (HMD) includes the operation interface unit 11 , for example, this operation may be performed by an angular rate sensor or the like included in the HMD detecting the angle of the head of the user, or this operation may be performed by the operation interface unit 11 detecting the line-of-sight of the user.
  • HMD head mounted display
  • the machine learning model 13 selects a plurality of pieces of content information corresponding to the second point P 2 .
  • the operation interface unit 11 presents the plurality of pieces of content information corresponding to the second point P 2 selected by the machine learning model 13 to the user. Moreover, the operation interface unit 11 encourages the user to select specific content information from the plurality of pieces of content information.
  • the second point P 2 indicating desired emotion information is selected in units of millimeters by intuition of the user. Therefore, the position of the second point P 2 varies. Therefore, by the plurality of pieces of content information being presented, the influence of this variation is reduced.
  • a plurality of second points P 2 may be selected.
  • the operation interface unit 11 presents a plurality of pieces of content information corresponding to the plurality of respective second points to a user.
  • the operation interface unit 11 may encourage input of one coordinate, or may encourage continuous or stepwise input of a plurality of coordinates.
  • FIG. 11 is a diagram illustrating an example of a screen displayed on the operation interface unit 11 according to the one embodiment of the present technology.
  • a first point P 1 is displayed similarly to the fourth embodiment.
  • a user who has objectively recognized a current own emotion inputs a route R indicating desired emotion information by moving the first point P 1 .
  • the route R is formed by continuous or stepwise input of a plurality of coordinates.
  • the user can move the first point P 1 to a second point P 2 via a first via point V 1 and a second via point V 2 .
  • This operation can be performed by a touch operation or the like similarly to the fourth embodiment.
  • the machine learning model 13 selects a plurality of pieces of content information corresponding to one or a plurality of via points included in the route R. In an implementation illustrated in FIG. 11 , a plurality of pieces of content information corresponding to the first via point V 1 and a plurality of pieces of content information corresponding to the second via point V 2 are selected.
  • the operation interface unit 11 presents the plurality of pieces of content information corresponding to the first via point V 1 and the plurality of pieces of content information corresponding to the second via point V 2 selected by the machine learning model 13 to the user. Moreover, the operation interface unit 11 encourages the user to select specific content information from the plurality of pieces of content information.
  • the machine learning model 13 on the basis of the plurality of pieces of content information presented to the user corresponding to predetermined desired emotion information in the route R (first via point V 1 and second via point V 2 in the present implementation) and content information selected by the user from the plurality of pieces of content information, performs machine learning so as to present the content information suitable for the desired emotion information.
  • the user can move from the first point P 1 straight to the second point P 2 .
  • the fourth embodiment is used, for example, in a case where the user wants to immediately feel an emotion indicated at the second point P 2 .
  • the user can move from the first point P 1 to the second point P 2 while detouring around.
  • the fifth embodiment is used, for example, in a case where the user wants to feel a final emotion while feeling various emotions.
  • a specific example in a case where a nervous user wants to exert his/her performance, there is a case where the performance can be exerted more by viewing contents by which the user feels relaxed and then viewing contents by which the user feels excited than by immediately viewing contents by which the user feels excited.
  • content information can be flexibly presented according to the characteristic of the user.
  • the machine learning model 13 may select a plurality of pieces of content information corresponding to the shape of the route R and the second point P 2 .
  • the machine learning model 13 can select, for example, a plurality of pieces of content information corresponding to the degree of the curve included in the shape.
  • the operation interface unit 11 presents the plurality of pieces of content information corresponding to the shape of the route R and the second point P 2 selected by the machine learning model 13 to the user. Moreover, the operation interface unit 11 encourages the user to select specific content information from the plurality of pieces of content information.
  • the machine learning model 13 on the basis of the plurality of pieces of content information presented to the user corresponding to the shape of the route R and the second point P 2 and content information selected by the user from the plurality of pieces of content information, performs machine learning so as to present the content information suitable for the desired emotion information.
  • a plurality of pieces of content information presented corresponding to the second point P 2 is different between a case of moving from the first point P 1 straight to the second point P 2 and a case of moving from the first point P 1 to the second point P 2 while detouring around. Therefore, content information in which the process of changing emotions of a user is considered is presented to the user.
  • FIG. 12 is a block diagram illustrating a configuration of a content presentation device 200 according to one embodiment of the present technology.
  • the content presentation device 200 can include, for example, an operation interface unit 11 , a storage unit 12 , a machine learning model 13 , and a control unit 14 .
  • the machine learning model 13 performs machine learning so as to present content information suitable for emotion information. More specifically, the machine learning model 13 , on the basis of a plurality of pieces of content information presented to a user corresponding to the emotion information and content information selected by the user from the plurality of pieces of content information, performs machine learning so as to present the content information suitable for the emotion information.
  • the technology related to the content presentation system 100 described above may be used. Therefore, repeated description of the operation interface unit 11 , the storage unit 12 , the machine learning model 13 , and the control unit 14 is omitted.
  • a content presentation method includes a computer device that holds content information associated with emotion information indicating an emotion of a user.
  • FIG. 13 is a flowchart illustrating an example of the procedure of the content presentation method according to the one embodiment of the present technology.
  • the content presentation method at least includes, on the basis of a plurality of pieces of content information presented to the user corresponding to desired emotion information indicating emotion information desired by the user and content information selected by the user from the plurality of pieces of content information, performing machine learning so as to present the content information suitable for the emotion information by the computer device (step S 1 ).
  • the technology according to another embodiment described above may be used. Therefore, repeated description of the technology described in the embodiments described above is omitted.
  • the content presentation method according to the present embodiment can be implemented by using software and hardware. Specifically, for example, a CPU included in hardware reads a program for implementing the content presentation method according to the present embodiment, whereby the content presentation method according to the present embodiment can be implemented.
  • a content presentation system including a computer device that holds content information associated with emotion information indicating an emotion of a user
  • the content presentation system according to [4] or [5] further including an emotion estimation unit,
  • the content presentation system according to any one of [1] to [6] further including an operation interface unit,
  • a content presentation device that holds content information associated with emotion information indicating an emotion of a user, the content presentation device at least including
  • a content presentation method including a computer device that holds content information associated with emotion information indicating an emotion of a user, the content presentation method at least including,

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Public Health (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Psychiatry (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Data Mining & Analysis (AREA)
  • Cardiology (AREA)
  • Physiology (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Computation (AREA)
  • Animal Behavior & Ethology (AREA)
  • Surgery (AREA)
  • Veterinary Medicine (AREA)
  • Computing Systems (AREA)
  • Epidemiology (AREA)
  • Software Systems (AREA)
  • Primary Health Care (AREA)
  • Computational Linguistics (AREA)
  • Social Psychology (AREA)
  • Psychology (AREA)
  • Hospice & Palliative Care (AREA)
  • Fuzzy Systems (AREA)
  • Educational Technology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Developmental Disabilities (AREA)
  • Signal Processing (AREA)
  • Child & Adolescent Psychology (AREA)

Abstract

A content presentation system, a content presentation device, and a content presentation method that reduce a burden on a user and present suitable contents to the user with high accuracy are provided. The present technology provides a content presentation system including a computer device that holds content information associated with emotion information indicating an emotion of a user, in which the computer device at least includes a machine learning model that, on the basis of a plurality of pieces of content information presented to the user corresponding to desired emotion information indicating emotion information desired by the user and content information selected by the user from the plurality of pieces of content information, performs machine learning so as to present the content information suitable for the emotion information.

Description

CROSS REFERENCE TO RELATED APPLICATIONS
This application is a national stage application under 35 U.S.C. 371 and claims the benefit of PCT Application No. PCT/JP2021/018923, having an international filing date of 19 May 2021, which designated the United States, which PCT application claimed the benefit of Japanese Patent Application No. 2020-109359, filed 25 Jun. 2020, the entire disclosures of each of which are incorporated herein by reference.
TECHNICAL FIELD
The present technology relates to a content presentation system, a content presentation device, and a content presentation method.
BACKGROUND ART
Conventionally, technology has been disclosed in which contents associated with biometric information, behavior information, attribute information, and the like related to a user are presented to the user.
For example, Patent Document 1 discloses “a preference determination system including a biometric information measurement unit that measures biometric information of a user, a behavior information measurement unit that measures behavior information of the user, an attribute information input unit that inputs attribute information of the user, a database that stores past biometric information, behavior information, attribute information, and preference items of a user in association with each other, and a preference determination unit that acquires biometric information measured by the biometric information measurement unit, behavior information measured by the behavior information measurement unit, and attribute information input by the attribute information input unit, and determines preference of the user by collating the acquired biometric information, the behavior information, and the attribute information with the database”. Patent Document 1 describes technology of determining user's preference by a machine learning method and presenting recommended contents or the like to the user.
For example, Patent Document 2 discloses “a purchasing behavior management device that gives an instruction for purchasing behavior of food, the purchasing behavior management device including a basic information acquisition unit that acquires basic information that is information regarding a user, and a purchasing behavior determination unit that determines an intake state of various nutritional components of a user on the basis of basic information acquired by the basic information acquisition unit described above and determines propriety of purchasing behavior on the basis of a result of the determination, in which the basic information acquisition unit described above continuously acquires, as the basic information described above, at least biometric information of a user and purchasing behavior information indicating a content of purchasing behavior by a user”. Patent Document 2 describes technology of advising a user on food purchase behavior suitable for the user.
CITATION LIST Patent Document
  • Patent Document 1: Japanese Patent Application Laid-Open No. 2014-219937
  • Patent Document 2: Japanese Patent Application Laid-Open No. 2007-328464
SUMMARY OF THE INVENTION Problems to be Solved by the Invention
However, for example, a machine learning model used in Patent Document 1 and the like is required to have high generalization performance. The generalization performance refers to performance in which determination can be performed appropriately and accurately even on the basis of information obtained from an unlearned user. Patent Document 1 does not disclose this generalization performance.
Furthermore, for example, in the technology disclosed in Patent Document 2 and the like, information such as things a user likes or dislikes, for example, is required to be registered. However, such information changes depending on the context. Therefore, information corresponding to various contexts, and there is an issue that a burden on a user is large.
Therefore, a main object of the present technology is to provide a content presentation system, a content presentation device, and a content presentation method that reduce a burden on a user and present suitable contents to the user with high accuracy.
Solutions to Problems
The present technology provides a content presentation system including a computer device that holds content information associated with emotion information indicating an emotion of a user, in which the computer device at least includes a machine learning model that, on the basis of a plurality of pieces of content information presented to the user corresponding to desired emotion information indicating emotion information desired by the user and content information selected by the user from the plurality of pieces of content information, performs machine learning so as to present the content information suitable for the emotion information.
The emotion information and the content information may be associated with each other on the basis of a coordinate system having elements of the emotion as coordinate axes.
The coordinate system may be a Russell's circumplex model.
Current emotion information indicating the emotion that is current of the user may be estimated on the basis of biometric information obtained from sensing information detected by a biometric sensor.
The current emotion information may be corrected on the basis of context information related to a context of the user obtained from sensing information detected by a context sensor.
An emotion estimation unit may be further included and the emotion estimation unit may perform machine learning so as to estimate the emotion information suitable for the biometric information on the basis of the biometric information.
An operation interface unit may be further included and the operation interface unit may encourage input of the desired emotion information.
The operation interface unit may present the content information corresponding to the desired emotion information and/or encourage selection of the content information.
The operation interface unit may encourage continuous or stepwise input of the desired emotion information, and the machine learning model, on the basis of a plurality of pieces of the content information presented to the user corresponding to the desired emotion information that is predetermined in a route formed by the input and content information selected by the user from the plurality of pieces of content information, may perform machine learning so as to present the content information suitable for the desired emotion information.
The operation interface unit may encourage continuous or stepwise input of the desired emotion information, and the machine learning model, on the basis of a plurality of pieces of the content information presented to the user corresponding to a shape of a route formed by the input and the desired emotion information and content information selected by the user from the plurality of pieces of content information, may perform machine learning so as to present the content information suitable for the desired emotion information.
Furthermore, the present technology provides a content presentation device that holds content information associated with emotion information indicating an emotion of a user, the content presentation device at least including a machine learning model that, on the basis of a plurality of pieces of content information presented to the user corresponding to desired emotion information indicating emotion information desired by the user and content information selected by the user from the plurality of pieces of content information, performs machine learning so as to present the content information suitable for the emotion information.
Furthermore, the present technology provides a content presentation method including a computer device that holds content information associated with emotion information indicating an emotion of a user, the content presentation method at least including, on the basis of a plurality of pieces of content information presented to the user corresponding to desired emotion information indicating emotion information desired by the user and content information selected by the user from the plurality of pieces of content information, performing machine learning so as to present the content information suitable for the emotion information by the computer device.
BRIEF DESCRIPTION OF DRAWINGS
FIG. 1 is a block diagram illustrating a configuration of a content presentation system 100 according to one embodiment of the present technology.
FIG. 2 is a conceptual diagram illustrating an example of an emotion model according to the one embodiment of the present technology.
FIG. 3 is a flowchart illustrating an example of a procedure of the content presentation system 100 according to the one embodiment of the present technology.
FIG. 4 is a block diagram illustrating a hardware configuration of a computer device 1 according to the one embodiment of the present technology.
FIG. 5 is a block diagram illustrating a configuration of a content presentation system 100 according to one embodiment of the present technology.
FIG. 6 is a flowchart illustrating an example of a procedure of an emotion estimation unit 15 according to the one embodiment of the present technology.
FIG. 7 is a flowchart illustrating an example of a procedure of the content presentation system 100 according to the one embodiment of the present technology.
FIG. 8 is a block diagram illustrating a configuration of a content presentation system 100 according to one embodiment of the present technology.
FIG. 9 is a flowchart illustrating an example of a procedure of the content presentation system 100 according to the one embodiment of the present technology.
FIG. 10 is a diagram illustrating an example of a screen displayed on an operation interface unit 11 according to one embodiment of the present technology.
FIG. 11 is a diagram illustrating an example of a screen displayed on an operation interface unit 11 according to one embodiment of the present technology.
FIG. 12 is a block diagram illustrating a configuration of a content presentation device 200 according to one embodiment of the present technology.
FIG. 13 is a flowchart illustrating an example of a procedure of a content presentation method according to one embodiment of the present technology.
 MODE FOR CARRYING OUT THE INVENTION
Hereinafter, preferred embodiments for carrying out the present technology will be described. The embodiments described below illustrate examples of representative embodiments of the present technology, and the scope of the present technology is not narrowly interpreted by these. Furthermore, each drawing is a schematic view, and is not necessarily strictly illustrated.
The description of the present technology will be given in the following order.
1. First Embodiment of Present Technology (Example 1 of Content Presentation System)
    • (1) Outline of Present Embodiment
    • (2) Emotion Model
    • (3) Flowchart
    • (4) Hardware Configuration
2. Second Embodiment of Present Technology (Example 2 of Content Presentation System)
3. Third Embodiment of Present Technology (Example 3 of Content Presentation System)
4. Fourth Embodiment of Present Technology (Example 4 of Content Presentation System)
5. Fifth Embodiment of Present Technology (Example 5 of Content Presentation System)
6. Sixth Embodiment of Present Technology (Content Presentation Device)
7. Seventh Embodiment of Present Technology (Content Presentation Method)
1. First Embodiment of Present Technology (Example 1 of Content Presentation System) (1) Outline of Present Embodiment
A content presentation system according to one embodiment of the present technology can be used, for example, for a user to control his/her emotion. An emotion refers to overall mental process, and include affect, mood, sentiment, and the like. Examples of an emotion include happiness, relaxation, lethargy, anxiety, tension, and the like.
According to the present technology, a user can control his/her own emotion. For example, a user who feels tense can feel relaxed by viewing contents such as a moving image that makes the user feel relaxed.
The content presentation system according to the one embodiment of the present technology includes a computer device that holds content information associated with emotion information indicating an emotion of a user. The computer device encourages a user to input emotion information desired by the user (desired emotion information). The computer device presents content information associated with the emotion information to the user.
The content information includes, for example, information such as a scent, temperature, and lighting in addition to a moving image, a still image, voice, music, text, and the like. Note that the scent includes, in addition to a scent that can be perceived by a human as a scent, a scent that cannot be perceived by a human as a scent but is inhaled to exert some action on a human. For example, a medical sedative to be inhaled, gas that is odorless and acts on a physical condition of a human by being inhaled, such as oxygen or carbon dioxide, or the like is also included in the scent.
A configuration of the content presentation system according to a first embodiment of the present technology will be described with reference to FIG. 1 . FIG. 1 is a block diagram illustrating a configuration of a content presentation system 100 according to the one embodiment of the present technology. As illustrated in FIG. 1 , the content presentation system 100 according to the one embodiment of the present technology includes a computer device 1. The computer device 1 can include, for example, an operation interface unit 11, a storage unit 12, a machine learning model 13, and a control unit 14.
The operation interface unit 11 encourages a user to input desired emotion information. The operation interface unit 11 can be implemented by using, for example, a touch panel or the like. The operation interface unit 11 can acquire desired emotion information by recognizing a touch operation of a user.
Alternatively, the operation interface unit 11 can be implemented by using, for example, a microphone or the like. The operation interface unit 11 can acquire desired emotion information by recognizing spoken voice of a user.
Alternatively, the operation interface unit 11 can be implemented by using, for example, a camera or the like. The operation interface unit 11 can acquire desired emotion information by performing image recognition on facial expression, a line-of-sight, and the like of a user.
The storage unit 12 holds content information associated with emotion information. In addition, the storage unit 12 may hold, for example, parameters of the machine learning model 13, history information related to user operations, attribute information related to a user, and the like.
The machine learning model 13 performs machine learning so as to present content information suitable for emotion information. More specifically, the machine learning model 13, on the basis of a plurality of pieces of content information presented to a user corresponding to the emotion information and content information selected by the user from the plurality of pieces of content information, performs machine learning so as to present the content information suitable for the emotion information.
The control unit 14 controls operation of the operation interface unit 11, the storage unit 12, and the machine learning model 13.
Here, the fact that a plurality of pieces of content information is selected will be described. In a case where the machine learning model 13 is generated, a data set in which emotion information and content information related to a plurality of people are associated with each other is used. In a case where a user to be learned by the machine learning model 13 is not included in the plurality of people, there is a possibility that the presented content information is not optimal for emotion information of the user since the characteristic of the user has not been learned.
Therefore, by presenting a plurality of pieces of content information to a user, the content presentation system 100 encourages the user to select optimal content information from the plurality of pieces of content information. The plurality of pieces of presented content information may be ranked in a suitable order for emotion information of a user. By content information being suitably selected for an emotion of a user, the machine learning model 13 can learn content information suitable for the emotion of the user. As a result, the content presentation system 100 can present content information suitable for emotion information of the user. The presented content information is customized for the user.
What the machine learning model 13 should figure out is to “present content information suitable for emotion information”. Determination as to whether or not it is suitable can be made, for example, by a difference between content information presented to a user and content information selected by the user. As the difference is smaller, it can be said that contents suitable for the emotion information can be presented.
The content information presented to a user may be, for example, content information suitable for the characteristic of the user or content information that tends to be suitable for a large number of users. The former corresponds to, for example, favorite music or the like. The latter corresponds to, for example, popular music or the like. The content information is obtained from history information related to selection of content information or the like.
Note that the emotion information may be obtained from a biometric sensor, or may be obtained by the operation interface unit 11 encouraging a user to input. Details will be described below.
The machine learning model 13 performs machine learning so as to reduce this difference. The machine learning technique is not particularly specified, but for example, parameters (weighting coefficients) of a neural network that is the machine learning model 13 may be updated by reinforcement learning. The machine learning model 13 can be implemented to obtain a higher reward as the difference is smaller to perform reinforcement learning.
Alternatively, the difference may be quantified. The machine learning model 13 may update this difference by machine learning.
According to the present technology, a user only needs to select content information suitable for his/her own emotion. The user does not need to register attribute information, for example, such as his/her age, sex, and favorite food in advance. Therefore, a burden on a user is small.
According to the present technology, learning can be performed even for an unlearned user, and suitable content information is presented with high accuracy. Therefore, generalization performance of the machine learning model 13 is high.
Note that these effects are similar in other embodiments described below. Therefore, in the description of other embodiments, repeated description is omitted.
(2) Emotion Model
The association between emotion information and content information will be described with reference to FIG. 2 . FIG. 2 is a conceptual diagram illustrating an example of an emotion model according to the one embodiment of the present technology.
FIG. 2 illustrates a coordinate system including elements of an emotion as coordinate axes. A two-dimensional coordinate system is illustrated in which the horizontal axis represents valence of “pleasant—unpleasant” and the vertical axis represents arousal. The elements of an emotion are arranged in an annular shape. For example, similar emotions such as “happy” and “contented” are arranged at positions close to each other on the annulus. On the other hand, for example, dissimilar emotions such as “happy” and “sad” are arranged at opposing positions on the annulus.
The storage unit 12 includes this coordinate system. This coordinate system is conventionally called Russell's circumplex model. This Russell's circumplex model can be used for associating emotion information with content information. A coordinate system that is not a Russell's circumplex model may be used for associating emotion information with content information. For example, Millenson's three-dimensional model or the like may be used for associating emotion information with content information.
Emotion information related to a user can be mapped to a specific coordinate with reference to the two axes. The intensity of an emotion related to a user can be indicated by a distance from an intersection X of the two axes to the specific coordinate. As a result, an emotion related to a user can be quantitatively evaluated.
Emotion information and content information are associated with coordinate information. As a result, emotion information and content information can be associated. In a case where a coordinate related to emotion information is selected, content information related to the coordinate is selected.
(3) Flowchart
A procedure of the content presentation system 100 will be described with reference to FIG. 3 . FIG. 3 is a flowchart illustrating an example of the procedure of the content presentation system 100 according to the one embodiment of the present technology.
As illustrated in FIG. 3 , first, in step S11, the machine learning model 13 selects a plurality of pieces of content information associated with obtained emotion information. The plurality of pieces of content information may be ranked according to the emotion information, for example.
In step S12, the operation interface unit 11 presents the selected plurality of pieces of content information to a user. The operation interface unit 11 encourages the user to select specific content information from the plurality of pieces of content information.
In step S13, the machine learning model 13, on the basis of the plurality of pieces of content information presented to the user and the content information selected by the user from the plurality of pieces of content information, performs machine learning so as to present the content information suitable for the emotion information.
A specific example will be described. A coordinate related to emotion information desired by a user is assumed to be (0, −10). The machine learning model 13 selects a plurality of pieces of content information associated with the coordinate or a coordinate near the coordinate (step S11). The operation interface unit 11 presents the plurality of pieces of content information to the user and encourages selection (step S12). In a case where a coordinate related to content information selected from the plurality of pieces of content information is (0, −15), the machine learning model 13 corrects the coordinate related to the content information to (0, −10) (step S13).
(4) Hardware Configuration
A hardware configuration of the computer device 1 will be described with reference to FIG. 4 . FIG. 4 is a block diagram illustrating the hardware configuration of the computer device 1 according to the one embodiment of the present technology. As illustrated in FIG. 4 , the computer device 1 can include a CPU 101, a storage 102, a random access memory (RAM) 103, a communication interface 104, and a touch panel 105 as components. Each of the components is connected by, for example, a bus as a data transmission path.
The CPU 101 is implemented by, for example, a microcomputer, and controls each of the components of the computer device 1. The CPU 101 can function as, for example, the control unit 14, the machine learning model 13, and the like. The machine learning model 13 can be implemented by, for example, a program. The CPU 101 functions by reading this program.
The storage 102 stores control data and the like such as a program and operation parameters used by the CPU 101. The storage 102 can be implemented by using, for example, a hard disk drive (HDD), a solid state drive (SSD), or the like. The storage 102 can function as, for example, the storage unit 12.
The RAM 103 temporarily stores, for example, a program, and the like executed by the CPU 101.
The communication interface 104 has a function of communicating via an information communication network 3 using communication technology such as Wi-Fi, Bluetooth (registered trademark), or long term evolution (LTE) for example.
The touch panel 105 encourages a user to perform an operation by a touch operation. The touch panel 105 can function as, for example, the operation interface unit 11.
The computer device 1 may be, for example, a smartphone terminal, a tablet terminal, a mobile phone terminal, a personal digital assistant (PDA), a personal computer (PC), a portable music player, a portable game machine, or a wearable terminal (head mounted display: HMD, glasses-type HMD, watch-type terminal, band-type terminal, and the like).
A program for implementing the machine learning model 13 and the like may be stored in another computer device of the content presentation system 100 or a computer system. In this case, the content presentation system 100 can use a cloud service that provides the function of this program. Examples of the cloud service include software as a service (SaaS), infrastructure as a service (IaaS), platform as a service (PaaS), and the like.
Furthermore, the program can be stored using various types of non-transitory computer readable media and supplied to the computer. The non-transitory computer readable media include various types of tangible storage media. Examples of the non-transitory computer readable media include a magnetic recording medium (for example, flexible disk, magnetic tape, or hard disk drive), a magneto-optical recording medium (for example, magneto-optical disk), a compact disc read only memory (CD-ROM), a CD-R, a CD-R/W, and a semiconductor memory (for example, mask ROM, programmable ROM (PROM), erasable PROM (EPROM), flash ROM, or random access memory (RAM)). Furthermore, the program described above may be supplied to the computer by various types of transitory computer readable media. Examples of the transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer readable medium can supply the program described above to the computer via a wired communication path such as an electric wire and an optical fiber or a wireless communication path.
Note that the technology used in the present embodiment can also be used in other embodiments described below. The similar applies to other embodiments.
2. Second Embodiment of Present Technology (Example 2 of Content Presentation System)
A content presentation system 100 according to one embodiment of the present technology can include a biometric sensor or the like to estimate a current emotion of a user. This will be described with reference to FIG. 5 . FIG. 5 is a block diagram illustrating a configuration of the content presentation system 100 according to the one embodiment of the present technology.
As illustrated in FIG. 5 , the content presentation system 100 according to the one embodiment of the present technology can further include a biometric sensor 2 and an emotion estimation unit 15. The emotion estimation unit 15 is included in a computer device 1. The biometric sensor 2 and the computer device 1 are connected via an information communication network 3.
The biometric sensor can acquire biometric information related to a user. The biometric information includes, for example, a heart rate, body temperature, blood pressure, blood oxygen concentration, respiration, water, blood glucose, an electrocardiogram, brain waves, and the like.
Conventionally, for example, technology for coding facial actions by image recognition (facial action coding system: FACS, and the like) or technology called affective computing for estimating an emotion on the basis of biometric information such as a lie detector have been used. Similarly, the emotion estimation unit 15 estimates current emotion information indicating a current emotion of a user on the basis of biometric information obtained from sensing information detected by the biometric sensor 2.
An example of processing of the emotion estimation unit 15 will be described with reference to FIG. 6 . FIG. 6 is a flowchart illustrating the example of a procedure of the emotion estimation unit 15 according to the one embodiment of the present technology.
As illustrated in FIG. 6 , first, in step S21, the emotion estimation unit 15 performs preprocessing on sensing information detected by the biometric sensor 2. For example, the emotion estimation unit 15 decomposes the sensing information or removes unnecessary information such as noise.
In step S22, the emotion estimation unit 15 extracts features included in the sensing information to obtain biometric information. These features may be predefined or may be automatically extracted by, for example, deep learning.
In step S23, the emotion estimation unit 15 estimates emotion information on the basis of the biometric information.
In estimating the emotion information, the emotion estimation unit 15 can perform machine learning so as to estimate emotion information suitable for the biometric information on the basis of the biometric information. The emotion estimation unit 15 can be implemented by using, for example, a neural network on which supervised learning is performed. This supervised learning can be performed, for example, by using biometric information as input and emotion information as output.
Although the biometric information of a user has a characteristic depending on the user, according to the present technology, suitable emotion information can be estimated by the characteristic being incorporated.
The emotion estimation unit 15 may perform machine learning so as to estimate emotion information suitable for the biometric information on the basis of biometric information in a case where a user is viewing contents or the like. As a result, the degree of association between the emotion information and the content information is stronger, and content information more suitable for the emotion information is presented.
Furthermore, the emotion estimation unit 15 may use, for example, logistic regression, support vector machine, Bayesian estimation, decision tree, random forest, k-nearest neighbor algorithm, principal component analysis, k-means clustering, or the like.
The current emotion information estimated by the emotion estimation unit 15 is presented in the emotion model illustrated in FIG. 2 . As a result, the user can objectively grasp his/her current emotion. As a result, the user can easily select a desired emotion. For example, in a case where a current emotion is a near-lethargic emotion even through the user is at work, the user may select an emotion that enhances concentration.
A procedure of the content presentation system 100 will be described with reference to FIG. 7 . FIG. 7 is a flowchart illustrating an example of the procedure of the content presentation system 100 according to the one embodiment of the present technology.
As illustrated in FIG. 7 , first, in step S31, the biometric sensor 2 detects sensing information from a user and acquires biometric information.
In step S32, the emotion estimation unit 15 estimates current emotion information related to the user on the basis of the biometric information.
In step S33, the operation interface unit 11 presents the current emotion information to the user.
In step S34, the operation interface unit 11 encourages input of the emotion information desired by the user.
Since the procedure of steps S35 to S37 is similar to that of steps S11 to S13 in FIG. 3 , detailed description thereof is omitted.
3. Third Embodiment of Present Technology (Example 3 of Content Presentation System)
The biometric information obtained in the second embodiment may have different features depending on a context of a user. For example, in a case where a heart rate in a case where a user is sitting is compared with a heart rate in a case where the user is standing, the heart rate in a case where the user is standing tends to be higher. Therefore, in a case where the emotion estimation unit 15 is generated on the basis of biometric information in a case where the user is sitting, an emotion in a case where the user is standing may be erroneously estimated as “nervous”. Therefore, biometric information is preferably corrected according to the context of a user.
A content presentation system 100 according to one embodiment of the present technology can include a context sensor or the like to estimate a current context of a user. The context sensor obtains context information indicating information related to a context such as posture or a location of the user. The context information obtains context information such as, for example, whether the user is indoors or outdoors and whether the user is meeting a friend, shopping, walking, or running.
The context sensor can be implemented by using, for example, an angular rate sensor, an acceleration sensor, an inertial measurement unit (IMU), a global positioning system (GPS) positioning unit, a microphone, an ambient light sensor, a schedule management application, or the like. For example, by using an angular rate sensor, an acceleration sensor, an inertial sensor, or the like, context information such as whether the user is walking or walking is obtained. For example, by using a GPS positioning unit, a microphone, an ambient light sensor, or the like, context information such as whether the user is indoors or outdoors is obtained. For example, by using a schedule management application or the like, context information such as whether the user is meeting a friend can be obtained.
A configuration of the content presentation system 100 according to the one embodiment of the present technology will be described with reference to FIG. 8 . FIG. 8 is a block diagram illustrating the configuration of the content presentation system 100 according to the one embodiment of the present technology.
As illustrated in FIG. 8 , the content presentation system 100 according to the one embodiment of the present technology can further include a context sensor 4 and a context estimation unit 16. The context estimation unit 16 is included in a computer device 1. The context sensor 4 and the computer device 1 are connected via an information communication network 3.
The context estimation unit 16 estimates context information on the basis of sensing information detected by the context sensor 4.
The context estimation unit 16 can be implemented by using, for example, a neural network on which supervised learning is performed. This supervised learning can be performed, for example, by using sensing information as input and context information as output.
An emotion estimation unit 15 can correct current emotion information on the basis of the context information. Using the example of the heart rate described above, in a case where a heart rate is high because a user is standing, the emotion estimation unit 15 may subtract the heart rate. Alternatively, in a case where the emotion estimation unit 15 is generated, the emotion estimation unit 15 may be generated using the subtracted heart rate as an explanatory variable.
Alternatively, the emotion estimation unit 15 may correct, for example, the intersection X of the coordinate system illustrated in FIG. 2 on the basis of the context information. As a result, for example, the intensity of an emotion or the like is corrected.
Alternatively, the coordinate system and the context information may be associated with each other. For example, a coordinate system in a case where a user is sitting may be different from a coordinate system in a case where the user is standing.
Alternatively, in a case where biometric information before correction is an m-dimensional (for example, three-dimension of a heart rate, a sweat rate, and respiration) vector and context information is an n-dimensional vector, biometric information after the correction may be an m*n-dimensional vector.
Moreover, in estimating emotion information, the emotion estimation unit 15 can perform machine learning so as to estimate emotion information suitable for the biometric information on the basis of content information in addition to the biometric information. The emotion estimation unit 15 can be implemented by using, for example, a neural network on which supervised learning is performed. This supervised learning can be performed, for example, by using biometric information and content information as input and emotion information as output.
Alternatively, the emotion estimation unit 15 may perform machine learning on the basis of a data set in which emotion information and content information for every context are associated with each other.
Although an error occurs in biometric information of a user depending on the context of the user, according to the present technology, suitable emotion information can be estimated by the context being incorporated.
A procedure of the content presentation system 100 will be described with reference to FIG. 9 . FIG. 9 is a flowchart illustrating an example of the procedure of the content presentation system 100 according to the one embodiment of the present technology.
As illustrated in FIG. 9 , first, in step S41, a biometric sensor 2 detects sensing information from a user and acquires biometric information.
In step S42, the emotion estimation unit 15 estimates current emotion information related to the user on the basis of the biometric information.
In step S43, the context estimation unit 16 estimates context information on the basis of sensing information detected by the context sensor 4.
In step S44, the emotion estimation unit 15 corrects the current emotion information on the basis of the context information.
Since the procedure of steps S45 to S49 is similar to that of steps S33 to S37 in FIG. 7 , detailed description thereof is omitted.
4. Fourth Embodiment of Present Technology (Example 4 of Content Presentation System)
As an operation interface unit 11 that encourages a user to input desired emotion information, for example, a microphone or a camera may be used, but here, an operation interface unit 11 in a case where a display is used as one embodiment will be described.
An operation method via the operation interface unit 11 will be described with reference to FIG. 10 . FIG. 10 is a diagram illustrating an example of a screen displayed on the operation interface unit 11 according to the one embodiment of the present technology. Similarly to FIG. 2 , a coordinate system including elements of an emotion as coordinate axes is illustrated.
As illustrated in FIG. 10 , current emotion information indicating a current emotion of a user is displayed at a first point P1.
With the first point P1, the user who has objectively recognized the current own emotion can move the first point P1 to a second point P2 indicating desired emotion information. This operation can be performed by a touch operation as long as the operation interface unit 11 includes a touch panel. Alternatively, this operation may be performed by using a mouse. Alternatively, in a case where a head mounted display (HMD) includes the operation interface unit 11, for example, this operation may be performed by an angular rate sensor or the like included in the HMD detecting the angle of the head of the user, or this operation may be performed by the operation interface unit 11 detecting the line-of-sight of the user.
The machine learning model 13 selects a plurality of pieces of content information corresponding to the second point P2.
The operation interface unit 11 presents the plurality of pieces of content information corresponding to the second point P2 selected by the machine learning model 13 to the user. Moreover, the operation interface unit 11 encourages the user to select specific content information from the plurality of pieces of content information.
Here, the fact that a plurality of pieces of content information is selected will be described. The second point P2 indicating desired emotion information is selected in units of millimeters by intuition of the user. Therefore, the position of the second point P2 varies. Therefore, by the plurality of pieces of content information being presented, the influence of this variation is reduced.
Note that a plurality of second points P2 may be selected. At this time, the operation interface unit 11 presents a plurality of pieces of content information corresponding to the plurality of respective second points to a user.
5. Fifth Embodiment of Present Technology (Example 5 of Content Presentation System)
Subsequently, an operation interface unit 11 in a case where a display is used as one embodiment will be described.
In a case of encouraging input of desired emotion information, the operation interface unit 11 may encourage input of one coordinate, or may encourage continuous or stepwise input of a plurality of coordinates.
An operation method via the operation interface unit 11 will be described with reference to FIG. 11 . FIG. 11 is a diagram illustrating an example of a screen displayed on the operation interface unit 11 according to the one embodiment of the present technology.
As illustrated in FIG. 11 , a first point P1 is displayed similarly to the fourth embodiment.
With the first point P1, a user who has objectively recognized a current own emotion inputs a route R indicating desired emotion information by moving the first point P1. The route R is formed by continuous or stepwise input of a plurality of coordinates. The user can move the first point P1 to a second point P2 via a first via point V1 and a second via point V2. This operation can be performed by a touch operation or the like similarly to the fourth embodiment.
Note that the number of via points is not limited to two.
The machine learning model 13 selects a plurality of pieces of content information corresponding to one or a plurality of via points included in the route R. In an implementation illustrated in FIG. 11 , a plurality of pieces of content information corresponding to the first via point V1 and a plurality of pieces of content information corresponding to the second via point V2 are selected.
The operation interface unit 11 presents the plurality of pieces of content information corresponding to the first via point V1 and the plurality of pieces of content information corresponding to the second via point V2 selected by the machine learning model 13 to the user. Moreover, the operation interface unit 11 encourages the user to select specific content information from the plurality of pieces of content information.
The machine learning model 13, on the basis of the plurality of pieces of content information presented to the user corresponding to predetermined desired emotion information in the route R (first via point V1 and second via point V2 in the present implementation) and content information selected by the user from the plurality of pieces of content information, performs machine learning so as to present the content information suitable for the desired emotion information.
In the fourth embodiment, the user can move from the first point P1 straight to the second point P2. The fourth embodiment is used, for example, in a case where the user wants to immediately feel an emotion indicated at the second point P2.
On the other hand, in this fifth embodiment, the user can move from the first point P1 to the second point P2 while detouring around. The fifth embodiment is used, for example, in a case where the user wants to feel a final emotion while feeling various emotions. In description in which a specific example is used, in a case where a nervous user wants to exert his/her performance, there is a case where the performance can be exerted more by viewing contents by which the user feels relaxed and then viewing contents by which the user feels excited than by immediately viewing contents by which the user feels excited. According to the present technology, content information can be flexibly presented according to the characteristic of the user.
Moreover, the machine learning model 13 may select a plurality of pieces of content information corresponding to the shape of the route R and the second point P2. The machine learning model 13 can select, for example, a plurality of pieces of content information corresponding to the degree of the curve included in the shape.
The operation interface unit 11 presents the plurality of pieces of content information corresponding to the shape of the route R and the second point P2 selected by the machine learning model 13 to the user. Moreover, the operation interface unit 11 encourages the user to select specific content information from the plurality of pieces of content information.
The machine learning model 13, on the basis of the plurality of pieces of content information presented to the user corresponding to the shape of the route R and the second point P2 and content information selected by the user from the plurality of pieces of content information, performs machine learning so as to present the content information suitable for the desired emotion information.
According to the present embodiment, a plurality of pieces of content information presented corresponding to the second point P2 is different between a case of moving from the first point P1 straight to the second point P2 and a case of moving from the first point P1 to the second point P2 while detouring around. Therefore, content information in which the process of changing emotions of a user is considered is presented to the user.
6. Sixth Embodiment of Present Technology (Content Presentation Device)
A configuration of a content presentation device according to the first embodiment of the present technology will be described with reference to FIG. 12 . FIG. 12 is a block diagram illustrating a configuration of a content presentation device 200 according to one embodiment of the present technology.
As illustrated in FIG. 12 , the content presentation device 200 according to the one embodiment of the present technology can include, for example, an operation interface unit 11, a storage unit 12, a machine learning model 13, and a control unit 14.
The machine learning model 13 performs machine learning so as to present content information suitable for emotion information. More specifically, the machine learning model 13, on the basis of a plurality of pieces of content information presented to a user corresponding to the emotion information and content information selected by the user from the plurality of pieces of content information, performs machine learning so as to present the content information suitable for the emotion information.
For the content presentation device 200, the technology related to the content presentation system 100 described above may be used. Therefore, repeated description of the operation interface unit 11, the storage unit 12, the machine learning model 13, and the control unit 14 is omitted.
7. Seventh Embodiment of Present Technology (Content Presentation Method)
A content presentation method according to one embodiment of the present technology includes a computer device that holds content information associated with emotion information indicating an emotion of a user.
A procedure of the content presentation method according to the one embodiment of the present technology will be described with reference to FIG. 13 . FIG. 13 is a flowchart illustrating an example of the procedure of the content presentation method according to the one embodiment of the present technology.
As illustrated in FIG. 13 , the content presentation method according to the present embodiment at least includes, on the basis of a plurality of pieces of content information presented to the user corresponding to desired emotion information indicating emotion information desired by the user and content information selected by the user from the plurality of pieces of content information, performing machine learning so as to present the content information suitable for the emotion information by the computer device (step S1).
In the content presentation method according to the present embodiment, the technology according to another embodiment described above may be used. Therefore, repeated description of the technology described in the embodiments described above is omitted.
The content presentation method according to the present embodiment can be implemented by using software and hardware. Specifically, for example, a CPU included in hardware reads a program for implementing the content presentation method according to the present embodiment, whereby the content presentation method according to the present embodiment can be implemented.
In addition to this, the configurations described in the embodiments described above can be selected or changed as appropriate to other configurations without departing from the gist of the present technology.
Note that the effects described in the present specification are merely examples and are not limited thereto, and other effects may be provided.
Note that the present technology can also have the following configurations.
[1]
A content presentation system including a computer device that holds content information associated with emotion information indicating an emotion of a user,
    • in which the computer device at least includes a machine learning model that, on the basis of a plurality of pieces of content information presented to the user corresponding to desired emotion information indicating emotion information desired by the user and content information selected by the user from the plurality of pieces of content information, performs machine learning so as to present the content information suitable for the emotion information.
      [2]
The content presentation system according to [1],
    • in which the emotion information and the content information are associated with each other on the basis of a coordinate system having elements of the emotion as coordinate axes.
      [3]
The content presentation system according to [2],
    • in which the coordinate system is a Russell's circumplex model.
      [4]
The content presentation system according to any one of [1] to [3],
    • in which current emotion information indicating the emotion that is current of the user is estimated on the basis of biometric information obtained from sensing information detected by a biometric sensor.
      [5]
The content presentation system according to [4],
    • in which the current emotion information is corrected on the basis of context information related to a context of the user obtained from sensing information detected by a context sensor.
      [6]
The content presentation system according to [4] or [5] further including an emotion estimation unit,
    • in which the emotion estimation unit that performs machine learning so as to estimate the emotion information suitable for the biometric information on the basis of the biometric information.
      [7]
The content presentation system according to any one of [1] to [6] further including an operation interface unit,
    • in which the operation interface unit encourages input of the desired emotion information.
      [8]
The content presentation system according to [7],
    • in which the operation interface unit presents the content information corresponding to the desired emotion information and/or encourages selection of the content information.
      [9]
The content presentation system according to [7] or [8],
    • in which the operation interface unit encourages continuous or stepwise input of the desired emotion information, and
    • the machine learning model, on the basis of a plurality of pieces of the content information presented to the user corresponding to the desired emotion information that is predetermined in a route formed by the input and content information selected by the user from the plurality of pieces of content information, performs machine learning so as to present the content information suitable for the desired emotion information.
      [10]
The content presentation system according to any one of [7] to [9],
    • in which the operation interface unit encourages continuous or stepwise input of the desired emotion information, and
    • the machine learning model, on the basis of a plurality of pieces of the content information presented to the user corresponding to a shape of a route formed by the input and the desired emotion information and content information selected by the user from the plurality of pieces of content information, performs machine learning so as to present the content information suitable for the desired emotion information.
      [11]
A content presentation device that holds content information associated with emotion information indicating an emotion of a user, the content presentation device at least including
    • a machine learning model that, on the basis of a plurality of pieces of content information presented to the user corresponding to desired emotion information indicating emotion information desired by the user and content information selected by the user from the plurality of pieces of content information, performs machine learning so as to present the content information suitable for the emotion information.
      [12]
A content presentation method including a computer device that holds content information associated with emotion information indicating an emotion of a user, the content presentation method at least including,
    • on the basis of a plurality of pieces of content information presented to the user corresponding to desired emotion information indicating emotion information desired by the user and content information selected by the user from the plurality of pieces of content information, performing machine learning so as to present the content information suitable for the emotion information by the computer device.
REFERENCE SIGNS LIST
    • 100 Content presentation system
    • 1 Computer device
    • 11 Operation interface unit
    • 12 Storage unit
    • 13 Machine learning model
    • 14 Control unit
    • 15 Emotion estimation unit
    • 16 Context estimation unit
    • 2 Biometric sensor
    • 3 Information communication network
    • 4 Context sensor
    • P1 First point
    • P2 Second point
    • R Route
    • V1 First via point
    • V2 Second via point
    • 200 Content presentation device
    • S1 Machine learning

Claims (9)

The invention claimed is:
1. A content presentation system comprising:
a computer device that holds content information associated with emotion information indicating an emotion of a user; and
an operation interface unit,
wherein the operation interface unit encourages continuous or stepwise input of desired emotion information, and wherein the computer device at least includes a machine learning model that, on a basis of a plurality of pieces of content information presented to the user corresponding to the desired emotion information that is predetermined in a route formed by the continuous or stepwise input and content information selected by the user from the plurality of pieces of content information presented to the user, performs machine learning to present content information suitable for the desired emotion information.
2. The content presentation system according to claim 1,
wherein the emotion information and the content information are associated with each other on a basis of a coordinate system having elements of the emotion as coordinate axes.
3. The content presentation system according to claim 2,
wherein the coordinate system is a Russell's circumplex model.
4. The content presentation system according to claim 1,
wherein current emotion information indicating the user's current emotion is estimated on a basis of biometric information obtained from sensing information detected by a biometric sensor.
5. The content presentation system according to claim 4,
wherein the current emotion information is corrected on a basis of context information related to a context of the user obtained from sensing information detected by a context sensor.
6. The content presentation system according to claim 4 further comprising an emotion estimation unit,
wherein the emotion estimation unit performs machine learning as to estimate the current emotion information.
7. The content presentation system according to claim 1,
wherein the operation interface unit presents the content information corresponding to the desired emotion information and/or encourages selection of the content information.
8. A content presentation device that holds content information associated with emotion information indicating an emotion of a user, the content presentation device at least comprising:
an operation interface unit, wherein the operation interface unit encourages continuous or stepwise input of desired emotion information; and
a machine learning model that, on a basis of a plurality of pieces of content information presented to the user corresponding to a shape of a route formed by the continuous or stepwise input, the desired emotion information, and content information selected by the user from the plurality of pieces of content information presented to the user, performs machine learning to present content information suitable for the desired emotion information.
9. A content presentation method comprising a computer device that holds content information associated with emotion information indicating an emotion of a user, the content presentation method at least comprising:
receiving continuous or stepwise input of desired emotion information; and
on a basis of a plurality of pieces of content information presented to the user corresponding to a shape of a route formed by the continuous or stepwise input, the desired emotion information, and content information selected by the user from the plurality of pieces of content information presented to the user, performing machine learning to present content information suitable for the desired emotion information by the computer device.
US18/001,760 2020-06-25 2021-05-19 Content presentation system, content presentation device, and content presentation method Active US11983305B2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2020-109359 2020-06-25
JP2020109359 2020-06-25
PCT/JP2021/018923 WO2021261123A1 (en) 2020-06-25 2021-05-19 Content presentation system, content presentation device and content presentation method

Publications (2)

Publication Number Publication Date
US20230244297A1 US20230244297A1 (en) 2023-08-03
US11983305B2 true US11983305B2 (en) 2024-05-14

Family

ID=79282469

Family Applications (1)

Application Number Title Priority Date Filing Date
US18/001,760 Active US11983305B2 (en) 2020-06-25 2021-05-19 Content presentation system, content presentation device, and content presentation method

Country Status (3)

Country Link
US (1) US11983305B2 (en)
EP (1) EP4163924A4 (en)
WO (1) WO2021261123A1 (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007328464A (en) 2006-06-06 2007-12-20 Sharp Corp Buying activity management device, control method, control program and computer-readable recording medium with the control program recorded thereon
JP2008204193A (en) 2007-02-20 2008-09-04 Nippon Telegr & Teleph Corp <Ntt> Content retrieval/recommendation method, content retrieval/recommendation device, and content retrieval/recommendation program
JP2014219937A (en) 2013-05-10 2014-11-20 パナソニック株式会社 Taste determination system
EP3287075A1 (en) 2015-04-23 2018-02-28 Sony Corporation Information processing device, control method, and program
US20180336276A1 (en) 2017-05-17 2018-11-22 Panasonic Intellectual Property Management Co., Ltd. Computer-implemented method for providing content in accordance with emotional state that user is to reach
WO2019017124A1 (en) 2017-07-19 2019-01-24 パナソニックＩｐマネジメント株式会社 Drowsiness estimating device and awakening inducing device
US20190187870A1 (en) * 2017-12-20 2019-06-20 International Business Machines Corporation Utilizing biometric feedback to allow users to scroll content into a viewable display area
JP2019208576A (en) 2018-05-31 2019-12-12 株式会社デンソー Emotion data acquisition device and emotion operation device

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007328464A (en) 2006-06-06 2007-12-20 Sharp Corp Buying activity management device, control method, control program and computer-readable recording medium with the control program recorded thereon
JP2008204193A (en) 2007-02-20 2008-09-04 Nippon Telegr & Teleph Corp <Ntt> Content retrieval/recommendation method, content retrieval/recommendation device, and content retrieval/recommendation program
JP2014219937A (en) 2013-05-10 2014-11-20 パナソニック株式会社 Taste determination system
EP3287075A1 (en) 2015-04-23 2018-02-28 Sony Corporation Information processing device, control method, and program
US20180336276A1 (en) 2017-05-17 2018-11-22 Panasonic Intellectual Property Management Co., Ltd. Computer-implemented method for providing content in accordance with emotional state that user is to reach
JP2018195043A (en) 2017-05-17 2018-12-06 パナソニックＩｐマネジメント株式会社 Content providing method, content providing device, and content providing program
WO2019017124A1 (en) 2017-07-19 2019-01-24 パナソニックＩｐマネジメント株式会社 Drowsiness estimating device and awakening inducing device
US20190187870A1 (en) * 2017-12-20 2019-06-20 International Business Machines Corporation Utilizing biometric feedback to allow users to scroll content into a viewable display area
JP2019208576A (en) 2018-05-31 2019-12-12 株式会社デンソー Emotion data acquisition device and emotion operation device

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
English machine translation of Japanese patent publication JP 2008-204193 (Year: 2008). *
International Search Report and Written Opinion prepared by the Japan Patent Office dated Jun. 30, 2021, for International Application No. PCT/JP2021/018923, 2 pgs.
Kawashima et al., "Deep Reinforcement Learning for Recommendation System," 32nd Annual Conference of the Japanese Society for Artificial Intelligence, Jul. 6, 2018, 4 pages (with English abstract).

Also Published As

Publication number Publication date
WO2021261123A1 (en) 2021-12-30
EP4163924A1 (en) 2023-04-12
US20230244297A1 (en) 2023-08-03
EP4163924A4 (en) 2023-11-29

Similar Documents

Publication Publication Date Title
US9955902B2 (en) Notifying a user about a cause of emotional imbalance
US10261947B2 (en) Determining a cause of inaccuracy in predicted affective response
Ayata et al. Emotion based music recommendation system using wearable physiological sensors
JP6815486B2 (en) Mobile and wearable video capture and feedback platform for the treatment of mental illness
KR102520627B1 (en) Apparatus and method and for recommending products
US11334804B2 (en) Cognitive music selection system and method
US10198505B2 (en) Personalized experience scores based on measurements of affective response
US10223613B2 (en) Machine intelligent predictive communication and control system
US20160350801A1 (en) Method for analysing comprehensive state of a subject
US20180101776A1 (en) Extracting An Emotional State From Device Data
US20200342979A1 (en) Distributed analysis for cognitive state metrics
CN107392124A (en) Emotion identification method, apparatus, terminal and storage medium
US20190108191A1 (en) Affective response-based recommendation of a repeated experience
US20150379774A1 (en) System and method for dynamically generating contextual and personalized digital content
US11483618B2 (en) Methods and systems for improving user experience
WO2015091893A1 (en) System and method for topic-related detection of the emotional state of a person
US11914784B1 (en) Detecting emotions from micro-expressive free-form movements
US20220101146A1 (en) Neural network training with bias mitigation
US20180130372A1 (en) System and method for generating an adaptive embodied conversational agent configured to provide interactive virtual coaching to a subject
CN105955490A (en) Information processing method based on augmented reality, information processing device based on augmented reality and mobile terminal
WO2020077011A1 (en) Systems and methods for generating a product recommendation in a virtual try-on session
JP2014219937A (en) Taste determination system
Medjden et al. Adaptive user interface design and analysis using emotion recognition through facial expressions and body posture from an RGB-D sensor
Bianco et al. A smart mirror for emotion monitoring in home environments
US11983305B2 (en) Content presentation system, content presentation device, and content presentation method

Legal Events

Date Code Title Description
AS Assignment

Owner name: SONY GROUP CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ISHIKAWA, TAKANORI;SASAKI, RYO;AOKI, YUTA;SIGNING DATES FROM 20221114 TO 20221117;REEL/FRAME:062087/0156

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE