CN112487980B - Micro-expression-based treatment method, device, system and computer-readable storage medium - Google Patents
Micro-expression-based treatment method, device, system and computer-readable storage medium Download PDFInfo
- Publication number
- CN112487980B CN112487980B CN202011374590.6A CN202011374590A CN112487980B CN 112487980 B CN112487980 B CN 112487980B CN 202011374590 A CN202011374590 A CN 202011374590A CN 112487980 B CN112487980 B CN 112487980B
- Authority
- CN
- China
- Prior art keywords
- target object
- micro
- expression
- medical record
- identifying
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000011282 treatment Methods 0.000 title claims abstract description 104
- 238000000034 method Methods 0.000 title claims abstract description 45
- 208000024891 symptom Diseases 0.000 claims abstract description 103
- 201000010099 disease Diseases 0.000 claims abstract description 83
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims abstract description 83
- 230000001815 facial effect Effects 0.000 claims abstract description 58
- 230000014509 gene expression Effects 0.000 claims abstract description 32
- 238000011269 treatment regimen Methods 0.000 claims abstract description 19
- 230000006996 mental state Effects 0.000 claims abstract description 17
- 238000012549 training Methods 0.000 claims description 34
- 230000003938 response to stress Effects 0.000 claims description 25
- 230000000694 effects Effects 0.000 claims description 16
- 238000002560 therapeutic procedure Methods 0.000 claims description 13
- 238000012216 screening Methods 0.000 claims description 6
- 238000003745 diagnosis Methods 0.000 abstract description 5
- 239000003814 drug Substances 0.000 description 11
- 229940079593 drug Drugs 0.000 description 9
- 238000004891 communication Methods 0.000 description 7
- 210000004709 eyebrow Anatomy 0.000 description 6
- 210000000744 eyelid Anatomy 0.000 description 6
- 230000001225 therapeutic effect Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 206010033557 Palpitations Diseases 0.000 description 2
- 208000007271 Substance Withdrawal Syndrome Diseases 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 210000000554 iris Anatomy 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000003340 mental effect Effects 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 238000007637 random forest analysis Methods 0.000 description 2
- 241000251468 Actinopterygii Species 0.000 description 1
- 208000004179 Oral Leukoplakia Diseases 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000011840 criminal investigation Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000008921 facial expression Effects 0.000 description 1
- 210000001061 forehead Anatomy 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 210000000214 mouth Anatomy 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 201000008557 oral mucosa leukoplakia Diseases 0.000 description 1
- 230000001575 pathological effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H10/00—ICT specially adapted for the handling or processing of patient-related medical or healthcare data
- G16H10/60—ICT specially adapted for the handling or processing of patient-related medical or healthcare data for patient-specific data, e.g. for electronic patient records
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Epidemiology (AREA)
- Medical Informatics (AREA)
- Primary Health Care (AREA)
- Public Health (AREA)
- Medical Treatment And Welfare Office Work (AREA)
Abstract
The invention discloses a micro-expression-based treatment method, which comprises the following steps: collecting a first facial image of a target object, and identifying a first micro-expression of the target object based on the first facial image; acquiring first medical record information of the target object, and identifying a first illness symptom of the target object based on the first medical record information and the first micro expression; a first treatment regimen of the target subject is determined based on the first symptom of the condition. The invention also discloses a treatment device, a treatment system and a computer readable storage medium based on the micro-expression. When judging the disease symptoms of the target object, the invention combines the micro expression besides judging according to the medical record information, and can reflect the mental state of the target object, thereby judging the disease symptoms of the target object from two angles of body and mind, improving the diagnosis accuracy and matching more effective treatment scheme for the target object.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a micro-expression-based treatment method, a micro-expression-based treatment device, a micro-expression-based treatment system and a computer-readable storage medium.
Background
A large number of research facts indicate that the crowd in the high stress environment is a psychological crisis high incidence group. Therefore, the psychological condition of the part of people can be timely known and monitored, which is beneficial to preventing the people from suffering from the disease. The psychological condition is judged through the micro-expressions, so that the criminal investigation efficiency can be greatly improved, and the property loss can be saved. Aiming at the exit and entrance gateway, the micro expression technology assists the personnel to check, and the potential safety hazard can be refused to the outside of the national gate.
Therefore, the people can be screened and distinguished accurately by using big data analysis and artificial intelligence technology, and psychological hidden trouble people can be found timely, so that the people are prevented from suffering from the psychological hidden trouble, and the technology is an urgent and important technical task facing the current society.
Disclosure of Invention
The invention mainly aims to provide a micro-expression-based treatment method, a micro-expression-based treatment device, a micro-expression-based treatment system and a computer-readable storage medium, aiming at improving diagnosis accuracy.
In order to achieve the above object, the present invention provides a micro-expression-based therapeutic method, which includes the following steps:
collecting a first facial image of a target object, and identifying a first micro-expression of the target object based on the first facial image;
acquiring first medical record information of the target object, and identifying a first illness symptom of the target object based on the first medical record information and the first micro expression;
determining a first treatment regimen for the target subject based on the first symptom of the condition;
Wherein the step of identifying the first condition symptom of the target object based on the first medical record information and the first micro-expression includes:
identifying a disease type corresponding to the first medical record information;
Inputting the disease type and the first microexpressions into a pre-constructed model to identify a first disease symptom of a target subject;
Before the step of inputting the disease type and the first micro-expression into a pre-constructed model to identify the first disease symptom of the target object, the micro-expression-based treatment method further includes:
Tracking and collecting a second facial image of a training object, and identifying a second micro-expression of the training object based on the second facial image;
Acquiring second medical record information of the training object, and tracking and recording second disease symptoms, a second treatment scheme and treatment effects of the training object, wherein the second medical record information comprises disease types;
screening the micro-expression to be trained, the disease symptom to be trained and the treatment scheme to be trained from the second micro-expression, the second disease symptom and the second treatment scheme respectively based on the treatment effect;
Based on the second medical record information, the micro-expression to be trained, the disease symptom to be trained and the treatment scheme to be trained, performing supervision modeling by adopting a tree model to obtain the pre-constructed model;
The step of acquiring a first facial image of a target object and identifying a first micro-expression of the target object based on the first facial image includes:
Acquiring a first facial image of a target object based on machine vision, and identifying characteristic points of the first facial image, wherein the characteristic points comprise eyes and mouths;
Determining an identification area corresponding to the feature points, and identifying the gesture of the feature object in the identification area;
based on the gesture, identifying a first micro-expression of the target object;
The step of obtaining the first medical record information of the target object includes:
acquiring a medical image picture of the target object, and identifying the medical image picture to determine first medical record information of the target object;
Or alternatively
And acquiring medical record text of the target object, and identifying the medical record text to determine first medical record information of the target object.
Optionally, the step of determining a first treatment regimen for the target subject based on the first symptom of the condition comprises:
predicting a stress response of the target subject based on the first symptom of the condition;
A treatment regimen of the target subject is determined based on the first symptom of the condition and the stress response.
Optionally, the step of predicting a stress response of the target subject based on the first symptom of the condition comprises:
Judging the mental state of the target object based on the first micro-expression, wherein the first micro-expression at least comprises two frames;
based on the first symptom and the mental state, a stress response of the target subject is predicted.
In addition, to achieve the above object, the present invention also provides a micro-expression based therapeutic apparatus, including:
The first recognition module is used for collecting a first facial image of a target object and recognizing a first micro expression of the target object based on the first facial image;
The second identifying module is used for acquiring first medical record information of the target object and identifying a first illness symptom of the target object based on the first medical record information and the first micro expression;
a determination module for determining a first treatment regimen for the target subject based on the first symptom of the condition;
The training module is used for tracking and collecting a second facial image of the training object and identifying a second micro expression of the training object based on the second facial image;
the first identification module is specifically configured to:
Acquiring a first facial image of a target object based on machine vision, and identifying characteristic points of the first facial image, wherein the characteristic points comprise eyes and mouths;
Determining an identification area corresponding to the feature points, and identifying the gesture of the feature object in the identification area;
based on the gesture, identifying a first micro-expression of the target object;
The second identifying module is specifically configured to:
identifying a disease type corresponding to the first medical record information;
Inputting the disease type and the first microexpressions into a pre-constructed model to identify a first disease symptom of a target subject;
Wherein, the second identification module is further configured to:
acquiring a medical image picture of the target object, and identifying the medical image picture to determine first medical record information of the target object;
Or alternatively
Acquiring medical record text of the target object, and identifying the medical record text to determine first medical record information of the target object;
Wherein, training module is still used for:
Acquiring second medical record information of the training object, and tracking and recording second disease symptoms, a second treatment scheme and treatment effects of the training object, wherein the second medical record information comprises disease types;
screening the micro-expression to be trained, the disease symptom to be trained and the treatment scheme to be trained from the second micro-expression, the second disease symptom and the second treatment scheme respectively based on the treatment effect;
And performing supervision modeling by adopting a tree model based on the second medical record information, the micro expression to be trained, the disease symptom to be trained and the treatment scheme to be trained so as to obtain the pre-constructed model.
In addition, to achieve the above object, the present invention also provides a micro-expression based therapeutic system, including: the device comprises a memory, a processor and a micro-expression based treatment program stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the micro-expression based treatment method as described above.
In addition, in order to achieve the above object, the present invention also provides a computer-readable storage medium having stored thereon a micro-expression based therapeutic program which, when executed by a processor, implements the steps of the micro-expression based therapeutic method as described above.
The invention provides a treatment method based on micro-expressions, which is characterized by collecting a first facial image of a target object and identifying a first micro-expression of the target object based on the first facial image; acquiring first medical record information of the target object, and identifying a first illness symptom of the target object based on the first medical record information and the first micro expression; a first treatment regimen of the target subject is determined based on the first symptom of the condition. The invention also discloses a treatment device, a treatment system and a computer readable storage medium based on the micro-expression. When judging the disease symptoms of the target object, the invention combines the micro expression besides judging according to the medical record information, and can reflect the mental state of the target object, thereby judging the disease symptoms of the target object from two angles of body and mind, improving the diagnosis accuracy and matching more effective treatment scheme for the target object.
Drawings
FIG. 1 is a schematic diagram of a system architecture of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart of a first embodiment of a micro-expression based therapeutic method according to the present invention;
the achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Referring to fig. 1, fig. 1 is a schematic system architecture diagram of a hardware running environment according to an embodiment of the present invention.
As shown in fig. 1, the system may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.
Those skilled in the art will appreciate that the system architecture shown in fig. 1 is not limiting of the system and may include more or fewer components than shown, or certain components may be combined, or a different arrangement of components.
As shown in fig. 1, an operating system, a network communication module, a user interface module, and a micro-expression based treatment program may be included in a memory 1005, which is a computer storage medium.
The operating system is a program for managing and controlling the micro-expression based treatment system and software resources, and supports the operation of a network communication module, a user interface module, a micro-expression based treatment program and other programs or software; the network communication module is used to manage and control the network interface 1002; the user interface module is used to manage and control the user interface 1003.
In the micro-expression based therapy system shown in fig. 1, the micro-expression based therapy system invokes a micro-expression based therapy program stored in a memory 1005 through a processor 1001 and performs operations in various embodiments of the micro-expression based therapy method described below.
Based on the hardware structure, the embodiment of the invention based on the micro-expression treatment method is provided.
Referring to fig. 2, fig. 2 is a flowchart of a first embodiment of a micro-expression based therapeutic method according to the present invention, the method includes the following steps:
Step S10, a first facial image of a target object is acquired, and a first micro expression of the target object is identified based on the first facial image;
step S20, acquiring first medical record information of the target object, and identifying a first illness state symptom of the target object based on the first medical record information and the first micro expression;
Step S30, determining a first treatment regimen of the target subject based on the first symptom of the condition.
The micro-expression-based treatment method of the embodiment can be applied to a micro-expression-based treatment system for detecting the real intention in various industries such as security check, interrogation, clinical medicine (doctor-patient communication), advertisement and consumption (monitoring the real attitude of consumers), and the like.
It can be understood that in the actual process of drug rehabilitation, the psychological state is different due to different disease symptoms of each target object, so that the receiving performance of each target object on drug rehabilitation is different, and the object is a psychological crisis high incidence group, so that the target object needs to be monitored and judged, and a treatment scheme is timely given.
According to the micro-expression-based treatment method, a facial image of a target object (a drug rehabilitation institute student or a volunteer) is acquired in real time through machine vision, then micro-expressions of the target object are identified from the facial image through machine learning, then disease symptoms of the target object are identified according to the micro-expressions and medical record information of the target object, and a treatment scheme is matched for the target object.
The following will explain each step in detail:
Step S10, a first facial image of a target object is acquired, and a first micro expression of the target object is identified based on the first facial image.
In this embodiment, the treatment system acquires a first facial image of the target object through machine vision, such as a camera, and identifies a first microexpression of the target object from the first target image. In the specific implementation, the space where the target object is located, such as a room, is provided with a camera and other acquisition devices, the camera and other acquisition devices can work in real time or at fixed time, video pictures or images of the space where the camera and other acquisition devices are located are acquired in real time or at fixed time, the treatment system identifies pictures containing the target object from the acquired video pictures or images, then locates facial images of the target object, and then identifies micro expressions of the target object.
Further, in an embodiment, step S10 includes:
Step a1, acquiring a first facial image of a target object based on machine vision, and identifying characteristic points of the first facial image, wherein the characteristic points comprise eyes and a mouth;
In an embodiment, to increase the recognition efficiency, the treatment system first recognizes key points, i.e. feature points, of the first facial image after acquiring the first facial image of the target object based on machine vision in the process of recognizing the micro-expression of the target object, where the feature points include eyes and mouth.
Step a2, determining an identification area corresponding to the feature points, and identifying the gesture of the feature object in the identification area;
Then, the recognition area corresponding to the feature point is determined, and it can be understood that the judgment of the human micro-expression is mainly concentrated on the eyes, mouth, eyebrows, cheeks and other parts, while other parts of the human face, such as the forehead, chin and the like, have no great reference value, so that the whole facial image of the target object only needs to pick out the recognition area to be recognized, and the other areas do not need to be recognized, thereby achieving the technical effect of shortening the recognition time.
Specifically, the positions of the eyes and the mouth in the first facial image are determined, the lengths of the eyes and the mouth are determined, the lengths are taken as the radius, the eyes and the mouth are taken as the starting points, a circle is drawn, and the obtained area is the identification area.
At this time, only the gesture of the feature object in the recognition area, including the eyebrow, eyelid, mouth, etc., need be recognized, and other areas need not be recognized.
And a step a3 of identifying a first micro-expression of the target object based on the gesture.
And finally, according to the gesture of the characteristic object, identifying the first micro expression of the target object.
Specifically, the upper eyelid is lifted to expose more upper edges of the irises, and if the mouth is large, the micro expression of the target object is surprise;
The eyebrows are wrinkled and raised, the inner sides of the eyebrows are twisted, the upper eyelids are lifted, the irises are exposed, and the micro expression of the target object is fear;
Lifting the upper lip, and forming nose lip grooves on two sides of the nose wing, so that the micro expression of the target object is averse;
when the eyebrow is raised, the whole eyebrow keeps being pressed down, and the micro expression of the target object is sad;
while the upper eyelid is lifted, the lower eyelid is tightened, so that the microexpressions of the target object are anger;
The eyes are squinted, fish tail lines are formed, the lower eyelid is tightened, lifted, raised, the mouth corners are tilted, nose and lip furrows are formed by stretching, cheeks are raised, and the micro expression of the target object is pleasant and the like.
That is, in an embodiment, the treatment system only needs to identify a few areas during the process of identifying the facial expression of the target object, and does not need to identify the whole facial image, so that the identification range is reduced, and the identification time is shortened.
Step S20, obtaining first medical record information of the target object, and identifying a first disease symptom of the target object based on the first medical record information and the first micro-expression.
In this embodiment, the treatment system obtains the first medical record information of the target object, and identifies the first disease symptom of the target object through the first medical record information and the first micro-expression, that is, when the first disease symptom of the target object is determined, the physical condition of the target object is determined from the medical record information, and the mental condition of the target object is determined by combining the micro-expression, so that the physical and mental conditions of the target object are combined, and the identified disease symptom of the target object is more accurate.
Specifically, in an embodiment, the step of obtaining the first medical record information of the target object includes:
Step b1, acquiring a medical image picture of the target object, and identifying the medical image picture to determine first medical record information of the target object;
In an embodiment, the treatment system identifies the medical image by acquiring the medical image of the target object, thereby identifying the first medical record information of the target object, and specifically, may process the medical image of CT, MRI, etc. by a video image processing (CV) technique, thereby identifying the medical record information of the target object, where the medical record information includes a disease type.
Or alternatively
And b2, acquiring medical record text of the target object, and identifying the medical record text to determine first medical record information of the target object.
In an embodiment, the treatment system identifies the first medical record information of the target object by acquiring the medical record text of the target object and identifying the medical record text, specifically, the case record, the detection scale and other medical record texts are firstly subjected to word segmentation and entity identification (patient behavior, disease type, medicine and treatment scheme) to be arranged into data in a corresponding form, symptoms and scores in the detection scale are extracted, and the disease probability of the target object in different mental states, namely the first medical record information, is predicted by combining the diagnosis result and treatment effect of the final target object through a Natural Language Processing (NLP) algorithm.
Further, in an embodiment, the step of identifying the first condition symptom of the target object based on the first medical record information and the first micro-expression includes:
Step b3, identifying the disease type corresponding to the first medical record information;
In an embodiment, the disease type corresponding to the first medical record information is identified by the identification method in the step b1 or the step b 2.
And b4, inputting the disease type and the first micro-expression into a pre-constructed model to identify a first disease symptom of the target object.
Then, the disease type and the first micro-expression are input into a pre-constructed model, so that a first disease symptom of the target object is identified.
Wherein, the pre-constructed model can be selected as a tree model, such as a random forest model and the like.
Thus, prior to step b4, the microexpressive based treatment method further comprises:
Tracking and collecting a second facial image of a training object, and identifying a second micro-expression of the training object based on the second facial image;
Acquiring second medical record information of the training object, and tracking and recording second disease symptoms, a second treatment scheme and treatment effects of the training object, wherein the second medical record information comprises disease types;
screening the micro-expression to be trained, the disease symptom to be trained and the treatment scheme to be trained from the second micro-expression, the second disease symptom and the second treatment scheme respectively based on the treatment effect;
And performing supervision modeling by adopting a tree model based on the second medical record information, the micro expression to be trained, the disease symptom to be trained and the treatment scheme to be trained so as to obtain the pre-constructed model.
That is, the pre-constructed model is obtained through data statistics and machine learning, so that in the subsequent recognition process, only relevant parameters, such as micro-expressions, illness symptoms and the like, are input into the model, and the corresponding recognition result can be obtained.
Specifically, the second facial image of the training object is tracked and collected, then the second micro-expression of the training object is identified, in addition, the second disease symptoms and the second treatment scheme of the training object and the treatment effect are tracked and recorded, the treatment effective data are screened out from the treatment effect, namely the micro-expression to be trained, the disease symptoms to be trained and the treatment scheme to be trained are screened out, namely the input and the output of the model are obtained at the moment, and the model is trained.
In the training process, tree models such as the rogue regression, the SVM, the lightgbm and the random forest are adopted for supervision modeling, so that a pre-constructed model is obtained through training.
Step S30, determining a first treatment regimen of the target subject based on the first symptom of the condition.
In this embodiment, after the first disease symptoms of the target object are determined, the first treatment scheme of the target object can be determined, specifically, a mapping table of the disease symptoms and the treatment schemes can be established in advance, and which disease symptoms correspond to which treatment schemes can be declared; the deep learning mode can also be adopted, and the disease symptoms and the treatment schemes in a large amount of case data are continuously learned, so that the treatment schemes corresponding to the current disease symptoms can be automatically judged.
It should be noted that the symptoms of the disease can be in one-to-one relationship with the treatment regimen, one-to-many relationship, or many-to-one relationship. Moreover, the terms first, second, etc. are used herein only to distinguish one application from another of the same technical feature.
The method comprises the steps of collecting a first facial image of a target object, and identifying a first micro expression of the target object based on the first facial image; acquiring first medical record information of the target object, and identifying a first illness symptom of the target object based on the first medical record information and the first micro expression; a first treatment regimen of the target subject is determined based on the first symptom of the condition. The invention also discloses a treatment device, a treatment system and a computer readable storage medium based on the micro-expression. When judging the disease symptoms of the target object, the invention combines the micro expression besides judging according to the medical record information, and can reflect the mental state of the target object, thereby judging the disease symptoms of the target object from two angles of body and mind, improving the diagnosis accuracy and matching more effective treatment scheme for the target object.
Further, based on the first embodiment of the micro-expression based treatment method of the present invention, a second embodiment of the micro-expression based treatment method of the present invention is provided.
The second embodiment of the micro-expression based treatment method is different from the first embodiment of the micro-expression based treatment method in that step S30 includes:
step c1, predicting a stress response of the target subject based on the first disease symptom;
Step c2, determining a treatment regimen for said subject based on said first symptom of the condition and said stress response.
The treatment system of the embodiment can also predict the stress response of the target object according to the disease symptoms of the target object, thereby timely adjusting the treatment scheme and improving the matching degree of the treatment scheme.
The following will explain each step in detail:
Step c1, predicting a stress response of the target subject based on the first symptom of the condition.
In this embodiment, after the first disease symptoms of the target object are determined, the treatment system predicts the subsequent symptoms of the current target object, i.e. stress response, according to the general pathological knowledge of the first disease symptoms, if the target object is tic after taking a certain drug for drug withdrawal, the target object can be predicted to have stress response such as oral leukoplakia in the future.
Further, in an embodiment, step c1 includes:
Step c11, judging the mental state of the target object based on the first micro-expression, wherein the first micro-expression at least comprises two frames;
In one embodiment, the stress response of the target object is predicted by combining the mental states of the target object, so that the mental states of the current target object need to be determined according to at least two frames of micro-expressions, such as anger, pain, calm and other micro-expressions of the target object caused by drug withdrawal, so as to determine the acceptance degree of the target object to the drug.
Step c12, predicting a stress response of the target subject based on the first symptom of the condition and the mental state.
In one embodiment, the stress response of the target subject is predicted by the symptoms and mental state, e.g., the symptoms of the target subject are palpitations and the mental state is fear; the symptoms of the illness state of the target object are palpitation, mental state is calm, the predicted stress response of the former target object is different, the former target object is possibly worsened and is not suitable for aggravating medicines, namely the target object has large stress response, the latter target object is possibly not worsened, and medicines, namely the target object has small stress response and the like.
Step c2, determining a treatment regimen for said subject based on said first symptom of the condition and said stress response.
In this embodiment, the treatment system may select a treatment regimen more suitable for the target subject according to the first condition and the stress, such as a large stress, a proper reduction in the dosage of the relevant drug, a small stress, a proper increase in the dosage of the relevant drug, and so on.
The treatment system of the embodiment can also predict the stress response of the target object according to the disease symptoms of the target object, thereby timely adjusting the treatment scheme, matching a more proper treatment scheme for the target object and improving the matching degree of the treatment scheme.
The invention also provides a micro-expression-based treatment device. The invention relates to a micro-expression-based therapeutic device, which comprises:
The first recognition module is used for collecting a first facial image of a target object and recognizing a first micro expression of the target object based on the first facial image;
The second identifying module is used for acquiring first medical record information of the target object and identifying a first illness symptom of the target object based on the first medical record information and the first micro expression;
a determination module for determining a first treatment regimen of the target subject based on the first symptom of the condition.
Optionally, the second identifying module is further configured to:
identifying a disease type corresponding to the first medical record information;
inputting the disease type and the first micro-expression into a pre-constructed model to identify a first condition symptom of a target subject.
Optionally, the micro-expression based therapeutic device further comprises a training module for:
Tracking and collecting a second facial image of a training object, and identifying a second micro-expression of the training object based on the second facial image;
Acquiring second medical record information of the training object, and tracking and recording second disease symptoms, a second treatment scheme and treatment effects of the training object, wherein the second medical record information comprises disease types;
screening the micro-expression to be trained, the disease symptom to be trained and the treatment scheme to be trained from the second micro-expression, the second disease symptom and the second treatment scheme respectively based on the treatment effect;
And performing supervision modeling by adopting a tree model based on the second medical record information, the micro expression to be trained, the disease symptom to be trained and the treatment scheme to be trained so as to obtain the pre-constructed model.
Optionally, the determining module is further configured to:
predicting a stress response of the target subject based on the first symptom of the condition;
A treatment regimen of the target subject is determined based on the first symptom of the condition and the stress response.
Optionally, the determining module is further configured to:
Judging the mental state of the target object based on the first micro-expression, wherein the first micro-expression at least comprises two frames;
based on the first symptom and the mental state, a stress response of the target subject is predicted.
Optionally, the first identification module is further configured to:
Acquiring a first facial image of a target object based on machine vision, and identifying characteristic points of the first facial image, wherein the characteristic points comprise eyes and mouths;
Determining an identification area corresponding to the feature points, and identifying the gesture of the feature object in the identification area;
And based on the gesture, identifying a first micro-expression of the target object.
Optionally, the second identifying module is further configured to:
acquiring a medical image picture of the target object, and identifying the medical image picture to determine first medical record information of the target object;
Or alternatively
And acquiring medical record text of the target object, and identifying the medical record text to determine first medical record information of the target object.
The invention also provides a computer readable storage medium.
The computer readable storage medium of the present invention stores a micro-expression based treatment program which, when executed by a processor, implements the steps of the micro-expression based treatment method described above.
The method implemented when the micro-expression based therapeutic program running on the processor is executed may refer to various embodiments of the micro-expression based therapeutic method of the present invention, and will not be described herein.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) as described above, comprising several instructions for causing an end system (which may be a robot, a computer, a server, or a network system, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein, or any application, directly or indirectly, in the field of other related technology.
Claims (6)
1. A microexpressive based treatment method, characterized in that the microexpressive based treatment method comprises:
collecting a first facial image of a target object, and identifying a first micro-expression of the target object based on the first facial image;
acquiring first medical record information of the target object, and identifying a first illness symptom of the target object based on the first medical record information and the first micro expression;
determining a first treatment regimen for the target subject based on the first symptom of the condition;
Wherein the step of identifying the first condition symptom of the target object based on the first medical record information and the first micro-expression includes:
identifying a disease type corresponding to the first medical record information;
Inputting the disease type and the first microexpressions into a pre-constructed model to identify a first disease symptom of a target subject;
Before the step of inputting the disease type and the first micro-expression into a pre-constructed model to identify the first disease symptom of the target object, the micro-expression-based treatment method further includes:
Tracking and collecting a second facial image of a training object, and identifying a second micro-expression of the training object based on the second facial image;
Acquiring second medical record information of the training object, and tracking and recording second disease symptoms, a second treatment scheme and treatment effects of the training object, wherein the second medical record information comprises disease types;
screening the micro-expression to be trained, the disease symptom to be trained and the treatment scheme to be trained from the second micro-expression, the second disease symptom and the second treatment scheme respectively based on the treatment effect;
Based on the second medical record information, the micro-expression to be trained, the disease symptom to be trained and the treatment scheme to be trained, performing supervision modeling by adopting a tree model to obtain the pre-constructed model;
The step of acquiring a first facial image of a target object and identifying a first micro-expression of the target object based on the first facial image includes:
Acquiring a first facial image of a target object based on machine vision, and identifying characteristic points of the first facial image, wherein the characteristic points comprise eyes and mouths;
Determining an identification area corresponding to the feature points, and identifying the gesture of the feature object in the identification area;
based on the gesture, identifying a first micro-expression of the target object;
The step of obtaining the first medical record information of the target object includes:
acquiring a medical image picture of the target object, and identifying the medical image picture to determine first medical record information of the target object;
Or alternatively
And acquiring medical record text of the target object, and identifying the medical record text to determine first medical record information of the target object.
2. The microexpressive therapy method according to claim 1, wherein said step of determining a first treatment regimen for said target subject based on said first condition symptoms comprises:
predicting a stress response of the target subject based on the first symptom of the condition;
A treatment regimen of the target subject is determined based on the first symptom of the condition and the stress response.
3. The microexpressive therapy method according to claim 2, wherein said step of predicting the stress response of said target subject based on said first condition symptoms comprises:
Judging the mental state of the target object based on the first micro-expression, wherein the first micro-expression at least comprises two frames;
based on the first symptom and the mental state, a stress response of the target subject is predicted.
4. A microexpressive based treatment device, comprising:
The first recognition module is used for collecting a first facial image of a target object and recognizing a first micro expression of the target object based on the first facial image;
The second identifying module is used for acquiring first medical record information of the target object and identifying a first illness symptom of the target object based on the first medical record information and the first micro expression;
a determination module for determining a first treatment regimen for the target subject based on the first symptom of the condition;
The training module is used for tracking and collecting a second facial image of the training object and identifying a second micro expression of the training object based on the second facial image;
the first identification module is specifically configured to:
Acquiring a first facial image of a target object based on machine vision, and identifying characteristic points of the first facial image, wherein the characteristic points comprise eyes and mouths;
Determining an identification area corresponding to the feature points, and identifying the gesture of the feature object in the identification area;
based on the gesture, identifying a first micro-expression of the target object;
The second identifying module is specifically configured to:
identifying a disease type corresponding to the first medical record information;
Inputting the disease type and the first microexpressions into a pre-constructed model to identify a first disease symptom of a target subject;
Wherein, the second identification module is further configured to:
acquiring a medical image picture of the target object, and identifying the medical image picture to determine first medical record information of the target object;
Or alternatively
Acquiring medical record text of the target object, and identifying the medical record text to determine first medical record information of the target object;
Wherein, training module is still used for:
Acquiring second medical record information of the training object, and tracking and recording second disease symptoms, a second treatment scheme and treatment effects of the training object, wherein the second medical record information comprises disease types;
screening the micro-expression to be trained, the disease symptom to be trained and the treatment scheme to be trained from the second micro-expression, the second disease symptom and the second treatment scheme respectively based on the treatment effect;
And performing supervision modeling by adopting a tree model based on the second medical record information, the micro expression to be trained, the disease symptom to be trained and the treatment scheme to be trained so as to obtain the pre-constructed model.
5. A microexpressive therapy system, the microexpressive therapy system comprising: a memory, a processor, and a micro-expression based therapy program stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the micro-expression based therapy method of any one of claims 1 to 3.
6. A computer readable storage medium, characterized in that it has stored thereon a micro-expression based treatment program, which when executed by a processor implements the steps of the micro-expression based treatment method according to any of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011374590.6A CN112487980B (en) | 2020-11-30 | 2020-11-30 | Micro-expression-based treatment method, device, system and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011374590.6A CN112487980B (en) | 2020-11-30 | 2020-11-30 | Micro-expression-based treatment method, device, system and computer-readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112487980A CN112487980A (en) | 2021-03-12 |
| CN112487980B true CN112487980B (en) | 2024-06-14 |
Family
ID=74937667
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011374590.6A Active CN112487980B (en) | 2020-11-30 | 2020-11-30 | Micro-expression-based treatment method, device, system and computer-readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112487980B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113643807B (en) * | 2021-10-14 | 2021-12-28 | 江苏润无声科技有限公司 | Dialysis treatment matching method and system based on feature analysis |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107242876A (en) * | 2017-04-20 | 2017-10-13 | 合肥工业大学 | A kind of computer vision methods for state of mind auxiliary diagnosis |
| CN109767321A (en) * | 2018-12-18 | 2019-05-17 | 深圳壹账通智能科技有限公司 | Question answering process optimization method, device, computer equipment and storage medium |
| CN109785977A (en) * | 2018-12-18 | 2019-05-21 | 平安科技（深圳）有限公司 | Automated information input method, system, device and storage medium |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| ES2964705T3 (en) * | 2016-05-06 | 2024-04-09 | Univ Leland Stanford Junior | Mobile and portable video capture and feedback platforms for the therapy of mental disorders |
| US10515393B2 (en) * | 2016-06-30 | 2019-12-24 | Paypal, Inc. | Image data detection for micro-expression analysis and targeted data services |
| KR101984410B1 (en) * | 2017-12-19 | 2019-05-30 | 건국대학교 산학협력단 | A method and apparatus for lighting therapy based on emotianal state |
| KR102009844B1 (en) * | 2018-02-06 | 2019-08-12 | 한림대학교 산학협력단 | Facial expression recognition device and management service server for dementia patient using the same |
| CN208580582U (en) * | 2018-02-27 | 2019-03-05 | 苏州康捷医疗股份有限公司 | A kind of recognition of face assisting in diagnosis and treatment system |
| JP2020057111A (en) * | 2018-09-28 | 2020-04-09 | パナソニックＩｐマネジメント株式会社 | Facial expression determination system, program and facial expression determination method |
| CN109657554B (en) * | 2018-11-21 | 2022-12-20 | 腾讯科技（深圳）有限公司 | Image identification method and device based on micro expression and related equipment |
| CN109685672A (en) * | 2018-12-13 | 2019-04-26 | 平安医疗健康管理股份有限公司 | Extremely medical recognition methods, device, terminal and computer readable storage medium |
| CN109830280A (en) * | 2018-12-18 | 2019-05-31 | 深圳壹账通智能科技有限公司 | Psychological aided analysis method, device, computer equipment and storage medium |
| CN109697421A (en) * | 2018-12-18 | 2019-04-30 | 深圳壹账通智能科技有限公司 | Evaluation method, device, computer equipment and storage medium based on micro- expression |
| CN109766859B (en) * | 2019-01-17 | 2023-12-19 | 平安科技（深圳）有限公司 | Campus monitoring method, device, equipment and storage medium based on micro-expressions |
| CN110047588A (en) * | 2019-03-18 | 2019-07-23 | 平安科技（深圳）有限公司 | Method of calling, device, computer equipment and storage medium based on micro- expression |
| CN111553311A (en) * | 2020-05-13 | 2020-08-18 | 吉林工程技术师范学院 | Micro-expression recognition robot and control method thereof |
| CN111933239A (en) * | 2020-09-17 | 2020-11-13 | 泰康保险集团股份有限公司 | Data processing method, device, system and storage medium |
-
2020
- 2020-11-30 CN CN202011374590.6A patent/CN112487980B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107242876A (en) * | 2017-04-20 | 2017-10-13 | 合肥工业大学 | A kind of computer vision methods for state of mind auxiliary diagnosis |
| CN109767321A (en) * | 2018-12-18 | 2019-05-17 | 深圳壹账通智能科技有限公司 | Question answering process optimization method, device, computer equipment and storage medium |
| CN109785977A (en) * | 2018-12-18 | 2019-05-21 | 平安科技（深圳）有限公司 | Automated information input method, system, device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112487980A (en) | 2021-03-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110647812B (en) | Tumble behavior detection processing method and device, computer equipment and storage medium | |
| Werner et al. | Towards pain monitoring: Facial expression, head pose, a new database, an automatic system and remaining challenges | |
| JP4401079B2 (en) | Subject behavior analysis | |
| JP2022512044A (en) | Automatic image-based skin diagnosis using deep learning | |
| JP2020518894A (en) | Person identification system and method | |
| Hu et al. | Deep neural network-based speaker-aware information logging for augmentative and alternative communication | |
| CN109492595B (en) | Behavior prediction method and system suitable for fixed group | |
| WO2017143952A1 (en) | Human face detection method | |
| Lu et al. | Automated strabismus detection for telemedicine applications | |
| Dadiz et al. | Detecting depression in videos using uniformed local binary pattern on facial features | |
| WO2023097780A1 (en) | Classification method and device for classifying patient‑ventilator asynchrony phenomenon in mechanical ventilation process | |
| CN112487980B (en) | Micro-expression-based treatment method, device, system and computer-readable storage medium | |
| CN110755091A (en) | Personal mental health monitoring system and method | |
| CN114242243A (en) | User health assessment method, device, equipment and storage medium | |
| Singh et al. | Prediction of pain intensity using multimedia data | |
| US20240050006A1 (en) | System and method for prediction and control of attention deficit hyperactivity (adhd) disorders | |
| CN113033387A (en) | Intelligent assessment method and system for automatically identifying chronic pain degree of old people | |
| CN116759075A (en) | Psychological disorder inquiry method, device, equipment and medium | |
| CN116130088A (en) | Multi-mode face diagnosis method, device and related equipment | |
| US20220101655A1 (en) | System and method of facial analysis | |
| Zurita et al. | Fitness-for-duty classification using temporal sequences of iris periocular images | |
| Gu et al. | AI-Driven Depression Detection Algorithms from Visual and Audio Cues | |
| Hsu et al. | Extraction of visual facial features for health management | |
| AU2021104681A4 (en) | An Autonomous Early Detection analysis of User Drowsiness through Facial Recognition using Artificial Intelligence | |
| Seong et al. | Face Recognition and Physiological Signal for Impaired Drivers: A Review |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |