CN110477907B - Modeling method for intelligently assisting in recognizing epileptic seizures - Google Patents
Modeling method for intelligently assisting in recognizing epileptic seizures Download PDFInfo
- Publication number
- CN110477907B CN110477907B CN201910641833.9A CN201910641833A CN110477907B CN 110477907 B CN110477907 B CN 110477907B CN 201910641833 A CN201910641833 A CN 201910641833A CN 110477907 B CN110477907 B CN 110477907B
- Authority
- CN
- China
- Prior art keywords
- epileptic
- human body
- machine learning
- image
- algorithm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/369—Electroencephalography [EEG]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/40—Detecting, measuring or recording for evaluating the nervous system
- A61B5/4076—Diagnosing or monitoring particular conditions of the nervous system
- A61B5/4094—Diagnosing or monitoring seizure diseases, e.g. epilepsy
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Public Health (AREA)
- Surgery (AREA)
- Veterinary Medicine (AREA)
- General Health & Medical Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- Psychiatry (AREA)
- Physiology (AREA)
- Neurology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Neurosurgery (AREA)
- Evolutionary Computation (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Psychology (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of intelligent auxiliary detection, and particularly relates to an intelligent auxiliary method for recognizing epileptic seizures. The invention utilizes a machine learning method to convert the clinical diagnosis experience of experts into feature extraction and algorithm logic learning, thereby constructing a machine learning model to realize intelligent auxiliary identification of epileptic seizure. The invention can liberate doctors from the heavy image reading work, improve the diagnosis efficiency of the doctors and simultaneously avoid the situation that different doctors have different reading results.
Description
Technical Field
The invention belongs to the technical field of intelligent auxiliary recognition, and particularly relates to a method for intelligently and auxiliarily recognizing epileptic seizures.
Background
Epilepsy is one of common chronic diseases of the nervous system, can last for years, even decades, about 1% of people all over the world suffer from epilepsy, and about 900 thousands of epilepsy patients exist in China. The brain injury of the epileptic patient further influences the intelligence of the patient due to the long-term abnormal brain discharge, and the abilities of reading characters, expressing languages and the like are weak, so that the epileptic patient seriously influences the physical and mental status, the family status, the social and economic status and the like of the patient.
Video electroencephalogram analysis is a key step for assisting in diagnosing the type of the epileptic seizure, and the step mainly depends on manual interpretation of clinicians at present, but manual identification and clinical analysis have the limitations of strong subjectivity, low efficiency, consumption of a large number of clinical resources and the like.
Artificial intelligence, particularly deep learning, develops rapidly in recent years, so that the machine learning method is utilized to convert expert clinical diagnosis experience into feature extraction and algorithm logic learning, a machine learning model is constructed to realize intelligent auxiliary identification of epileptic seizure, the method has very important significance, the burdensome and boring burden of a clinician on the work of analyzing video monitoring images can be relieved, and the utilization rate of clinical resources and the efficiency and accuracy of clinical auxiliary diagnosis can be improved.
Disclosure of Invention
The invention aims to provide an intelligent auxiliary epileptic seizure identification method with high identification efficiency and high identification accuracy.
In order to achieve the purpose, the technical scheme of the invention is as follows:
a method for intelligently assisting in identifying an epileptic seizure comprises the following steps:
s1, constructing a training video data set by utilizing the video electroencephalogram data of the epileptic;
s2, marking epileptic seizures on the training video data set according to the epileptic pathology criterion;
s3, establishing a human body structure model, marking the parts related to the epileptic seizure, and extracting key nodes of the human body;
s4, extracting action characteristic values, recovering tracks among nodes of the occluded key points, reconstructing the occluded part, further reconstructing a human body node track phase space, analyzing an action mode by adopting a multi-node target tracking method, and extracting spatio-temporal interest points according to characteristics;
s5, performing machine learning classification on the training video data set, dividing the data into epileptic seizure action and non-epileptic seizure action, training a machine learning model, and determining parameters of the machine learning model;
s6, acquiring the video data of the patient, and importing the video data into a machine learning model for identification;
and S7, outputting the recognition result.
Preferably, the specific implementation manner of S3 is as follows:
s3.1, detecting an epileptic patient image and segmenting an example by adopting a Mask R-CNN algorithm expanded based on a Faster R-CNN classification algorithm;
and S3.2, performing human body key node detection on the segmented epileptic image by adopting CPM, and expressing picture texture information and spatial information by using a convolution layer.
Preferably, the occluded keypoint trajectory recovery in S4 is implemented as follows:
s4.1, extracting focusing information from the pictures, extracting image features by using a feature extraction algorithm, calculating Euclidean distance between the two picture features, and performing feature point matching, so that an image pair with the required feature point matching number is found, and the feature points are transmitted in a chain manner in a matching pair, so that a motion track is formed;
s4.2, selecting a good image pair as initialization, carrying out first BA on the two images selected by initialization, and then circularly adding a new image to carry out new BA until no suitable image which can be continuously added exists, so as to obtain camera estimation parameters and scene geometric information;
and S4.3, combining the camera estimation parameters and scene geometric information obtained in S4.2 with the human body model constructed in S3 to obtain a complete human body skeleton structure model.
Preferably, in S4, when extracting the spatio-temporal interest points, the LSTM master network is used to extract the features, the temporal attention subnetwork is used to assign appropriate weights to different video frames, and the spatial attention subnetwork is used to assign appropriate weights to different joint points.
Preferably, the machine learning classification in S5 is performed by using an SVM algorithm or an ELM algorithm.
The invention has the beneficial effects that:
(1) the invention provides an automatic interpretation method of video electroencephalogram data of epileptics, which can enable doctors to be liberated from heavy data interpretation by automatically interpreting the video electroencephalogram data, improve the diagnosis efficiency of the doctors and simultaneously avoid the situation that different doctors have different interpretation results;
(2) according to the invention, the problem that the shielded node cannot be identified can be solved by adopting the BA algorithm to carry out track recovery on the shielded node, the interference of the shielded node on the identification result is avoided, and the accuracy of the identification result is improved;
(3) the invention adopts Mask R-CNN algorithm to detect and segment human body, and adopts CPM to detect key nodes; extracting the characteristic value by adopting an LSTM main network; adopting a time domain attention subnetwork to allocate proper weights to different video frames; adopting a spatial domain attention subnetwork to assign proper weights to different joint points; the accuracy and precision of recognition can be improved.
Drawings
FIG. 1 is a flow chart of an embodiment of the present invention;
FIG. 2 is a diagram of an example segmentation framework for Mask R-CNN;
FIG. 3 is a diagram of the valid acceptor domain of CPM;
FIG. 4 is a schematic diagram of RSC algorithm optimization;
fig. 5 is a block diagram of the network architecture of this patent.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to specific embodiments, and it should be understood that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments that can be derived by one of ordinary skill in the art from the embodiments disclosed herein are intended to be within the scope of the present invention.
Example 1
As shown in fig. 1, a method for intelligently assisting in identifying an epileptic seizure includes the following steps:
s1, constructing a training video data set by utilizing the video electroencephalogram data of the epileptic, wherein the video electroencephalogram data of the epileptic is selected from an epileptic video electroencephalogram database of a medical institution;
s2, marking epileptic seizures on the training video data set according to epileptic pathology criteria, wherein the epileptic seizures are mainly expressed as sucking mouths, four limb muscles are sometimes spastic, the head or the body turns to one side or deflects to one side, or the head or the body is limited to one limb or the sucking mouths or spasms and the like;
s3, establishing a human body structure model, marking the parts related to the epileptic seizure, and extracting human body key nodes, wherein the human body key nodes comprise a head, shoulders, elbows, wrists, mouths, eyes and the like;
s4, extracting action characteristic values, recovering tracks among nodes of the occluded key points, reconstructing the occluded part, further reconstructing a human body node track phase space, analyzing an action mode by adopting a multi-node target tracking method, and extracting spatio-temporal interest points according to characteristics;
s5, performing machine learning classification on the training video data set, dividing the data into epileptic seizure action and non-epileptic seizure action, training a machine learning model, and determining parameters of the machine learning model;
s6, acquiring video data of the epileptic, and importing the video data into a trained machine learning model for recognition;
and S7, outputting the recognition result.
The specific implementation manner of extracting the human body key nodes in S3 is as follows:
s3.1, adopting a Mask R-CNN algorithm expanded based on a Faster R-CNN classification algorithm to carry out human body detection and example segmentation, wherein an example segmentation frame of the Mask R-CNN is shown in figure 2;
s3.2, performing human body key node detection on the segmented human body by adopting CPM, and expressing texture information and spatial information by using a convolution map layer, wherein the main network structure is divided into a plurality of stages, a first stage generates a preliminary key point detection effect, and a plurality of subsequent stages take prediction output of the previous stage and features extracted from an original image as input, so that the key point detection effect is further improved, wherein an effective acceptance domain of CPM is shown in FIG. 3.
The specific implementation manner of the occluded key point trajectory recovery in S4 is as follows:
s4.1, extracting information of a focusing attention area from the picture, extracting image features by using a feature extraction algorithm, calculating Euclidean distance between the two picture features, and performing feature point matching, so that an image pair with the required feature point matching number is found, and the feature points are transmitted in a chain manner in a matching pair, so that an action track is formed; in matching each set of images, epipolar geometry is computed, the F matrix is estimated and the matching pairs are refined by Random Sample Consensus (RSC) optimization. Wherein, the RSC algorithm optimization schematic diagram is shown in FIG. 4;
s4.2, the selected image pair is used as initialization, the first BA is carried out on the two images selected by initialization, then new BA is added in a circulating mode until no proper image which can be added continuously exists, and camera estimation parameters and scene geometric information are obtained.
And S4.3, combining the camera estimation parameters and scene geometric information obtained in S4.2 with the human body model constructed in S3 to obtain a complete human body skeleton structure model.
When the spatial and temporal interest points are extracted from the features in S4, the LSTM master network is used to extract the feature values, the temporal attention sub-network is used to assign appropriate weights to different video frames, the spatial attention sub-network is used to assign appropriate weights to different skeletal joint points, and the LSTM master network is combined with the temporal attention sub-network and the spatial attention sub-network to identify the abnormal seizures and non-seizures of epileptic patients, wherein a network structure diagram is shown in fig. 5.
In the step S5, the machine learning classification layer performs learning classification by using an SVM algorithm or an ELM algorithm. The core of the SVM algorithm is to find an optimal hyperplane, map a low-dimensional nonlinear separable sample space to a high-dimensional linear separable feature space, and utilize an optimal hyperplane formula: w x + b is 0. The ELM algorithm is a machine learning algorithm based on a feedforward neural network, and is mainly characterized in that hidden layer node parameters can be randomly or artificially given without adjustment, only output weights need to be calculated in the learning process, the hidden layer node parameters and the output layer node parameters do not depend on iterative computation, but can be independent of an objective function or a training data set by solving an equation set, and the core formula is as follows: and H beta is T', a weight matrix w is randomly generated, and an implicit layer output matrix H and an output weight beta are calculated and used for classifying normal discharge and abnormal discharge.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, but rather as the intention of all modifications, equivalents, improvements, and equivalents falling within the spirit and scope of the invention.
Claims (4)
1. A modeling method for intelligently assisting in recognizing epileptic seizures comprises the following steps:
s1, constructing a training video data set by utilizing the video electroencephalogram data of the epileptic;
s2, marking epileptic seizures on the training video data set according to the epileptic pathology criterion;
s3, establishing a human body structure model, marking the parts related to the epileptic seizure, and extracting key nodes of the human body;
s4, extracting action characteristic values, recovering tracks among nodes of the occluded key points, reconstructing the occluded part, further reconstructing a human body node track phase space, analyzing an action mode by adopting a multi-node target tracking method, and extracting spatio-temporal interest points according to characteristics;
the specific implementation manner of the occluded key point trajectory recovery in S4 is as follows:
s4.1, extracting focusing information from the pictures, extracting image features by using a feature extraction algorithm, calculating the Euclidean distance between the two picture features, and performing feature point matching, so as to find out an image pair with the matching number of feature points reaching the requirement, and transferring the feature points in a chain manner in a matching pair, thereby forming an action track of the epileptic;
s4.2, selecting a good image pair as initialization, carrying out a first light beam adjustment method on the two images selected by initialization, and then circularly adding a new image to carry out a new light beam adjustment method until no suitable image which can be continuously added exists, so as to obtain camera estimation parameters and scene geometric information; s4.3, combining the camera estimation parameters and scene geometric information obtained in the S4.2 with the human body structure model constructed in the S3 to obtain a complete human body bone structure model;
and S5, performing machine learning classification on the training video data set, classifying the data into seizure action and non-seizure action, training a machine learning model, and determining parameters of the machine learning model.
2. The modeling method for intelligently assisting in identifying the epileptic seizure as claimed in claim 1, wherein the S3 is implemented as follows:
s3.1, detecting and segmenting the epileptic patient image by adopting a Mask R-CNN algorithm expanded based on a Faster R-CNN classification algorithm;
and S3.2, performing human body key node detection on the segmented epileptic patient image by adopting a convolution attitude network, and expressing texture information and spatial information by using a convolution image layer.
3. The modeling method for intelligent assistance in recognizing epileptic seizures as claimed in claim 1, wherein the feature extraction in S4 is performed by using a long-short term memory main network when the feature is extracted as the spatio-temporal interest point, and the temporal attention sub-network is used to assign proper weights to different video frames and the spatial attention sub-network is used to assign proper weights to different joint points.
4. The modeling method for intelligently and auxiliarily recognizing epileptic seizure as claimed in claim 1, wherein the machine learning classification in S5 is performed by using SVM algorithm or ELM algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910641833.9A CN110477907B (en) | 2019-07-16 | 2019-07-16 | Modeling method for intelligently assisting in recognizing epileptic seizures |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910641833.9A CN110477907B (en) | 2019-07-16 | 2019-07-16 | Modeling method for intelligently assisting in recognizing epileptic seizures |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110477907A CN110477907A (en) | 2019-11-22 |
| CN110477907B true CN110477907B (en) | 2022-05-03 |
Family
ID=68547377
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910641833.9A Active CN110477907B (en) | 2019-07-16 | 2019-07-16 | Modeling method for intelligently assisting in recognizing epileptic seizures |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110477907B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111462887B (en) * | 2020-03-31 | 2023-08-29 | 首都医科大学宣武医院 | Wearable epileptic digital assistant system |
| CN114099173B (en) * | 2020-06-09 | 2023-05-09 | 首都医科大学宣武医院 | Data acquisition system and method for sudden symptoms |
| CN113159100B (en) * | 2021-02-19 | 2023-03-31 | 湖南第一师范学院 | Circuit fault diagnosis method, circuit fault diagnosis device, electronic equipment and storage medium |
| CN113017567A (en) * | 2021-03-01 | 2021-06-25 | 西安交通大学医学院第一附属医院 | Deep learning technology-based children epilepsy detection method |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7643655B2 (en) * | 2000-11-24 | 2010-01-05 | Clever Sys, Inc. | System and method for animal seizure detection and classification using video analysis |
| CN104680499A (en) * | 2015-03-24 | 2015-06-03 | 江南大学 | Moving target shielding and restoring method based on Steiner point |
| CN105760861B (en) * | 2016-03-29 | 2020-05-12 | 华东师范大学 | Epileptic seizure monitoring method and system based on depth data |
| CN108647645B (en) * | 2018-05-11 | 2020-10-09 | 广州飞宇智能科技有限公司 | Multi-modal epilepsy diagnosis system and method based on video analysis |
| CN109376681B (en) * | 2018-11-06 | 2021-09-03 | 广东工业大学 | Multi-person posture estimation method and system |
-
2019
- 2019-07-16 CN CN201910641833.9A patent/CN110477907B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110477907A (en) | 2019-11-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110477907B (en) | Modeling method for intelligently assisting in recognizing epileptic seizures | |
| Chen et al. | Patient-specific pose estimation in clinical environments | |
| CN105069434B (en) | A kind of human action Activity recognition method in video | |
| CN109635721B (en) | Video human body falling detection method and system based on track weighted depth convolution order pooling descriptor | |
| CN109902565B (en) | Multi-feature fusion human behavior recognition method | |
| Sarkar et al. | 3D Human Action Recognition: Through the eyes of researchers | |
| Ahmedt-Aristizabal et al. | Deep motion analysis for epileptic seizure classification | |
| CN111401106A (en) | Behavior identification method, device and equipment | |
| Lu et al. | Nas-hr: Neural architecture search for heart rate estimation from face videos | |
| WO2023116736A1 (en) | Video-data-based auxiliary screening system for tourette syndrome | |
| CN111210415B (en) | Method for detecting facial expression hypo of Parkinson patient | |
| CN106529441B (en) | Depth motion figure Human bodys' response method based on smeared out boundary fragment | |
| Li et al. | Graph diffusion convolutional network for skeleton based semantic recognition of two-person actions | |
| Zhou et al. | Intelligent recognition of medical motion image combining convolutional neural network with Internet of Things | |
| CN115346272A (en) | Real-time tumble detection method based on depth image sequence | |
| CN115661856A (en) | User-defined rehabilitation training monitoring and evaluating method based on Lite-HRNet | |
| Guan | Application of logistic regression algorithm in the diagnosis of expression disorder in Parkinson's disease | |
| Wang et al. | Hierarchical Style-Aware Domain Generalization for Remote Physiological Measurement | |
| CN113159002A (en) | Facial expression recognition method based on self-attention weight auxiliary module | |
| CN113705670A (en) | Brain image classification method and device based on magnetic resonance imaging and deep learning | |
| Qiu et al. | 3D EMSU‐Net: A framework for automatic segmentation of brain tumors | |
| CN115154828A (en) | Brain function remodeling method, system and equipment based on brain-computer interface technology | |
| CN113033359B (en) | Self-supervision-based pre-training and facial paralysis grading modeling and grading method and system | |
| Ishwerlal et al. | Lung disease classification using chest X ray image: An optimal ensemble of classification with hybrid training | |
| Le-Tien et al. | Graph-based signal processing to convolutional neural networks for medical image segmentation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |