CN116417135A - Processing method and device for predicting early Alzheimer's disease type based on brain image - Google Patents
Processing method and device for predicting early Alzheimer's disease type based on brain image Download PDFInfo
- Publication number
- CN116417135A CN116417135A CN202310132964.0A CN202310132964A CN116417135A CN 116417135 A CN116417135 A CN 116417135A CN 202310132964 A CN202310132964 A CN 202310132964A CN 116417135 A CN116417135 A CN 116417135A
- Authority
- CN
- China
- Prior art keywords
- brain
- type
- classification
- feature
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 210000004556 brain Anatomy 0.000 title claims abstract description 278
- 208000024827 Alzheimer disease Diseases 0.000 title claims abstract description 79
- 238000003672 processing method Methods 0.000 title claims abstract description 9
- 238000012545 processing Methods 0.000 claims abstract description 81
- 238000000605 extraction Methods 0.000 claims abstract description 74
- 208000010877 cognitive disease Diseases 0.000 claims abstract description 67
- 238000000034 method Methods 0.000 claims abstract description 42
- 238000007781 pre-processing Methods 0.000 claims abstract description 15
- 230000004927 fusion Effects 0.000 claims abstract description 7
- 239000013598 vector Substances 0.000 claims description 79
- 230000006870 function Effects 0.000 claims description 65
- 230000004913 activation Effects 0.000 claims description 45
- 238000004364 calculation method Methods 0.000 claims description 22
- 230000015654 memory Effects 0.000 claims description 21
- 208000027061 mild cognitive impairment Diseases 0.000 claims description 21
- 238000013527 convolutional neural network Methods 0.000 claims description 20
- 238000007499 fusion processing Methods 0.000 claims description 14
- 230000011218 segmentation Effects 0.000 claims description 11
- 230000007787 long-term memory Effects 0.000 claims description 10
- 230000006403 short-term memory Effects 0.000 claims description 9
- 230000008030 elimination Effects 0.000 claims description 6
- 238000003379 elimination reaction Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 3
- 238000002610 neuroimaging Methods 0.000 claims 1
- 238000003384 imaging method Methods 0.000 description 17
- 238000010586 diagram Methods 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 208000028698 Cognitive impairment Diseases 0.000 description 4
- 238000002595 magnetic resonance imaging Methods 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 230000008447 perception Effects 0.000 description 3
- 230000002093 peripheral effect Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 230000032683 aging Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 210000002418 meninge Anatomy 0.000 description 1
- 238000012821 model calculation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012634 optical imaging Methods 0.000 description 1
- 210000004129 prosencephalon Anatomy 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 208000011580 syndromic disease Diseases 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/40—Detecting, measuring or recording for evaluating the nervous system
- A61B5/4076—Diagnosing or monitoring particular conditions of the nervous system
- A61B5/4088—Diagnosing of monitoring cognitive diseases, e.g. Alzheimer, prion diseases or dementia
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0033—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
- A61B5/004—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room adapted for image acquisition of a particular organ or body part
- A61B5/0042—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room adapted for image acquisition of a particular organ or body part for the brain
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves
- A61B5/055—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/30—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for calculating health indices; for individual health risk assessment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/03—Recognition of patterns in medical or anatomical images
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Health & Medical Sciences (AREA)
- Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- Medical Informatics (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Artificial Intelligence (AREA)
- Theoretical Computer Science (AREA)
- Biomedical Technology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Pathology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Veterinary Medicine (AREA)
- Databases & Information Systems (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- General Physics & Mathematics (AREA)
- Heart & Thoracic Surgery (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Neurology (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Radiology & Medical Imaging (AREA)
- Multimedia (AREA)
- Psychiatry (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Physiology (AREA)
- Fuzzy Systems (AREA)
- Primary Health Care (AREA)
- Signal Processing (AREA)
- Epidemiology (AREA)
- High Energy & Nuclear Physics (AREA)
- Developmental Disabilities (AREA)
- Hospice & Palliative Care (AREA)
- Child & Adolescent Psychology (AREA)
- Psychology (AREA)
Abstract
The embodiment of the invention relates to a processing method and a device for predicting early Alzheimer's disease type based on brain images, wherein the method comprises the following steps: receiving a first brain image set; preprocessing brain images based on the data enhancement model; extracting the three-dimensional structural features of the whole brain based on the feature extraction model and carrying out feature fusion treatment on the two extracted features; performing classification prediction based on the cognitive disorder classification prediction model; when the second classification probability is the maximum classification probability and exceeds a preset threshold value, performing classification prediction based on the early Alzheimer disease classification prediction model; and performing prediction data output processing. The invention can improve the prediction convenience and the prediction efficiency.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a processing method and device for predicting early Alzheimer's disease type based on brain images.
Background
Mild cognitive impairment (Mild Cognitive Impairment, MCI) is classified into two types, amnestic MCI (acmi) and non-amnestic MCI (naMCI), where acmi is a cognitive dysfunction syndrome that occurs before early Alzheimer's Disease (AD) and is a transitional phase between normal aging and early Alzheimer's disease. At present, forgetting type mild cognitive impairment and early Alzheimer's disease can be accurately identified based on brain image technology, but the identification process requires intervention of human factors, that is to say, old people or families of the old people need to regularly hold related brain images to carry out hospital registration, queuing and consultation, which is inconvenient for the old people or families of the old people with inconvenient actions.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a processing method, a device and electronic equipment based on early stage Alzheimer's disease type prediction of brain images, which are used for carrying out image enhancement and denoising on multi-class brain images (conventional T2 structural imaging and full brain 3D rapid brain structural high-resolution imaging) through a data enhancement model to obtain corresponding multi-class enhanced images, carrying out feature extraction and fusion processing on the multi-class enhanced images through a feature extraction model to obtain corresponding full brain structural features, carrying out classification prediction on the obtained full brain structural features according to a preset normal crowd brain image structural feature normal model, a forgetting type mild cognitive impairment crowd brain image structural feature normal model and a non-forgetting type mild cognitive impairment crowd brain image structural feature normal model through a cognitive impairment classification prediction model to obtain corresponding prediction types (normal type, aMCI type and naMCI type), and further carrying out classification prediction on the full brain structural features according to the preset normal crowd image structural features normal model and the early stage Alzheimer's disease brain image structural features when the prediction types are aMCI type to obtain corresponding prediction types (normal crowd type and early stage classification AD type). The invention combines the brain image technology and the artificial intelligence model technology to provide a data processing mechanism for performing aMCI and early AD prediction on the currently obtained brain image based on a preset multi-type brain image structural feature template (normal mode), and the prediction convenience and the prediction efficiency can be improved by the invention.
To achieve the above object, a first aspect of the embodiments of the present invention provides a method for predicting early stage alzheimer's disease type based on brain images, the method comprising:
receiving a first brain image set; the first brain image set comprises a first type of brain image and a second type of brain image; the first brain image is a conventional T2 structure image, and the second brain image is a full brain 3D rapid brain structure high-resolution image;
performing brain image preprocessing on the first and second brain images based on a preset data enhancement model to generate corresponding first and second enhancement images;
based on a preset feature extraction model, carrying out full brain three-dimensional structural feature extraction processing on the first type and the second type of enhanced images respectively, and carrying out feature fusion processing on the two obtained extracted features to generate a corresponding first full brain structural feature map;
based on a preset cognitive disorder classification prediction model, performing classification prediction processing on the first whole brain structure feature map according to a preset normal population brain image structure feature normal model, a forgetting type mild cognitive disorder population brain image structure feature normal model and a non-forgetting type mild cognitive disorder population brain image structure feature normal model to generate a corresponding first classification vector; the first classification vector comprises first, second and third classification probabilities; the prediction type corresponding to the first classification probability is a normal type, the prediction type corresponding to the second classification probability is a forgetting type mild cognitive impairment type, and the prediction type corresponding to the third classification probability is a non-forgetting type mild cognitive impairment type;
When the second classification probability is the maximum classification probability of the first, second and third classification probabilities and exceeds a preset first probability threshold, performing classification prediction processing on the first whole brain structural feature map based on a preset early Alzheimer's disease classification prediction model according to a preset normal population brain image structural feature normal model and an early Alzheimer's disease population brain image structural feature normal model to generate a corresponding second classification vector; the second classification vector comprises a fourth classification probability and a fifth classification probability; the prediction type corresponding to the fourth classification probability is a normal type, and the prediction type corresponding to the fifth classification probability is an early-stage Alzheimer disease type;
and carrying out prediction data output processing according to the first classification vector and the second classification vector.
Preferably, the first brain image is a 2D slice image sequence composed of a plurality of 2D slice images with different depths; the second brain image is a 3D image;
the first type of enhanced image is a 2D slice image sequence which consists of a plurality of 2D enhanced slice images with different depths; the second type of enhanced image is a 3D enhanced image;
the data enhancement model comprises a first data enhancement network and a second data enhancement network; the first data enhancement network and the second data enhancement network are realized based on an automatic encoder network;
The feature extraction model comprises a first feature extraction network, a second feature extraction network and a first feature fusion module; the first feature extraction network comprises a first 2D convolutional neural network and a first long-term and short-term memory network; the second feature extraction network comprises a first plane segmentation unit, a second 2D convolutional neural network and a second long-term and short-term memory network;
the cognitive disorder classification prediction model comprises a first, a second and a third fully connected networks and a corresponding first, second and third activation functions; the first fully-connected network is connected with the first activation function; the second fully-connected network is connected with the second activation function; the third full connection network is connected with the third activation function;
the early Alzheimer disease classification prediction model comprises a fourth fully-connected network, a fifth fully-connected network, a fourth activation function and a fifth activation function; the fourth fully-connected network is connected with the fourth activation function; the fifth fully-connected network is connected with the fifth activation function.
Preferably, the preprocessing of brain images of the first type and the second type based on the preset data enhancement model to generate corresponding first type and second type enhancement images specifically includes:
2D image noise elimination and contrast and brightness adjustment processing are respectively carried out on each 2D slice image of the first type brain image based on the first data enhancement network of the data enhancement model to generate a corresponding 2D enhancement slice image, and the corresponding first type enhancement image is formed by all the obtained 2D enhancement slice images according to corresponding depth ordering;
and performing 3D image noise elimination and contrast and brightness adjustment processing on the second type brain image based on the second data enhancement network of the data enhancement model to generate a corresponding 3D enhanced image, and taking the obtained 3D enhanced image as the corresponding second type enhanced image.
Preferably, the method for extracting the full-brain three-dimensional structural features of the first type and the second type of enhanced images based on a preset feature extraction model and performing feature fusion processing on the two obtained extracted features to generate a corresponding first full-brain structural feature map specifically includes:
inputting the first type of enhanced images into the first feature extraction network of the feature extraction model, and performing 2D image feature extraction processing on each 2D enhanced slice image of the first type of enhanced images by the first 2D convolutional neural network to generate a corresponding shape of 1 XH 1 ×W 1 ×C 1 Is composed of all the obtained first slice feature images with the corresponding shape of D 1 ×H 1 ×W 1 ×C 1 The first feature map is subjected to depth feature extraction processing by the first long-term and short-term memory network according to the depth direction to obtain a corresponding shape D 1 ×H 1 ×W 1 ×C 1 Is a second feature map of (2); d (D) 1 、H 1 、W 1 、C 1 Respectively obtaining a depth dimension parameter, a height dimension parameter, a width dimension parameter and a channel dimension parameter of the first slice feature map;
inputting the second type of enhanced images into the second feature extraction network of the feature extraction model, performing 2D plane segmentation processing on the second type of enhanced images along the depth direction by the first plane segmentation unit according to a preset unit depth to generate corresponding first plane graphs, and performing 2D image feature extraction processing on each first plane graph by the second 2D convolutional neural network to generate corresponding 1 XH shape 2 ×W 2 ×C 2 Is composed of all the obtained first plane feature images and has a corresponding shape of D 2 ×H 2 ×W 2 ×C 2 The second long-short-term memory network performs depth feature extraction processing on the third feature map according to the depth direction to obtain a corresponding shape D 2 ×H 2 ×W 2 ×C 2 Is a fourth feature map of (2); d (D) 2 、H 2 、W 2 、C 2 D, respectively, depth dimension parameter, height dimension parameter, width dimension parameter and channel dimension parameter of the first plane feature map 2 =D 1 、H 2 =H 1 、W 2 =W 1 ;
Inputting the second feature map and the fourth feature map into the first feature fusion module of the feature extraction model to perform feature fusion processing to obtain a corresponding first whole brain structure feature map; the shape of the first whole brain structure feature map is D 3 ×H 3 ×W 3 ×C 3 ,D 3 、H 3 、W 3 、C 3 D is the depth dimension parameter, the height dimension parameter, the width dimension parameter and the channel dimension parameter of the first whole brain structure feature map respectively 3 =D 1 、H 3 =H 1 、W 3 =W 1 。
Preferably, the classifying and predicting model based on preset cognitive disorder performs classifying and predicting processing on the first whole brain structure feature map according to a preset normal population brain image structure feature normal model, a forgetting type mild cognitive disorder population brain image structure feature normal model and a non-forgetting type mild cognitive disorder population brain image structure feature normal model to generate a corresponding first classification vector, and specifically includes:
inputting the first full brain structure feature map into the first, second and third full connection networks of the cognitive disorder classification prediction model respectively, carrying out full connection difference calculation on the first full brain structure feature map and the normal crowd brain image structure feature normal mode by the first full connection network to obtain a corresponding first feature vector, carrying out full connection difference calculation on the first full brain structure feature map and the forgetting type mild cognitive disorder crowd brain image structure feature normal mode by the second full connection network to obtain a corresponding second feature vector, and carrying out full connection difference calculation on the first full brain structure feature map and the non-forgetting type mild cognitive disorder crowd brain image structure feature normal mode by the third full connection network to obtain a corresponding third feature vector;
Inputting the first, second and third feature vectors into the first, second and third activation functions of the cognitive disorder classification prediction model respectively, performing function calculation on the first feature vector by the first activation function to obtain corresponding first classification probability, performing function calculation on the second feature vector by the second activation function to obtain corresponding second classification probability, and performing function calculation on the third feature vector by the third activation function to obtain corresponding third classification probability;
and forming the corresponding first classification vector by the obtained first, second and third classification probabilities.
Preferably, the classifying and predicting model based on the preset early stage alzheimer's disease performs classifying and predicting processing on the first whole brain structural feature map according to a preset normal population brain image structural feature normal model and a preset early stage alzheimer's disease population brain image structural feature normal model to generate a corresponding second classification vector, and specifically includes:
respectively inputting the first full-brain structural feature map into the fourth full-connection network and the fifth full-connection network of the early Alzheimer's disease classification prediction model, performing full-connection differential computation on the first full-brain structural feature map and the normal crowd brain image structural feature normal model by the fourth full-connection network to obtain a corresponding fourth feature vector, and performing full-connection differential computation on the first full-brain structural feature map and the early Alzheimer's disease crowd brain image structural feature normal model by the fifth full-connection network to obtain a corresponding fifth feature vector;
Inputting the fourth and fifth feature vectors into the fourth and fifth activation functions of the early Alzheimer's disease classification prediction model respectively, performing function calculation on the fourth feature vector by the fourth activation function to obtain the corresponding fourth classification probability, and performing function calculation on the fifth feature vector by the fifth activation function to obtain the corresponding fifth classification probability;
and forming the corresponding second classification vector by the fourth classification probability and the fifth classification probability.
A second aspect of the embodiments of the present invention provides an apparatus for implementing the processing method for predicting early alzheimer's disease type based on brain image according to the first aspect, where the apparatus includes: the device comprises a data receiving module, a preprocessing module, a feature extraction module, a first classification prediction module, a second classification prediction module and a data output module;
the data receiving module is used for receiving a first brain image set; the first brain image set comprises a first type of brain image and a second type of brain image; the first brain image is a conventional T2 structure image, and the second brain image is a full brain 3D rapid brain structure high-resolution image;
the preprocessing module is used for preprocessing the brain images of the first type and the second type based on a preset data enhancement model to generate corresponding first type and second type enhancement images;
The feature extraction module is used for respectively carrying out full-brain three-dimensional structural feature extraction processing on the first type of enhanced images and the second type of enhanced images based on a preset feature extraction model, and carrying out feature fusion processing on the two obtained extracted features to generate a corresponding first full-brain structural feature map;
the first classification prediction module is used for performing classification prediction processing on the first whole brain structure feature map based on a preset cognitive disorder classification prediction model according to a preset normal population brain image structure feature normal model, a forgetting type mild cognitive disorder population brain image structure feature normal model and a non-forgetting type mild cognitive disorder population brain image structure feature normal model to generate a corresponding first classification vector; the first classification vector comprises first, second and third classification probabilities; the prediction type corresponding to the first classification probability is a normal type, the prediction type corresponding to the second classification probability is a forgetting type mild cognitive impairment type, and the prediction type corresponding to the third classification probability is a non-forgetting type mild cognitive impairment type;
the second classification prediction module is used for performing classification prediction processing on the first whole brain structural feature map according to a preset normal population brain image structural feature normal model and a preset early Alzheimer's disease population brain image structural feature normal model based on a preset early Alzheimer's disease classification prediction model when the second classification probability is the maximum classification probability of the first, second and third classification probabilities and exceeds a preset first probability threshold; the second classification vector comprises a fourth classification probability and a fifth classification probability; the prediction type corresponding to the fourth classification probability is a normal type, and the prediction type corresponding to the fifth classification probability is an early-stage Alzheimer disease type;
The data output module is used for carrying out prediction data output processing according to the first classification vector and the second classification vector.
A third aspect of an embodiment of the present invention provides an electronic device, including: memory, processor, and transceiver;
the processor is configured to couple to the memory, and read and execute the instructions in the memory, so as to implement the method steps described in the first aspect;
the transceiver is coupled to the processor and is controlled by the processor to transmit and receive messages.
The embodiment of the invention provides a processing method, a device and electronic equipment based on early Alzheimer's disease type prediction of brain images, which are used for carrying out image enhancement and denoising on multi-type brain images (conventional T2 structural imaging and full brain 3D rapid brain structural high-resolution imaging) through a data enhancement model to obtain corresponding multi-type enhanced images, carrying out feature extraction and fusion processing on the multi-type enhanced images through a feature extraction model to obtain corresponding full brain structural features, carrying out classification prediction on the obtained full brain structural features according to a preset normal crowd brain image structural feature normal model and a forgetting type mild cognitive impairment crowd brain image structural feature normal model through a cognitive impairment classification prediction model to obtain corresponding prediction types (normal types, aMCI types and naI types), and further using an early Alzheimer's disease classification prediction model to classify the full brain structural features according to the preset normal crowd brain image structural feature normal model and the early Alzheimer's disease crowd brain structural feature normal model when the prediction types are aMCI types to obtain corresponding prediction types (AD types and early prediction types). The invention combines the brain image technology and the artificial intelligence model technology to provide a data processing mechanism for performing aMCI and early AD prediction on the currently obtained brain image based on a preset multi-type brain image structural feature template (normal mode), and the prediction convenience and the prediction efficiency are improved through the invention.
Drawings
Fig. 1 is a schematic diagram of a treatment method for predicting early alzheimer's disease type based on brain images according to a first embodiment of the present invention;
fig. 2 is a block diagram of a processing device for predicting early alzheimer's disease based on brain images according to a second embodiment of the present invention;
fig. 3 is a schematic structural diagram of an electronic device according to a third embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
An embodiment of the present invention provides a method for predicting early-stage alzheimer's disease based on brain images, as shown in fig. 1, which is a schematic diagram of a method for predicting early-stage alzheimer's disease based on brain images, and the method mainly includes the following steps:
Step 1, receiving a first brain image set;
the first brain image set comprises a first type of brain images and a second type of brain images; the first brain image is a conventional T2 structure image, and the second brain image is a full brain 3D rapid brain structure high-resolution image; the first brain image is a 2D slice image sequence composed of a plurality of 2D slice images with different depths; the second type of brain image is a 3D image.
Here, the first brain image set in the embodiment of the present invention includes two types of brain image data: conventional T2 structure imaging, namely a first type brain image, and whole brain 3D rapid brain structure high-resolution imaging, namely a second type brain image; the conventional T2 structure imaging is here a T2 weighted imaging, also called T2WI imaging, highlighting differences in T2 relaxation (transverse relaxation) of tissue in brain magnetic resonance imaging (Magnetic Resonance Imaging, MRI), from the well-known MRI imaging principle we know that the conventional T2 structure imaging is actually a 2D slice image sequence consisting of 2D slice images at different tomographic depths; the whole brain 3D fast brain structure high resolution imaging here is a forebrain 3D structure imaging obtained by fast, high resolution 3D optical imaging technique, which imaging is a 3D image.
Step 2, performing brain image preprocessing on the first type and the second type of brain images based on a preset data enhancement model to generate corresponding first type and second type of enhancement images;
wherein the data enhancement model comprises a first data enhancement network and a second data enhancement network; the first and second data enhancement networks are implemented based on an Auto Encoder network (AE); the first type of enhanced image is a 2D slice image sequence which consists of a plurality of 2D enhanced slice images with different depths; the second type of enhanced image is a 3D enhanced image;
the method specifically comprises the following steps: step 21, performing 2D image noise elimination and contrast and brightness adjustment processing on each 2D slice image of the first brain image based on a first data enhancement network of the data enhancement model to generate a corresponding 2D enhanced slice image, and forming a corresponding first enhanced image by all obtained 2D enhanced slice images according to corresponding depth ordering;
here, according to the disclosed structure of the AE network, the first data enhancement network is composed of an encoder and a decoder, the first data enhancement network sends the 2D slice image to the encoder for feature encoding, then sends the encoding result to the decoder for reducing noise features in the encoding features, and simultaneously optimally adjusts brightness and contrast features respectively, so that the processing effect of simultaneously reducing noise and enhancing the image is achieved;
And step 22, performing 3D image noise elimination and contrast and brightness adjustment processing on the second type brain image based on the second data enhancement network of the data enhancement model to generate a corresponding 3D enhanced image, and taking the obtained 3D enhanced image as the corresponding second type enhanced image.
Here, according to the disclosed structure of the AE network, the second data enhancement network is composed of two parts of an encoder and a decoder, the second data enhancement network sends the second brain image to the encoder for feature encoding, and then sends the encoding result to the decoder for reducing noise features in the encoding features and simultaneously optimally adjusting brightness and contrast features, so that the processing effect of simultaneously reducing noise and enhancing images is achieved.
Step 3, based on a preset feature extraction model, performing full-brain three-dimensional structural feature extraction processing on the first type and the second type of enhanced images respectively, and performing feature fusion processing on the two obtained extracted features to generate a corresponding first full-brain structural feature map;
the feature extraction model comprises a first feature extraction network, a second feature extraction network and a first feature fusion module; the first feature extraction network comprises a first 2D convolutional neural network (2D-Convolutional Neural Networks, 2D-CNN) and a first Long Short-Term Memory network (LSTM); the second feature extraction network comprises a first plane segmentation unit, a second 2D convolutional neural network and a second long-term and short-term memory network;
The method specifically comprises the following steps: step 31, inputting the first type of enhanced image into a first feature extraction network of a feature extraction model, and inputting the first type of enhanced image by a first 2D convolutional neural network2D image feature extraction processing is performed on each 2D enhanced slice image of the slice image to generate a corresponding shape of 1 XH 1 ×W 1 ×C 1 Is composed of all the obtained first slice feature images and has a corresponding shape of D 1 ×H 1 ×W 1 ×C 1 The first feature map is subjected to depth feature extraction processing according to the depth direction by a first long-short-term memory network to obtain a corresponding shape D 1 ×H 1 ×W 1 ×C 1 Is a second feature map of (2);
wherein D is 1 、H 1 、W 1 、C 1 The depth dimension parameter, the height dimension parameter, the width dimension parameter and the channel dimension parameter of the first slice feature map are respectively;
here, in the embodiment of the present invention, 2D image feature extraction is performed on each 2D enhanced slice image of the first type of enhanced image through the first 2D convolutional neural network of the first feature extraction network, so that only one planar feature can be obtained, and no continuous feature or associated feature in the depth dimension exists; the long-term memory network and the short-term memory network can correct the current characteristic based on the history characteristic, so that the embodiment of the invention is connected with a long-term memory network, namely the first long-term memory network, after the first 2D convolutional neural network, so that each layer of characteristic can be reset on the depth channel of the first characteristic map based on the history characteristic to obtain a second characteristic map, and the second characteristic map is actually a 3D whole brain structure characteristic deduced from the enhanced image imaged by the conventional T2 structure;
Step 32, inputting the second type of enhanced images into a second feature extraction network of a feature extraction model, performing 2D plane segmentation processing on the second type of enhanced images along the depth direction by a first plane segmentation unit according to a preset unit depth to generate corresponding first plane diagrams, and performing 2D image feature extraction processing on each first plane diagram by a second 2D convolutional neural network to generate corresponding 1 XH shape 2 ×W 2 ×C 2 Is formed by all the obtained first plane characteristic diagrams and has the corresponding shape of D 2 ×H 2 ×W 2 ×C 2 And (2) a third feature map ofThe second long-short-term memory network performs depth feature extraction processing on the third feature map according to the depth direction to obtain a corresponding shape D 2 ×H 2 ×W 2 ×C 2 Is a fourth feature map of (2);
wherein D is 2 、H 2 、W 2 、C 2 D is the depth dimension parameter, the height dimension parameter, the width dimension parameter and the channel dimension parameter of the first plane feature map respectively 2 =D 1 、H 2 =H 1 、W 2 =W 1 ;
Here, the second type of enhanced image is a 3D image, and conventionally, a 3D-CNN network is used to directly perform 3D feature extraction on the second type of enhanced image, but the model calculation amount in the conventional processing manner is huge, and the calculation period is also longer; in order to solve the problem, the embodiment of the invention replaces the conventional 3D-CNN network by using a first plane segmentation unit, a second 2D convolutional neural network and a second long-short-term memory network, namely, firstly, the first plane segmentation unit is used for slicing the second type of enhanced images along the depth direction to obtain a plurality of plane images similar to slice images, namely, a first plane image, then, 2D image feature extraction is carried out on each first plane image based on the second 2D convolutional neural network similar to the first 2D convolutional neural network, and then, the second long-short-term memory network is used for resetting each layer of features on the depth channel of a third feature image based on historical features to obtain a fourth feature image which is actually a 3D whole brain structure feature deduced from the enhanced images of the whole brain 3D rapid brain structure high resolution imaging;
Step 33, inputting the second feature map and the fourth feature map into a first feature fusion module of a feature extraction model to perform feature fusion processing to obtain a corresponding first whole brain structure feature map;
wherein the shape of the first whole brain structure feature map is D 3 ×H 3 ×W 3 ×C 3 ,D 3 、H 3 、W 3 、C 3 D is the depth dimension parameter, the height dimension parameter, the width dimension parameter and the channel dimension parameter of the first whole brain structure feature map respectively 3 =D 1 、H 3 =H 1 、W 3 =W 1 。
Step 4, based on a preset cognitive disorder classification prediction model, performing classification prediction processing on the first whole brain structure feature map according to a preset normal population brain image structure feature normal model, a forgetting type mild cognitive disorder population brain image structure feature normal model and a non-forgetting type mild cognitive disorder population brain image structure feature normal model to generate a corresponding first classification vector;
wherein the cognitive disorder classification prediction model comprises first, second and third fully connected networks and corresponding first, second and third activation functions; the first full connection network is connected with a first activation function; the second full connection network is connected with a second activation function; the third full connection network is connected with a third activation function; the first classification vector comprises first, second and third classification probabilities; the prediction type corresponding to the first classification probability is a normal type, the prediction type corresponding to the second classification probability is a forgetting type mild cognitive impairment type, and the prediction type corresponding to the third classification probability is a non-forgetting type mild cognitive impairment type;
The method specifically comprises the following steps: step 41, respectively inputting the first full brain structure feature map into a first full-connection network, a second full-connection network and a third full-connection network of a cognitive disorder classification prediction model, carrying out full-connection differential computation on the first full brain structure feature map and a normal crowd brain image structure feature normal model by the first full-connection network to obtain a corresponding first feature vector, carrying out full-connection differential computation on the first full brain structure feature map and a forgetting type mild cognitive disorder crowd brain image structure feature normal model by the second full-connection network to obtain a corresponding second feature vector, and carrying out full-connection differential computation on the first full brain structure feature map and a non-forgetting type mild cognitive disorder crowd brain image structure feature normal model by the third full-connection network to obtain a corresponding third feature vector;
step 42, inputting the first, second and third feature vectors into the first, second and third activation functions of the cognitive disorder classification prediction model respectively, performing function calculation on the first feature vector by the first activation function to obtain corresponding first classification probability, performing function calculation on the second feature vector by the second activation function to obtain corresponding second classification probability, and performing function calculation on the third feature vector by the third activation function to obtain corresponding third classification probability;
And step 43, forming a corresponding first classification vector by the obtained first, second and third classification probabilities.
Here, the model structure of the cognitive impairment classification prediction model resembles a multi-layer perceptual network (Multilayer Perceptron, MLP) supporting multi-class output; the normal crowd brain image structural feature normal mode, the forgetting type mild cognitive impairment crowd brain image structural feature normal mode and the non-forgetting type mild cognitive impairment crowd brain image structural feature normal mode are 3D brain structural templates of three types of crowds prepared in advance, the three types of 3D brain structural templates are general brain structural templates obtained by acquiring and constructing 3D brain structural feature databases of the three types of crowds in advance and summarizing common brain structural features of 3D brain structural feature data in the various 3D brain structural feature databases, and the common brain structural feature summarizing mode can be a brain map or a meninges processing mode disclosed by each authority and is not repeated herein; the application of the multi-layer perception network can be used for carrying out corresponding classification or multi-classification prediction on the input variable based on one or more reference templates, and the implementation of the invention uses a cognitive disorder classification prediction model to carry out three-classification prediction (normal type, aMCI type and naMCI type) on the input first whole brain structure feature map according to three brain structure templates by utilizing the multi-layer perception network.
Step 5, when the second classification probability is the maximum classification probability of the first, second and third classification probabilities and exceeds a preset first probability threshold, performing classification prediction processing on the first whole brain structural feature map based on a preset early Alzheimer's disease classification prediction model according to a preset normal population brain image structural feature normal model and an early Alzheimer's disease population brain image structural feature normal model to generate a corresponding second classification vector;
the early Alzheimer disease classification prediction model comprises a fourth fully-connected network, a fifth fully-connected network, a fourth activation function and a fifth activation function; the fourth full-connection network is connected with a fourth activation function; the fifth full connection network is connected with a fifth activation function; the second classification vector comprises a fourth classification probability and a fifth classification probability; the prediction type corresponding to the fourth classification probability is a normal type, and the prediction type corresponding to the fifth classification probability is an early Alzheimer disease type;
based on a preset early Alzheimer's disease classification prediction model, performing classification prediction processing on the first whole brain structure feature map according to a preset normal population brain image structure feature normal model and an early Alzheimer's disease population brain image structure feature normal model to generate a corresponding second classification vector, wherein the method specifically comprises the following steps:
Step 51, the first full brain structure feature map is respectively input into a fourth full-connection network and a fifth full-connection network of the early Alzheimer's disease classification prediction model, the fourth full-connection network carries out full-connection differential computation on the first full-brain structure feature map and the normal crowd brain image structure feature normal model to obtain a corresponding fourth feature vector, and the fifth full-connection network carries out full-connection differential computation on the first full-brain structure feature map and the early Alzheimer's disease crowd brain image structure feature normal model to obtain a corresponding fifth feature vector;
step 52, inputting the fourth and fifth feature vectors into the fourth and fifth activation functions of the early Alzheimer's disease classification prediction model, respectively, performing function calculation on the fourth feature vector by the fourth activation function to obtain a corresponding fourth classification probability, and performing function calculation on the fifth feature vector by the fifth activation function to obtain a corresponding fifth classification probability;
and step 53, forming corresponding second classification vectors by the obtained fourth and fifth classification probabilities.
Here, the first probability threshold is a preset larger probability threshold; the model structure of the early Alzheimer disease classification prediction model is similar to a multi-layer perception network supporting multi-classification output; the brain image structural feature normal model of the early-stage Alzheimer's disease crowd is a 3D brain structural template of the pair of early-stage Alzheimer's disease crowd prepared in advance, and the 3D brain structural template is a general brain structural template obtained by acquiring and constructing a 3D brain structural feature database of the early-stage Alzheimer's disease crowd in advance and summarizing the common brain structural features of the 3D brain structural feature data in the 3D brain structural feature database; the application of the multi-layer sensing network can be used for carrying out corresponding classification or multi-classification prediction on the input variable based on one or more reference templates, and the application of the invention in the early Alzheimer disease classification prediction model is that the multi-layer sensing network is used for carrying out classification prediction (normal type and early AD type) on the input first whole brain structure characteristic diagram according to two brain structure templates.
Step 6, performing prediction data output processing according to the first and second classification vectors;
the method specifically comprises the following steps: the first classification probability and the corresponding prediction type form corresponding first prediction data, the second classification probability and the corresponding prediction type form corresponding second prediction data, the third classification probability and the corresponding prediction type form corresponding third prediction data, the fifth classification probability and the corresponding prediction type form corresponding fourth prediction data, and the obtained first, second, third and fourth prediction data form corresponding prediction data sets to be output.
Fig. 2 is a block diagram of a processing apparatus for predicting early alzheimer's disease type based on brain image according to a second embodiment of the present invention, where the apparatus is a terminal device or a server for implementing the foregoing method embodiment, or may be an apparatus capable of enabling the foregoing terminal device or the server to implement the foregoing method embodiment, and for example, the apparatus may be an apparatus or a chip system of the foregoing terminal device or the server. As shown in fig. 2, the apparatus includes: a data receiving module 201, a preprocessing module 202, a feature extraction module 203, a first classification prediction module 204, a second classification prediction module 205, and a data output module 206.
The data receiving module 201 is configured to receive a first brain image set; the first brain image set comprises a first type of brain image and a second type of brain image; the first brain image is a conventional T2 structure image, and the second brain image is a full brain 3D rapid brain structure high resolution image.
The preprocessing module 202 is configured to perform brain image preprocessing on the first and second brain images based on a preset data enhancement model to generate corresponding first and second enhanced images.
The feature extraction module 203 is configured to perform feature extraction processing on the first type and the second type of enhanced images in the whole brain three-dimensional structure based on a preset feature extraction model, and perform feature fusion processing on the obtained two extracted features to generate a corresponding first whole brain structural feature map.
The first classification prediction module 204 is configured to perform classification prediction processing on the first whole brain structure feature map based on a preset cognitive disorder classification prediction model according to a preset normal population brain image structure feature normal model, a forgetting type mild cognitive disorder population brain image structure feature normal model, and a non-forgetting type mild cognitive disorder population brain image structure feature normal model to generate a corresponding first classification vector; the first classification vector comprises first, second and third classification probabilities; the prediction type corresponding to the first classification probability is a normal type, the prediction type corresponding to the second classification probability is a forgetting type mild cognitive impairment type, and the prediction type corresponding to the third classification probability is a non-forgetting type mild cognitive impairment type.
The second classification prediction module 205 is configured to perform classification prediction processing on the first whole brain structure feature map to generate a corresponding second classification vector based on a preset early-stage alzheimer's disease classification prediction model according to a preset normal population brain image structure feature normal model and an early-stage alzheimer's disease population brain image structure feature normal model when the second classification probability is the maximum classification probability of the first, second and third classification probabilities and exceeds a preset first probability threshold; the second classification vector comprises a fourth classification probability and a fifth classification probability; the prediction type corresponding to the fourth classification probability is a normal type, and the prediction type corresponding to the fifth classification probability is an early-stage Alzheimer disease type.
The data output module 206 is configured to perform prediction data output processing according to the first and second classification vectors.
The processing device for predicting early alzheimer's disease type based on brain image provided by the embodiment of the invention can execute the method steps in the method embodiment, and the implementation principle and the technical effect are similar and are not repeated here.
It should be noted that, it should be understood that the division of the modules of the above apparatus is merely a division of a logic function, and may be fully or partially integrated into a physical entity or may be physically separated. And these modules may all be implemented in software in the form of calls by the processing element; or can be realized in hardware; the method can also be realized in a form of calling software by a processing element, and the method can be realized in a form of hardware by a part of modules. For example, the data receiving module may be a processing element that is set up separately, may be implemented in a chip of the above apparatus, or may be stored in a memory of the above apparatus in the form of program codes, and may be called by a processing element of the above apparatus to execute the functions of the above determining module. The implementation of the other modules is similar. In addition, all or part of the modules can be integrated together or can be independently implemented. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in a software form.
For example, the modules above may be one or more integrated circuits configured to implement the methods above, such as: one or more specific integrated circuits (Application Specific Integrated Circuit, ASIC), or one or more digital signal processors (Digital Signal Processor, DSP), or one or more field programmable gate arrays (Field Programmable Gate Array, FPGA), etc. For another example, when a module above is implemented in the form of a processing element scheduler code, the processing element may be a general purpose processor, such as a central processing unit (Central Processing Unit, CPU) or other processor that may invoke the program code. For another example, the modules may be integrated together and implemented in the form of a System-on-a-chip (SOC).
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces, in whole or in part, the processes or functions described in connection with the foregoing method embodiments. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (e.g., coaxial cable, fiber optic, digital subscriber line ((Digital Subscriber Line, DSL)), or wireless (e.g., infrared, wireless, bluetooth, microwave, etc.) means, the computer-readable storage medium may be any available medium that can be accessed by the computer or a data storage device such as a server, data center, etc., that contains an integration of one or more available media, the available media may be magnetic media (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid state disk, SSD), etc.
Fig. 3 is a schematic structural diagram of an electronic device according to a third embodiment of the present invention. The electronic device may be the aforementioned terminal device or server, or may be a terminal device or server connected to the aforementioned terminal device or server for implementing the method of the embodiment of the present invention. As shown in fig. 3, the electronic device may include: a processor 301 (e.g., a CPU), a memory 302, a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transceiving actions of the transceiver 303. The memory 302 may store various instructions for performing the various processing functions and implementing the processing steps described in the method embodiments previously described. Preferably, the electronic device according to the embodiment of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to implement communication connections between the elements. The communication port 306 is used for connection communication between the electronic device and other peripheral devices.
The system bus 305 referred to in fig. 3 may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, or the like. The system bus may be classified into an address bus, a data bus, a control bus, and the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus. The communication interface is used to enable communication between the database access apparatus and other devices (e.g., clients, read-write libraries, and read-only libraries). The Memory may comprise random access Memory (Random Access Memory, RAM) and may also include Non-Volatile Memory (Non-Volatile Memory), such as at least one disk Memory.
The processor may be a general-purpose processor, including a Central Processing Unit (CPU), a network processor (Network Processor, NP), a graphics processor (Graphics Processing Unit, GPU), etc.; but may also be a digital signal processor DSP, an application specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.
The embodiment of the invention provides a processing method, a device and electronic equipment based on early Alzheimer's disease type prediction of brain images, which are used for carrying out image enhancement and denoising on multi-type brain images (conventional T2 structural imaging and full brain 3D rapid brain structural high-resolution imaging) through a data enhancement model to obtain corresponding multi-type enhanced images, carrying out feature extraction and fusion processing on the multi-type enhanced images through a feature extraction model to obtain corresponding full brain structural features, carrying out classification prediction on the obtained full brain structural features according to a preset normal crowd brain image structural feature normal model and a forgetting type mild cognitive impairment crowd brain image structural feature normal model through a cognitive impairment classification prediction model to obtain corresponding prediction types (normal types, aMCI types and naI types), and further using an early Alzheimer's disease classification prediction model to classify the full brain structural features according to the preset normal crowd brain image structural feature normal model and the early Alzheimer's disease crowd brain structural feature normal model when the prediction types are aMCI types to obtain corresponding prediction types (AD types and early prediction types). The invention combines the brain image technology and the artificial intelligence model technology to provide a data processing mechanism for performing aMCI and early AD prediction on the currently obtained brain image based on a preset multi-type brain image structural feature template (normal mode), and the prediction convenience and the prediction efficiency are improved through the invention.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of function in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (8)
1. A method of treating early stage alzheimer's disease type based on brain imaging prediction, the method comprising:
receiving a first brain image set; the first brain image set comprises a first type of brain image and a second type of brain image; the first brain image is a conventional T2 structure image, and the second brain image is a full brain 3D rapid brain structure high-resolution image;
performing brain image preprocessing on the first and second brain images based on a preset data enhancement model to generate corresponding first and second enhancement images;
based on a preset feature extraction model, carrying out full brain three-dimensional structural feature extraction processing on the first type and the second type of enhanced images respectively, and carrying out feature fusion processing on the two obtained extracted features to generate a corresponding first full brain structural feature map;
based on a preset cognitive disorder classification prediction model, performing classification prediction processing on the first whole brain structure feature map according to a preset normal population brain image structure feature normal model, a forgetting type mild cognitive disorder population brain image structure feature normal model and a non-forgetting type mild cognitive disorder population brain image structure feature normal model to generate a corresponding first classification vector; the first classification vector comprises first, second and third classification probabilities; the prediction type corresponding to the first classification probability is a normal person type, the prediction type corresponding to the second classification probability is a forgetting type mild cognitive impairment type, and the prediction type corresponding to the third classification probability is a non-forgetting type mild cognitive impairment type;
When the second classification probability is the maximum classification probability of the first, second and third classification probabilities and exceeds a preset first probability threshold, performing classification prediction processing on the first whole brain structural feature map based on a preset early Alzheimer's disease classification prediction model according to a preset normal population brain image structural feature normal model and an early Alzheimer's disease population brain image structural feature normal model to generate a corresponding second classification vector; the second classification vector comprises a fourth classification probability and a fifth classification probability; the prediction type corresponding to the fourth classification probability is a normal person type, and the prediction type corresponding to the fifth classification probability is an early Alzheimer disease type;
and carrying out prediction data output processing according to the first classification vector and the second classification vector.
2. The method according to claim 1, wherein the method comprises the steps of,
the first brain image is a 2D slice image sequence and consists of a plurality of 2D slice images with different depths; the second brain image is a 3D image;
the first type of enhanced image is a 2D slice image sequence which consists of a plurality of 2D enhanced slice images with different depths; the second type of enhanced image is a 3D enhanced image;
The data enhancement model comprises a first data enhancement network and a second data enhancement network; the first data enhancement network and the second data enhancement network are realized based on an automatic encoder network;
the feature extraction model comprises a first feature extraction network, a second feature extraction network and a first feature fusion module; the first feature extraction network comprises a first 2D convolutional neural network and a first long-term and short-term memory network; the second feature extraction network comprises a first plane segmentation unit, a second 2D convolutional neural network and a second long-term and short-term memory network;
the cognitive disorder classification prediction model comprises a first, a second and a third fully connected networks and a corresponding first, second and third activation functions; the first fully-connected network is connected with the first activation function; the second fully-connected network is connected with the second activation function; the third full connection network is connected with the third activation function;
the early Alzheimer disease classification prediction model comprises a fourth fully-connected network, a fifth fully-connected network, a fourth activation function and a fifth activation function; the fourth fully-connected network is connected with the fourth activation function; the fifth fully-connected network is connected with the fifth activation function.
3. The method for processing early-stage alzheimer's disease based on brain image prediction according to claim 2, wherein the pre-processing the first and second brain images based on a preset data enhancement model to generate corresponding first and second enhanced images specifically comprises:
2D image noise elimination and contrast and brightness adjustment processing are respectively carried out on each 2D slice image of the first type brain image based on the first data enhancement network of the data enhancement model to generate a corresponding 2D enhancement slice image, and the corresponding first type enhancement image is formed by all the obtained 2D enhancement slice images according to corresponding depth ordering;
and performing 3D image noise elimination and contrast and brightness adjustment processing on the second type brain image based on the second data enhancement network of the data enhancement model to generate a corresponding 3D enhanced image, and taking the obtained 3D enhanced image as the corresponding second type enhanced image.
4. The method for processing early-stage alzheimer's disease type based on brain image prediction according to claim 2, wherein the method for processing the first and second type of enhanced images based on a preset feature extraction model respectively performs feature extraction processing on the whole brain three-dimensional structure and performs feature fusion processing on the obtained two extracted features to generate a corresponding first whole brain structure feature map, specifically comprising:
Inputting the first type of enhanced images into the first feature extraction network of the feature extraction model, and performing 2D image feature extraction processing on each 2D enhanced slice image of the first type of enhanced images by the first 2D convolutional neural network to generate a corresponding shape of 1 XH 1 ×W 1 ×C 1 Is composed of all the obtained first slice feature images with the corresponding shape of D 1 ×H 1 ×W 1 ×C 1 The first feature map is subjected to depth feature extraction processing by the first long-term and short-term memory network according to the depth direction to obtain a corresponding shape D 1 ×H 1 ×W 1 ×C 1 Is a second feature map of (2); d (D) 1 、H 1 、W 1 、C 1 Respectively obtaining a depth dimension parameter, a height dimension parameter, a width dimension parameter and a channel dimension parameter of the first slice feature map;
inputting the second type of enhanced images into the second feature extraction network of the feature extraction model, performing 2D plane segmentation processing on the second type of enhanced images along the depth direction by the first plane segmentation unit according to a preset unit depth to generate corresponding first plane graphs, and performing 2D image feature extraction processing on each first plane graph by the second 2D convolutional neural network to generate corresponding 1 XH shape 2 ×W 2 ×C 2 Is composed of all the obtained first plane feature images and has a corresponding shape of D 2 ×H 2 ×W 2 ×C 2 The second long-short-term memory network performs depth feature extraction processing on the third feature map according to the depth direction to obtain a corresponding shape D 2 ×H 2 ×W 2 ×C 2 Is a fourth feature map of (2); d (D) 2 、H 2 、W 2 、C 2 D, respectively, depth dimension parameter, height dimension parameter, width dimension parameter and channel dimension parameter of the first plane feature map 2 =D 1 、H 2 =H 1 、W 2 =W 1 ;
Inputting the second feature map and the fourth feature map into the first feature fusion module of the feature extraction model to perform feature fusion processing to obtain a corresponding first whole brain structure feature map; the shape of the first whole brain structure feature map is D 3 ×H 3 ×W 3 ×C 3 ,D 3 、H 3 、W 3 、C 3 D is the depth dimension parameter, the height dimension parameter, the width dimension parameter and the channel dimension parameter of the first whole brain structure feature map respectively 3 =D 1 、H 3 =H 1 、W 3 =W 1 。
5. The method for processing early stage alzheimer's disease type based on brain image prediction according to claim 2, wherein the classification prediction processing is performed on the first whole brain structure feature map to generate a corresponding first classification vector based on a preset cognitive disorder classification prediction model according to a preset normal population brain image structure feature normal model, a forgetting type mild cognitive disorder population brain image structure feature normal model and a non-forgetting type mild cognitive disorder population brain image structure feature normal model, and the method specifically comprises:
Inputting the first full brain structure feature map into the first, second and third full connection networks of the cognitive disorder classification prediction model respectively, carrying out full connection difference calculation on the first full brain structure feature map and the normal crowd brain image structure feature normal mode by the first full connection network to obtain a corresponding first feature vector, carrying out full connection difference calculation on the first full brain structure feature map and the forgetting type mild cognitive disorder crowd brain image structure feature normal mode by the second full connection network to obtain a corresponding second feature vector, and carrying out full connection difference calculation on the first full brain structure feature map and the non-forgetting type mild cognitive disorder crowd brain image structure feature normal mode by the third full connection network to obtain a corresponding third feature vector;
inputting the first, second and third feature vectors into the first, second and third activation functions of the cognitive disorder classification prediction model respectively, performing function calculation on the first feature vector by the first activation function to obtain corresponding first classification probability, performing function calculation on the second feature vector by the second activation function to obtain corresponding second classification probability, and performing function calculation on the third feature vector by the third activation function to obtain corresponding third classification probability;
And forming the corresponding first classification vector by the obtained first, second and third classification probabilities.
6. The method for processing early stage alzheimer's disease type based on brain image prediction according to claim 2, wherein the classifying and predicting process is performed on the first whole brain structure feature map to generate a corresponding second classification vector based on a preset early stage alzheimer's disease classification and prediction model according to a preset normal population brain image structure feature normal model and an early stage alzheimer's disease population brain image structure feature normal model, and the method specifically comprises:
respectively inputting the first full-brain structural feature map into the fourth full-connection network and the fifth full-connection network of the early Alzheimer's disease classification prediction model, performing full-connection differential computation on the first full-brain structural feature map and the normal crowd brain image structural feature normal model by the fourth full-connection network to obtain a corresponding fourth feature vector, and performing full-connection differential computation on the first full-brain structural feature map and the early Alzheimer's disease crowd brain image structural feature normal model by the fifth full-connection network to obtain a corresponding fifth feature vector;
inputting the fourth and fifth feature vectors into the fourth and fifth activation functions of the early Alzheimer's disease classification prediction model respectively, performing function calculation on the fourth feature vector by the fourth activation function to obtain the corresponding fourth classification probability, and performing function calculation on the fifth feature vector by the fifth activation function to obtain the corresponding fifth classification probability;
And forming the corresponding second classification vector by the fourth classification probability and the fifth classification probability.
7. An apparatus for implementing the brain image prediction-based early stage alzheimer's disease type processing method of any one of claims 1-6, said apparatus comprising: the device comprises a data receiving module, a preprocessing module, a feature extraction module, a first classification prediction module, a second classification prediction module and a data output module;
the data receiving module is used for receiving a first brain image set; the first brain image set comprises a first type of brain image and a second type of brain image; the first brain image is a conventional T2 structure image, and the second brain image is a full brain 3D rapid brain structure high-resolution image;
the preprocessing module is used for preprocessing the brain images of the first type and the second type based on a preset data enhancement model to generate corresponding first type and second type enhancement images;
the feature extraction module is used for respectively carrying out full-brain three-dimensional structural feature extraction processing on the first type of enhanced images and the second type of enhanced images based on a preset feature extraction model, and carrying out feature fusion processing on the two obtained extracted features to generate a corresponding first full-brain structural feature map;
The first classification prediction module is used for performing classification prediction processing on the first whole brain structure feature map based on a preset cognitive disorder classification prediction model according to a preset normal population brain image structure feature normal model, a forgetting type mild cognitive disorder population brain image structure feature normal model and a non-forgetting type mild cognitive disorder population brain image structure feature normal model to generate a corresponding first classification vector; the first classification vector comprises first, second and third classification probabilities; the prediction type corresponding to the first classification probability is a normal person type, the prediction type corresponding to the second classification probability is a forgetting type mild cognitive impairment type, and the prediction type corresponding to the third classification probability is a non-forgetting type mild cognitive impairment type;
the second classification prediction module is used for performing classification prediction processing on the first whole brain structural feature map according to a preset normal population brain image structural feature normal model and a preset early Alzheimer's disease population brain image structural feature normal model based on a preset early Alzheimer's disease classification prediction model when the second classification probability is the maximum classification probability of the first, second and third classification probabilities and exceeds a preset first probability threshold; the second classification vector comprises a fourth classification probability and a fifth classification probability; the prediction type corresponding to the fourth classification probability is a normal person type, and the prediction type corresponding to the fifth classification probability is an early Alzheimer disease type;
The data output module is used for carrying out prediction data output processing according to the first classification vector and the second classification vector.
8. An electronic device, comprising: memory, processor, and transceiver;
the processor being operative to couple with the memory, read and execute instructions in the memory to implement the method of any one of claims 1-6;
the transceiver is coupled to the processor and is controlled by the processor to transmit and receive messages.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310132964.0A CN116417135B (en) | 2023-02-17 | 2023-02-17 | Processing method and device for predicting early Alzheimer's disease type based on brain image |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310132964.0A CN116417135B (en) | 2023-02-17 | 2023-02-17 | Processing method and device for predicting early Alzheimer's disease type based on brain image |
Publications (2)
Publication Number | Publication Date |
---|---|
CN116417135A true CN116417135A (en) | 2023-07-11 |
CN116417135B CN116417135B (en) | 2024-03-08 |
Family
ID=87050549
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202310132964.0A Active CN116417135B (en) | 2023-02-17 | 2023-02-17 | Processing method and device for predicting early Alzheimer's disease type based on brain image |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN116417135B (en) |
Citations (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107506797A (en) * | 2017-08-25 | 2017-12-22 | 电子科技大学 | One kind is based on deep neural network and multi-modal image alzheimer disease sorting technique |
CN111738302A (en) * | 2020-05-28 | 2020-10-02 | 华南理工大学 | System for classifying and diagnosing Alzheimer disease based on multi-modal data |
CN112470143A (en) * | 2018-09-12 | 2021-03-09 | 株式会社Fronteo | Dementia prediction device, prediction model generation device, and dementia prediction program |
CN112767341A (en) * | 2021-01-14 | 2021-05-07 | 广州中医药大学第一附属医院 | Deep learning-based method for classifying cognitive impairment of brain structure of type 2 diabetes patient |
CN114187925A (en) * | 2020-09-14 | 2022-03-15 | 北京脑智蓝海科技有限公司 | Dementia early evaluation method and device based on voice characteristics |
CN114202668A (en) * | 2020-08-31 | 2022-03-18 | 上海宽带技术及应用工程研究中心 | Method, system, medium, and apparatus for processing clinical data of alzheimer's disease |
CN114359164A (en) * | 2021-12-10 | 2022-04-15 | 中国科学院深圳先进技术研究院 | Method and system for automatically predicting Alzheimer disease based on deep learning |
CN114870191A (en) * | 2022-07-08 | 2022-08-09 | 北京智精灵科技有限公司 | Cognitive assessment improving method and system based on personality difference |
CN114999629A (en) * | 2022-04-28 | 2022-09-02 | 安徽大学 | AD early prediction method, system and device based on multi-feature fusion |
CN115064260A (en) * | 2022-06-01 | 2022-09-16 | 上海全景医学影像诊断中心有限公司 | Nuclear magnetic data-based mild cognitive impairment prediction method based on multi-connection mode fusion |
US20220367056A1 (en) * | 2020-12-09 | 2022-11-17 | MS Technologies | System and method for diagnostics and prognostics of mild cognitive impairment using deep learning |
CN115587977A (en) * | 2022-09-29 | 2023-01-10 | 中南民族大学 | Alzheimer disease pathological area positioning and classification prediction method |
-
2023
- 2023-02-17 CN CN202310132964.0A patent/CN116417135B/en active Active
Patent Citations (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107506797A (en) * | 2017-08-25 | 2017-12-22 | 电子科技大学 | One kind is based on deep neural network and multi-modal image alzheimer disease sorting technique |
CN112470143A (en) * | 2018-09-12 | 2021-03-09 | 株式会社Fronteo | Dementia prediction device, prediction model generation device, and dementia prediction program |
CN111738302A (en) * | 2020-05-28 | 2020-10-02 | 华南理工大学 | System for classifying and diagnosing Alzheimer disease based on multi-modal data |
CN114202668A (en) * | 2020-08-31 | 2022-03-18 | 上海宽带技术及应用工程研究中心 | Method, system, medium, and apparatus for processing clinical data of alzheimer's disease |
CN114187925A (en) * | 2020-09-14 | 2022-03-15 | 北京脑智蓝海科技有限公司 | Dementia early evaluation method and device based on voice characteristics |
US20220367056A1 (en) * | 2020-12-09 | 2022-11-17 | MS Technologies | System and method for diagnostics and prognostics of mild cognitive impairment using deep learning |
CN112767341A (en) * | 2021-01-14 | 2021-05-07 | 广州中医药大学第一附属医院 | Deep learning-based method for classifying cognitive impairment of brain structure of type 2 diabetes patient |
CN114359164A (en) * | 2021-12-10 | 2022-04-15 | 中国科学院深圳先进技术研究院 | Method and system for automatically predicting Alzheimer disease based on deep learning |
CN114999629A (en) * | 2022-04-28 | 2022-09-02 | 安徽大学 | AD early prediction method, system and device based on multi-feature fusion |
CN115064260A (en) * | 2022-06-01 | 2022-09-16 | 上海全景医学影像诊断中心有限公司 | Nuclear magnetic data-based mild cognitive impairment prediction method based on multi-connection mode fusion |
CN114870191A (en) * | 2022-07-08 | 2022-08-09 | 北京智精灵科技有限公司 | Cognitive assessment improving method and system based on personality difference |
CN115587977A (en) * | 2022-09-29 | 2023-01-10 | 中南民族大学 | Alzheimer disease pathological area positioning and classification prediction method |
Non-Patent Citations (1)
Title |
---|
刘昭: "基于海马体影像组学的AD辅诊模型的研究与实现", 《中国优秀硕士学位论文全文数据库 医药卫生科技》 * |
Also Published As
Publication number | Publication date |
---|---|
CN116417135B (en) | 2024-03-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10726244B2 (en) | Method and apparatus detecting a target | |
CN109325954B (en) | Image segmentation method and device and electronic equipment | |
CN110574077A (en) | Image analysis device and method using virtual three-dimensional deep neural network | |
CN112070781B (en) | Processing method and device of craniocerebral tomography image, storage medium and electronic equipment | |
CN111444744A (en) | Living body detection method, living body detection device, and storage medium | |
CN111814589A (en) | Part recognition method and related equipment and device | |
WO2021082819A1 (en) | Image generation method and apparatus, and electronic device | |
CN110032931B (en) | Method and device for generating countermeasure network training and removing reticulation and electronic equipment | |
CN112766040B (en) | Method, device, apparatus and readable storage medium for detecting residual bait | |
CN110991412A (en) | Face recognition method and device, storage medium and electronic equipment | |
CN115631112B (en) | Building contour correction method and device based on deep learning | |
CN112489063A (en) | Image segmentation method, and training method and device of image segmentation model | |
CN115410030A (en) | Target detection method, target detection device, computer equipment and storage medium | |
CN110084742B (en) | Parallax map prediction method and device and electronic equipment | |
CN113971732A (en) | Small target detection method and device, readable storage medium and electronic equipment | |
CN114037893A (en) | High-resolution remote sensing image building extraction method based on convolutional neural network | |
CN116417135B (en) | Processing method and device for predicting early Alzheimer's disease type based on brain image | |
CN112101296A (en) | Face registration method, face verification method, device and system | |
CN111461303A (en) | Digital core reconstruction method and system based on generation of antagonistic neural network | |
CN116258873A (en) | Position information determining method, training method and device of object recognition model | |
CN115689947A (en) | Image sharpening method, system, electronic device and storage medium | |
CN113808021B (en) | Image processing and model training method and device and electronic equipment | |
CN115564810A (en) | Image registration method and device | |
CN113920099A (en) | Polyp segmentation method and device, computer equipment and storage medium | |
CN112767260A (en) | Image quality improving method and device, computer equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |