CN112446875A - AMD grading system based on macular attention mechanism and uncertainty - Google Patents
AMD grading system based on macular attention mechanism and uncertainty Download PDFInfo
- Publication number
- CN112446875A CN112446875A CN202011453517.8A CN202011453517A CN112446875A CN 112446875 A CN112446875 A CN 112446875A CN 202011453517 A CN202011453517 A CN 202011453517A CN 112446875 A CN112446875 A CN 112446875A
- Authority
- CN
- China
- Prior art keywords
- blood vessel
- image
- uncertainty
- macular
- optic disc
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007246 mechanism Effects 0.000 title claims abstract description 21
- 230000011218 segmentation Effects 0.000 claims abstract description 129
- 210000004204 blood vessel Anatomy 0.000 claims abstract description 127
- 230000003902 lesion Effects 0.000 claims abstract description 31
- 238000000034 method Methods 0.000 claims abstract description 28
- 238000013135 deep learning Methods 0.000 claims abstract description 22
- 238000012549 training Methods 0.000 claims description 42
- 238000004364 calculation method Methods 0.000 claims description 30
- 230000008569 process Effects 0.000 claims description 11
- 238000005070 sampling Methods 0.000 claims description 11
- 238000013527 convolutional neural network Methods 0.000 claims description 10
- 238000004422 calculation algorithm Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 9
- 238000013528 artificial neural network Methods 0.000 claims description 7
- 238000012360 testing method Methods 0.000 claims description 7
- 238000003062 neural network model Methods 0.000 claims description 6
- 230000002207 retinal effect Effects 0.000 claims description 6
- 230000003321 amplification Effects 0.000 claims description 5
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 5
- 238000000605 extraction Methods 0.000 claims description 4
- 210000004220 fundus oculi Anatomy 0.000 claims description 4
- 210000002189 macula lutea Anatomy 0.000 claims description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 230000004807 localization Effects 0.000 claims 5
- 208000002780 macular degeneration Diseases 0.000 abstract description 14
- 238000001514 detection method Methods 0.000 abstract description 12
- 208000035719 Maculopathy Diseases 0.000 abstract description 4
- 206010064930 age-related macular degeneration Diseases 0.000 description 10
- 230000002792 vascular Effects 0.000 description 8
- 238000007781 pre-processing Methods 0.000 description 5
- 238000012935 Averaging Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 201000010099 disease Diseases 0.000 description 3
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 3
- 208000038015 macular disease Diseases 0.000 description 3
- 210000003733 optic disk Anatomy 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 206010012601 diabetes mellitus Diseases 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 210000001525 retina Anatomy 0.000 description 2
- 210000001210 retinal vessel Anatomy 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- 201000004569 Blindness Diseases 0.000 description 1
- 238000000342 Monte Carlo simulation Methods 0.000 description 1
- 206010047513 Vision blurred Diseases 0.000 description 1
- 206010047555 Visual field defect Diseases 0.000 description 1
- 238000013531 bayesian neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000002427 irreversible effect Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000019612 pigmentation Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 231100000216 vascular lesion Toxicity 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
- G06F18/24155—Bayesian classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30101—Blood vessel; Artery; Vein; Vascular
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Bioinformatics & Computational Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Molecular Biology (AREA)
- Evolutionary Biology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Eye Examination Apparatus (AREA)
Abstract
The invention provides an AMD grading system based on a macular attention mechanism and uncertainty, and relates to the technical field of maculopathy detection. The method comprises the steps of obtaining a video disc segmentation image and a blood vessel segmentation image by using a segmentation network model, obtaining a macular region image according to the obtained video disc segmentation image and the blood vessel segmentation image, obtaining a multi-channel image by using an attention network submodule, extracting features by using a Bayesian deep learning classification network model, and outputting four groups of probability values and one group of noise corresponding to four lesion types through multiple dropout Monte Carlo; and the classification network module gives the accidental uncertainty and the model uncertainty while finally outputting a model classification result. The safety performance of the model is ensured.
Description
Technical Field
The invention relates to the technical field of maculopathy detection, in particular to an AMD grading system based on a maculopathy attention mechanism and uncertainty.
Background
The macula lutea is a circular area with a diameter of about 5-6 mm between the posterior pole and the temporal side vascular arches, and diseases caused by various causes and occurring in this area are called macular diseases. Macular diseases, such as Age-Related Macular Degeneration (AMD), have severely affected the quality of life and work of millions of people worldwide. The macula lutea disease has multiple types and complex characteristics, is difficult to diagnose in early stage, and can cause irreversible blurred vision, deformed vision and visual field defect if the macular diseases are not diagnosed in time and treated properly, and can cause blindness most seriously.
The existing yellow spot detection methods are all based on a deep learning algorithm;
however, the existing yellow spot detection system cannot provide the accuracy of the detection result, and has potential safety hazard.
Disclosure of Invention
Technical problem to be solved
Aiming at the defects of the prior art, the invention provides an AMD grading system based on a macular attention mechanism and uncertainty, and solves the problems that the existing maculopathy detection system based on a deep learning algorithm cannot give out the accuracy of a detection result and has potential safety hazards.
(II) technical scheme
In order to achieve the purpose, the invention is realized by the following technical scheme:
an AMD grading system based on macular attention mechanism and uncertainty, comprising:
the blood vessel optic disc segmentation module is used for segmenting the fundus color image to obtain a binarized optic disc segmentation image and a blood vessel segmentation image;
the macular region positioning module is used for obtaining an image of the macular region based on the optic disc segmentation image and the blood vessel segmentation image;
the classification network module is used for inputting the fundus color map, the corresponding optic disc segmentation image, the blood vessel segmentation image and the macular region image into a trained deep learning classification network model based on an attention mechanism, fusing the fundus color map, the corresponding blood vessel segmentation image and the macular region image by using an attention network to obtain a multi-channel image, performing feature extraction on the multi-channel image by using a main neural network, and outputting four groups of probability values and one group of noise corresponding to four lesion types by using a plurality of dropout Monte Carlo; and acquiring the mean value and the variance of the four groups of probability values, taking the lesion type with the maximum probability mean value as a final classification result, taking the mean value of noise as accidental uncertainty, and taking the variance sum as model uncertainty.
Further, the classification network module includes:
the attention network submodule is used for fusing the fundus color image, the corresponding blood vessel segmentation image and the macular area image to obtain a multi-channel image;
the main neural network submodule is used for extracting the characteristics of the multi-channel image by using the trained main neural network model and outputting four groups of probability values and one group of noise corresponding to four lesion types through dropout Monte Carlo for multiple times;
and the classification result output submodule is used for acquiring the mean value and the variance of four groups of probability values, taking the lesion type with the maximum probability mean value as a final classification result, taking the mean value of noise as accidental uncertainty and taking the sum of variance as model uncertainty.
Further, the attention network sub-module is a third-order attention module, and
the residual attention mechanism is:
Hi,c(x)＝(1+Mi,c(x))×Fi,c(x)
Hi,c(x) Is the output of the attention network submodule, Fi,c(x) Is a picture tensor feature of the previous layer, Mi,c(x) Is the attention parameter of the soft mask.
Furthermore, the trained main neural network model is a Bayes deep learning classification network model, and 5 output nodes are set, wherein the output nodes comprise 4 classification nodes corresponding to lesion classification and a noise node corresponding to image noise;
the multichannel image is a 512 × 512 × 5 multichannel image formed by fusing a 512 × 512 macular region image, a binarized blood vessel segmentation image and a 512 × 512 × 3 fundus image;
and training the Bayesian deep learning classification network model comprises the following steps:
A. using Cross-Entrophy Loss + accidental uncertainty (apparent uncertainty) + model uncertainty (model uncertainty) Loss as a Loss function;
B. using a sub-sampling gradient descent algorithm as a learning algorithm of the convolutional neural network;
C. training 300 epochs in each convolutional neural network, setting the batch _ size to be 2, setting the initial learning rate to be 0.001, setting the attenuation coefficient to be 0.995, and stopping training when the final learning rate is reduced to 0.0001 and is not reduced any more, and performing optimization training by using an Adam optimizer;
D. diagnostic markers are grade 0, 1, 2 and 3, corresponding to healthy, mild, moderate and severe grades, respectively; in the training process, performing data amplification on training data;
E. during testing, through dropout Monte Carlo for multiple times, prediction distribution is obtained, namely, four groups of probability values corresponding to four lesion types.
Further, the macular region locating module includes:
the optic disc parameter calculating unit is used for carrying out ellipse fitting on the optic disc in the optic disc segmentation image to obtain the center and the radius of the optic disc, and the long axis and the short axis of the ellipse;
the macular region interest area calculation unit is used for determining the macular region interest area on the blood vessel segmentation image based on the optic disc radius and expanding the optic disc in the optic disc segmentation image;
the quadrant dividing unit is used for dividing the macular region into four quadrants based on the long axis and the short axis of the ellipse;
the first main vessel arch calculation unit is used for performing Venn difference on the blood vessel segmentation image and the expanded optic disc segmentation image, performing 8 connected domain calculation, reserving an area with the pixel number of the connected domain not less than 30 to obtain a connected domain calculation graph, and finding the connected domain with the largest pixel number of the connected domain as a first main vessel arch;
the blood vessel primary and secondary sequencing unit is used for taking the center of the optic disc as the center, taking the blood vessel widths at the optic disc radius positions of a plurality of preset multiples in the connected domain calculation graph, and carrying out linear combination by using corresponding first preset proportionality coefficients to obtain the average width of each blood vessel in the connected domain calculation graph; and the pixel number of the connected domain and the average width of the blood vessels are linearly combined based on a second preset proportionality coefficient, and the blood vessels are sorted according to the result.
The alternative main vessel arch calculation unit is used for determining the quadrant where the preferred main vessel arch is located; the device is also used for sequentially judging whether each blood vessel and the preferred main vessel arch are in the same left and right quadrant and in different up and down quadrants or not based on the main and secondary ordering of the blood vessels, if so, the alternative main vessel arch is obtained, and otherwise, the next blood vessel is judged;
the macular region calculation unit is used for making a straight line where the long axis is located based on the blood vessel segmentation image after the two main blood vessel arches are positioned, making a parallel line of the straight line where the long axis is located at a position which is 5 times of the radius of the optic disc on one side of the two main blood vessel arches, and forming a closed region with the two main blood vessel arches; performing circle fitting on the closed area, wherein the circle center is used as the center of the macular area, and the circle fitting result is used for positioning the macular area;
the closed region generating unit is used for drawing a perpendicular line from the main vessel arch to the straight line on which the long axis is positioned to the closest point of the straight line on which the long axis is positioned to the straight line on which the long axis is positioned when the straight line on which the long axis is positioned, the parallel line of the straight line on which the long axis is positioned and the two main vessel arches cannot form a closed region; and then a closed area is formed by the straight line of the long axis, the parallel line and the perpendicular line of the straight line of the long axis and the two main vessel arches.
Further, the expansion multiple of the optic disc is 1.2 times, and a square with the optic disc as the center is made with the 16 times radius of the optic disc as the side length to serve as the interested area.
Further, the width of the blood vessel is the distance from the outer edge of the blood vessel to the center line of the blood vessel; the preset multiples are 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9 and 2.0, and the corresponding first preset proportionality coefficients are 0.3, 0.1, 0.06 and 0.06; and the second preset proportionality coefficients corresponding to the pixel number of the connected domain and the average width of the blood vessel are 0.3 and 0.7.
Further, the yellow spot area position calculating module includes:
the macula lutea positioning unit is used for making a straight line where the long axis is located based on the blood vessel images after the two main blood vessel arches are positioned, making a parallel line of the straight line where the long axis is located at a position of 5 times of the radius of the optic disc on one side of the two main blood vessel arches, and forming a closed area with the two main blood vessel arches; and performing circle fitting on the closed area, wherein the center of the circle is used as the center of the macular area, and the circle fitting result is used for positioning the macular area.
Further, the macular region locating module further includes:
the closed region generating unit is used for drawing a perpendicular line from the main vessel arch to the straight line on which the long axis is positioned to the closest point of the straight line on which the long axis is positioned to the straight line on which the long axis is positioned when the straight line on which the long axis is positioned, the parallel line of the straight line on which the long axis is positioned and the two main vessel arches cannot form a closed region; and then a closed area is formed by the straight line of the long axis, the parallel line and the perpendicular line of the straight line of the long axis and the two main vessel arches.
Further, the system further comprises:
and the classification result credibility judging module is used for judging that the prediction credibility of the fundus oculi color map is low when the model uncertainty or the accidental uncertainty is higher than the average uncertainty of the training set by a threshold value, wherein the calculation method of the average uncertainty of the training set is used for calculating the uncertainty of each fundus oculi color map and then calculating the expectation.
(III) advantageous effects
The present invention provides an AMD classification system based on macular attention mechanism and uncertainty. Compared with the prior art, the method has the following beneficial effects:
the method comprises the steps of obtaining an optic disc segmentation image and a blood vessel segmentation image by using a blood vessel optic disc segmentation module, calculating a macular region according to the obtained optic disc segmentation image and the blood vessel segmentation image, fusing an RGB original image, the blood vessel segmentation image and the corresponding preprocessed macular region into a multi-channel image by using a classification network module, extracting the characteristics of the multi-channel image, and outputting four groups of probability values and one group of noise corresponding to four lesion types through dropout Monte Carlo for multiple times; acquiring the mean value and the variance of the four groups of probability values, taking the lesion type with the maximum probability mean value as a final classification result, taking the mean value of noise as accidental uncertainty, and taking the variance sum as model uncertainty; and the classification module gives the accidental uncertainty and the model uncertainty while finally outputting a model classification result. The safety performance of the model is ensured, when a diagnostic image can not give a result in a very certain way, whether the human expert needs to diagnose again or not can be determined through two uncertainties, and the model is safer and more reliable in clinical use.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a system block diagram of an embodiment of the present invention;
FIG. 2 is a flow chart of the training of the optic disc segmentation network and the blood vessel segmentation network according to the embodiment of the present invention;
FIG. 3 is a schematic diagram of locating a macular region in accordance with an embodiment of the present invention;
fig. 4 is a schematic diagram of a classification network according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described, and it is obvious that the described embodiments are a part of the embodiments of the present invention, but not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the application solves the problems that the existing yellow spot disease detection system based on a deep learning algorithm cannot provide accuracy of detection results and has potential safety hazards by providing an AMD grading system based on a macular attention mechanism and uncertainty.
In order to solve the technical problems, the general idea of the embodiment of the application is as follows: obtaining an optic disc segmentation image and a blood vessel segmentation image by using a blood vessel optic disc segmentation module, calculating a macular region according to the obtained optic disc segmentation image and the blood vessel segmentation image, fusing an RGB original image, the blood vessel segmentation image and the corresponding preprocessed macular region into a multi-channel image by using a classification network module, extracting the characteristics of the multi-channel image, and outputting four groups of probability values and one group of noise corresponding to four lesion types through multiple dropout Monte Carlo; acquiring the mean value and the variance of the four groups of probability values, taking the lesion type with the maximum probability mean value as a final classification result, taking the mean value of noise as accidental uncertainty, and taking the variance sum as model uncertainty; and the classification module gives the accidental uncertainty and the model uncertainty while finally outputting a model classification result. The safety performance of the model is ensured, when a diagnostic image can not give a result in a very certain way, whether the human expert needs to diagnose again or not can be determined through two uncertainties, and the model is safer and more reliable in clinical use.
In order to better understand the technical solution, the technical solution will be described in detail with reference to the drawings and the specific embodiments.
Example 1:
as shown in FIG. 1, the present invention provides an AMD staging system based on macular attention mechanism and uncertainty, the system comprising:
the blood vessel optic disc segmentation module is used for segmenting the fundus color image to obtain a binarized optic disc segmentation image and a blood vessel segmentation image;
the macular region positioning module is used for obtaining an image of the macular region based on the optic disc segmentation image and the blood vessel segmentation image;
the classification network module is used for inputting the fundus color map, the corresponding optic disc segmentation image, the blood vessel segmentation image and the macular region image into a trained deep learning classification network model based on an attention mechanism, fusing the fundus color map, the corresponding blood vessel segmentation image and the macular region image by using an attention network to obtain a multi-channel image, performing feature extraction on the multi-channel image by using a main neural network, and outputting four groups of probability values and one group of noise corresponding to four lesion types by using a plurality of dropout Monte Carlo; and acquiring the mean value and the variance of the four groups of probability values, taking the lesion type with the maximum probability mean value as a final classification result, taking the mean value of noise as accidental uncertainty, and taking the variance sum as model uncertainty.
The beneficial effect of this embodiment does:
the method comprises the steps of obtaining optic disc segmentation images and blood vessel segmentation images by utilizing two trained segmentation network models, calculating a macular region according to the obtained optic disc segmentation images and the blood vessel segmentation images, fusing a fundus color image, the corresponding blood vessel segmentation images and the macular region images by utilizing an attention network submodule to obtain a multi-channel image, extracting features by utilizing the trained Bayes deep learning classification network model, and outputting four groups of probability values and one group of noise corresponding to four lesion types through multiple dropout Monte Carlo; acquiring the mean value and the variance of the four groups of probability values, taking the lesion type with the maximum probability mean value as a final classification result, taking the mean value of noise as accidental uncertainty, and taking the variance sum as model uncertainty; and the classification network module gives the accidental uncertainty and the model uncertainty while finally outputting a model classification result. The safety performance of the model is ensured, when a diagnostic image can not give a result in a very certain way, whether the human expert needs to diagnose again or not can be determined through two uncertainties, and the model is safer and more reliable in clinical use.
The following describes the implementation process of the embodiment of the present invention in detail:
(1) vascular optic disc segmentation module:
the vascular optic disk segmentation module uses an architecture of 2 convolutional neural networks. The first convolutional neural network is a vessel segmentation network, and after training, a new image can be output after inputting a retinal fundus image, and the pixel intensity range of the image is between 0 and 1. Each pixel value represents the probability that it belongs to a retinal blood vessel, and the vessel segmentation process can effectively eliminate the changes in pigmentation, illumination, and non-vascular lesions. And the second convolutional neural network is an optic disc segmentation network, and after training, a fundus image is input, so that optic disc segmentation probability similar to that of the first network can be output.
The steps of constructing the optic disc segmentation network model and the blood vessel segmentation network model comprise:
1-1) firstly, labels containing optic disc pixel level labels and labels containing blood vessel pixel level labels need to be obtained, and eyeground images can be labeled by an ophthalmology eyeground specialist.
1-2) the preprocessing module preprocesses the fundus image; specifically, the preprocessing comprises uniformly scaling the size to 512 × 512 by non-deforming; and then all the images are subjected to pixel level normalization through mean value reduction and variance removal. Due to the fact that the segmentation of two focuses can be influenced by the uneven illumination of the fundus image, the robustness is higher by adopting the self-adaptive histogram equalization processing and gamma correction.
1-3) the optic disc segmentation network model and the blood vessel segmentation network model both adopt a blood vessel segmentation network model of a Refine-Unet structure, because the proportion of the optic disc and the blood vessel occupied on each image is very small, but the detection of the optic disc and the blood vessel is crucial to the classification of diabetic retina, the image-to-image segmentation can be realized by adopting the Refine-Unet structure, after the model is trained, an eyeground image is input, the corresponding optic disc or the blood vessel can be automatically segmented, and the output size is completely consistent with the input size; secondly, the segmentation of tiny objects can be realized, and the segmentation accuracy of the optic disc or the blood vessel is improved as much as possible. The model output is a probability map with each pixel between 0 and 1 indicating the probability that the pixel belongs to a disc or vessel.
Because the input is the preprocessed fundus color image, the input channel is 3, the output size is 512 multiplied by 512, and the times of the model keeping of down sampling and up sampling of the referred-Unet network are the same, so that the final output size is completely consistent with the input size. And in the process of cascading and downsampling the feature maps with the same size, the cascaded feature maps in the upsampling process are cascaded with the downsampling medium map after being subjected to convolution once, so that the better performance of feature fusion is ensured. In addition, the model output is multi-scale output, in the up-sampling branch, after each up-sampling, the output is performed through two times of convolution, multi-scale loss is calculated through different scale outputs and corresponding scale labels, the loss supervision effect is enhanced, and the model segmentation performance is improved.
1-4) dividing the preprocessed fundus image containing the optic disc pixel level annotation and the fundus image containing the blood vessel pixel level annotation into a training set, a verification set and a test set according to the proportion of 70%, 15% and 15% respectively.
When training the optic disc segmentation network model and the blood vessel segmentation network model:
the preprocessed 512 × 512 × 3 fundus images are input, and the optic disc segmentation network model and the blood vessel segmentation network model are trained respectively.
As shown in FIG. 2, the segmentation network training module trains the vessel and optic disk segmentation network on the training set separated from the vessel and optic disk segmentation data set
Specific details and parameters of training include:
A. using a multi-scale Cross-Entropy Loss function, each scale being referred to as Cross-enhancement Loss (Softmax Loss), the total Loss function is L ═ Σ Li(1≤i≤4)，Li＝Cross-Entropy Loss(Yi,Pi)；
B. Because GPU computing resources are limited, a sub-sampling mode is used for training the model;
C. the epoch of the training is set to 1000, the batch _ size is set to 2, the initial learning rate is set to 0.001, the attenuation coefficient is 0.995, and the training is stopped when the final learning rate drops to 0.0001.
D. Optimization training was performed using an Adam optimizer.
E. And L2 Weight Decay regularization is added in each layer of convolution operation, so that the overfitting problem caused by overlarge Weight parameters is prevented.
F. Because the amount of training data is not large, in order to improve the performance, data amplification processing is carried out during training, wherein the data amplification processing comprises random rotation, inversion, random Gaussian noise addition and histogram equalization.
G. After training two network models of the optic disc and the blood vessel, the weight is fixed.
Finally, the structures of the trained optic disc segmentation network model and the trained blood vessel segmentation network model are shown in table 1:
TABLE 1
(2) Yellow spot area positioning module
Yellow spot area orientation module includes:
the optic disc parameter calculating unit is used for carrying out ellipse fitting on the optic disc in the optic disc segmentation image to obtain the center and the radius of the optic disc, and the long axis and the short axis of the ellipse;
the macular region interest area calculation unit is used for determining the macular region interest area on the blood vessel segmentation image based on the optic disc radius and expanding the optic disc in the optic disc segmentation image;
and the quadrant dividing unit is used for dividing the macular region into four quadrants based on the long axis and the short axis of the ellipse. The left and right of the major axis are respectively nasal side and temporal side, and the left and right of the minor axis are respectively two groups of main vascular arches.
Further, the expansion multiple of the optic disc is 1.2 times, and a square with the optic disc as the center is made with the 16 times radius of the optic disc as the side length to serve as the interested area.
A method for determining a macular region of interest based on a disk segmentation image may be implemented.
Since the positions of the blood vessel, optic disc and fundus image are consistent, the determined region positions can be mutually converted on each image. For example, a macular region of interest obtained by optic disc segmentation image can be located on the blood vessel segmentation image. The fitted ellipse, major axis, minor axis, vessel, etc. are also similar.
The first main vessel arch calculation unit is used for performing Venn difference on the blood vessel segmentation image and the expanded optic disc segmentation image, performing 8 connected domain calculation, reserving an area with the pixel number of the connected domain not less than 30 to obtain a connected domain calculation graph, and finding the connected domain with the largest pixel number of the connected domain as a first main vessel arch; the preferred main vessel arch occupies one quadrant.
The blood vessel primary and secondary sequencing unit is used for taking the center of the optic disc as the center, taking the blood vessel widths at the optic disc radius positions of a plurality of preset multiples in the connected domain calculation graph, and carrying out linear combination by using corresponding first preset proportionality coefficients to obtain the average width of each blood vessel in the connected domain calculation graph; and the pixel number of the connected domain and the average width of the blood vessels are linearly combined based on a second preset proportionality coefficient, and the blood vessels are sorted according to the result.
The alternative main vessel arch calculation unit is used for determining the quadrant where the preferred main vessel arch is located; and the method is also used for sequentially judging whether each blood vessel and the preferred main vessel arch are in the same left and right quadrant and in the different up and down quadrant based on the primary and secondary ordering of the blood vessels, if so, obtaining the alternative main vessel arch, and otherwise, judging the next blood vessel.
Wherein, the width of the blood vessel is the distance from the outer edge of the blood vessel to the central line of the blood vessel; the preset multiples are 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9 and 2.0, and the corresponding first preset proportionality coefficients are 0.3, 0.1, 0.06 and 0.06; and the second preset proportionality coefficients corresponding to the pixel number of the connected domain and the average width of the blood vessel are 0.3 and 0.7.
The acquisition of the first-selected main vessel arch and the candidate main vessel arch, namely the temporotemporal subtopic vessel arch can be realized.
As shown in fig. 3, the macular region calculation unit is configured to make a straight line where the long axis is located based on the blood vessel segmentation image after the two main blood vessel arches are positioned, make a parallel line of the straight line where the long axis is located at a position 5 times the radius of the optic disc on one side of the two main blood vessel arches, and form a closed region with the two main blood vessel arches; and performing circle fitting on the closed area, wherein the center of the circle is used as the center of the macular area, and the circle fitting result is used for positioning the macular area.
The closed region generating unit is used for drawing a perpendicular line from the main vessel arch to the straight line on which the long axis is positioned to the closest point of the straight line on which the long axis is positioned to the straight line on which the long axis is positioned when the straight line on which the long axis is positioned, the parallel line of the straight line on which the long axis is positioned and the two main vessel arches cannot form a closed region; and then a closed area is formed by the straight line of the long axis, the parallel line and the perpendicular line of the straight line of the long axis and the two main vessel arches.
And finally, determining the positions of the corresponding blood vessel segmentation images based on the positions of the two main blood vessel arches in the region of interest of the macular region, and positioning the macular region by combining the long axis and the radius.
(3) Classification network module
As shown in fig. 4, the classification network module includes:
the attention network submodule is used for fusing the fundus color image, the corresponding blood vessel segmentation image and the macular area image to obtain a multi-channel image;
the main neural network submodule is used for extracting the characteristics of the multi-channel image by using the trained main neural network model and outputting four groups of probability values and one group of noise corresponding to four lesion types through dropout Monte Carlo for multiple times;
and the classification result output submodule is used for acquiring the mean value and the variance of four groups of probability values, taking the lesion type with the maximum probability mean value as a final classification result, taking the mean value of noise as accidental uncertainty and taking the sum of variance as model uncertainty.
The attention network sub-module is a three-stage attention module (3-stage attention module), each of which can be divided into two branches, the upper branch is called a main branch (trunk branch), and the upper branch is a basic residual error network (ResNet) structure. The lower branch is a soft mask branch (soft mask branch), and the main part included in the soft mask branch is the residual attention learning mechanism. The attention mechanism is composed of down sampling and up sampling, and residual unit.
The residual attention mechanism is:
Hi,c(x)＝(1+Mi,c(x))×Fi,c(x)
Hi,c(x) Is the output of the attention network submodule, Fi,c(x) Is a picture tensor feature of the previous layer, Mi,c(x) Is the attention parameter of the soft mask. This constitutes a residual attention module, which can input the picture features together with the features after attention enhancement into the next module. The F function can select different functions, and results of different attention domains can be obtained.
f1(xi,c) The method is characterized in that a direct sigmoid activation function of a picture feature tensor is the attention of a mixed domain;
f2(xi,c) The image feature tensor is directly subjected to global averaging pooling (global averaging), so that the attention of the channel domain is obtained (analogy to SEnet [5 ]])；
f3(xi,c) The method is an activation function for averaging the image feature tensor in the channel domain, and similarly ignores the information of the channel domain, thereby obtaining the attention of the spatial domain.
The main neural network model is a Bayes deep learning classification network model, and the weight and bias of the common deep convolution neural network are changed from fixed values to distribution. Training for bayesian deep learning networks, i.e. given a training set D { (X)1,Y1),…,(XN,YN) The posterior distribution p (W | X, Y) of the model weights is obtained by the bayes formula:
the prior distribution p (w) is set to a standard normal distribution. However, the marginal distribution p (Y | X) needs to be integrated over the whole W, the deep learning convolutional network has a large number of layers and weight parameters, the space formed by the weights is very complex, and the integration result is difficult to obtain, so that the true weight posterior probability distribution cannot be obtained.
Approximating the model posterior distribution by using variational inference, i.e. using a simple point distribution qθ(W) to approximate the distribution p (W | X, Y) of the posterior probability, in order to be able to make the approximate posterior distribution as close as possible to the true posterior distribution, the KL divergence between the two is calculated and optimized to be as small as possible. Minimizing KL divergence is equivalent to minimizing negative ELBO (negative evidence lower bound), i.e., minimizing ^ qθ(W)log p(Y|X,W)dW+KL[qθ(W)|p(W)]Finally, the optimal approximate posterior distribution is obtained
However, the variation method for approximating the posterior in the bayesian neural network greatly increases the number of parameters in calculation, and the posterior distribution of the optimized variation inference method is equivalent to the dropout regularization network model, so that the dropout regularization method is used as the bayesian approximation method to obtain the model posterior distribution.
After obtaining the approximate posterior distribution of the model, when testing and using, because the approximate posterior distribution is more complicated, the dropout Monte Carlo simulation means (for example, 50 forward transmissions) is adopted to carry out sampling acquisition
Distribution of classification results below. And obtaining the classification result and the model uncertainty by obtaining the mean value and the variance of the classification result.
3-1) similarly, acquiring fundus images marked with four lesion types; the fundus image may be annotated for AMD grading by an ophthalmologist.
3-2) before input into the network model, the same pre-processing module performs the same pre-processing on the fundus image for the same reason.
And 3-3) the optic disc segmentation module and the blood vessel segmentation module respectively use the trained optic disc segmentation network model and the trained blood vessel segmentation network model to segment the preprocessed fundus image labeled with the four lesion types to obtain an optic disc segmentation image and a blood vessel segmentation image.
3-4) identifying the macular region of each image from the optic disc segmentation image and the blood vessel segmentation image obtained in the above step through an macular region calculation module;
and 3-5) obtaining a classification result and uncertainty by using a Bayes deep learning classification network through information fusion and an attention mechanism on the RGB and macular region detection results.
3-6) in training the Bayesian deep learning classification network model:
the 512 × 512 macular region detection result image, the binarized blood vessel image, and the 512 × 512 × 3 fundus color photograph are fused into a 512 × 512 × 5 multi-channel image as input.
The following method is adopted in training the Bayes deep learning classification network model:
A. Cross-Entrophy Loss + accidental uncertainty (apparent uncertainty) + model uncertainty Loss (L ═ L) was usedCEL+Laleatoric+Lmodle) As a function of the loss.
B. Using a sub-sampling gradient descent algorithm as a learning algorithm of the convolutional neural network;
C. each convolutional neural network trains 300 epochs, and the training parameters and the optimizer are the same as those of the segmentation model;
D. diagnostic markers for diabetic retinal fundus data were rated from 0, 1, 2 and 3 for healthy, mild, moderate and severe grade, respectively. In the training process, data amplification is carried out on training data, so that the model performance is more robust.
E. During testing, through dropout Monte Carlo for multiple times, as shown in Table 2, prediction distribution, namely four groups of probability values corresponding to four lesion types, is obtained;
TABLE 2
During testing, the lesion type with the highest probability mean is a final classification result, the variance is model uncertainty, and the model noise node learns that the image noise mean is the accidental uncertainty of the image. And comparing the result obtained by comparing the test set with the average uncertainty of the training set by comprehensively considering the two uncertainties obtained from the model to obtain the credibility of the result, wherein the calculation method of the average uncertainty of the training set is to calculate the uncertainty of each image and then calculate the expectation.
The specific layer structure of the network structure of the finally trained Bayesian deep learning classification network model is shown in Table 3;
TABLE 3
(4) After the trained optic disc segmentation network model, the blood vessel segmentation network model and the Bayes deep learning classification network model are obtained, the system can be used for classification, and the process is as shown in FIG. 4:
s1, acquiring fundus images to be classified;
s2, preprocessing the fundus image;
s3, the vascular optic disc segmentation module respectively takes the preprocessed fundus images to be classified as the input of the trained optic disc segmentation network model and the trained vascular segmentation network model to respectively obtain optic disc segmentation images and vascular segmentation images;
s4, the macular region positioning module carries out macular region positioning based on the optic disc segmentation image and the blood vessel segmentation image to obtain an macular region image;
s5, the classification network module takes the fundus image to be classified, the corresponding optic disc segmentation image, the blood vessel segmentation image and the corresponding preprocessed macular region as input, the fundus image, the corresponding optic disc segmentation image, the blood vessel segmentation image and the corresponding preprocessed macular region are fused into a multi-channel image through the attention network sub-module, the trained Bayes deep learning classification network model carries out feature extraction on the multi-channel image, and four groups of probability values and one group of noise corresponding to four lesion types are output through multiple dropout Monte Carlo; and the classification result output submodule acquires the mean and the variance of the four groups of probability values, the lesion type with the maximum probability mean is used as a final classification result, the mean of noise is used as the accidental uncertainty, and the variance sum is used as the model uncertainty.
S6, when the model uncertainty or the accidental uncertainty is higher than the average uncertainty of the training set by 50%, judging that the prediction credibility of the image is low, and submitting the fundus image to an ophthalmologist for re-diagnosis; the average uncertainty of the training set is calculated by calculating the uncertainty of each image and then calculating the expectation.
In summary, compared with the prior art, the invention has the following beneficial effects:
firstly, obtaining a optic disc segmentation image and a blood vessel segmentation image by using two trained segmentation network models, calculating a macular region according to the obtained optic disc segmentation image and the blood vessel segmentation image, fusing a fundus color image, the corresponding blood vessel segmentation image and the macular region image by using an attention network submodule to obtain a multi-channel image, extracting features by using the trained Bayes deep learning classification network model, and outputting four groups of probability values and one group of noise corresponding to four lesion types through multiple dropout Monte Carlo; acquiring the mean value and the variance of the four groups of probability values, taking the lesion type with the maximum probability mean value as a final classification result, taking the mean value of noise as accidental uncertainty, and taking the variance sum as model uncertainty; and the classification network module gives the accidental uncertainty and the model uncertainty while finally outputting a model classification result. The safety performance of the model is ensured, when a diagnostic image can not give a result in a very certain way, whether the human expert needs to diagnose again or not can be determined through two uncertainties, and the model is safer and more reliable in clinical use.
Automatically segmenting retinal blood vessels and optic discs in the fundus image by adopting a convolutional neural network; calculating parameters of the optic disc and parameters of the blood vessels based on the segmentation result; based on the parameters of the blood vessels and optic disc, the main vascular arch structure of the retinal result is determined, from which the macular region position is derived. Automatic macular region positioning based on the retina structure is achieved, and time loss of manual labeling of each image is saved.
It should be noted that, through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform. With this understanding, the above technical solutions may be embodied in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments. In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. An AMD grading system based on macular attention mechanism and uncertainty, the system comprising:
the blood vessel optic disc segmentation module is used for segmenting the fundus color image to obtain a binarized optic disc segmentation image and a blood vessel segmentation image;
the macular region positioning module is used for obtaining an image of the macular region based on the optic disc segmentation image and the blood vessel segmentation image;
the classification network module is used for inputting the fundus color map, the corresponding optic disc segmentation image, the blood vessel segmentation image and the macular region image into a trained deep learning classification network model based on an attention mechanism, fusing the fundus color map, the corresponding blood vessel segmentation image and the macular region image by using an attention network to obtain a multi-channel image, performing feature extraction on the multi-channel image by using a main neural network, and outputting four groups of probability values and one group of noise corresponding to four lesion types by using a plurality of dropout Monte Carlo; and acquiring the mean value and the variance of the four groups of probability values, taking the lesion type with the maximum probability mean value as a final classification result, taking the mean value of noise as accidental uncertainty, and taking the variance sum as model uncertainty.
2. The AMD ranking system based on macular attention and uncertainty as recited in claim 1 wherein the classification network module comprises:
the attention network submodule is used for fusing the fundus color image, the corresponding blood vessel segmentation image and the macular area image to obtain a multi-channel image;
the main neural network submodule is used for extracting the characteristics of the multi-channel image by using the trained main neural network model and outputting four groups of probability values and one group of noise corresponding to four lesion types through dropout Monte Carlo for multiple times;
and the classification result output submodule is used for acquiring the mean value and the variance of four groups of probability values, taking the lesion type with the maximum probability mean value as a final classification result, taking the mean value of noise as accidental uncertainty and taking the sum of variance as model uncertainty.
3. The macular attention mechanism and uncertainty-based AMD classification system of claim 2 wherein the attention network submodule is a third order attention module and
the residual attention mechanism is:
Hi,c(x)＝(1+Mi,c(x))×Fi,c(x)
Hi,c(x) Is the output of the attention network submodule, Fi,c(x) Is a picture tensor feature of the previous layer, Mi,c(x) Is the attention parameter of the soft mask.
4. The macular attention mechanism and uncertainty-based AMD classification system of claim 2 wherein the trained master neural network model is a bayesian deep learning classification network model, and the number of output nodes is 5, including 4 classification nodes corresponding to lesion classification and one noise node corresponding to image noise;
the multichannel image is a 512 × 512 × 5 multichannel image formed by fusing a 512 × 512 macular region image, a binarized blood vessel segmentation image and a 512 × 512 × 3 fundus image;
and training the Bayesian deep learning classification network model comprises the following steps:
A. cross-control Loss + occasional uncertainty + model uncertainty Loss as a Loss function;
B. using a sub-sampling gradient descent algorithm as a learning algorithm of the convolutional neural network;
C. training 300 epochs in each convolutional neural network, setting the batch _ size to be 2, setting the initial learning rate to be 0.001, setting the attenuation coefficient to be 0.995, and stopping training when the final learning rate is reduced to 0.0001 and is not reduced any more, and performing optimization training by using an Adam optimizer;
D. diagnostic markers are grade 0, 1, 2 and 3, corresponding to healthy, mild, moderate and severe grades, respectively; in the training process, performing data amplification on training data;
E. during testing, through dropout Monte Carlo for multiple times, prediction distribution is obtained, namely, four groups of probability values corresponding to four lesion types.
5. The AMD classification system based on macular attention and uncertainty as recited in claim 1 wherein the macular region locating module comprises:
the optic disc parameter calculating unit is used for carrying out ellipse fitting on the optic disc in the optic disc segmentation image to obtain the center and the radius of the optic disc, and the long axis and the short axis of the ellipse;
the macular region interest area calculation unit is used for determining the macular region interest area on the blood vessel segmentation image based on the optic disc radius and expanding the optic disc in the optic disc segmentation image;
the quadrant dividing unit is used for dividing the macular region into four quadrants based on the long axis and the short axis of the ellipse;
the first main vessel arch calculation unit is used for performing Venn difference on the blood vessel segmentation image and the expanded optic disc segmentation image, performing 8 connected domain calculation, reserving an area with the pixel number of the connected domain not less than 30 to obtain a connected domain calculation graph, and finding the connected domain with the largest pixel number of the connected domain as a first main vessel arch;
the blood vessel primary and secondary sequencing unit is used for taking the center of the optic disc as the center, taking the blood vessel widths at the optic disc radius positions of a plurality of preset multiples in the connected domain calculation graph, and carrying out linear combination by using corresponding first preset proportionality coefficients to obtain the average width of each blood vessel in the connected domain calculation graph; the pixel number of the connected domain and the average width of the blood vessels are linearly combined based on a second preset proportionality coefficient, and the blood vessels are sorted according to the result;
the alternative main vessel arch calculation unit is used for determining the quadrant where the preferred main vessel arch is located; the device is also used for sequentially judging whether each blood vessel and the preferred main vessel arch are in the same left and right quadrant and in different up and down quadrants or not based on the main and secondary ordering of the blood vessels, if so, the alternative main vessel arch is obtained, and otherwise, the next blood vessel is judged;
the macular region calculation unit is used for making a straight line where the long axis is located based on the blood vessel segmentation image after the two main blood vessel arches are positioned, making a parallel line of the straight line where the long axis is located at a position which is 5 times of the radius of the optic disc on one side of the two main blood vessel arches, and forming a closed region with the two main blood vessel arches; performing circle fitting on the closed area, wherein the circle center is used as the center of the macular area, and the circle fitting result is used for positioning the macular area;
the closed region generating unit is used for drawing a perpendicular line from the main vessel arch to the straight line on which the long axis is positioned to the closest point of the straight line on which the long axis is positioned to the straight line on which the long axis is positioned when the straight line on which the long axis is positioned, the parallel line of the straight line on which the long axis is positioned and the two main vessel arches cannot form a closed region; and then a closed area is formed by the straight line of the long axis, the parallel line and the perpendicular line of the straight line of the long axis and the two main vessel arches.
6. The macular region localization system according to claim 5, wherein the expansion factor of the optic disc is 1.2 times, and a square with the optic disc as the center is made with a 16-times radius of the optic disc as a side length as the region of interest.
7. The macular region localization system based on a retinal structure according to claim 5, wherein the blood vessel width is a distance from an outer edge of the blood vessel to a centerline of the blood vessel; the preset multiples are 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9 and 2.0, and the corresponding first preset proportionality coefficients are 0.3, 0.1, 0.06 and 0.06; and the second preset proportionality coefficients corresponding to the pixel number of the connected domain and the average width of the blood vessel are 0.3 and 0.7.
8. The retinal structure-based macular region localization system of claim 3, wherein the macular region location calculation module comprises:
the macula lutea positioning unit is used for making a straight line where the long axis is located based on the blood vessel images after the two main blood vessel arches are positioned, making a parallel line of the straight line where the long axis is located at a position of 5 times of the radius of the optic disc on one side of the two main blood vessel arches, and forming a closed area with the two main blood vessel arches; and performing circle fitting on the closed area, wherein the center of the circle is used as the center of the macular area, and the circle fitting result is used for positioning the macular area.
9. The macular region localization system based on a retinal structure according to claim 5, wherein the macular region localization module further comprises:
the closed region generating unit is used for drawing a perpendicular line from the main vessel arch to the straight line on which the long axis is positioned to the closest point of the straight line on which the long axis is positioned to the straight line on which the long axis is positioned when the straight line on which the long axis is positioned, the parallel line of the straight line on which the long axis is positioned and the two main vessel arches cannot form a closed region; and then a closed area is formed by the straight line of the long axis, the parallel line and the perpendicular line of the straight line of the long axis and the two main vessel arches.
10. The AMD classification system based on macular attention and uncertainty as recited in claim 1 further comprising:
and the classification result credibility judging module is used for judging that the prediction credibility of the fundus oculi color map is low when the model uncertainty or the accidental uncertainty is higher than the average uncertainty of the training set by a threshold value, wherein the calculation method of the average uncertainty of the training set is used for calculating the uncertainty of each fundus oculi color map and then calculating the expectation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011453517.8A CN112446875A (en) | 2020-12-11 | 2020-12-11 | AMD grading system based on macular attention mechanism and uncertainty |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011453517.8A CN112446875A (en) | 2020-12-11 | 2020-12-11 | AMD grading system based on macular attention mechanism and uncertainty |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112446875A true CN112446875A (en) | 2021-03-05 |
Family
ID=74740518
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011453517.8A Pending CN112446875A (en) | 2020-12-11 | 2020-12-11 | AMD grading system based on macular attention mechanism and uncertainty |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112446875A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113768460A (en) * | 2021-09-10 | 2021-12-10 | 北京鹰瞳科技发展股份有限公司 | Fundus image analysis system and method and electronic equipment |
| JP2022186569A (en) * | 2021-06-03 | 2022-12-15 | 宏碁智醫股▲ふん▼有限公司 | Electronic device and method for training classification model for age-related macular degeneration |
| JP2022186568A (en) * | 2021-06-03 | 2022-12-15 | 宏碁智醫股▲ふん▼有限公司 | Electronic device and method for training classification model for age-related macular degeneration |
| CN116384448A (en) * | 2023-04-10 | 2023-07-04 | 中国人民解放军陆军军医大学 | CD severity grading system based on hybrid high-order asymmetric convolution network |
-
2020
- 2020-12-11 CN CN202011453517.8A patent/CN112446875A/en active Pending
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2022186569A (en) * | 2021-06-03 | 2022-12-15 | 宏碁智醫股▲ふん▼有限公司 | Electronic device and method for training classification model for age-related macular degeneration |
| JP2022186568A (en) * | 2021-06-03 | 2022-12-15 | 宏碁智醫股▲ふん▼有限公司 | Electronic device and method for training classification model for age-related macular degeneration |
| JP7229313B2 (en) | 2021-06-03 | 2023-02-27 | 宏碁智醫股▲ふん▼有限公司 | Electronic device and method for training a classification model for age-related macular degeneration |
| JP7247292B2 (en) | 2021-06-03 | 2023-03-28 | 宏碁智醫股▲ふん▼有限公司 | Electronic device and method for training a classification model for age-related macular degeneration |
| CN113768460A (en) * | 2021-09-10 | 2021-12-10 | 北京鹰瞳科技发展股份有限公司 | Fundus image analysis system and method and electronic equipment |
| CN113768460B (en) * | 2021-09-10 | 2023-11-14 | 北京鹰瞳科技发展股份有限公司 | Fundus image analysis system, fundus image analysis method and electronic equipment |
| CN116384448A (en) * | 2023-04-10 | 2023-07-04 | 中国人民解放军陆军军医大学 | CD severity grading system based on hybrid high-order asymmetric convolution network |
| CN116384448B (en) * | 2023-04-10 | 2023-09-12 | 中国人民解放军陆军军医大学 | CD severity grading system based on hybrid high-order asymmetric convolution network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112016626B (en) | Uncertainty-based diabetic retinopathy classification system | |
| CN112446875A (en) | AMD grading system based on macular attention mechanism and uncertainty | |
| CN112132817B (en) | Retina blood vessel segmentation method for fundus image based on mixed attention mechanism | |
| CN109635862B (en) | Sorting method for retinopathy of prematurity plus lesion | |
| CN110197493A (en) | Eye fundus image blood vessel segmentation method | |
| CN111667490B (en) | Fundus picture cup optic disc segmentation method | |
| CN113768461B (en) | Fundus image analysis method, fundus image analysis system and electronic equipment | |
| Devda et al. | Pathological myopia image analysis using deep learning | |
| CN112541923B (en) | Cup optic disk segmentation method based on fundus image data set migration learning | |
| CN113160226A (en) | Two-way guide network-based classification segmentation method and system for AMD lesion OCT image | |
| Wang et al. | Towards an extended EfficientNet-based U-Net framework for joint optic disc and cup segmentation in the fundus image | |
| CN112957005A (en) | Automatic identification and laser photocoagulation region recommendation algorithm for fundus contrast image non-perfusion region | |
| CN113610842A (en) | OCT image retina detachment and splitting automatic segmentation method based on CAS-Net | |
| CN113887662A (en) | Image classification method, device, equipment and medium based on residual error network | |
| Mai et al. | Few-shot transfer learning for hereditary retinal diseases recognition | |
| Bakır et al. | Using transfer learning technique as a feature extraction phase for diagnosis of cataract disease in the eye | |
| CN111968107B (en) | Uncertainty-based retinopathy plus lesion classification system for premature infants | |
| Phridviraj et al. | A bi-directional Long Short-Term Memory-based Diabetic Retinopathy detection model using retinal fundus images | |
| CN112581439A (en) | Macular region positioning system based on retina structure | |
| Ang et al. | Computer-based identification of diabetic maculopathy stages using fundus images | |
| Li et al. | Glaucoma detection: Joint segmentation and classification framework via deep ensemble network | |
| CN114998300A (en) | Corneal ulcer classification method based on multi-scale information fusion network | |
| Cheng et al. | Structure-preserving guided retinal image filtering for optic disc analysis | |
| Joseph et al. | Early detection of at-risk keratoplasties and prediction of future corneal graft rejection from pre-diagnosis endothelial cell images | |
| Nageswari et al. | Automatic Detection and Classification of Diabetic Retinopathy using Modified UNET |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |