CN116152229A - Method for constructing diabetic retinopathy diagnosis model and diagnosis model - Google Patents
Method for constructing diabetic retinopathy diagnosis model and diagnosis model Download PDFInfo
- Publication number
- CN116152229A CN116152229A CN202310397924.9A CN202310397924A CN116152229A CN 116152229 A CN116152229 A CN 116152229A CN 202310397924 A CN202310397924 A CN 202310397924A CN 116152229 A CN116152229 A CN 116152229A
- Authority
- CN
- China
- Prior art keywords
- loss function
- relative position
- diabetic retinopathy
- image
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30041—Eye; Retina; Ophthalmic
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method for constructing a diabetic retinopathy diagnosis model and the diagnosis model, and belongs to the technical field of deep learning. The construction method comprises the following steps: acquiring a fundus image and preprocessing to obtain an input image; inputting the input image into a backbone network for feature extraction to obtain a feature image containing advanced features; designing an improved classifier, inputting the feature map into the improved classifier, and obtaining a classification prediction result and a regression prediction result through calculation; designing a relative position loss function additional term, wherein the classification prediction result in the previous step is used for calculating a cross entropy loss function, and the regression prediction result is used for calculating the relative position loss function additional term; and combining the cross entropy loss function and the relative position loss function additional term to obtain a final joint loss function so as to construct a diagnosis model. The model constructed by the invention can learn the continuity information of the diabetic retinopathy process, and the model performance is obviously improved.
Description
Technical Field
The invention relates to the technical field of deep learning, in particular to a method for constructing a diabetic retinopathy diagnosis model and the diagnosis model.
Background
In recent years, diabetic retinopathy (Diabetic Retinopathy, abbreviated as DR) is one of the most common diabetic ophthalmic disease complications, and is a preventable disease. In clinic, the eye specialist with high experience is mainly relied on to observe and evaluate the color fundus images to determine the disease stage grade, wherein the pathology stage grade adopts DR international five stage standard to make an optimized diagnosis and treatment scheme. The deep learning can effectively extract pathological features in the image according to the marked data and obtain a better diagnosis result, but the actual situation can face a plurality of challenges.
The problem of DR staging has a significant feature in that the disease progression is continuous. However, the existing DR stage method based on deep learning generally regards the problem as a classification task with discrete categories, and ignores the continuity attribute of the lesion process. This corresponds to discarding a significant portion of the prior knowledge during modeling, which undoubtedly negatively affects the performance of the model. The main reason for this is that the model end classifier cannot directly model and learn the continuity information of the labels in the training data due to the structural limitation of the model end classifier.
Disclosure of Invention
Aiming at the problems, the invention aims to provide a method for constructing a diabetic retinopathy diagnosis model and the diagnosis model, and aims to improve the performance of the model by designing a brand-new classifier structure and a loss function so that the model can learn the continuity information of a DR pathological change process.
In order to solve the technical problems, the invention provides the following scheme:
in one aspect, a method for constructing a diabetic retinopathy diagnostic model is provided, comprising the steps of:
acquiring a fundus image and preprocessing to obtain an input image;
inputting the input image into a backbone network for feature extraction to obtain a feature image containing advanced features;
designing an improved classifier, inputting the feature map into the improved classifier, and obtaining a classification prediction result and a regression prediction result through calculation;
designing a relative position loss function additional term, wherein the classification prediction result in the previous step is used for calculating a cross entropy loss function, and the regression prediction result in the previous step is used for calculating the relative position loss function additional term;
and combining the cross entropy loss function and the relative position loss function additional term to obtain a final joint loss function so as to construct a diagnosis model.
Preferably, the step of preprocessing the fundus image specifically includes:
firstly, removing black areas around fundus images;
secondly, the sizes of fundus images are regulated uniformly, and the resolution of the regulated images is 256 multiplied by 256;
finally, fundus image enhancement is performed using the following formula to improve the brightness and contrast differences of fundus images:
in the formula (1),for the preprocessed fundus image, +.>Representing standard deviation +.>Gaussian convolution of>Representing pixel points in the fundus image; equation (2) is a weighted sum of pre-and post-enhancement fundus images, wherein the parameters used +.>, />, />,/>4, -4, 10 and 128, respectively.
Preferably, a ResNet50 network containing ImageNet pre-training parameters is selected as the backbone network for the encoder to construct the diagnostic model.
Preferably, the process of designing an improved classifier is as follows:
feature graphs obtained via backbone network are fed into full connection layer and vector representation with dimension of 6 is outputThe components areThe method comprises the steps of carrying out a first treatment on the surface of the Wherein->After Softmax the probability of i for the input sample label is obtained +.>,/>The method comprises the steps of carrying out a first treatment on the surface of the The Softmax calculation procedure is as follows:
classification results given to input samplesObtained by subscript of the largest element in P, the formula is:
here, theConsider as the regression prediction value of the input sample; />The greater the value representing the greater the extent of pathology of the sample; considering the value range of the input sample label of 0-4, for the convenience of calculation, the predicted value +.>Amplified to obtain:
The regression prediction value is regarded as the regression prediction value of the lesion degree of the input sample.
Preferably, the process of designing the relative position loss function additional term is as follows:
assume any two input samples of a training setAnd->Their label is->And->Regression prediction value of lesion degree is +.>And->Their tag distance and predicted value distance are +.>Andthe method comprises the steps of carrying out a first treatment on the surface of the To avoid interference of the symbol to the model training process, the absolute values of the two distances are taken when the distance difference is calculated:
likewise, consider cancellationThe influence of the sign of (2) on the model is verified by experiments, and finally selectedSelect pair->Taking the square as the relative position loss;
for the model training process, one batch containsIn the case of several input samples, the distance difference between any two input samples is calculated, and +.>Secondary times; then taking the average as a relative position loss function additional term for this batch +.>:
The final joint loss function consists of the cross entropy loss function and the relative position loss function additional term together, and is specifically shown as follows:
wherein the method comprises the steps ofFor a super parameter adjusting the ratio of the two loss functions, +.>Is a cross entropy loss function.
In another aspect, a diagnostic model of diabetic retinopathy is provided, comprising:
the preprocessing unit is used for preprocessing the acquired fundus image to obtain an input image;
the backbone network unit is used for extracting the characteristics of the input image to obtain a characteristic image containing advanced characteristics;
the improved classifier unit is used for calculating the feature map by utilizing the improved classifier to respectively obtain a classification prediction result and a regression prediction result;
and the loss function unit is used for calculating a cross entropy loss function by using the classification prediction result, calculating a relative position loss function additional term by using the regression prediction result, and combining the two to obtain a final joint loss function.
Preferably, the preprocessing unit is specifically configured to:
firstly, removing black areas around fundus images;
secondly, the sizes of fundus images are regulated uniformly, and the resolution of the regulated images is 256 multiplied by 256;
finally, fundus image enhancement is performed using the following formula to improve the brightness and contrast differences of fundus images:
in the formula (1),for the preprocessed fundus image, +.>Representing standard deviation +.>Gaussian convolution of>Representing pixel points in the fundus image; equation (2) is a weighted sum of pre-and post-enhancement fundus images, wherein the parameters used +.>, />, />,/>4, -4, 10 and 128, respectively.
Preferably, in the backbone network unit, a ResNet50 network containing ImageNet pre-training parameters is selected as a backbone network of the encoder forming the diagnostic model.
Preferably, in the improved classifier unit, the feature map obtained via the backbone network is fed into the fully connected layer, and a vector representation with dimension of 6 is outputThe component is->The method comprises the steps of carrying out a first treatment on the surface of the Wherein->After Softmax the probability of i for the input sample label is obtained +.>,/>The method comprises the steps of carrying out a first treatment on the surface of the The Softmax calculation procedure is as follows:
classification results given to input samplesObtained by subscript of the largest element in P, the formula is:
here, theConsider as the regression prediction value of the input sample; />The greater the value representing the greater the extent of pathology of the sample; considering the value range of the input sample label of 0-4, for the convenience of calculation, the predicted value +.>Amplified to obtain:
The regression prediction value is regarded as the regression prediction value of the lesion degree of the input sample.
Preferably, in the loss function unit, the relative position loss function additional term is designed as follows:
assume any two input samples of a training setAnd->They areThe label of (2) is->And->Regression prediction value of lesion degree is +.>And->Their tag distance and predicted value distance are +.>Andthe method comprises the steps of carrying out a first treatment on the surface of the To avoid interference of the symbol to the model training process, the absolute values of the two distances are taken when the distance difference is calculated:
likewise, consider cancellationThe influence of the sign of (2) on the model is verified by experiments, and the final selection pair is +>Taking the square as the relative position loss;
for the model training process, one batch containsIn the case of several input samples, the distance difference between any two input samples is calculated, and +.>Secondary times; then taking the average as a relative position loss function additional term for this batch +.>:
The final joint loss function consists of the cross entropy loss function and the relative position loss function additional term together, and is specifically shown as follows:
wherein the method comprises the steps ofFor a super parameter adjusting the ratio of the two loss functions, +.>Is a cross entropy loss function.
Compared with the prior art, the technical scheme provided by the invention has the following beneficial effects:
according to the method for constructing the diabetic retinopathy diagnosis model and the diagnosis model, provided by the invention, the model can learn the continuity information of the DR pathological change process by designing a brand-new classifier structure and a loss function, so that the performance of the model is obviously improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a general flow chart of a method of constructing a diagnostic model of diabetic retinopathy;
FIGS. 2 a-2 c are contrast plots showing fundus image preprocessing before and after preprocessing;
fig. 3 is a schematic diagram of the structure of a double-ended classifier.
While particular structures have been shown for purposes of clarity and understanding of the embodiments of the invention, it is not intended to limit the invention to the particular structures and environments disclosed, and modifications within the scope of the invention will occur to those skilled in the art.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. It will be apparent that the described embodiments are some, but not all, embodiments of the invention. All other embodiments, which can be made by a person skilled in the art without creative efforts, based on the described embodiments of the present invention fall within the protection scope of the present invention.
The embodiment of the invention firstly provides a method for constructing a diabetic retinopathy diagnosis model, and referring to fig. 1, the method comprises the following steps:
s1, acquiring a fundus image and preprocessing to obtain an input image;
s2, inputting the input image into a backbone network for feature extraction to obtain a feature image containing advanced features;
s3, designing an improved classifier, inputting the feature map into the improved classifier, and obtaining a classification prediction result and a regression prediction result through calculation;
s4, designing a relative position loss function additional item, wherein the classification prediction result in the previous step is used for calculating a cross entropy loss function, and the regression prediction result in the previous step is used for calculating the relative position loss function additional item;
s5, combining the cross entropy loss function and the relative position loss function additional term to obtain a final joint loss function so as to construct a diagnosis model.
In the present embodiment, this newly constructed diagnostic model is designated as RL-ResNet. The RL-res net is mainly composed of a backbone network and an improved double-end classifier, and after the preprocessed fundus image is input into the backbone network, features are extracted through the backbone network, and become a set of feature images containing advanced features, and the feature images are sent into the improved classifier. In the classifier, the feature map is calculated to obtain a classification prediction result and a regression prediction result, and the two results are used for calculating a cross entropy loss function and a relative position loss function addition term designed by the invention, wherein the cross entropy loss function and the relative position loss function addition term are used for modeling continuous information in the label. The two are combined to obtain a final joint loss function so as to realize the training of model parameters.
In particular, since fundus images have differences in illumination conditions and photographing apparatuses during sampling, there is a great difference in size and color between pictures. Meanwhile, some pictures have the problems of overexposure, more noise and the like. In order to train a network using samples of consistent color range, pre-processing and enhancement of the acquired fundus image is required.
In step S1, the step of preprocessing the fundus image specifically includes:
firstly, removing black areas around fundus images; black areas around the fundus image have a great negative influence on the classification result, and therefore need to be removed;
secondly, the sizes of fundus images are adjusted uniformly, and the resolution of the images after adjustment in the embodiment of the invention is 256×256;
finally, fundus image enhancement is performed using the following formula to improve the brightness and contrast differences of fundus images:
in the formula (1),for the preprocessed fundus image, +.>Representing standard deviation +.>Gaussian convolution of>Representing pixel points in the fundus image; equation (2) is a weighted sum of pre-and post-enhancement fundus images, wherein the parameters used +.>, />, />,/>4, -4, 10 and 128, respectively.
Fig. 2 a-2 c show the preprocessing results of three different fundus images. As can be seen from the figure, by the above pretreatment, the black areas around the fundus image become gray, reducing the influence on the classification result. The sharpness of blood vessels and lesion areas and the contrast with surrounding areas are significantly improved.
Further, in the embodiment of the invention, a ResNet50 network containing ImageNet pre-training parameters is selected as a backbone network of the diagnostic model formed by the encoder.
The ResNet50 network has proven to be very efficient in extracting image high-level semantic information, which is critical to medical image tasks. Therefore, the invention selects the ResNet50 network containing the ImageNet pre-training parameters as an encoder to form the backbone network for providing the diagnosis model, so that the backbone network has better initialization parameters and the convergence speed of the backbone network in the training process is accelerated.
The improved classifier designed in the invention is a double-end classifier, and the structure of the classifier is shown in figure 3.
Feature graphs obtained via backbone network are fed into full connection layer and vector representation with dimension of 6 is outputThe components areThe method comprises the steps of carrying out a first treatment on the surface of the Wherein->After Softmax the probability of i for the input sample label is obtained +.>,/>The method comprises the steps of carrying out a first treatment on the surface of the The Softmax calculation procedure is as follows:
classification results given to input samplesObtained by subscript of the largest element in P, the formula is:
existing deep learning-based DR staging methods typically use P-dependent cross entropy loss functions for network training, a process that does not require the use of。/>Calculation is only required when using a trained model. If want to model the tag continuity informationThe use of +.>Related information. But the gradient information of the corresponding term of the loss function cannot be transferred to the shallow layer of the model through the process shown in the formula (4). That is, if use +.>Modeling tag continuity information and constructing corresponding loss function additional items, wherein the part of information cannot influence the training process of the model, namely, the newly added additional items are invalid. The invention thus envisages +_ in figure 3>Corresponding structure. />
here, theI.e. can be regarded as regression predictions of the input samples; />The greater the value representing the greater the extent of pathology of the sample; for example->Corresponding to DR0 stage, and->Then the corresponding DR4 stage.
Taking into account the input samplesThe value range of the label is 0-4, and the predicted value is calculated for convenienceAmplified to get->:
The regression prediction value is regarded as the regression prediction value of the lesion degree of the input sample. For example->Indicating that the extent of lesions in the input samples is between DR2 and DR3, closer to DR2.
In the embodiment of the invention, the relative distance between regression predicted values of paired samples is used for modeling the lesion continuity, and a loss function additional term, namely a relative position loss function additional term (Relative Position Loss, rePLoss) is calculated, which is used for calculating the relative position loss functionAnd (3) representing.
The process of designing the relative position loss function additional term is as follows:
assume any two input samples of a training setAnd->Their label is->And->Regression prediction value of lesion degree is +.>And->Their tag distance and predicted value distance are +.>Andthe method comprises the steps of carrying out a first treatment on the surface of the To avoid interference of the symbol to the model training process, the absolute values of the two distances are taken when the distance difference is calculated:
likewise, consider cancellationThe influence of the sign of (2) on the model is verified by experiments, and the final selection pair is +>The relative position loss is obtained as a square.
For the model training process, a Batch (Batch) containsIn the case of several input samples, the distance difference between any two input samples is calculated, and +.>Secondary times; then taking the average as a relative position loss function additional term for this batch +.>:
The final joint loss function consists of the cross entropy loss function and the relative position loss function additional term together, and is specifically shown as follows:
wherein the method comprises the steps ofFor a super parameter adjusting the ratio of the two loss functions, +.>Is a cross entropy loss function.
The calculation procedure for a single sample is as follows:
the onehot representation, which is a sample tag, when the sample belongs to category i, i.e. the sample tag is i,otherwise->;/>Is classified as +.>Probability of individual categories. For a batch +.>Sample number->For each of them->Average value of (2):
aiming at the defects of the prior art, the invention designs a brand-new classifier structure and a loss function, so that the model can learn the DR pathological change process continuity information, thereby obviously improving the model performance.
Accordingly, embodiments of the present invention also provide a diagnostic model of diabetic retinopathy, the diagnostic model comprising:
the preprocessing unit is used for preprocessing the acquired fundus image to obtain an input image;
the backbone network unit is used for extracting the characteristics of the input image to obtain a characteristic image containing advanced characteristics;
the improved classifier unit is used for calculating the feature map by utilizing the improved classifier to respectively obtain a classification prediction result and a regression prediction result;
and the loss function unit is used for calculating a cross entropy loss function by using the classification prediction result, calculating a relative position loss function additional term by using the regression prediction result, and combining the two to obtain a final joint loss function.
Further, the preprocessing unit is specifically configured to:
firstly, removing black areas around fundus images;
secondly, the sizes of fundus images are regulated uniformly, and the resolution of the regulated images is 256 multiplied by 256;
finally, fundus image enhancement is performed using the following formula to improve the brightness and contrast differences of fundus images:
in the formula (1),for the preprocessed fundus image, +.>Representing standard deviation +.>Gaussian convolution of>Representing pixel points in the fundus image; equation (2) is a weighted sum of pre-and post-enhancement fundus images, wherein the parameters used +.>, />, />,/>4, -4, 10 and 128, respectively.
Further, in the backbone network unit, a ResNet50 network containing ImageNet pre-training parameters is selected as a backbone network of the diagnostic model formed by the encoder.
Further, in the improved classifier unit, the feature map obtained through the backbone network is sent to the full connection layer, and a vector representation with 6 dimensions is outputThe component is->The method comprises the steps of carrying out a first treatment on the surface of the Wherein->After Softmax the probability of i for the input sample label is obtained +.>,/>The method comprises the steps of carrying out a first treatment on the surface of the The Softmax calculation procedure is as follows: />
Classification results given to input samplesObtained by subscript of the largest element in P, the formula is:
here, theConsider as the regression prediction value of the input sample; />The greater the value representing the greater the extent of pathology of the sample; considering the value range of the input sample label of 0-4, for the convenience of calculation, the predicted value +.>Amplified to obtain:
The regression prediction value is regarded as the regression prediction value of the lesion degree of the input sample.
Further, in the loss function unit, the relative position loss function additional term is designed as follows:
assume any two input samples of a training setAnd->Their label is->And->Regression prediction value of lesion degree is +.>And->Their tag distance and predicted value distance are +.>Andthe method comprises the steps of carrying out a first treatment on the surface of the To avoid interference of the symbol to the model training process, the absolute values of the two distances are taken when the distance difference is calculated:
likewise, consider cancellationThe influence of the sign of (2) on the model is verified by experiments, and the final selection pair is +>The relative position loss is obtained as a square.
For the model training process, one batch containsIn the case of several input samples, the distance difference between any two input samples is calculated, and +.>Secondary times; then taking the average as a relative position loss function additional term for this batch +.>:
The final joint loss function consists of the cross entropy loss function and the relative position loss function additional term together, and is specifically shown as follows:
wherein the method comprises the steps ofFor a super parameter adjusting the ratio of the two loss functions, +.>Is a cross entropy loss function.
The calculation procedure for a single sample is as follows:
onehot, which is a sample tag, indicates that when the sample belongs to category i, i.e. the sample tag is i,/>Otherwise->;/>Is classified as +.>Probability of individual categories. For a batch +.>Sample number->For each of them->Average value of (2):
according to the diagnosis model provided by the invention, through designing a brand-new classifier structure and a loss function, the model can learn the DR pathological change process continuity information, so that the performance of the model is obviously improved.
Embodiments of the present invention also provide an electronic device that may vary greatly in configuration or performance, and may include one or more processors (central processing units, CPU) and one or more memories, where the memories store at least one instruction that is loaded and executed by the processors to implement the steps of the method for constructing a diabetic retinopathy diagnostic model described above.
In an exemplary embodiment, a computer readable storage medium, such as a memory comprising instructions executable by a processor in a terminal to perform the method of constructing a diabetic retinopathy diagnostic model described above, is also provided. For example, the computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or terminal device comprising the element.
References in the specification to "one embodiment," "an example embodiment," "some embodiments," etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the relevant art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.
The invention is intended to cover any alternatives, modifications, equivalents, and variations that fall within the spirit and scope of the invention. In the following description of preferred embodiments of the invention, specific details are set forth in order to provide a thorough understanding of the invention, and the invention will be fully understood to those skilled in the art without such details. In other instances, well-known methods, procedures, flows, components, circuits, and the like have not been described in detail so as not to unnecessarily obscure aspects of the present invention.
Those of ordinary skill in the art will appreciate that all or a portion of the steps in implementing the methods of the embodiments described above may be implemented by a program that instructs associated hardware, and the program may be stored on a computer readable storage medium, such as: ROM/RAM, magnetic disks, optical disks, etc.
The foregoing description of the preferred embodiments of the invention is not intended to limit the invention to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the invention are intended to be included within the scope of the invention.
Claims (10)
1. The method for constructing the diabetic retinopathy diagnosis model is characterized by comprising the following steps of:
acquiring a fundus image and preprocessing to obtain an input image;
inputting the input image into a backbone network for feature extraction to obtain a feature image containing advanced features;
designing an improved classifier, inputting the feature map into the improved classifier, and obtaining a classification prediction result and a regression prediction result through calculation;
designing a relative position loss function additional term, wherein the classification prediction result in the previous step is used for calculating a cross entropy loss function, and the regression prediction result in the previous step is used for calculating the relative position loss function additional term;
and combining the cross entropy loss function and the relative position loss function additional term to obtain a final joint loss function so as to construct a diagnosis model.
2. The method for constructing a diabetic retinopathy diagnostic model according to claim 1, wherein the step of preprocessing the fundus image specifically includes:
firstly, removing black areas around fundus images;
secondly, the sizes of fundus images are regulated uniformly, and the resolution of the regulated images is 256 multiplied by 256;
finally, fundus image enhancement is performed using the following formula to improve the brightness and contrast differences of fundus images:
in the formula (1),for the preprocessed fundus image, +.>Representing standard deviation +.>Is a gaussian convolution of (c) and,representing pixel points in the fundus image; equation (2) is a weighted sum of pre-and post-enhancement fundus images, wherein the parameters used +.>, />, />,/>4, -4, 10 and 128, respectively.
3. The method for constructing a diagnostic model of diabetic retinopathy according to claim 1, wherein a network of ResNet50 containing image Net pre-training parameters is selected as a backbone network of the diagnostic model.
4. The method of constructing a diagnostic model of diabetic retinopathy according to claim 1, wherein the process of designing an improved classifier is as follows:
feature graphs obtained via backbone network are fed into full connection layer and vector representation with dimension of 6 is outputThe components areThe method comprises the steps of carrying out a first treatment on the surface of the Wherein->After Softmax the probability of i for the input sample label is obtained +.>,/>The method comprises the steps of carrying out a first treatment on the surface of the The Softmax calculation procedure is as follows:
classification results given to input samplesObtained by subscript of the largest element in P, the formula is:
here, theConsider as the regression prediction value of the input sample; />The greater the value representing the greater the extent of pathology of the sample; considering the value range of the input sample label of 0-4, for the convenience of calculation, the predicted value +.>Amplified to get->:
5. The method for constructing a diabetic retinopathy diagnostic model according to claim 1, wherein the process of designing the relative position loss function additional term is as follows:
assume any two input samples of a training setAnd->Their label is->And->Regression prediction value of lesion degree is +.>And->Their tag distance and predicted value distance are +.>And->The method comprises the steps of carrying out a first treatment on the surface of the To avoid interference of the symbol to the model training process, the absolute values of the two distances are taken when the distance difference is calculated:
likewise, consider cancellationThe influence of the sign of (2) on the model is verified by experiments, and the final selection pair is +>Taking the square as the relative position loss;
for the model training process, one batch containsIn the case of several input samples, the distance difference between any two input samples is calculated, and +.>Secondary times; and then averaged as an additional term to the relative position loss function of the batch:
The final joint loss function consists of the cross entropy loss function and the relative position loss function additional term together, and is specifically shown as follows:
6. A diagnostic model of diabetic retinopathy, comprising:
the preprocessing unit is used for preprocessing the acquired fundus image to obtain an input image;
the backbone network unit is used for extracting the characteristics of the input image to obtain a characteristic image containing advanced characteristics;
the improved classifier unit is used for calculating the feature map by utilizing the improved classifier to respectively obtain a classification prediction result and a regression prediction result;
and the loss function unit is used for calculating a cross entropy loss function by using the classification prediction result, calculating a relative position loss function additional term by using the regression prediction result, and combining the two to obtain a final joint loss function.
7. The diabetic retinopathy diagnostic model according to claim 6, wherein the preprocessing unit is specifically configured to:
firstly, removing black areas around fundus images;
secondly, the sizes of fundus images are regulated uniformly, and the resolution of the regulated images is 256 multiplied by 256;
finally, fundus image enhancement is performed using the following formula to improve the brightness and contrast differences of fundus images:
in the formula (1),for the preprocessed fundus image, +.>Representing standard deviation +.>Is a gaussian convolution of (c) and,representing pixel points in the fundus image; equation (2) is a weighted sum of pre-and post-enhancement fundus images, wherein the parameters used +.>, />, />,/>4, -4, 10 and 128, respectively.
8. The diabetic retinopathy diagnostic model according to claim 6, wherein the backbone network unit selects a ResNet50 network containing image Net pre-training parameters as an encoder to construct a backbone network of the diagnostic model.
9. The diabetic retinopathy diagnostic model according to claim 6, wherein in the improved classifier unit, the feature map obtained via the backbone network is fed into the fully connected layer and a vector representation with dimension 6 is outputThe components areThe method comprises the steps of carrying out a first treatment on the surface of the Wherein->After Softmax the probability of i for the input sample label is obtained +.>,/>The method comprises the steps of carrying out a first treatment on the surface of the The Softmax calculation procedure is as follows:
for input sampleThe classification result givenObtained by subscript of the largest element in P, the formula is:
here, theConsider as the regression prediction value of the input sample; />The greater the value representing the greater the extent of pathology of the sample; considering the value range of the input sample label of 0-4, for the convenience of calculation, the predicted value +.>Amplified to get->:
10. The diabetic retinopathy diagnostic model according to claim 6, wherein the loss function unit designs a relative position loss function additional term as follows:
assume any two input samples of a training setAnd->Their label is->And->Regression prediction value of lesion degree is +.>And->Their tag distance and predicted value distance are +.>And->The method comprises the steps of carrying out a first treatment on the surface of the To avoid interference of the symbol to the model training process, the absolute values of the two distances are taken when the distance difference is calculated:
likewise, consider cancellationThe influence of the sign of (2) on the model is verified by experiments, and the final selection pair is +>Taking the square as the relative position loss;
for the model training process, one batch containsIn the case of several input samples, the distance difference between any two input samples is calculated, and +.>Secondary times; and then averaged as an additional term to the relative position loss function of the batch:
The final joint loss function consists of the cross entropy loss function and the relative position loss function additional term together, and is specifically shown as follows:
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310397924.9A CN116152229B (en) | 2023-04-14 | 2023-04-14 | Method for constructing diabetic retinopathy diagnosis model and diagnosis model |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310397924.9A CN116152229B (en) | 2023-04-14 | 2023-04-14 | Method for constructing diabetic retinopathy diagnosis model and diagnosis model |
Publications (2)
Publication Number | Publication Date |
---|---|
CN116152229A true CN116152229A (en) | 2023-05-23 |
CN116152229B CN116152229B (en) | 2023-07-11 |
Family
ID=86350944
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202310397924.9A Active CN116152229B (en) | 2023-04-14 | 2023-04-14 | Method for constructing diabetic retinopathy diagnosis model and diagnosis model |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN116152229B (en) |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108960257A (en) * | 2018-07-06 | 2018-12-07 | 东北大学 | A kind of diabetic retinopathy grade stage division based on deep learning |
US20190110753A1 (en) * | 2017-10-13 | 2019-04-18 | Ai Technologies Inc. | Deep learning-based diagnosis and referral of ophthalmic diseases and disorders |
CN113902743A (en) * | 2021-12-08 | 2022-01-07 | 武汉爱眼帮科技有限公司 | Method and device for identifying diabetic retinopathy based on cloud computing |
CN114271763A (en) * | 2021-12-20 | 2022-04-05 | 合肥中纳医学仪器有限公司 | Mask RCNN-based gastric cancer early identification method, system and device |
CN114387201A (en) * | 2021-04-08 | 2022-04-22 | 透彻影像科技(南京)有限公司 | Cytopathic image auxiliary diagnosis system based on deep learning and reinforcement learning |
CN114445620A (en) * | 2022-01-13 | 2022-05-06 | 国网江苏省电力有限公司苏州供电分公司 | Target segmentation method for improving Mask R-CNN |
CN114492574A (en) * | 2021-12-22 | 2022-05-13 | 中国矿业大学 | Pseudo label loss unsupervised countermeasure domain adaptive picture classification method based on Gaussian uniform mixing model |
CN114529819A (en) * | 2022-02-23 | 2022-05-24 | 合肥学院 | Household garbage image recognition method based on knowledge distillation learning |
WO2022134362A1 (en) * | 2020-12-24 | 2022-06-30 | 平安科技(深圳)有限公司 | Training method and apparatus for optic disc image classification model, device, and storage medium |
-
2023
- 2023-04-14 CN CN202310397924.9A patent/CN116152229B/en active Active
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190110753A1 (en) * | 2017-10-13 | 2019-04-18 | Ai Technologies Inc. | Deep learning-based diagnosis and referral of ophthalmic diseases and disorders |
CN108960257A (en) * | 2018-07-06 | 2018-12-07 | 东北大学 | A kind of diabetic retinopathy grade stage division based on deep learning |
WO2022134362A1 (en) * | 2020-12-24 | 2022-06-30 | 平安科技(深圳)有限公司 | Training method and apparatus for optic disc image classification model, device, and storage medium |
CN114387201A (en) * | 2021-04-08 | 2022-04-22 | 透彻影像科技(南京)有限公司 | Cytopathic image auxiliary diagnosis system based on deep learning and reinforcement learning |
CN113902743A (en) * | 2021-12-08 | 2022-01-07 | 武汉爱眼帮科技有限公司 | Method and device for identifying diabetic retinopathy based on cloud computing |
CN114271763A (en) * | 2021-12-20 | 2022-04-05 | 合肥中纳医学仪器有限公司 | Mask RCNN-based gastric cancer early identification method, system and device |
CN114492574A (en) * | 2021-12-22 | 2022-05-13 | 中国矿业大学 | Pseudo label loss unsupervised countermeasure domain adaptive picture classification method based on Gaussian uniform mixing model |
CN114445620A (en) * | 2022-01-13 | 2022-05-06 | 国网江苏省电力有限公司苏州供电分公司 | Target segmentation method for improving Mask R-CNN |
CN114529819A (en) * | 2022-02-23 | 2022-05-24 | 合肥学院 | Household garbage image recognition method based on knowledge distillation learning |
Non-Patent Citations (3)
Title |
---|
QIAN LIU ET AL.: "DCA-Net: Dual-branch contextual-aware network for auxiliary localization and segmentation of parathyroid glands", BIOMEDICAL SIGNAL PROCESSING AND CONTROL, pages 1 - 13 * |
欧阳继红 等: "糖尿病视网膜病变分期双分支混合注意力决策网络", 吉林大学学报(工学版), vol. 52, no. 3, pages 648 - 656 * |
荣晶施 等: "基于对称性理论的医学图像多阶段分类算法", 计算机学报, vol. 38, no. 9, pages 1809 - 1824 * |
Also Published As
Publication number | Publication date |
---|---|
CN116152229B (en) | 2023-07-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Wang et al. | A deep learning algorithm using CT images to screen for Corona Virus Disease (COVID-19) | |
WO2020151307A1 (en) | Automatic lesion recognition method and device, and computer-readable storage medium | |
CN111938569A (en) | Eye ground multi-disease classification detection method based on deep learning | |
CN110689025A (en) | Image recognition method, device and system, and endoscope image recognition method and device | |
EP4187484A1 (en) | Cbd-net-based medical endoscopic image denoising method | |
WO2021159811A1 (en) | Auxiliary diagnostic apparatus and method for glaucoma, and storage medium | |
CN113643261B (en) | Lung disease diagnosis method based on frequency attention network | |
CN114287878A (en) | Diabetic retinopathy focus image identification method based on attention model | |
Zhen et al. | Assessment of central serous chorioretinopathy depicted on color fundus photographs using deep learning | |
Panda et al. | Deep convolutional neural network-based patch classification for retinal nerve fiber layer defect detection in early glaucoma | |
Hussein et al. | Retinex theory for color image enhancement: A systematic review | |
CN115578783B (en) | Device and method for identifying eye diseases based on eye images and related products | |
CN113870270A (en) | Eyeground image cup and optic disc segmentation method under unified framework | |
CN110660048B (en) | Leather surface defect detection method based on shape characteristics | |
CN110473176B (en) | Image processing method and device, fundus image processing method and electronic equipment | |
Singh et al. | Optimized convolutional neural network for glaucoma detection with improved optic-cup segmentation | |
CN110827963A (en) | Semantic segmentation method for pathological image and electronic equipment | |
Al-Ameen | Contrast Enhancement of Digital Images Using an Improved Type-II Fuzzy Set-Based Algorithm. | |
CN116152229B (en) | Method for constructing diabetic retinopathy diagnosis model and diagnosis model | |
CN111931544B (en) | Living body detection method, living body detection device, computing equipment and computer storage medium | |
CN113140291A (en) | Image segmentation method and device, model training method and electronic equipment | |
Erwin et al. | The augmentation data of retina image for blood vessel segmentation using U-Net convolutional neural network method | |
Anjugam et al. | Study of Deep Learning Approaches for Diagnosing Covid-19 Disease using Chest CT Images | |
Listyalina et al. | Joint dice and intersection over union losses for deep optical disc segmentation | |
Valério et al. | Lesions multiclass classification in endoscopic capsule frames |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |