CN113780478B - Activity classification model training method, classification method, device, equipment and medium - Google Patents
Activity classification model training method, classification method, device, equipment and medium Download PDFInfo
- Publication number
- CN113780478B CN113780478B CN202111248118.2A CN202111248118A CN113780478B CN 113780478 B CN113780478 B CN 113780478B CN 202111248118 A CN202111248118 A CN 202111248118A CN 113780478 B CN113780478 B CN 113780478B
- Authority
- CN
- China
- Prior art keywords
- choroidal neovascularization
- image
- loss value
- neural network
- sample data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000000694 effects Effects 0.000 title claims abstract description 161
- 238000000034 method Methods 0.000 title claims abstract description 90
- 238000013145 classification model Methods 0.000 title claims abstract description 84
- 238000012549 training Methods 0.000 title claims abstract description 65
- 208000005590 Choroidal Neovascularization Diseases 0.000 claims abstract description 139
- 206010060823 Choroidal neovascularisation Diseases 0.000 claims abstract description 139
- 238000012545 processing Methods 0.000 claims abstract description 57
- 238000013528 artificial neural network Methods 0.000 claims abstract description 48
- 238000003062 neural network model Methods 0.000 claims abstract description 40
- 238000011282 treatment Methods 0.000 claims abstract description 25
- 238000005728 strengthening Methods 0.000 claims abstract description 14
- 238000011176 pooling Methods 0.000 claims description 23
- 230000009467 reduction Effects 0.000 claims description 18
- 239000013598 vector Substances 0.000 claims description 12
- 238000000605 extraction Methods 0.000 claims description 10
- 238000010606 normalization Methods 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000012935 Averaging Methods 0.000 claims description 2
- 238000013473 artificial intelligence Methods 0.000 abstract description 16
- 238000005516 engineering process Methods 0.000 description 16
- 238000004891 communication Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 11
- 239000000243 solution Substances 0.000 description 11
- 239000002872 contrast media Substances 0.000 description 9
- 230000006870 function Effects 0.000 description 9
- 238000003058 natural language processing Methods 0.000 description 8
- 238000012014 optical coherence tomography Methods 0.000 description 8
- 238000012797 qualification Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 210000001519 tissue Anatomy 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000002347 injection Methods 0.000 description 2
- 239000007924 injection Substances 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 210000001525 retina Anatomy 0.000 description 2
- 210000003583 retinal pigment epithelium Anatomy 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 206010002198 Anaphylactic reaction Diseases 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000036783 anaphylactic response Effects 0.000 description 1
- 208000003455 anaphylaxis Diseases 0.000 description 1
- 230000003416 augmentation Effects 0.000 description 1
- 238000001574 biopsy Methods 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000003759 clinical diagnosis Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000001497 fibrovascular Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003907 kidney function Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000002792 vascular Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The embodiment relates to the technical field of artificial intelligence, in particular to an activity classification model training method, an activity classification model training device and an activity classification model training medium. The activity classification model training method comprises the following steps: acquiring an image sample dataset of choroidal neovascularization; inputting each image sample data into a preset neural network model; extracting the characteristics of each image sample data through a plurality of network layers of the first neural network to obtain a characteristic image; performing choroidal neovascularization activity classification processing on the characteristic image through a second neural network to obtain a first loss value; performing attention strengthening treatment on the characteristic image through a preset attention strengthening model to obtain a second loss value; and training the neural network model according to the first loss value and the second loss value to obtain an activity classification model of the choroidal neovascularization. The scheme realizes automatic classification of the activity of the choroidal neovascularization and improves the efficiency of recognizing the activity of the choroidal neovascularization.
Description
Technical Field
The application relates to the technical field of artificial intelligence, in particular to an activity classification model training method, an activity classification device, an activity classification equipment and an activity classification medium.
Background
Choroidal neovascularization (Choroidal Neovascularization, CNV) is a fibrous vascular tissue that can be caused by a variety of etiologies to proliferate under or above the retinal pigment epithelium. For inactive CNV only follow-up observations or conservative treatments are required, but for active CNV, immediate injection treatment is required because the leakage caused by it can seriously affect vision.
In the related art, judgment of CNV activity can be achieved by injecting a contrast agent onto the retina, however, there is a side effect of the contrast agent, which increases the risk of human illness.
Disclosure of Invention
The main purpose of the disclosed embodiments is to provide an activity classification model training method, a classification method, a device, equipment and a medium, which can conveniently establish an activity classification model for classifying the activity of the choroidal neovascularization, thereby realizing automatic classification of the activity of the choroidal neovascularization and improving the efficiency of recognizing the activity of the choroidal neovascularization.
To achieve the above object, a first aspect of the embodiments of the present disclosure proposes an activity classification model training method for choroidal neovascularization, comprising:
acquiring an image sample dataset of choroidal neovascularization; wherein the image sample data set comprises a plurality of image sample data;
Inputting each image sample data of the image sample data set into a preset neural network model; wherein the neural network model comprises a first neural network and a second neural network;
Extracting the characteristics of each image sample data through a plurality of network layers of the first neural network to obtain a characteristic image;
Performing choroidal neovascularization activity classification processing on the characteristic images through a second neural network to obtain a first loss value of each image sample data; wherein the first loss value is used to characterize the activity of choroidal neovascularization;
Performing attention strengthening treatment on the characteristic images through a preset attention strengthening model to obtain a second loss value of each image sample data; wherein the second loss value is used to characterize the activity of choroidal neovascularization;
training the neural network model according to the first loss value and the second loss value to obtain an activity classification model of the choroidal neovascularization; wherein the activity classification model is used to classify choroidal neovascularization activity.
In some implementations, the number of network layers includes: a dimension-reducing convolution layer and at least one moving flip bottleneck convolution layer;
Feature extraction is performed on each image sample data through a plurality of network layers of the first neural network to obtain a feature image, including:
Performing dimension reduction convolution processing on each image sample data through a dimension reduction convolution layer to obtain a dimension reduction convolution image;
and carrying out mobile overturning bottleneck convolution processing on the dimension reduction convolution image through the mobile overturning bottleneck convolution layer to obtain a characteristic image.
In some embodiments, the second neural network comprises: a global average pooling layer, a full connection layer and a normalized classification layer;
Performing choroidal neovascularization activity classification processing on the characteristic image through a second neural network to obtain a first loss value of each image sample data, wherein the first loss value comprises:
carrying out pooling treatment on the characteristic images through a global average pooling layer to obtain pooling vectors;
carrying out full connection processing on the pooling vector through a full connection layer to obtain a preliminary classification value;
And carrying out normalization processing on the preliminary classification value through a normalization classification layer to obtain a first loss value.
In some embodiments, an actual frame is arranged in the image sample data, and the actual frame is used for marking the actual position of the leakage area of the image sample data;
Performing attention enhancement processing on the feature image through a preset attention enhancement model to obtain a second loss value of each image sample data, wherein the second loss value comprises:
upsampling the feature image to obtain a predicted frame; the prediction frame is used for marking the position of the predicted seepage region of the image sample data;
And carrying out loss calculation on the predicted frame and the actual frame to obtain a second loss value.
In some embodiments, training the neural network model based on the first loss value and the second loss value to obtain an activity classification model of choroidal neovascularization comprises:
Weighting the first loss value and the second loss value according to a preset weight to obtain a target loss value;
and training the neural network model according to the target loss value to obtain an activity classification model of the choroidal neovascularization.
To achieve the above object, a second aspect of the embodiments of the present disclosure proposes an image classification method, including:
acquiring an image to be detected;
Inputting the image to be detected into an activity classification model for classification treatment to obtain the activity type of the choroidal neovascularization; wherein the activity classification model is trained according to the method of any one of the embodiments of the first aspect.
To achieve the above object, a third aspect of the embodiments of the present disclosure proposes an activity classification model training apparatus for choroidal neovascularization, comprising:
A sample acquisition module for acquiring an image sample dataset of choroidal neovascularization; wherein the image sample data set comprises a plurality of image sample data;
The input module is used for inputting each image sample data of the image sample data set into a preset neural network model; wherein the neural network model comprises a first neural network and a second neural network;
the feature extraction module is used for extracting features of each image sample data through a plurality of network layers of the first neural network to obtain a feature image;
The classification processing module is used for performing choroidal neovascularization activity classification processing on the characteristic images through a second neural network to obtain a first loss value of each image sample data; wherein the first loss value is used to characterize the activity of choroidal neovascularization;
The strengthening processing module is used for carrying out attention strengthening processing on the characteristic images through a preset attention strengthening model to obtain a second loss value of each image sample data; wherein the second loss value is used to characterize the activity of choroidal neovascularization;
the training processing module is used for training the neural network model according to the first loss value and the second loss value to obtain an activity classification model of the choroidal neovascularization; wherein the activity classification model is used to classify choroidal neovascularization activity.
To achieve the above object, a fourth aspect of the embodiments of the present disclosure proposes an image classification apparatus, including:
the image acquisition module is used for acquiring an image to be detected;
The image classification module is used for inputting the image to be detected into the activity classification model for classification processing to obtain the activity category of the choroidal neovascularization; wherein the activity classification model is trained according to the method of any one of the embodiments of the first aspect.
To achieve the above object, a fifth aspect of the embodiments of the present disclosure proposes an electronic device, including:
At least one memory;
At least one processor;
at least one program;
the program is stored in the memory, and the processor executes at least one program to implement:
the method of any one of the embodiments of the first aspect; or alternatively
The method as in the embodiments of the second aspect.
To achieve the above object, a sixth aspect of the embodiments of the present disclosure proposes a storage medium that is a computer-readable storage medium storing computer-executable instructions for causing a computer to execute:
the method of any one of the embodiments of the first aspect; or alternatively
The method as in the embodiments of the second aspect.
The method, the device, the equipment and the medium for training the activity classification model provided by the embodiment of the disclosure are used for inputting each image sample data into a first neural network, extracting the characteristics of each image sample data through a plurality of network layers of the first neural network to obtain a corresponding characteristic image, and then classifying the choroidal neovascularization activity of the characteristic image according to a second neural network to obtain a first loss value representing the activity of the choroidal neovascularization; and, leading in attention branches of the leakage area, adopting a preset attention enhancement model to carry out attention enhancement treatment on the characteristic image, and obtaining a second loss value representing the activity of the choroidal neovascularization; and training the neural network model according to the first loss value and the second loss value to obtain an activity classification model of the choroidal neovascularization. According to the choroidal neovascularization activity classification model obtained by the scheme, the choroidal neovascularization activity is automatically classified, the side effect of using a contrast agent is avoided, the cost for judging the choroidal neovascularization activity is reduced, the qualification requirement for judging the choriocapillaris activity by a doctor is reduced, and the attention enhancement treatment is carried out on the characteristic image by using the attention enhancement model due to the addition of the attention branches of the seepage region, so that the choroidal neovascularization activity classification model can better judge the choriocapillaris activity.
Drawings
FIG. 1 is a flow chart of an activity classification model training method for choroidal neovascularization provided by an embodiment of the present application;
Fig. 2 is a flowchart of a specific method of step S103 in fig. 1;
FIG. 3 is a flowchart of a specific method of step S104 in FIG. 1;
fig. 4 is a flowchart of a specific method of step S105 in fig. 1;
FIG. 5 is a flowchart of a specific method of step S106 in FIG. 1;
FIG. 6 is a schematic diagram of a specific application scenario of an activity classification model training method provided by an embodiment of the present application;
FIG. 7 is a flow chart of an image classification method provided by an embodiment of the present application;
FIG. 8 is a block diagram of an activity classification model training apparatus for choroidal neovascularization provided by an embodiment of the present application;
fig. 9 is a schematic hardware structure of an electronic device according to an embodiment of the disclosure.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
It should be noted that although functional block division is performed in a device diagram and a logic sequence is shown in a flowchart, in some cases, the steps shown or described may be performed in a different order than the block division in the device, or in the flowchart. The terms first, second and the like in the description and in the claims and in the above-described figures, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the application only and is not intended to be limiting of the application.
First, several nouns involved in the present application are parsed:
Artificial intelligence (ARTIFICIAL INTELLIGENCE, AI): is a new technical science for researching and developing theories, methods, technologies and application systems for simulating, extending and expanding the intelligence of people; artificial intelligence is a branch of computer science that attempts to understand the nature of intelligence and to produce a new intelligent machine that can react in a manner similar to human intelligence, research in this field including robotics, language recognition, image recognition, natural language processing, and expert systems. Artificial intelligence can simulate the information process of consciousness and thinking of people. Artificial intelligence is also a theory, method, technique, and application system that utilizes a digital computer or digital computer-controlled machine to simulate, extend, and expand human intelligence, sense the environment, acquire knowledge, and use knowledge to obtain optimal results.
Natural language processing (natural language processing, NLP): NLP is a branch of artificial intelligence that is a interdisciplinary of computer science and linguistics, and is often referred to as computational linguistics, and is processed, understood, and applied to human languages (e.g., chinese, english, etc.). Natural language processing includes parsing, semantic analysis, chapter understanding, and the like. Natural language processing is commonly used in the technical fields of machine translation, handwriting and print character recognition, voice recognition and text-to-speech conversion, information retrieval, information extraction and filtering, text classification and clustering, public opinion analysis and viewpoint mining, and the like, and relates to data mining, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research, linguistic research related to language calculation, and the like.
Medical cloud (Medical closed): the medical cloud is based on new technologies such as cloud computing, mobile technology, multimedia, 4G communication, big data, internet of things and the like, and a medical health service cloud platform is created by combining the medical technology and using 'cloud computing', so that medical resource sharing and medical range expansion are realized. Because the cloud computing technology is applied to combination, the medical cloud improves the efficiency of medical institutions, and residents can conveniently seek medical advice. Like reservation registration, electronic medical records, medical insurance and the like of the traditional hospital are products of combination of cloud computing and medical field, and the medical cloud also has the advantages of data security, information sharing, dynamic expansion and overall layout.
Optical coherence tomography (Optical CoherenceTomography, OCT): OCT is a new type of tomographic imaging technology that has been developed faster in recent years, and in particular, has an attractive application prospect in biological tissue biopsy and imaging, and has been tried to be applied to clinical diagnosis of ophthalmology, dentistry and dermatology, and has been a breakthrough in the following technique of X-CT and MRI, and has been developed rapidly in recent years.
EfficientNetB0 network model: EFFICIENTNETS the network model structure is proposed in paper EFFICIENTNET: RETHINKING MODEL SCALING FOR CONVOLUTIONAL NEURAL NETWORKS by engineers of *** brain, tanming star and chief scientist Quoc v.le. The underlying network architecture of the model is designed using neural network architecture search (neural architecture search). The EfficientNetB network model is the basic network model in the EFFICIENTNETS series, which model takes into account both the accuracy and the speed of the predictions. EfficientNetB0 a network model includes a plurality of repeated mobile rollover bottleneck convolution (mobile inverted bottleneck convolution, MBCONV) modules, a convolution layer (CONV), a batch normalization layer (Batch Normalization, BN), swish activation functions, a global average pooling layer (global average pooling, GAP), a full connection layer (full connection layer, FC), a Softmax classification layer, and the like. Among them, the mobile rollover bottleneck convolution (mobile inverted bottleneck convolution, MBCONV) module is the core idea of the EfficientNetB0 network model.
Softmax function: the Softmax function is a normalized exponential function that can "compress" a K-dimensional vector z containing arbitrary real numbers into another K-dimensional real vector σ (z) such that each element ranges between (0, 1) and the sum of all elements is 1, which is commonly used in multi-classification problems.
GIoU loss: the joint loss generalized intersection (Generalized Intersection over Union loss, GIoU loss) may be a function of the loss. GIoU is a method for calculating loss of frame prediction from IoU, and in the fields of target detection and the like, the predicted frame and the actual marked frame need to be compared to calculate loss.
Convex set (convex set): in convex geometry, a convex set is a subset of affine space that is closed under a convex combination. More specifically, in Euclidean space, a convex set is for each point in a set, each point on the straight line segment connecting that point is also in the set. For example, the cube is a convex set, but any hollow or dimpled, e.g., crescent shape, is not a convex set.
Union: given two sets A and B, all elements of the two sets A and B are combined to form a set called the union of the set A and the set B, and the union is named as A U B and is named as A and B.
Intersection of: let A, B be two sets, set composed of all elements belonging to set A and to set B, called intersection of set A and set B, denoted A.U.B, read A.C.B.
The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Wherein artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) is the theory, method, technique, and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend, and expand human intelligence, sense the environment, acquire knowledge, and use knowledge to obtain optimal results.
The training method and the image classification method for the activity classification model of the choroidal neovascularization provided by the embodiment of the application can be applied to artificial intelligence. Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.
Choroidal neovascularization (Choroidal Neovascularization, CNV): CNV is a fibrovascular tissue that can be caused by a variety of causes to proliferate under or on the retinal pigment epithelium. For inactive CNV only follow-up observations or conservative treatments are required, but for active CNV, immediate injection treatment is required because the leakage caused by it can seriously affect vision. Currently, in clinical medicine, when judging the activity of the choroidal neovascularization, a contrast agent is injected onto the retina to obtain a corresponding contrast image, and then a doctor can judge the activity of the choroidal neovascularization by referring to the contrast image. However, the contrast agent has side effects, which easily cause anaphylaxis or damage of renal function in human body, and increase the risk of human body diseases; meanwhile, the judging mode is low in efficiency and has a large qualification requirement on doctors.
Based on the above, the embodiment of the application provides an activity classification model training method, a classification method, a device, equipment and a medium, so as to realize automatic classification of the activity of the choroidal neovascularization, avoid side effects caused by using a contrast agent, reduce the cost of judging the activity of the choroidal neovascularization, reduce the qualification requirement of a doctor on judging the activity of the choroidal neovascularization, and accelerate the efficiency of identifying the activity of the choroidal neovascularization.
The method, the device and the equipment for training the activity classification model provided by the embodiment of the disclosure are specifically described through the following embodiments, and the method for training the activity classification model in the embodiment of the disclosure is described first.
The embodiment of the application provides an activity classification model training method, and relates to the technical field of artificial intelligence. The activity classification model training method provided by the embodiment of the application can be applied to a terminal, a server and software running in the terminal or the server. In some embodiments, the terminal may be a smart phone, tablet, notebook, desktop, or smart watch, etc.; the server side can be configured as an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, and a cloud server for providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), basic cloud computing services such as big data and artificial intelligent platforms and the like; the software may be an application or the like that implements the activity classification model training method, but is not limited to the above form.
Embodiments of the present disclosure are operational with numerous general purpose or special purpose computer system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
Referring to fig. 1, a detailed description will be given of a specific process of the training method for the activity classification model of choroidal neovascularization according to the present application with reference to fig. 1.
As shown in fig. 1, in a first aspect, some embodiments of the present application provide an activity classification model training method for choroidal neovascularization, comprising steps S101, S102, S103, S104, S105, and S106. These six steps are described in detail below, and it should be understood that the activity classification model training method for choroidal neovascularization includes, but is not limited to, these six steps.
Step S101: acquiring an image sample dataset of choroidal neovascularization; wherein the image sample data set comprises a number of image sample data.
Specifically, in step S101, the image sample data of the choroidal neovascularization may be obtained by the OCT scanning device as an OCT scanning image of the macular region of the examiner, or may be a choroidal neovascularization image downloaded directly through a medical cloud server, to which the present application is not particularly limited.
It should be noted that, whether the image sample data is an OCT scan image obtained by the OCT scanning device or a choroidal neovascularization image downloaded by the medical cloud server. The image sample data sets should have been classified, and each image sample data set needs to be labeled. I.e. the image sample dataset is divided into two parts, active CNV and inactive CNV, and for active CNV it is necessary to label the location of its leakage area with a rectangular label box. The inactive CNV may be denoted by the numeral "0", the active CNV may be denoted by the numeral "1", or the active CNV may be distinguished from the inactive CNV by other labeling means, and the present application is not particularly limited thereto.
To facilitate the model training process in the subsequent steps, each image sample data in the image sample dataset is typically preprocessed prior to model training to convert each image sample data into a standard training image of the same resolution in both width and height. The specific operation is as follows:
And performing edge filling processing on each image sample data by using a certain standard pixel, and then scaling to a certain preset resolution by using a bilinear difference algorithm to obtain a standard training image. For example: pixel "0" is filled in the edge position of each image sample data, so that the resolution of each image sample data is 768×768, and then the image sample data is scaled to a resolution of 512×512 by a linear difference algorithm. But may also be scaled to other resolutions, such as 256 x 256, and are not described in detail herein due to their similarity in operation.
Step S102: inputting each image sample data of the image sample data set into a preset neural network model; wherein the neural network model comprises: a first neural network and a second neural network.
In step S102, the preset neural network model may be efficientNetB a 0 neural network model, and may be other neural network models, which is not particularly limited in this application, and in the embodiment of the present application, a efficientNetB a neural network model is taken as an example, and the training method of the activity classification model for choroidal neovascularization in the embodiment of the present application is described in detail.
Step S103: and extracting the characteristics of each image sample data through a plurality of network layers of the first neural network to obtain a characteristic image.
Step S104: performing choroidal neovascularization activity classification processing on the characteristic images through a second neural network to obtain a first loss value of each image sample data; wherein the first loss value is used to characterize the activity of choroidal neovascularization.
Step S105: performing attention strengthening treatment on the characteristic images through a preset attention strengthening model to obtain a second loss value of each image sample data; wherein the second loss value is used to characterize the activity of choroidal neovascularization.
Step S106: training the neural network model according to the first loss value and the second loss value to obtain an activity classification model of the choroidal neovascularization; wherein the activity classification model is used to classify choroidal neovascularization activity.
According to the training method for the activity classification model of the choroidal neovascularization, each image sample data is input into a first neural network, feature extraction is carried out on each image sample data through a plurality of network layers of the first neural network to obtain a corresponding feature image, and then the choroidal neovascularization activity classification processing is carried out on the feature image according to a second neural network to obtain a first loss value representing the choroidal neovascularization activity; and, leading in attention branches of the leakage area, adopting a preset attention enhancement model to carry out attention enhancement treatment on the characteristic image, and obtaining a second loss value representing the activity of the choroidal neovascularization; and training the neural network model according to the first loss value and the second loss value to obtain an activity classification model of the choroidal neovascularization. According to the activity classification model of the choroidal neovascularization, which is obtained by the scheme, an activity classification model for classifying the activity of the choroidal neovascularization is established, the automatic classification of the activity of the choroidal neovascularization is realized, the side effect of using a contrast agent is avoided, the cost for judging the activity of the choroidal neovascularization is reduced, the qualification requirement of a doctor for judging the activity of the choroidal neovascularization is reduced, and because a leakage area attention branch is added, the attention enhancement treatment is carried out on a characteristic image by using an attention enhancement model, so that the activity classification model for the choroidal neovascularization, which is obtained by training, can better judge the activity of the choroidal neovascularization.
Referring to fig. 2, in some embodiments of the present application, the plurality of network layers includes a reduced dimension convolution layer and at least one mobile rollover bottleneck convolution layer, and step S103 includes, but is not limited to, step S201 and step S202. These two steps are described in detail below in conjunction with fig. 2.
Step S201: and performing dimension reduction convolution processing on each image sample data through the dimension reduction convolution layer to obtain a dimension reduction convolution image.
In step S201, after the image sample data is input into the first neural network, the dimension-reducing convolution processing is performed on the dimension-reducing convolution layer image sample data, so as to obtain a dimension-reducing convolution image. By performing dimension reduction convolution processing on the image sample data, irrelevant information can be reduced, and the subsequent training process is more focused on useful information.
Step S202: and carrying out mobile overturning bottleneck convolution processing on the dimension reduction convolution image through the mobile overturning bottleneck convolution layer to obtain a characteristic image.
In step S202, the mobile inversion bottleneck convolution layer is set to perform mobile inversion bottleneck convolution processing on the dimension reduction convolution image, so that the random depth of the model can be increased, the time required by subsequent model training is reduced, and the performance of the model is improved.
It should be noted that, in general, a plurality of mobile inversion bottleneck convolution layers with the same parameters are adopted to perform mobile inversion bottleneck convolution processing on the dimension-reduced convolution image.
Referring to fig. 3, in some embodiments of the present application, the second neural network includes a global averaging pooling layer, a full connection layer, and a normalized classification layer, and step S104 includes, but is not limited to, step S301, step S302, and step S303. These three steps are described in detail below in conjunction with fig. 3.
Step S301: and carrying out pooling treatment on the characteristic images through a global average pooling layer to obtain pooling vectors.
Step S302: and carrying out full connection processing on the pooled vector through a full connection layer to obtain a preliminary classification value.
Step S303: and carrying out normalization processing on the preliminary classification value through a normalization classification layer to obtain a first loss value.
Specifically, in steps S301 to S303, the feature image obtained by extracting the features of the first neural network is input to a global average pooling layer to obtain a pooled vector, then the pooled vector passes through a full-connection layer to obtain a preliminary classification value, and the preliminary classification value is normalized to obtain a first loss value, where the first loss value characterizes the accuracy of the current neural network model in predicting the CNV image sample data. For example, the first loss value is 0.5, and the success rate of characterizing the current prediction accuracy is 50%. Or setting a certain prediction threshold, and when the first loss value exceeds the prediction threshold, considering the CNV image sample data input into the model as active CNV; otherwise, the CNV image sample data input into the model is an inactive CNV. Or may be characterized as predicting the probability that the CNV image sample data input into the model is an active CNV. If the first loss value is 0.98, the probability of characterizing CNV image sample data input into the model as active CNV is 98%.
It should be noted that, the normalization processing may be performed on the preliminary classification value by using a softmax classification function, or may be performed by using some other classification function, which is not particularly limited in this application.
Referring to fig. 4, in some embodiments of the present application, an actual frame is provided in the image sample data, and the actual frame is used to mark the actual position of the leakage area of the image sample data; step S105 includes, but is not limited to, step S401 and step S402.
Step S401: upsampling the feature image to obtain a predicted frame; the prediction frame is used for marking the position of the predicted leakage area of the image sample data.
Specifically, in step S401, a feature image obtained by extracting features of image sample data by using a first neural network is up-sampled to obtain a position of a blowby area, and then an external rectangular frame of the position of the blowby area is taken to obtain a prediction frame.
Step S402: and carrying out loss calculation on the predicted frame and the actual frame to obtain a second loss value.
In step S402, an actual frame characterizing the actual position of the blowby area of the image sample data and a predicted frame characterizing the predicted position of the blowby area of the image sample data are processed GIOU loss to obtain a second loss value. GIOU loss comprises the following specific processes:
a is used for representing a predicted frame, B is used for representing an actual frame, C is used for representing a minimum convex set of A, B, and the calculation process is as follows:
GIOU loss＝1-GIOU (3)
As can be seen from the calculation process of GIOU loss, in the case where the predicted frame and the actual frame are similar in shape, GIOU loss is smaller as the center distances of the predicted frame and the actual frame are closer. And the second loss value can represent the accuracy of the current neural network model to the CNV image sample data prediction.
Referring to fig. 5, in some embodiments of the application, step S106 includes, but is not limited to, step S501 and step S502.
Step S501: and carrying out weighting treatment on the first loss value and the second loss value according to preset weights to obtain a target loss value.
Specifically, in step S501, the first loss value and the second loss value are weighted according to a preset weight, so as to obtain a target loss value. For example, the target loss value=0.5×first loss value+0.5×second loss value. It is understood that the preset weight may be other weights, and the present application is not limited in particular, and may be set according to actual situations. For example, the prediction success rate of the image sample data is set by itself according to the neural network model and the attention enhancement model.
Step S502: and training the neural network model according to the target loss value to obtain an activity classification model of the choroidal neovascularization.
In step S502, the neural network model finely adjusts parameters of the neural network model according to actual conditions of the target loss value, so that the target loss value meets a preset condition, if the target loss value is continuously smaller than a certain preset loss threshold.
Referring to fig. 6, fig. 6 is a schematic diagram of a specific application scenario of the training method for the activity classification model of choroidal neovascularization according to some embodiments of the present application.
In fig. 6, a method for training an activity classification model for choroidal neovascularization in an embodiment of the present application is described in detail by using efficientNetB a network model as a preset neural network model. In fig. 6, CONV represents a dimension reduction convolution layer, MBCONV represents a moving flip bottleneck convolution layer.
As shown in fig. 6, the image sample data is first subjected to dimension reduction convolution through a CONV layer 3*3 to obtain a dimension reduction convolution image, and then subjected to moving inversion bottleneck convolution through MBCONV layers with different parameters to obtain a feature image. After the feature image is obtained, on one hand, training is carried out on the feature image by continuously using efficientNetB network models, dimension reduction convolution is carried out through a CONV layer of 1*1, pooling treatment is carried out through a global average pooling layer, full connection treatment is carried out through a full connection layer, and a first loss value is obtained through a softmax function. On the other hand, the thermodynamic diagram of the characteristic image is up-sampled, then a rectangular frame is connected externally, a predicted frame is obtained, and a second loss value is obtained by calculating GIOS loss of the predicted frame and the actual frame. And after the first loss value and the second loss value are obtained, weighting the first loss value and the second loss value to obtain a target loss value, wherein the target loss value represents the accuracy of the neural network model and the attention enhancement model to the prediction of the image sample data. After the target loss value is obtained, the neural network model finely adjusts parameters according to the target loss value to obtain an activity classification model of the choroidal neovascularization.
Referring to fig. 7, in a second aspect, some embodiments of the present application also provide an image classification method including, but not limited to, step S701 and step S702. These two steps are described in detail below in conjunction with fig. 7.
Step S701: and acquiring an image to be detected.
Step S702: inputting the image to be detected into an activity classification model for classification treatment to obtain the activity type of the choroidal neovascularization; wherein the activity classification model is trained according to the method of the embodiment of the first aspect.
In this embodiment, the image to be detected is directly input into the activity classification model of the choroidal neovascularization, so that classification of the choroidal neovascularization of the image to be detected can be realized.
According to the image classification model provided by the embodiment of the application, the automatic classification of the activity of the choroidal neovascularization is realized by using the activity classification model of the choroidal neovascularization, the side effect of using a contrast agent is avoided, the cost for judging the activity of the choroidal neovascularization is reduced, the qualification requirement for a doctor to judge the activity of the choroidal neovascularization is reduced, and the attention enhancement treatment is carried out on the characteristic image by using the attention enhancement model due to the addition of the attention branches of the leakage zone, so that the activity classification model of the choroidal neovascularization can better judge the activity of the choroidal neovascularization.
Referring to fig. 8, in a third aspect, some embodiments of the present application further provide an activity classification model training apparatus for choroidal neovascularization, comprising a sample acquisition module 801, an input module 802, a feature extraction module 803, a classification processing module 804, an augmentation processing module 805, and a training processing module 806.
Sample acquisition module 801: for acquiring an image sample dataset for acquiring choroidal neovascularization; wherein the image sample data set comprises a number of image sample data.
Input module 802: inputting each image sample data of the image sample data set into a preset neural network model; wherein the neural network model includes a first neural network and a second neural network.
Feature extraction module 803: and the characteristic extraction module is used for extracting the characteristics of each image sample data through a plurality of network layers of the first neural network to obtain a characteristic image.
Classification processing module 804: the method comprises the steps of performing choroidal neovascularization activity classification processing on a characteristic image through a second neural network to obtain a first loss value of each image sample data; wherein the first loss value is used to characterize the activity of choroidal neovascularization.
Reinforcement processing module 805: the method comprises the steps of performing attention enhancement processing on a characteristic image through a preset attention enhancement model to obtain a second loss value of each image sample data; wherein the second loss value is used to characterize the activity of choroidal neovascularization.
Training processing module 806: training the neural network model according to the first loss value and the second loss value to obtain an activity classification model of the choroidal neovascularization; wherein the activity classification model is used to classify choroidal neovascularization activity.
According to the training device for the activity classification model of the choroidal neovascularization, each image sample data is input into a first neural network, feature extraction is carried out on each image sample data through a plurality of network layers of the first neural network to obtain a corresponding feature image, and then the choroidal neovascularization activity classification processing is carried out on the feature image according to a second neural network to obtain a first loss value representing the activity of the choroidal neovascularization; and, leading in attention branches of the leakage area, adopting a preset attention enhancement model to carry out attention enhancement treatment on the characteristic image, and obtaining a second loss value representing the activity of the choroidal neovascularization; and training the neural network model according to the first loss value and the second loss value to obtain an activity classification model of the choroidal neovascularization. According to the choroidal neovascularization activity classification model obtained by the scheme, the choroidal neovascularization activity is automatically classified, the side effect of using a contrast agent is avoided, the cost for judging the choroidal neovascularization activity is reduced, the qualification requirement for judging the choriocapillaris activity by a doctor is reduced, and because the attention branches of a seepage region are added, the attention enhancement treatment is carried out on the characteristic image by using the attention enhancement model, so that the choroidal neovascularization activity classification model can better judge the choriocapillaris activity.
It should be noted that, the training device for the activity classification model of the choroidal neovascularization according to the embodiment of the present application corresponds to the training method for the activity classification model of the choroidal neovascularization, and the specific process refers to the training method for the activity classification model of the choroidal neovascularization, which is not described herein.
In a fourth aspect, some embodiments of the present application further provide an image classification apparatus, including an image acquisition module and an image classification module. The image acquisition module is used for acquiring an image to be detected; the image classification module is used for inputting the image to be detected into the activity classification model for classification processing to obtain the activity category of the choroidal neovascularization; wherein the activity classification model is trained according to the method of the embodiment of the first aspect.
The embodiment of the disclosure also provides an electronic device, including:
At least one memory;
At least one processor;
at least one program;
the program is stored in the memory, and the processor executes at least one program to implement the present disclosure to implement the activity classification model training method or the image classification method for choroidal neovascularization described above. The electronic device may be any intelligent terminal including a mobile phone, a tablet computer, a Personal digital assistant (Personal DIGITAL ASSISTANT, PDA), a vehicle-mounted computer, and the like.
An electronic device according to an embodiment of the present application is described in detail below with reference to fig. 9.
As shown in fig. 9, fig. 9 illustrates a hardware structure of an electronic device of another embodiment, the electronic device includes:
The processor 901 may be implemented by a general purpose central processing unit (Central Processing Unit, CPU), a microprocessor, an Application SPECIFIC INTEGRATED Circuit (ASIC), or one or more integrated circuits, etc. for executing related programs, so as to implement the technical solutions provided by the embodiments of the present disclosure;
The Memory 902 may be implemented in the form of a Read Only Memory (ROM), a static storage device, a dynamic storage device, or a random access Memory (Random Access Memory, RAM). The memory 902 may store an operating system and other application programs, and when the technical solutions provided in the embodiments of the present disclosure are implemented by software or firmware, relevant program codes are stored in the memory 902, and the processor 901 invokes the activity classification model training method or the image classification method for choroidal neovascularization to perform the embodiments of the present disclosure;
an input/output interface 903 for inputting and outputting information;
The communication interface 904 is configured to implement communication interaction between the device and other devices, and may implement communication in a wired manner (e.g. USB, network cable, etc.), or may implement communication in a wireless manner (e.g. mobile network, WIFI, bluetooth, etc.); and
A bus 905 that transfers information between the various components of the device (e.g., the processor 901, the memory 902, the input/output interface 903, and the communication interface 904);
Wherein the processor 901, the memory 902, the input/output interface 903 and the communication interface 904 are communicatively coupled to each other within the device via a bus 905.
The disclosed embodiments also provide a storage medium that is a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the above-described activity classification model training method or image classification method for choroidal neovascularization.
The memory, as a non-transitory computer readable storage medium, may be used to store non-transitory software programs as well as non-transitory computer executable programs. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory remotely located relative to the processor, the remote memory being connectable to the processor through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The embodiments described in the embodiments of the present disclosure are for more clearly describing the technical solutions of the embodiments of the present disclosure, and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure, and as those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present disclosure are equally applicable to similar technical problems.
It will be appreciated by those skilled in the art that the technical solutions shown in the figures do not limit the embodiments of the present disclosure, and may include more or fewer steps than shown, or may combine certain steps, or different steps.
The above described apparatus embodiments are merely illustrative, wherein the units illustrated as separate components may or may not be physically separate, i.e. may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
Those of ordinary skill in the art will appreciate that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof.
The terms "first," "second," "third," "fourth," and the like in the description of the application and in the above figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be understood that in the present application, "at least one (item)" means one or more, and "a plurality" means two or more. "and/or" for describing the association relationship of the association object, the representation may have three relationships, for example, "a and/or B" may represent: only a, only B and both a and B are present, wherein a, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b or c may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of elements is merely a logical functional division, and there may be additional divisions of actual implementation, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including multiple instructions for causing an electronic device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, an optical disk, or other various media capable of storing a program.
Preferred embodiments of the disclosed embodiments are described above with reference to the accompanying drawings, and thus do not limit the scope of the claims of the disclosed embodiments. Any modifications, equivalent substitutions and improvements made by those skilled in the art without departing from the scope and spirit of the embodiments of the present disclosure shall fall within the scope of the claims of the embodiments of the present disclosure.
Claims (9)
1. A method for training an activity classification model for choroidal neovascularization, comprising:
acquiring an image sample dataset of choroidal neovascularization; wherein the image sample dataset comprises a plurality of image sample data, each of the image sample data being labeled with a tag for characterizing active or inactive choroidal neovascularization;
inputting each image sample data of the image sample data set into a preset neural network model; wherein the neural network model comprises: a first neural network and a second neural network;
Extracting the characteristics of each image sample data through a plurality of network layers of the first neural network to obtain a characteristic image;
performing choroidal neovascularization activity classification processing on the characteristic image through the second neural network to obtain a first loss value of each image sample data; wherein the first loss value is used to characterize the choroidal neovascularization as either the active choroidal neovascularization or the inactive choroidal neovascularization;
Performing attention strengthening treatment on the characteristic images through a preset attention strengthening model to obtain a second loss value of each image sample data; wherein the second loss value is used to characterize the choroidal neovascularization as active choroidal neovascularization or inactive choroidal neovascularization;
Training the neural network model according to the first loss value and the second loss value to obtain an activity classification model of the choroidal neovascularization; wherein the activity classification model is used to classify choroidal neovascularization activity;
Wherein the second neural network comprises: the global averaging pooling layer, the full connection layer and the normalized classification layer perform choroidal neovascularization activity classification processing on the feature images through the second neural network to obtain a first loss value of each image sample data, and the method comprises the following steps:
carrying out pooling treatment on the characteristic images through the global average pooling layer to obtain pooling vectors;
performing full connection processing on the pooling vector through the full connection layer to obtain a preliminary classification value;
And normalizing the preliminary classification value through the normalization classification layer to obtain the first loss value.
2. The method of claim 1, wherein the plurality of network layers comprises: a dimension-reducing convolution layer and at least one moving flip bottleneck convolution layer;
The feature extraction is performed on each image sample data through a plurality of network layers of the first neural network to obtain a feature image, including:
performing dimension reduction convolution processing on each image sample data through the dimension reduction convolution layer to obtain a dimension reduction convolution image;
And carrying out mobile overturning bottleneck convolution processing on the dimension reduction convolution image through the mobile overturning bottleneck convolution layer to obtain a characteristic image.
3. The method according to any one of claims 1 to 2, wherein an actual frame is provided in the image sample data, and the actual frame is used for marking an actual blowby area position of the image sample data;
Performing attention enhancement processing on the feature image through a preset attention enhancement model to obtain a second loss value of each image sample data, including:
Upsampling the feature image to obtain a predicted frame; the prediction frame is used for marking the position of a predicted blowby area of the image sample data;
And carrying out loss calculation on the predicted frame and the actual frame to obtain a second loss value.
4. The method according to any one of claims 1 to 2, wherein the training the neural network model according to the first loss value and the second loss value to obtain an activity classification model of choroidal neovascularization comprises:
Weighting the first loss value and the second loss value according to preset weights to obtain a target loss value;
and training the neural network model according to the target loss value to obtain an activity classification model of the choroidal neovascularization.
5. A method of classifying images, the method comprising:
acquiring an image to be detected;
inputting the image to be detected into an activity classification model for classification treatment to obtain the activity type of the choroidal neovascularization; wherein the activity classification model is trained according to the method of any one of claims 1 to 4.
6. An activity classification model training device for choroidal neovascularization, comprising:
A sample acquisition module for acquiring an image sample dataset of choroidal neovascularization; wherein the image sample dataset comprises a plurality of image sample data, each of the image sample data being labeled with a tag for characterizing active or inactive choroidal neovascularization;
The input module is used for inputting each image sample data of the image sample data set into a preset neural network model; wherein the neural network model comprises a first neural network and a second neural network;
The feature extraction module is used for extracting features of each image sample data through a plurality of network layers of the first neural network to obtain a feature image;
the classification processing module is used for performing choroidal neovascularization activity classification processing on the characteristic images through the second neural network to obtain a first loss value of each image sample data; wherein the first loss value is used to characterize the choroidal neovascularization as either the active choroidal neovascularization or the inactive choroidal neovascularization;
the strengthening processing module is used for carrying out attention strengthening processing on the characteristic images through a preset attention strengthening model to obtain a second loss value of each image sample data; wherein the second loss value is used to characterize the choroidal neovascularization as either the active choroidal neovascularization or the inactive choroidal neovascularization;
The training processing module is used for training the neural network model according to the first loss value and the second loss value to obtain an activity classification model of the choroidal neovascularization; wherein the activity classification model is used to classify choroidal neovascularization activity;
wherein the second neural network comprises: the classification processing module is used for performing choroidal neovascularization activity classification processing on the characteristic image through the second neural network to obtain a first loss value of each image sample data, and comprises the following steps:
carrying out pooling treatment on the characteristic images through the global average pooling layer to obtain pooling vectors;
performing full connection processing on the pooling vector through the full connection layer to obtain a preliminary classification value;
And normalizing the preliminary classification value through the normalization classification layer to obtain the first loss value.
7. An image classification apparatus, comprising:
the image acquisition module is used for acquiring an image to be detected;
The image classification module is used for inputting the image to be detected into an activity classification model for classification processing to obtain the activity category of the choroidal neovascularization; wherein the activity classification model is trained according to the method of any one of claims 1 to 4.
8. An electronic device, comprising:
At least one memory;
At least one processor;
at least one program;
The program is stored in the memory, and the processor executes the at least one program to implement:
the method of any one of claims 1 to 4; or alternatively
The method of claim 5.
9. A storage medium that is a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions for causing a computer to perform:
the method of any one of claims 1 to 4; or alternatively
The method of claim 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111248118.2A CN113780478B (en) | 2021-10-26 | 2021-10-26 | Activity classification model training method, classification method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111248118.2A CN113780478B (en) | 2021-10-26 | 2021-10-26 | Activity classification model training method, classification method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113780478A CN113780478A (en) | 2021-12-10 |
| CN113780478B true CN113780478B (en) | 2024-05-28 |
Family
ID=78956609
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111248118.2A Active CN113780478B (en) | 2021-10-26 | 2021-10-26 | Activity classification model training method, classification method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113780478B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109376757A (en) * | 2018-09-06 | 2019-02-22 | 北京飞搜科技有限公司 | A kind of multi-tag classification method and system |
| CN110991506A (en) * | 2019-11-22 | 2020-04-10 | 高新兴科技集团股份有限公司 | Vehicle brand identification method, device, equipment and storage medium |
| CN111639755A (en) * | 2020-06-07 | 2020-09-08 | 电子科技大学中山学院 | Network model training method and device, electronic equipment and storage medium |
| CN111950643A (en) * | 2020-08-18 | 2020-11-17 | 创新奇智(上海)科技有限公司 | Model training method, image classification method and corresponding device |
| CN112288086A (en) * | 2020-10-30 | 2021-01-29 | 北京市商汤科技开发有限公司 | Neural network training method and device and computer equipment |
| CN113392875A (en) * | 2021-05-20 | 2021-09-14 | 广东工业大学 | Method, system and equipment for classifying fine granularity of image |
| CN113420848A (en) * | 2021-08-24 | 2021-09-21 | 深圳市信润富联数字科技有限公司 | Neural network model training method and device and gesture recognition method and device |
| CN113449840A (en) * | 2020-03-27 | 2021-09-28 | 南京人工智能高等研究院有限公司 | Neural network training method and device and image classification method and device |
-
2021
- 2021-10-26 CN CN202111248118.2A patent/CN113780478B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109376757A (en) * | 2018-09-06 | 2019-02-22 | 北京飞搜科技有限公司 | A kind of multi-tag classification method and system |
| CN110991506A (en) * | 2019-11-22 | 2020-04-10 | 高新兴科技集团股份有限公司 | Vehicle brand identification method, device, equipment and storage medium |
| CN113449840A (en) * | 2020-03-27 | 2021-09-28 | 南京人工智能高等研究院有限公司 | Neural network training method and device and image classification method and device |
| CN111639755A (en) * | 2020-06-07 | 2020-09-08 | 电子科技大学中山学院 | Network model training method and device, electronic equipment and storage medium |
| CN111950643A (en) * | 2020-08-18 | 2020-11-17 | 创新奇智(上海)科技有限公司 | Model training method, image classification method and corresponding device |
| CN112288086A (en) * | 2020-10-30 | 2021-01-29 | 北京市商汤科技开发有限公司 | Neural network training method and device and computer equipment |
| CN113392875A (en) * | 2021-05-20 | 2021-09-14 | 广东工业大学 | Method, system and equipment for classifying fine granularity of image |
| CN113420848A (en) * | 2021-08-24 | 2021-09-21 | 深圳市信润富联数字科技有限公司 | Neural network model training method and device and gesture recognition method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113780478A (en) | 2021-12-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10482603B1 (en) | Medical image segmentation using an integrated edge guidance module and object segmentation network | |
| Yap et al. | Deep learning in diabetic foot ulcers detection: A comprehensive evaluation | |
| US11373390B2 (en) | Generating scene graphs from digital images using external knowledge and image reconstruction | |
| US11200416B2 (en) | Methods and apparatuses for image detection, electronic devices and storage media | |
| CN112131393A (en) | Construction method of medical knowledge map question-answering system based on BERT and similarity algorithm | |
| US20220383661A1 (en) | Method and device for retinal image recognition, electronic equipment, and storage medium | |
| CN113724228A (en) | Tongue color and coating color identification method and device, computer equipment and storage medium | |
| CN112419326B (en) | Image segmentation data processing method, device, equipment and storage medium | |
| CN114897060B (en) | Training method and device for sample classification model, and sample classification method and device | |
| CN115239675A (en) | Training method of classification model, image classification method and device, equipment and medium | |
| CN113707299A (en) | Auxiliary diagnosis method and device based on inquiry session and computer equipment | |
| CN114758382A (en) | Face AU detection model establishing method and application based on adaptive patch learning | |
| CN113780478B (en) | Activity classification model training method, classification method, device, equipment and medium | |
| CN116453222B (en) | Target object posture determining method, training device and storage medium | |
| CN116702776A (en) | Multi-task semantic division method, device, equipment and medium based on cross-Chinese and western medicine | |
| CN111353302A (en) | Medical word sense recognition method and device, computer equipment and storage medium | |
| CN113792740B (en) | Artery and vein segmentation method, system, equipment and medium for fundus color illumination | |
| CN115036022A (en) | Health risk assessment method and system, computer device, and storage medium | |
| CN115132324A (en) | Mental health prediction method and device, electronic equipment and storage medium | |
| Liao et al. | Video Face Detection Technology and Its Application in Health Information Management System | |
| CN112802573A (en) | Medicine package recommendation method and device, computer system and readable storage medium | |
| CN115631196B (en) | Image segmentation method, model training method, device, equipment and storage medium | |
| Bhagwat et al. | Conversion of Sign Language to Text Using Machine Learning | |
| Khartheesvar et al. | Automatic Indian sign language recognition using MediaPipe holistic and LSTM network | |
| CN116416637A (en) | Medical document information extraction method and device, electronic equipment and readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |