CN117496214A - Medical image classification method, medical image classification device, computer equipment and storage medium - Google Patents
Medical image classification method, medical image classification device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN117496214A CN117496214A CN202311221660.8A CN202311221660A CN117496214A CN 117496214 A CN117496214 A CN 117496214A CN 202311221660 A CN202311221660 A CN 202311221660A CN 117496214 A CN117496214 A CN 117496214A
- Authority
- CN
- China
- Prior art keywords
- image
- endoscope
- images
- augmentation
- classification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 230000003416 augmentation Effects 0.000 claims abstract description 79
- 238000012549 training Methods 0.000 claims abstract description 45
- 238000007781 pre-processing Methods 0.000 claims abstract description 41
- 238000013145 classification model Methods 0.000 claims abstract description 37
- 238000012545 processing Methods 0.000 claims abstract description 26
- 238000012360 testing method Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 14
- 230000007797 corrosion Effects 0.000 claims description 10
- 238000005260 corrosion Methods 0.000 claims description 10
- 238000003062 neural network model Methods 0.000 claims description 10
- 230000004927 fusion Effects 0.000 claims description 7
- 230000000873 masking effect Effects 0.000 claims description 7
- 238000010606 normalization Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000000670 limiting effect Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 210000004204 blood vessel Anatomy 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000001839 endoscopy Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000002829 reductive effect Effects 0.000 description 1
- 238000013341 scale-up Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/09—Supervised learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
- G06V10/16—Image acquisition using multiple overlapping images; Image stitching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Medical Treatment And Welfare Office Work (AREA)
Abstract
The application relates to a medical image classification method, a device, a computer device and a storage medium, wherein the method comprises the following steps: acquiring an endoscope image set; performing preprocessing operation on an endoscope image set, wherein the preprocessing operation comprises the steps of performing image augmentation on each endoscope image in the endoscope image set to generate an endoscope image set subjected to image augmentation, and performing mask processing on the endoscope image set subjected to image augmentation to obtain corresponding mask images, wherein each mask image comprises a target color block in the endoscope image subjected to image augmentation; fusing and splicing each mask image and the corresponding endoscope image with the amplified image to obtain a target image set; training based on the target image set and the corresponding classification label to obtain an image classification model; and preprocessing the endoscopic images to be classified, and inputting the preprocessed endoscopic images into an image classification model to obtain classification results. The method solves the problem of inaccurate classification and identification of NBI images and white light images in the prior art, and improves the classification precision and accuracy of endoscopic images.
Description
Technical Field
The present disclosure relates to the field of image processing technologies, and in particular, to a medical image classification method, apparatus, computer device, and storage medium.
Background
Compared with the traditional electronic endoscope, the NBI (Narrow Band Imaging, NBI, endoscope narrow-band imaging) endoscope can obviously improve the contrast of microstructure on the surface of the tissue, so that the blood vessel morphology of the surface structure of the tissue is clearly displayed, and the early detection of cancerous events and focus positions is facilitated, but the white light image and the NBI image are mixed with the image shot by the endoscope in the prior art, so that the distinction between the white light image and the NBI image is necessary. However, manual classification and inspection is time consuming, and thus it is valuable to develop a method for automatically distinguishing white light images from NBI images.
Under NBI endoscopy, images appear as dark green submucosal vessels and tan diagonal vessels, and currently existing techniques distinguish white light images from NBI images according to the size of green pixel values, however, the technique is easily affected by image size. With the development of artificial intelligence and deep learning, the realization of distinguishing white light images from NBI images based on convolutional neural networks shows excellent performance. However, due to different objective conditions of image acquisition, the unavoidable existence of partial NBI image has the phenomenon of the same as that of white light image because of the smaller green pixel value, and the satisfactory effect cannot be obtained by directly carrying out image identification on the original image.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a medical image classification method, apparatus, computer device, and computer-readable storage medium capable of improving the classification accuracy and efficiency of NBI images and white light images.
In a first aspect, the present application provides a medical image classification method, the method comprising:
acquiring an endoscope image set;
performing preprocessing operation on the endoscope image set, wherein the preprocessing operation comprises the steps of performing image augmentation on each endoscope image in the endoscope image set to generate an endoscope image set subjected to image augmentation, and performing mask processing on the endoscope image set subjected to image augmentation to obtain corresponding mask images, wherein each mask image comprises a target color block in the endoscope image subjected to image augmentation; fusing and splicing each mask image and the corresponding endoscope image after image augmentation to obtain a target image set;
training based on the target image set and the corresponding classification label to obtain an image classification model;
and inputting the endoscopic images to be classified into the image classification model after the preprocessing operation to obtain a classification result.
In one embodiment, the masking the image-amplified endoscopic image set includes:
converting the endoscope images after the image augmentation into HSV images;
and extracting target color blocks in the endoscope images after image augmentation based on the set HSV color range values, setting other areas except the target color blocks to be black, and generating binarized images corresponding to the endoscope images after image augmentation.
In one embodiment, performing mask processing on the image-amplified endoscope image set to obtain corresponding mask images includes:
and performing corrosion operation and expansion operation on each binarized image extracted from the target color block, removing noise points on the target color block, and connecting break points of the target color block to obtain each corresponding mask image.
In one embodiment, the removing the noise point of the target color block, and connecting the breakpoint of the target color block includes:
and (3) performing corrosion operation on the target color block by using the following formula (1) to obtain a corroded target color block:
performing expansion operation on the corroded target color block by using the following formula (2) to obtain the mask image:
wherein X represents the target color patch on the binarized image, S represents a convolution kernel, X, y represents a position where a center of the convolution kernel moves in the binarized image, and Sxy represents an area of the binarized image covered by the convolution kernel when the center of the convolution kernel is at the (X, y) position.
In one embodiment, the fusing and stitching each mask image and the corresponding endoscope image to obtain the target image set includes:
performing bit operation on each mask image and the corresponding endoscope image after image augmentation to obtain an endoscope image set after bit operation;
and performing tensor operation and normalization processing on the endoscope images subjected to the bit operation and the corresponding endoscope images subjected to the image augmentation, and performing fusion and splicing on the images generated after the processing to obtain the target image set, wherein the target images in the target image set are six-channel images.
In one embodiment, the endoscope image set includes a training set and a test set including an NBI image and a white light image, and the training to obtain the image classification model based on the target image and the corresponding classification label includes:
and inputting the training set of the NBI image and the white light image in the target image set, the testing set of the NBI image and the white light image and the corresponding classification labels into a neural network model for training until the output precision of the neural network model reaches a set threshold or the iterative training times reach the set threshold, so as to obtain the image classification model.
In one embodiment, image augmentation of each of the endoscopic images in the set of endoscopic images comprises: and randomly cutting and/or scaling and/or turning the endoscope image set to obtain the endoscope image set with the amplified image.
In a second aspect, the present application also provides a medical image classification apparatus, the apparatus comprising:
the image acquisition module is used for acquiring an endoscope image;
the image preprocessing module is used for executing preprocessing operation on the endoscope image set, the preprocessing operation comprises the steps of carrying out image augmentation on each endoscope image in the endoscope image set to generate an endoscope image set after image augmentation, carrying out mask processing on the endoscope image set after image augmentation to obtain corresponding mask images, and the mask images comprise target color blocks in the endoscope image after image augmentation; fusing and splicing each mask image and the corresponding endoscope image after image augmentation to obtain a target image set;
the model training module is used for training to obtain an image classification model based on the target image set and the corresponding classification labels;
and the image classification module is used for inputting the endoscope images to be classified into the image classification model after the preprocessing operation is carried out, so as to obtain classification results.
In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the above-mentioned first aspect when executing the computer program.
In a fourth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the above-described aspects.
The medical image classification method, the medical image classification device, the computer equipment and the storage medium are used for acquiring an endoscope image set; performing preprocessing operation on the endoscope image set, wherein the preprocessing operation comprises the steps of performing image augmentation on each endoscope image in the endoscope image set to generate an endoscope image set subjected to image augmentation, and performing mask processing on the endoscope image set subjected to image augmentation to obtain corresponding mask images, wherein each mask image comprises a target color block in the endoscope image subjected to image augmentation; fusing and splicing each mask image and the corresponding endoscope image after image augmentation to obtain a target image set; training based on the target image set and the corresponding classification label to obtain an image classification model; and inputting the endoscope images to be classified into the image classification model after the preprocessing operation to obtain classification results, solving the problem of inaccurate classification and identification of NBI images and white light images in the prior art, and improving the classification precision and accuracy of the endoscope images.
Drawings
FIG. 1 is a diagram of an application environment for a method of classifying medical images according to one embodiment;
FIG. 2 is a flow chart of a method of classifying medical images according to one embodiment;
FIG. 3 is a schematic diagram of an image classification model in one embodiment;
FIG. 4 is a block diagram of a medical image classification apparatus according to one embodiment;
fig. 5 is an internal structural diagram of a computer device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
Unless defined otherwise, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this application belongs. Reference to "a," "an," "the," and similar terms herein do not denote a limitation of quantity, but rather denote the singular or plural. The terms "comprising," "including," "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to only those steps or elements but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. The terms "connected," "coupled," and the like in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The term "plurality" as used herein refers to two or more. "and/or" describes an association relationship of an association object, meaning that there may be three relationships, e.g., "a and/or B" may mean: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship. The terms "first," "second," "third," and the like, as used herein, are merely distinguishing between similar objects and not representing a particular ordering of objects.
The medical image classification method provided by the embodiment of the application can be applied to an application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The data storage system 106 may store image data that the server 104 needs to process. The data storage system 106 may be integrated on the server 104 or may be located on a cloud or other network server.
Acquiring an endoscopic image set on the terminal 102; performing preprocessing operation on the endoscope image set, wherein the preprocessing operation comprises the steps of performing image augmentation on each endoscope image in the endoscope image set to generate an endoscope image set subjected to image augmentation, and performing mask processing on the endoscope image set subjected to image augmentation to obtain corresponding mask images, wherein each mask image comprises a target color block in the endoscope image subjected to image augmentation; fusing and splicing each mask image and the corresponding endoscope image after image augmentation to obtain a target image set; training based on the target image set and the corresponding classification label to obtain an image classification model; and inputting the endoscopic images to be classified into the image classification model after the preprocessing operation to obtain a classification result.
The terminal 102 may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, etc. The server 104 may be implemented as a stand-alone server or as a server cluster of multiple servers.
In one embodiment, as shown in fig. 2, a medical image classification method is provided, and the method is applied to the terminal in fig. 1 for illustration, and includes the following steps:
s202, acquiring an endoscopic image set.
Wherein the endoscopic image set comprises a training set and a test set comprising NBI images and white light images.
S204, performing preprocessing operation on the endoscope image set, wherein the preprocessing operation comprises the steps of performing image augmentation on each endoscope image in the endoscope image set to generate an endoscope image set subjected to image augmentation, and performing mask processing on the endoscope image set subjected to image augmentation to obtain corresponding mask images, wherein each mask image comprises target color blocks in the endoscope image subjected to image augmentation; and carrying out fusion and splicing on each mask image and the corresponding endoscope image with the amplified image to obtain a target image set.
Specifically, the training set image and the test set image of the NBI image and the white light image in the endoscope image set are subjected to image augmentation, the training set and the test set are enlarged, the endoscope image set after image augmentation is obtained, mask processing is carried out on the endoscope image set after image augmentation, target color blocks in the endoscope image after image augmentation are extracted, and mask images corresponding to the training set after image augmentation and the test set are obtained. And fusing and splicing the mask images with the corresponding NBI images or the white light images to obtain a final target image set.
S206, training based on the target image set and the corresponding classification labels to obtain an image classification model.
Specifically, the NBI image and the white light image in the target image set are classified according to the NBI image tag and the white light image tag. The initial model is trained using a training set of NBI images and white light images, and features of the NBI images and white light images are learned, the features corresponding to the classification labels. Inputting the test set of the NBI image and the white light image into the trained initial model, and testing the classification precision of the trained initial model to obtain a final image classification model.
S208, inputting the endoscope images to be classified into the image classification model after the preprocessing operation to obtain classification results.
Specifically, the obtained endoscopic image to be classified is input into the image classification model after the preprocessing operation in S204, and a classification result of the endoscopic image to be classified is output, where the classification result is an NBI image or a white light image.
According to the medical image classification method, an endoscope image set is acquired; performing preprocessing operation on the endoscope image set, wherein the preprocessing operation comprises the steps of performing image augmentation on each endoscope image in the endoscope image set to generate an endoscope image set subjected to image augmentation, and performing mask processing on the endoscope image set subjected to image augmentation to obtain corresponding mask images, wherein each mask image comprises a target color block in the endoscope image subjected to image augmentation; fusing and splicing each mask image and the corresponding endoscope image after image augmentation to obtain a target image set; training based on the target image set and the corresponding classification label to obtain an image classification model; and inputting the endoscope images to be classified into the image classification model after the preprocessing operation to obtain classification results, solving the problem of inaccurate classification and identification of NBI images and white light images in the prior art, and improving the classification precision and accuracy of the endoscope images.
In one embodiment, the image augmentation of each of the endoscope images in the endoscope image set in S204 specifically includes the following:
and randomly cutting and/or scaling and/or turning the endoscope image set to obtain the endoscope image set with the amplified image.
Specifically, an endoscope image set is obtained, and image augmentation is carried out on an initial endoscope image through random cutting and/or scaling and/or random overturning, so that the endoscope image set is obtained.
For example, the NBI image and the white light image in the training set are randomly cropped and then scaled to a 224 pixel by 224 pixel size and randomly flipped horizontally. The NBI image and the white light image in the test set were scaled to 256 pixels by 256 pixel size and then center cropped to 224 pixels by 224 pixel size. The image diversity can be randomly increased by carrying out image augmentation on the training set, the scale of the training set is enlarged, and the dependence of the model on certain attributes is reduced, so that the generalization capability of the model is improved. For the test set, the image augmentation process may not be performed randomly, but simply to scale up the test set.
In this embodiment, by performing image augmentation on the initial endoscopic image, a training set and a testing set for training an image classification model are enlarged, and the accuracy of the image classification model is improved.
In one embodiment, masking the image-augmented endoscopic image set in S204 includes:
converting the endoscope images after the image augmentation into HSV images; and extracting target color blocks in the endoscope images after image augmentation based on the set HSV color range values, setting other areas except the target color blocks to be black, and generating binarized images corresponding to the endoscope images after image augmentation.
Specifically, each image-amplified endoscope image is an RGB image, each image-amplified endoscope image is converted into an HSV image, an HSV color range value of dark green to be extracted is set, an inRange function is called to extract dark green color blocks as target color blocks in each image-amplified endoscope image, other areas except the target color blocks are set as black backgrounds, the target color blocks are reserved as white prospects, and a corresponding binarized image is generated.
Specifically, since the dark green region does not exist on the white light image, the white light image is processed by the method of this embodiment to obtain an almost completely black image.
In this embodiment, by performing image format conversion on the endoscope image after image augmentation and performing target color block extraction on the converted image, the processed image can have more visual contrast between brightness and darkness compared with the original image, and the final classification precision of the image classification model is improved.
In one embodiment, the masking process is performed on the image-amplified endoscopic image set in S204, so as to obtain corresponding mask images, where the masking process further includes the following specific contents:
and performing corrosion operation and expansion operation on each binarized image extracted from the target color block, removing noise points on the target color block, and connecting break points of the target color block to obtain each corresponding mask image.
The method comprises the steps of removing noise points on the target color block, and connecting the breakpoint of the target color block, wherein the breakpoint comprises the following specific contents:
and (3) performing corrosion operation on the target color block by using the following formula (1) to obtain a corroded target color block:
performing expansion operation on the corroded target color block by using the following formula (2) to obtain the mask image:
wherein X represents the target color patch on the binarized image, S represents a convolution kernel, X, y represents a position where a center of the convolution kernel moves in the binarized image, and Sxy represents an area of the binarized image covered by the convolution kernel when the center of the convolution kernel is at the (X, y) position.
Specifically, the binary image is subjected to corrosion and expansion operation, noise points on the target color block are removed, the break points of the target color block are connected, a mask image corresponding to the binary image is obtained, and the area accuracy of the target color block is improved.
In one embodiment, in S204, the fusing and stitching the mask images and the corresponding image-amplified endoscopic images to obtain the target image set includes the following specific contents:
and performing bit operation on each mask image and the corresponding endoscope image after image augmentation to obtain an endoscope image set after bit operation. And performing tensor operation and normalization processing on the endoscope images subjected to the bit operation and the corresponding endoscope images subjected to the image augmentation, and performing fusion and splicing on the images generated after the processing to obtain the target image set, wherein the target images in the target image set are six-channel images.
Specifically, a bitwise_and function is called to perform bit operation on the mask image and the corresponding endoscope image set after image augmentation to obtain the endoscope image set after bit operation, and each endoscope image after bit operation only keeps other colors of dark green to be changed into black. And performing tensor operation and normalization processing on the endoscope image set subjected to bit operation and the RGB image of the endoscope image subjected to corresponding image augmentation, so that subsequent splicing is facilitated, and the convergence speed of the neural network can be continuously deepened by the normalized image. And fusing and splicing the processed images to obtain six-channel images, and forming a target image set.
Illustratively, a six-channel image obtained based on NBI images is provided, and RGB three channels of the two NBI images are spliced into the six-channel image, wherein one NBI image is an original endoscopic image, and the other NBI image is an image which is generated by masking the original endoscopic image and only keeps other areas of a dark green area.
In one embodiment, S206 trains to obtain an image classification model based on the target image and the corresponding classification label, and specifically includes the following:
and inputting the training set of the NBI image and the white light image in the target image set, the testing set of the NBI image and the white light image and the corresponding classification labels into a neural network model for training until the output precision of the neural network model reaches a set threshold or the iterative training times reach the set threshold, so as to obtain the image classification model.
Specifically, the neural network model is a ResNet-18 residual network, and as shown in fig. 3, the neural network model comprises a 7×7 convolution layer, a batch norm2d operator, 4 residual blocks, 2 ResNet blocks per layer, an adaptive streaming pool2d pooling layer, a flat layer and an FC full connection layer which are sequentially connected. And inputting the training set of the NBI image and the white light image in the target image set and the test set of the NBI image and the white light image into a ResNet-18 residual error network for training. And (5) after 10 times of iteration, finishing to obtain the image classification model, or obtaining the image classification model with the output classification precision reaching a set threshold after the iteration.
In one example embodiment, a medical image classification method for NBI images and white light images is provided, comprising in particular:
s1, dividing an acquired endoscope image set into a training set and a testing set, manually dividing an initial endoscope image in the training set into an NBI image and a white light image according to image characteristics, and manually dividing the image of the testing set into the NBI image and the white light image in the same way. And carrying out random cutting and/or scaling and/or overturning on the divided endoscopic image set to obtain the endoscopic image set with the amplified image.
S2, converting each endoscope image in the endoscope image set with the amplified image into an HSV image, setting an HSV color range value of dark green to be extracted, calling an inRange function to extract dark green color blocks as target color blocks in each endoscope image, setting other areas except the target color blocks as black backgrounds, reserving the target color blocks as white prospects, and generating corresponding binary images.
And S3, performing corrosion operation on the obtained binarized image of the target color block by using a formula (1) to obtain the corroded target color block.
And (3) performing expansion operation on the corroded target color block by using the formula (2), removing noise points on the target color block, connecting break points of the target color block, and smoothing the edge of the target color block to obtain a mask image.
Wherein X represents the target color patch on the binarized image, S represents a convolution kernel, X, y represents a position where a center of the convolution kernel moves in the binarized image, and Sxy represents an area of the binarized image covered by the convolution kernel when the center of the convolution kernel is at the (X, y) position.
S4, invoking a bitwise_and function to perform bit operation on the mask image and the corresponding endoscope image to obtain an endoscope image after bit operation, wherein the endoscope image after bit operation only keeps dark green and changes other colors into black. And performing tensor operation and normalization processing on the endoscope image after bit operation and the RGB image of the corresponding endoscope image respectively, and performing fusion and splicing on the processed image to obtain a six-channel image.
S5, inputting the six-channel image set obtained by the endoscopic image set in the S2-S5 step into a ResNet-18 residual error network for training. And after 10 times of iteration, finishing to obtain the image classification model.
S6, preprocessing the endoscope images to be classified in the S2-S4 mode, and inputting the preprocessed endoscope images into a complete image classification model to obtain classification results, wherein the classification results are NBI images or white light images.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides a medical image classification device for realizing the medical image classification method. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitations in the embodiments of the medical image classification device or devices provided below may be referred to above for the limitations of the medical image classification method, and will not be repeated here.
In one embodiment, as shown in fig. 4, there is provided a medical image classification apparatus comprising: an image acquisition module 41, an image preprocessing module 42, a model training module 43, and an image classification module 44, wherein:
the image acquisition module 41 is used for acquiring an endoscopic image set.
An image preprocessing module 42, configured to perform a preprocessing operation on the endoscope image set, where the preprocessing operation includes performing image augmentation on each of the endoscope images in the endoscope image set, generating an image-augmented endoscope image set, and performing mask processing on the image-augmented endoscope image set to obtain corresponding mask images, where the mask images include target color patches in the image-augmented endoscope image; and carrying out fusion and splicing on each mask image and the corresponding endoscope image with the amplified image to obtain a target image set.
The model training module 43 is configured to train to obtain an image classification model based on the target image set and the corresponding classification labels.
The image classification module 44 is configured to perform the preprocessing operation on the endoscopic image to be classified, and then input the preprocessed image into the image classification model to obtain a classification result.
In one embodiment, the image preprocessing module 42 is further configured to: converting the endoscope images after the image augmentation into HSV images; and extracting target color blocks in the endoscope images after image augmentation based on the set HSV color range values, setting other areas except the target color blocks to be black, and generating binarized images corresponding to the endoscope images after image augmentation.
In one embodiment, the image preprocessing module 42 is further configured to: and performing corrosion operation and expansion operation on each binarized image extracted from the target color block, removing noise points on the target color block, and connecting break points of the target color block to obtain each corresponding mask image.
In one embodiment, the image preprocessing module 42 is further configured to: and (3) performing corrosion operation on the target color block by using the following formula (1) to obtain a corroded target color block:
performing expansion operation on the corroded target color block by using the following formula (2) to obtain the mask image:
wherein X represents the target color patch on the binarized image, S represents a convolution kernel, X, y represents a position where a center of the convolution kernel moves in the binarized image, and Sxy represents an area of the binarized image covered by the convolution kernel when the center of the convolution kernel is at the (X, y) position.
In one embodiment, the image preprocessing module 42 is further configured to: performing bit operation on each mask image and the corresponding endoscope image after image augmentation to obtain an endoscope image set after bit operation; and performing tensor operation and normalization processing on the endoscope images subjected to the bit operation and the corresponding endoscope images subjected to the image augmentation, and performing fusion and splicing on the images generated after the processing to obtain a target image set, wherein the target image in the target image set is a six-channel image.
In one embodiment, model training module 43 is further configured to: and inputting the training set of the NBI image and the white light image in the target image set, the testing set of the NBI image and the white light image and the corresponding classification labels into a neural network model for training until the output precision of the neural network model reaches a set threshold or the iterative training times reach the set threshold, so as to obtain the image classification model.
In one embodiment, the image preprocessing module 42 is further configured to: and randomly cutting and/or scaling and/or turning the endoscope image set to obtain the endoscope image set with the amplified image.
The various modules in the medical image classification apparatus described above may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a terminal, and the internal structure of which may be as shown in fig. 5. The computer device includes a processor, a memory, an input/output interface, a communication interface, a display unit, and an input means. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a medical image classification method. The display unit of the computer device is used for forming a visual picture, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be a key, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those skilled in the art that the structure shown in fig. 5 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, including a memory and a processor, where the memory stores a computer program, and the processor implements steps corresponding to the methods described in the above embodiments when the computer program is executed.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which when executed by a processor implements the steps corresponding to the methods described in the above embodiments.
It should be noted that, the data (including, but not limited to, data for analysis, stored data, displayed data, etc.) referred to in the present application are all data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data are required to comply with the related laws and regulations and standards of the related country and region.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the various embodiments provided herein may include at least one of relational databases and non-relational databases. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples only represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application shall be subject to the appended claims.
Claims (10)
1. A method of medical image classification, the method comprising:
acquiring an endoscope image set;
performing preprocessing operation on the endoscope image set, wherein the preprocessing operation comprises the steps of performing image augmentation on each endoscope image in the endoscope image set to generate an endoscope image set subjected to image augmentation, and performing mask processing on the endoscope image set subjected to image augmentation to obtain corresponding mask images, wherein each mask image comprises a target color block in the endoscope image subjected to image augmentation; fusing and splicing each mask image and the corresponding endoscope image after image augmentation to obtain a target image set;
training to obtain an image classification model based on the target image set and the corresponding classification label;
and inputting the endoscopic images to be classified into the image classification model after the preprocessing operation to obtain a classification result.
2. The medical image classification method according to claim 1, wherein masking the image-augmented endoscopic image set comprises:
converting the endoscope images after the image augmentation into HSV images;
and extracting target color blocks in the endoscope images after image augmentation based on the set HSV color range values, setting other areas except the target color blocks to be black, and generating binarized images corresponding to the endoscope images after image augmentation.
3. The medical image classification method according to claim 2, wherein masking the image-augmented endoscopic image set to obtain corresponding mask images comprises:
and performing corrosion operation and expansion operation on each binarized image extracted from the target color block, removing noise points on the target color block, and connecting break points of the target color block to obtain each corresponding mask image.
4. A medical image classification method according to claim 3, wherein said removing noise points of said target color patch, connecting break points of said target color patch comprises:
and (3) performing corrosion operation on the target color block by using the following formula (1) to obtain a corroded target color block:
performing expansion operation on the corroded target color block by using the following formula (2) to obtain the mask image:
wherein X represents the target color patch on the binarized image, S represents a convolution kernel, X, y represents a position where a center of the convolution kernel moves in the binarized image, and Sxy represents an area of the binarized image covered by the convolution kernel when the center of the convolution kernel is at the (X, y) position.
5. The medical image classification method according to claim 1, wherein the fusing and stitching each of the mask images and the corresponding image-augmented endoscopic image to obtain a target image set includes:
performing bit operation on each mask image and the corresponding endoscope image after image augmentation to obtain an endoscope image set after bit operation;
and performing tensor operation and normalization processing on the endoscope images subjected to the bit operation and the corresponding endoscope images subjected to the image augmentation, and performing fusion and splicing on the images generated after the processing to obtain the target image set, wherein the target images in the target image set are six-channel images.
6. The medical image classification method of claim 1, wherein the endoscopic image set comprises a training set and a test set comprising NBI images and white light images, and wherein training to obtain an image classification model based on the target images and corresponding classification labels comprises:
and inputting the training set of the NBI image and the white light image in the target image set, the testing set of the NBI image and the white light image and the corresponding classification labels into a neural network model for training until the output precision of the neural network model reaches a set threshold or the iterative training times reach the set threshold, so as to obtain the image classification model.
7. The medical image classification method according to claim 1, wherein image augmentation of each of the endoscopic images in the endoscopic image set comprises: and randomly cutting and/or scaling and/or turning the endoscope image set to obtain the endoscope image set with the amplified image.
8. A medical image classification apparatus, the apparatus comprising:
the image acquisition module is used for acquiring an endoscope image set;
the image preprocessing module is used for executing preprocessing operation on the endoscope image set, the preprocessing operation comprises the steps of carrying out image augmentation on each endoscope image in the endoscope image set to generate an endoscope image set after image augmentation, carrying out mask processing on the endoscope image set after image augmentation to obtain corresponding mask images, and the mask images comprise target color blocks in the endoscope image after image augmentation; fusing and splicing each mask image and the corresponding endoscope image after image augmentation to obtain a target image set;
the model training module is used for training to obtain an image classification model based on the target image set and the corresponding classification labels;
and the image classification module is used for inputting the endoscope images to be classified into the image classification model after the preprocessing operation is carried out, so as to obtain classification results.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 7 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311221660.8A CN117496214A (en) | 2023-09-20 | 2023-09-20 | Medical image classification method, medical image classification device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311221660.8A CN117496214A (en) | 2023-09-20 | 2023-09-20 | Medical image classification method, medical image classification device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117496214A true CN117496214A (en) | 2024-02-02 |
Family
ID=89683669
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311221660.8A Pending CN117496214A (en) | 2023-09-20 | 2023-09-20 | Medical image classification method, medical image classification device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117496214A (en) |
-
2023
- 2023-09-20 CN CN202311221660.8A patent/CN117496214A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI777092B (en) | Image processing method, electronic device, and storage medium | |
| Zhang et al. | Ensnet: Ensconce text in the wild | |
| CN110674804A (en) | Text image detection method and device, computer equipment and storage medium | |
| CN109919869A (en) | A kind of image enchancing method, device and storage medium | |
| US20220084165A1 (en) | System and method for single-modal or multi-modal style transfer and system for random stylization using the same | |
| Shete et al. | TasselGAN: An application of the generative adversarial model for creating field-based maize tassel data | |
| CN114155365A (en) | Model training method, image processing method and related device | |
| CN111353956A (en) | Image restoration method and device, computer equipment and storage medium | |
| CN114444565A (en) | Image tampering detection method, terminal device and storage medium | |
| CN115719416A (en) | Disconnecting link state identification method and device, computer equipment and storage medium | |
| CN110163095B (en) | Loop detection method, loop detection device and terminal equipment | |
| Bugeau et al. | Influence of color spaces for deep learning image colorization | |
| CN117409330A (en) | Aquatic vegetation identification method, aquatic vegetation identification device, computer equipment and storage medium | |
| CN113191189A (en) | Face living body detection method, terminal device and computer readable storage medium | |
| CN109658360B (en) | Image processing method and device, electronic equipment and computer storage medium | |
| CN116229130A (en) | Type identification method and device for blurred image, computer equipment and storage medium | |
| Huang et al. | Object‐Level Remote Sensing Image Augmentation Using U‐Net‐Based Generative Adversarial Networks | |
| CN117496214A (en) | Medical image classification method, medical image classification device, computer equipment and storage medium | |
| CN115713585A (en) | Texture image reconstruction method and device, computer equipment and storage medium | |
| Gandomkar et al. | Determining image processing features describing the appearance of challenging mitotic figures and miscounted nonmitotic objects | |
| US20240212239A1 (en) | Logo Labeling Method and Device, Update Method and System of Logo Detection Model, and Storage Medium | |
| CN115082345A (en) | Image shadow removing method and device, computer equipment and storage medium | |
| CN114820674A (en) | Arc contour extraction method, device, computer equipment and storage medium | |
| CN113032622A (en) | Novel medical video image acquisition and data management system | |
| Singla et al. | A Novel Approach to Image Colorization Algorithm using Convolutional Neural Networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |