CN114897914B - Semi-supervised CT image segmentation method based on countermeasure training - Google Patents
Semi-supervised CT image segmentation method based on countermeasure training Download PDFInfo
- Publication number
- CN114897914B CN114897914B CN202210259206.0A CN202210259206A CN114897914B CN 114897914 B CN114897914 B CN 114897914B CN 202210259206 A CN202210259206 A CN 202210259206A CN 114897914 B CN114897914 B CN 114897914B
- Authority
- CN
- China
- Prior art keywords
- image
- generator
- discriminator
- loss
- mask
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30061—Lung
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Apparatus For Radiation Diagnosis (AREA)
Abstract
The invention relates to the technical field of image processing, in particular to a semi-supervised CT image segmentation method based on countermeasure training, which comprises the following steps: firstly, acquiring a three-dimensional CT image of a lung, and establishing a diseased data set marked by a voxel level, an unlabeled diseased data set and a healthy data set; then, CT images in the three data sets are sequentially input to a generator and a divider, and a synthesized health image and a division mask are respectively obtained. Then splicing the mask area of the synthesized healthy image and the anti-mask area of the input image to obtain a recovered healthy image; the value of the mask area of the synthesized healthy image is set to 0 to obtain an anti-mask image. Finally, the two discriminators respectively monitor the restored health image and the anti-mask image in a countermeasure training mode to improve the segmentation of the segmenter and the effect of the generator. The invention realizes that a segmentation model with accurate performance is obtained by using a small number of voxel-level labeled samples, and the specially designed segmenter effectively improves the characteristic representation capability.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a semi-supervised CT image segmentation method based on countermeasure training.
Background
The background technology relates to: lung CT image segmentation and attention mechanisms.
1) Lung CT image segmentation
Pulmonary CT has a higher accuracy in diagnosing pulmonary diseases, and has been used for many tasks for pulmonary medical imaging using deep learning techniques. However, many previous works have been directed to classification tasks of lung images, which cannot reveal the location and size of lesion areas like segmentation tasks. Segmentation of the affected area using CT images can help radiologists to better quantify the lesion area. Quantitative analysis of the segmentation mask of the lesion area may yield a series of meaningful diagnostic results related to the lung.
Currently, deep learning has been primarily applied to CT segmentation tasks of the lung, but this approach requires a large amount of training data. Clinically, it is very difficult to annotate three-dimensional CT image data of the lung, and it takes more than 3 hours for radiologists to annotate a CT volume of the lung. Therefore, the scarcity of voxel-level labeled lung CT images is an important issue. Image synthesis and data enhancement can alleviate the problem of voxel level label loss. Active learning and self-learning can provide pseudo tags for unlabeled data to optimize the segmentation model. Generating a countermeasure network (GAN) and Class Activation Map (CAM) deals with the absence of pixel-level or voxel-level labels by training a segmentation model using weak labels, such as data of slice-level labels or volume-level labels with CT. However, training a model using pseudo tags may introduce noise that affects the performance of the model, and training the model only on data with weak tags tends to fail to achieve a satisfactory segmentation mask.
2) Attention mechanism
The basic idea of the attention mechanism in computer vision is to let the system learn to pay attention to and be able to ignore irrelevant information and concentrate on critical information. In recent years, attention models have been widely used in various fields such as image processing, speech recognition, and natural language processing. As deep learning progresses to today, it becomes increasingly important to build a neural network with a mechanism of attention. On the one hand, the neural network can autonomously learn the attention mechanism, and on the other hand, the attention mechanism can help understand the world seen by the neural network. In recent years, research work on a combination of deep learning and visual attention mechanisms has been mostly focused on using masks to form the attention mechanism. The principle of masking is to identify key features in the image data by another layer of new weights. Through learning and training, the deep neural network can learn the region needing to be noted in each new image.
Disclosure of Invention
The invention aims to solve the problem of label shortage of voxel level CT image data, and provides a semi-supervised CT image segmentation method based on countermeasure training.
The specific technical scheme for realizing the aim of the invention is as follows:
a semi-supervised CT image segmentation method based on countermeasure training, the method comprising the steps of:
step 1) obtaining a three-dimensional CT image of the lung, establishing a diseased data set V marked by voxel level, an unlabeled diseased data set D and a healthy data set H, and setting the marked diseased image as I v E V, unlabeled diseased image is I d E D, health image I h ∈H；
Step 2) for any one of the input images in the dataset, performing image preprocessing operations including image cropping, resampling and normalization to obtain a mapImage I i Inputting it into the generator to obtain the reconstruction result I of the generator g The goal of the generator is to compare I i Reconstruction into a health image I g The method comprises the steps of carrying out a first treatment on the surface of the Will I i Input to the divider to obtain a division mask
For the voxel-level labeled diseased dataset V, the segmentation mask is supervised using the real labels Y in dataset V>
Step 3) segmentation mask
The inverse is inverted to obtain an inverse mask->
Step 4) taking the generator result I g Mask prediction area of (i) that is
Part 1, < >>
And input image I i The inverse mask prediction region of->
Part 1, < >>
Adding the two images by elements to obtain a composite image I p ；
Step 5) taking the generator result I g Is the inverse mask prediction region of (1), i.e
And input image I i Is the inverse mask prediction region of->
Respectively obtain images I gh And image I ih ；
Step 6) constructing a semi-supervised CT image segmentation model, wherein the model comprises a generator G, a segmenter S and a discriminator D 1 Sum discriminator D 2 The method comprises the steps of carrying out a first treatment on the surface of the The input of the model is simultaneously input to a generator G and a divider S, the divider S obtains a division mask of the CT image, a voxel with a value of 1 represents a lesion area, and a voxel with a value of 0 represents a healthy area; the generator G is used for inputting a CT image, generating a CT image after the CT image is restored, and then combining the generated image with the segmentation mask and the model input image to generate a restored health image and an anti-mask image so as to approximate a real image to deceive the discriminator D 1 Sum discriminator D 2 The method comprises the steps of carrying out a first treatment on the surface of the Distinguishing device D 1 And D 2 The function of (a) is to judge whether the input image is from a real image; by competing with each other, generator G and arbiter D 1 And D 2 The weight is optimized in an iterative mode, so that the performance is improved;
the semi-supervised CT image segmentation model comprises four losses, namely a supervision loss, a reconstruction loss, a discrimination loss 1 and a discrimination loss 2; the supervision loss takes effect when the input CT image belongs to a diseased data set marked on a voxel level; the reconstruction loss is the mean square error MSE loss between the model input and the generator G results to supervise the generator G to produce a more realistic healthy CT image; judging that the loss 1 is a counterloss, judging that the input image is a real image or a synthesized image, and supervising the quality of the restored healthy image; judging that the loss 2 is a countering loss, judging whether the input image is from a real model input image or an image generated by the generator G, so as to monitor the quality of the anti-mask image; continuously optimizing the loss by using a model optimization method based on countermeasure training until the loss converges;
and 7) after model training is completed, giving a three-dimensional CT image to be segmented, and inputting the three-dimensional CT image to a segmenter S to obtain a three-dimensional segmentation mask of the voxel level of the image.
The step 1) specifically comprises the following steps:
step a1: acquiring a three-dimensional CT image sample through professional equipment to obtain an original data set;
step a2: dividing an original data set into a disease data set and a health data set, randomly sampling one fifth of the disease data, and marking by a manual expert to serve as a voxel-level marked disease data set V; the rest image data are used as an unlabeled diseased data set D, and all healthy image data form a healthy data set H;
step a3: sequentially inputting images in the diseased data set V, the unlabeled diseased data set D and the healthy data set H marked by voxel level to a divider and a generator, and marking an input CT image of a model as I i 。
The image preprocessing operation in the step 2) specifically includes:
(1) Image cropping
Image cropping cuts a three-dimensional medical image to its non-zero region, i.e., searching for a smallest three-dimensional bounding box in the image, the value outside the bounding box region being 0, and cropping the image using this bounding box;
(2) Resampling
Resampling can solve the problem of inconsistent actual spatial sizes represented by different voxel images in a three-dimensional CT dataset. In order to unify the resolution sizes of all CT images, resampling is used to scale the resolutions of different CT images so as to unify the resolutions to 0.5mm by 0.5mm;
(3) Normalization
In order to make each image have the same distribution of gray values, the minimum and maximum values of the CT image gray values are set to 300 and 3000; a value with a gray value less than 300 is increased to 300, and a value with a gray value greater than 3000 is decreased to 3000; the voxel values of the CT image are then normalized to values between [0,1], and then scaled to the [0,255] interval.
Generator G and arbiter D as described in step 6) 1 And D 2 The design of (a) follows the structure of the CycleGAN, and in addition, the full connection layer is used as a classification network in the last layer to obtain the final discrimination result, namely 1 or 0, wherein 1 represents the discriminatorThe input image is considered to be a true image, and 0 represents the image that the arbiter considers to be a generated or synthesized image.
The splitter S described in step 6) is based on 3DU-Net, and a feature enhancement module is designed in the splitter S, for enhancing the feature representation of the encoder, where the feature enhancement module includes channel attention and spatial attention. The method comprises the steps of carrying out a first treatment on the surface of the In order to balance between memory usage and segmentation accuracy, downsampling by a factor of 2 is used in the segmenter S; the divider S adopts a structure of pyramid pooling of dense hollow spaces, can use expansion rates with different sizes to combine features with different scales, and well realizes the reuse of the features; the expansion convolution (dilated convolution) in the divider S uses an expansion ratio of 3,6, 12; in order to improve the quality of the learned features, the features with the same scale are fused in a shortcut connection mode; furthermore, features of the same scale are fused by means of a quick connection, which improves the quality of the learned features. The number of channels is set to 16 in consideration of the memory usage of the 3D segmentation model.
Step 6) the model optimization method based on countermeasure training adopts a gradient descent algorithm based on an Adam optimizer to iteratively optimize the segmenter S, the generator G and the discriminant D 1 Sum discriminator D 2 Specifically comprising the following steps:
(1) Initialization arbiter D 1 And D 2 Weight parameters of (2); setting the iteration number iter to 1, and setting the maximum iteration number iter max ；
(2) Optimizing the segmenter S and the generator G: freezing discriminator D 1 And D 2 The parameters of the defreezing segmenter S and the generator G, calculating the loss and optimizing the model, and adding one to the item;
(3) Optimization discriminator D 1 : freezing divider S, generator G and discriminator D 2 Parameter of (D), thawing discriminator D 1 Calculating the loss and optimizing the model, and adding one to the item;
(4) Optimization discriminator D 2 : freezing divider S, generator G and discriminator D 1 Parameter of (D), thawing discriminator D 2 Is superior in calculating the parameters of the lossModeling, adding one to iter;
repeating (1) - (4), with the iter being greater than the iter max Or until the loss converges.
The design of the generator and the discriminator of the invention follows the structure of the CycleGAN, and in addition, the classifier is connected by using a full connection layer in the last layer to obtain the final discriminating result. The spectrum normalization operation is added to the arbiter. Regarding the memory usage of the 3D image, the number of basic channels is reduced from 64 to 16.
Compared with the prior art, the invention has the beneficial effects that:
1. the invention provides a semi-supervised CT image segmentation method based on countermeasure training, which is used for segmenting focus areas in a three-dimensional CT image of a lung. Compared with a two-dimensional divider, the three-dimensional divider can combine the image interlayer information, so that the continuity of change between interlayer image masks can be ensured. The invention can be trained using only a small number of voxel-level annotated data and the rest of the unlabeled data.
2. The invention optimizes the generator, the divider and the discriminator in the model in a countermeasure training mode, and enables the divider to learn the characteristic information of healthy lungs and focuses from the annotated data of the voxel level so as to gradually improve the segmentation performance of the model, thereby greatly reducing the number of voxel level labels required for training the model.
3. In order to better address the problem of low contrast between the affected areas of the lungs and normal tissue, the present invention contemplates a feature enhancement module to address the ambiguous boundary. In particular, channel attention is used to implicitly enhance contrast between features, highlighting boundary information of focal regions, spatial attention generating spatial attention is intended to highlight important regions. The feature enhancement module effectively enhances the feature representation of the lesion area by fusing the features.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a schematic diagram of an anti-semi-supervised model of the present invention;
FIG. 3 is a schematic view of a divider of the present invention;
FIG. 4 is a schematic diagram of a feature enhancement module of the present invention that combines channels and spatial attention.
Detailed Description
The present invention will be described in further detail with reference to specific embodiments and drawings. The procedures, conditions, experimental methods, etc. for carrying out the present invention are common knowledge and common knowledge in the art, except for the following specific references, and the present invention is not particularly limited.
The whole process of the present invention includes steps 1) -7), please refer to fig. 1 and 2. Fig. 1 is a flowchart of a semi-supervised CT image segmentation method based on countermeasure training according to the present invention, and fig. 2 is a schematic diagram of a countermeasure semi-supervised model.
The process of the invention comprises the following steps:
step 1) acquiring a three-dimensional CT image of a lung, establishing a diseased data set V marked by a voxel level, an unlabeled diseased data set D, a healthy data set S, and setting the marked diseased image as I v E V, unlabeled diseased image is I d E D, health image I h ∈S。
Step 2) for all input data, input to the generator to obtain a healthy reconstructed version I g Input to the divider to obtain a division mask
Diseased dataset I for voxel level labeling v Supervision of segmentation mask using real tag Y in dataset V>
Step 3) segmentation mask
The reverse is carried out to obtain->
Step 4) taking the generator result I g Mask prediction area (i.e
Part 1, < >>
) And input image I i Is the inverse mask prediction region of (i.e + ->
Part 1, < >>
) Adding the two images by elements to obtain a composite image I p 。
Specifically, an unlabeled disease data is set as I d E D, a piece of voxel-level labeled diseased data is I v E V, a piece of healthy image data is I h e.H. The lesion area predicted by the segmenter is replaced with the corresponding area reconstructed by the generator, while the uninfected area is preserved, called pseudo-healthy image I p . The pseudo-health image is calculated by the following formula:
1)
wherein φ is a function that generates a pseudo-healthy image;
is the probability segmentation mask predicted by the segmenter S; image I generated by a generator g ＝G(I d ；θ G )；θ S And theta G The learnable parameters of the segmenter S and the generator G, respectively.
Step 5) taking the generator result I g Is the inverse mask prediction region of (1)
And input image I i Is->
Respectively obtain I gh And I ih 。
Because there is no supervisory signal to constrain I g Which may lead to problems in that the quality of the generated image is uncontrollable, for image parts outside the lesion mask Y. Although they are not used to form the composite image, this affects the performance of the generator, which becomes a bottleneck to improving the final performance of the segmenter. The invention uses the reconstruction result of the generator to obtain I from the healthy area predicted by the divider g Generated health area I gh And I i Healthy area I in (1) ih ：
2)
Then I gh And I ih Is input to a discriminator D 2 ，I gh Known as synthetic healthy area image and I ih Known as a real healthy area image.
Step 6) constructing a three-dimensional countermeasure semi-supervision model, and iterating and optimizing a generator, a divider and a discriminator D 1 Sum discriminator D 2 Until the total loss converges. The countermeasure training can enable the model to train in a semi-supervised learning mode, namely the network can learn information which is beneficial to segmentation accuracy from unlabeled data.
To completely deceptive discriminant D 1 And D 2 The segmenter needs to segment all the infected areas and the generator needs to generate pseudo-healthy images in both the segmented predicted lesion area and the healthy area. In contrast, arbiter D 1 For distinguishing and real health image I d Discriminator D 2 To distinguish synthetic healthy area image I gh And a real healthy area image I ih . The model was trained in challenge mode by the following maximum and minimum, the challenge losses were as follows:
wherein θ is G And theta S The learnable parameters of generator G and segmenter S, respectively;
and->
Respectively the discriminant D 1 Sum discriminator D 1 Is a learning parameter of (a); />
Yang->
Is a loss function.
Marking the composite image as 0, true health image objective function
From the following components
5)
6) Wherein, the liquid crystal display device comprises a liquid crystal display device,
is D 1 Is a predicted result of (a); i p Calculated from formula (1); />
Representation discriminator D 1 Is a learning parameter of (a); />
And->
Representing mathematical expectations。
The synthetic healthy area is marked 0 and the real healthy area is marked 1, wherein the objective function
The calculation is as follows:
7)
wherein, the liquid crystal display device comprises a liquid crystal display device,
is a discriminator D 2 Is a predicted result of (a); image I gh And I sh Calculated from formulas 2 and 3, respectively; />
Representation D 2 Is provided.
The divider of the present invention is shown in fig. 3. Fig. 3 is a schematic diagram of a divider.
The present invention proposes a feature enhancement module that uses channel attention and spatial attention adaptive optimization features to optimize a segmenter, referring to fig. 4, fig. 4 is a schematic diagram of a feature enhancement module that combines channel and spatial attention. Specifically, the channel attention module learns global parameters to highlight useful boundary information. The spatial attention module calculates an attention weighting map of the lung region. Channel attention, spatial attention can improve the representational ability of features of the region of interest, thereby enabling highlighting important features and suppressing unnecessary features. Given intermediate feature mapping
As input, where C is the number of channels, D is the feature dimension, H is the image height, and W is the image width. The feature enhancement module sequentially deduces 1D channel attention +.>
And 2D spatial attention map->
The whole attention process can be summarized as:
channel attention: to enhance the contrast of features, spatial information on the feature map is first aggregated using an averaging pool and max pooling operation, generating two different spatial context descriptors, an averaging pooled feature and a max pooled feature. These two descriptors are forwarded to a multi-layer perceptron network of shared parameters, and the result is finally passed through a sigmoid function to produce the channel attention pattern M c (F) A. The invention relates to a method for producing a fibre-reinforced plastic composite The shared network is formed by a multi-layer perceptron (MLP) with a hidden layer. Channel attention mainly comprises an average pooling operation, a maximum pooling operation, a multi-layer perceptron sharing parameters and a sigmoid function. The intermediate optimization feature of channel attention generation is that
The calculation process is as follows:
8)
wherein F is the input feature of channel attention; sigma represents a sigmoid activation function; avgPool is an average pooling function; maxPool is the maximum pooling function.
Spatial attention: the purpose of spatial attention is to discard unimportant features and highlight features of interest that are beneficial in segmenting the covd 19 infection. Generating spatial attention map M using spatial relationships between features s (F') which is complementary to the channel attention. To calculate spatial attention, the average pooling and maximum pooling operations are first applied along the channel axis and concatenated to generate a valid feature descriptor. Applying pooling operations along the channel axis can effectively highlight feature regions of interest. On the concatenated feature descriptors, a convolutional layer is used to generate a spatial attention pattern that encodes the place information that needs to be emphasized or suppressed. The final optimization feature for spatial attention generation is F ",
wherein σ represents a sigmoid activation function; conv is a convolution operation of size 7x7x 7;
representing per-element multiplication. F 'is obtained by element-wise multiplication between F' and the attention map.
Due to the synthesized image I p Image I generated with prediction mask sum generator g Related, therefore
The gradients of (a) can be fed back to the divider S and generator G to optimize the parameters of the two modules. In addition, a basic segmentation penalty is added>
The difference between the output Y of S for measuring a small number of voxel level label samples and the true mask GT:
where CEL is cross entropy loss; />
Is voxel-level label data I v Is used for the prediction mask of (a); y is Y v Is voxel-level label data I v Is a real tag of (a). Prediction mask->
Wherein S (·) is a divider, θ S A learnable parameter representing the segmenter.
In order to improve the stability of the countermeasure in step 6), the present invention further sets an auxiliary constraint. First, health image I h Input into the divider and generator. Health image data I h Is (I) h ；θ S ) And health image data I h Is the true label Y of (2) h Cross entropy loss between is added to
Second, to further improve the performance of the generator, the method uses reconstruction loss
To constrain the output of the generator:
wherein MSE (·) is a mean square error function; g (I) h ；θ G ) Representation I h Inputting the reconstruction result obtained after the input to the generator G; θ G Is a learnable parameter of the generator G. Further, also to
Introducing additional loss function->
Wherein G (I) d ；θ G ) Representation I d Inputting the reconstruction result obtained after the input to the generator G;
is a discriminator D 1 Is provided.
By combining I d Is to generate an image I g Input D 1 To further improve the generation effect. When the input of the divider S is a healthy image, no forward propagation process for synthesis and discrimination is required, nor is calculation performed
In addition, the original disease image I d Input to a discriminator D 1 In order to maintain the discriminant D during training 1 Sensitivity and distinguishability to the lesion signal, the dropout rate was set to be fixed at 0.5. Constraint loss function +.>
Added to->
Distinguishing device D 1 The goal of (a) is to be able to distinguish between images of a patient and a healthy person:
in summary, the loss function is extended by adding four new losses as auxiliary constraints
And->
The final objective function is defined as follows:
wherein lambda is S Is used for balancing
And->
Is a super parameter of (a).
And 7) based on the trained segmenter, realizing CT image segmentation.
The protection of the present invention is not limited to the above embodiments. Variations and advantages that would occur to one skilled in the art are included in the invention without departing from the spirit and scope of the inventive concept, and the scope of the invention is defined by the appended claims.
Claims (6)
1. A semi-supervised CT image segmentation method based on countermeasure training is characterized by comprising the following steps:
step 1) obtaining a three-dimensional CT image of the lung, establishing a diseased data set V marked by voxel level, an unlabeled diseased data set D and a healthy data set H, and setting the marked diseased image as I v E V, unlabeled diseased image is I d E D, health image I h ∈H；
Step 2) for any one of the input images in the dataset, performing image preprocessing operations including image cropping, resampling and normalization to obtain an image I i Inputting it into the generator to obtain the reconstruction result I of the generator g The goal of the generator is to compare I i Reconstruction into a health image I g The method comprises the steps of carrying out a first treatment on the surface of the Will I i Input to the divider to obtain a division mask
For the voxel-level labeled diseased dataset V, the segmentation mask is supervised using the real labels Y in dataset V>
Step 3) segmentation mask
The inverse is inverted to obtain an inverse mask->
Step 4) taking the generator result I g Mask prediction area of (i) that is
Part 1, < >>
And input image I i The inverse mask prediction region of->
Part 1, < >>
Adding the two images by elements to obtain a composite image I p ；
Step 5) taking the generator result I g Is the inverse mask prediction region of (1), i.e
And input image I i Is the inverse mask prediction region of->
Respectively obtain images I gh And image I ih ；
Step 6) constructing a semi-supervised CT image segmentation model, wherein the model comprises a generator G, a segmenter S and a discriminator D 1 Sum discriminator D 2 The method comprises the steps of carrying out a first treatment on the surface of the The input of the model is simultaneously input to a generator G and a divider S, the divider S obtains a division mask of the CT image, a voxel with a value of 1 represents a lesion area, and a voxel with a value of 0 represents a healthy area; the generator G is used for inputting a CT image, generating a CT image after the CT image is restored, and then combining the generated image with the segmentation mask and the model input image to generate a restored health image and an anti-mask image so as to approximate a real image to deceive the discriminator D 1 Sum discriminator D 2 The method comprises the steps of carrying out a first treatment on the surface of the Distinguishing device D 1 And D 2 The function of (a) is to judge whether the input image is from a real image; by passing throughCompeting with each other, generator G and arbiter D 1 And D 2 The weight is optimized in an iterative mode, so that the performance is improved;
the semi-supervised CT image segmentation model comprises four losses, namely a supervision loss, a reconstruction loss, a discrimination loss 1 and a discrimination loss 2; the supervision loss takes effect when the input CT image belongs to a diseased data set marked on a voxel level; the reconstruction loss is the mean square error MSE loss between the model input and the generator G results to supervise the generator G to produce a more realistic healthy CT image; judging that the loss 1 is a counterloss, judging that the input image is a real image or a synthesized image, and supervising the quality of the restored healthy image; judging that the loss 2 is a countering loss, judging whether the input image is from a real model input image or an image generated by the generator G, so as to monitor the quality of the anti-mask image; continuously optimizing the loss by using a model optimization method based on countermeasure training until the loss converges;
and 7) after model training is completed, giving a three-dimensional CT image to be segmented, and inputting the three-dimensional CT image to a segmenter S to obtain a three-dimensional segmentation mask of the voxel level of the image.
2. The method for semi-supervised CT image segmentation based on countermeasure training as set forth in claim 1, wherein the step 1) specifically includes:
step a1: acquiring a three-dimensional CT image sample through professional equipment to obtain an original data set;
step a2: dividing an original data set into a disease data set and a health data set, randomly sampling one fifth of the disease data, and marking by a manual expert to serve as a voxel-level marked disease data set V; the remaining image data are used as unlabeled diseased data set D, and all healthy image data form a healthy data set H.
3. The method for semi-supervised CT image segmentation based on countermeasure training as set forth in claim 1, wherein the image preprocessing operation in step 2) specifically includes:
(1) Image cropping
Image cropping cuts a three-dimensional medical image to its non-zero region, i.e., searching for a smallest three-dimensional bounding box in the image, the value outside the bounding box region being 0, and cropping the image using this bounding box;
(2) Resampling
Scaling the different CT image resolutions using resampling to unify the resolutions to 0.5mm by 0.5mm;
(3) Normalization
Setting the minimum value and the maximum value of the gray value of the CT image to 300 and 3000; a value with a gray value less than 300 is increased to 300, and a value with a gray value greater than 3000 is decreased to 3000; the voxel values of the CT image are then normalized to obtain values between [0,1], and then the values between [0,1] are scaled to the [0,255] interval.
4. The method for semi-supervised CT image segmentation based on countermeasure training as recited in claim 1, wherein the generator G and the discriminator D of step 6) 1 And D 2 The design of (1) follows the structure of CycleGAN, and in addition, the fully connected layer is used as a classification network in the last layer to obtain the final discrimination result, namely 1 or 0, wherein 1 represents that the input image is considered to be a real image by the discriminator, and 0 represents that the input image is considered to be a generated or synthesized image by the discriminator.
5. The method for semi-supervised CT image segmentation based on countermeasure training according to claim 1, wherein the segmenter S of step 6) is based on 3DU-Net, and a feature enhancement module is designed in the segmenter S for enhancing the feature representation of the encoder, wherein the feature enhancement module includes channel attention and spatial attention; in order to balance between memory usage and segmentation accuracy, downsampling by a factor of 2 is used in the segmenter S; the divider S adopts a structure of pyramid pooling of dense hollow spaces, can use expansion rates with different sizes to combine features with different scales, and well realizes the reuse of the features; the expansion convolution (dilated convolution) in the divider S uses an expansion ratio of 3,6, 12; features with the same scale are fused in a shortcut connection mode; the number of channels was set to 16.
6. The method for semi-supervised CT image segmentation based on countermeasure training as set forth in claim 1, wherein the model optimization method based on countermeasure training of step 6) iteratively optimizes the segmenter S, the generator G, the discriminant D using a gradient descent algorithm based on Adam optimizer 1 Sum discriminator D 2 Specifically comprising the following steps:
(1) Initialization arbiter D 1 And D 2 Weight parameters of (2); setting the iteration number iter to 1, and setting the maximum iteration number iter max ；
(2) Optimizing the segmenter S and the generator G: freezing discriminator D 1 And D 2 The parameters of the defreezing segmenter S and the generator G, calculating the loss and optimizing the model, and adding one to the item;
(3) Optimization discriminator D 1 : freezing divider S, generator G and discriminator D 2 Parameter of (D), thawing discriminator D 1 Calculating the loss and optimizing the model, and adding one to the item;
(4) Optimization discriminator D 2 : freezing divider S, generator G and discriminator D 1 Parameter of (D), thawing discriminator D 2 Calculating the loss and optimizing the model, and adding one to the item;
repeating (1) - (4), with the iter being greater than the iter max Or until the loss converges.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210259206.0A CN114897914B (en) | 2022-03-16 | 2022-03-16 | Semi-supervised CT image segmentation method based on countermeasure training |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210259206.0A CN114897914B (en) | 2022-03-16 | 2022-03-16 | Semi-supervised CT image segmentation method based on countermeasure training |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114897914A CN114897914A (en) | 2022-08-12 |
| CN114897914B true CN114897914B (en) | 2023-07-07 |
Family
ID=82715871
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210259206.0A Active CN114897914B (en) | 2022-03-16 | 2022-03-16 | Semi-supervised CT image segmentation method based on countermeasure training |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114897914B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115187819B (en) * | 2022-08-23 | 2023-05-16 | 北京医准智能科技有限公司 | Training method and device for image classification model, electronic equipment and storage medium |
| CN115861252B (en) * | 2022-12-14 | 2023-09-22 | 深圳技术大学 | Semi-supervised medical image organ segmentation method based on countermeasure learning strategy |
| CN116739890B (en) * | 2023-06-26 | 2024-06-14 | 强联智创（北京）科技有限公司 | Method and equipment for training generation model for generating healthy blood vessel image |
| CN117078703B (en) * | 2023-08-30 | 2024-05-28 | 深圳扬奇医芯智能科技有限公司 | CT image segmentation method and system based on MRI guidance |
| CN117095395B (en) * | 2023-10-19 | 2024-02-09 | 北京智源人工智能研究院 | Model training method and device for heart ultrasonic image segmentation and segmentation method |
| CN117726642B (en) * | 2024-02-07 | 2024-05-31 | 中国科学院宁波材料技术与工程研究所 | High reflection focus segmentation method and device for optical coherence tomography image |
| CN117973496B (en) * | 2024-02-19 | 2024-07-05 | 苏州慧维智能医疗科技有限公司 | Training method, system and application of local lesion medical image generation model |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109949317B (en) * | 2019-03-06 | 2020-12-11 | 东南大学 | Semi-supervised image example segmentation method based on gradual confrontation learning |
| US11514694B2 (en) * | 2019-09-20 | 2022-11-29 | Samsung Electronics Co., Ltd. | Teaching GAN (generative adversarial networks) to generate per-pixel annotation |
| CN112150478B (en) * | 2020-08-31 | 2021-06-22 | 温州医科大学 | Method and system for constructing semi-supervised image segmentation framework |
| CN112419327B (en) * | 2020-12-10 | 2023-08-04 | 复旦大学附属肿瘤医院 | Image segmentation method, system and device based on generation countermeasure network |
| CN112734764A (en) * | 2021-03-31 | 2021-04-30 | 电子科技大学 | Unsupervised medical image segmentation method based on countermeasure network |
-
2022
- 2022-03-16 CN CN202210259206.0A patent/CN114897914B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN114897914A (en) | 2022-08-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114897914B (en) | Semi-supervised CT image segmentation method based on countermeasure training | |
| Li et al. | Deep learning attention mechanism in medical image analysis: Basics and beyonds | |
| Cong et al. | Boundary guided semantic learning for real-time COVID-19 lung infection segmentation system | |
| Ye et al. | Medical image diagnosis of prostate tumor based on PSP-Net+ VGG16 deep learning network | |
| WO2022205500A1 (en) | Method for constructing registration model for non-rigid multimodal medical image, and application thereof | |
| Xu et al. | Semi-supervised learning for fetal brain MRI quality assessment with ROI consistency | |
| Yang et al. | RADCU-Net: Residual attention and dual-supervision cascaded U-Net for retinal blood vessel segmentation | |
| CN117036288A (en) | Tumor subtype diagnosis method for full-slice pathological image | |
| Shi et al. | Dual dense context-aware network for hippocampal segmentation | |
| Liu et al. | AHU-MultiNet: Adaptive loss balancing based on homoscedastic uncertainty in multi-task medical image segmentation network | |
| Wang et al. | Automatic measurement of fetal head circumference using a novel GCN-assisted deep convolutional network | |
| Kong et al. | Data enhancement based on M2-Unet for liver segmentation in Computed Tomography | |
| Zhang et al. | GH-DDM: the generalized hybrid denoising diffusion model for medical image generation | |
| Li et al. | CorrDiff: Corrective Diffusion Model for Accurate MRI Brain Tumor Segmentation | |
| Wang et al. | SURVS: A Swin-Unet and game theory-based unsupervised segmentation method for retinal vessel | |
| Liu et al. | MRL-Net: multi-scale representation learning network for COVID-19 lung CT image segmentation | |
| He et al. | FRFCM clustering segmentation method for medical MR image feature diagnosis | |
| CN116665896A (en) | Model building method for predicting breast cancer axillary lymph node metastasis | |
| Wang et al. | Reliable joint segmentation of retinal edema lesions in oct images | |
| Zhang et al. | Nucleus image segmentation method based on GAN network and FCN model | |
| He et al. | Fetal cardiac ultrasound standard section detection model based on multitask learning and mixed attention mechanism | |
| Huang et al. | Efficient click-based interactive segmentation for medical image with improved Plain-ViT | |
| Gao et al. | MMGan: a multimodal MR brain tumor image segmentation method | |
| Zhang et al. | RotU-Net: An Innovative U-Net With Local Rotation for Medical Image Segmentation | |
| Li et al. | SSCK-Net: Spine segmentation in MRI based on cross attention and key-points recognition-assisted learner |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |