CN114937022A

CN114937022A - Novel coronary pneumonia disease detection and segmentation method

Info

Publication number: CN114937022A
Application number: CN202210608148.8A
Authority: CN
Inventors: 郭菲; 张南星; 李雪健; 马世强; 唐继军
Original assignee: Tianjin University
Current assignee: Tianjin University
Priority date: 2022-05-31
Filing date: 2022-05-31
Publication date: 2022-08-23
Anticipated expiration: 2042-05-31
Also published as: CN114937022B

Abstract

The invention discloses a novel coronary pneumonia disease detection and segmentation method, which comprises the following steps: s1, acquiring COVID-19lung CT lesion data, preprocessing the data, finding out the slices of the beginning and the end of the lung, expanding the slices outwards, and performing slicing processing to remove the slices without the focus, wherein the threshold range of the image is (-1000, 500); s2, processing the data, including randomly rotating the image, horizontally and vertically turning the image, histogram equalization technology and image normalization processing; s3, constructing an N-Net network structure; s4, optimizing the blocks of the convolution of two 3x3 of each layer in the N-Net, and simultaneously optimizing the blocks of the convolution of two 3x3 of each layer in the U-Net; and S5, merging the U-Net and the N-Net network structures into a final network structure NU-Net. The invention adopts the novel coronary pneumonia disease detection and segmentation method, and realizes effective segmentation of lung lesions by extracting characteristics of CT images of lungs infected with COVID-19.

Description

Novel coronary pneumonia disease detection and segmentation method

Technical Field

The invention relates to the technical field of computer deep learning image segmentation, in particular to a novel coronary pneumonia disease detection segmentation method.

Background

The widespread worldwide distribution of coronavirus disease (COVID-19) poses a health risk for survival throughout the world. Automated detection of lung infections via Computed Tomography (CT) images offers great potential for strengthening the traditional healthcare strategy for COVID-19. To control the spread of the disease, it is imperative to screen large numbers of suspected cases for appropriate quarantine and treatment.

In determining the severity of COVID-19, pulmonary abnormalities are a key factor in the clinical management of patients, potentially facilitating more timely and personalized medical intervention. Quantification of lesions may further provide tracking of disease progression and response to treatment strategies. Thus, improved treatment of COVID-19 begins with a clearer understanding of the disease state of the patient, and must include accurate identification, delineation, and quantification of lung lesions and disease phenotypes and patterns.

Currently, disease detection segmentation of the novel coronary pneumonitis COVID-19 still has many challenges, and although pulmonary imaging is crucial for early identification and treatment, segmentation of the infected region from CT slices faces several challenges, including high variation in the infection characteristics, low intensity contrast of infection to normal tissue. In addition, deep learning requires a large data set to obtain more efficient features, while it is impractical to collect a large amount of data in a short time, with a relatively small data set, which can lead to overfitting of the model, preventing training of the deep model.

Although UNet networks are very popular networks in the medical segmentation field in recent years, it has been found through research that UNet networks have poor performance in detecting fine tissue structures and cannot accurately segment boundary regions, which is caused by a large receptive field in such an incomplete network as UNet. With the increase of the network depth, the receptive field is larger, so that the network can pay more attention to high-level semantic information, and can learn less low-level features, but a small organization structure needs a smaller receptive field to obtain, and even if the UNet has a jump-link structure, the minimum receptive field is limited to the first-level network.

Disclosure of Invention

The invention aims to provide a novel coronary pneumonia disease detection and segmentation method, which solves the problems of poor performance and incapability of accurately segmenting boundary regions when a U-Net network detects a fine tissue structure, and realizes effective segmentation of lung lesions by extracting features from CT images of lungs infected with COVID-19.

In order to achieve the purpose, the invention provides a novel coronary pneumonia disease detection and segmentation method, which comprises the following steps:

s1, acquiring COVID-19lung CT lesion data, preprocessing the data, finding out the slices of the beginning and the end of the lung, expanding the slices outwards, and performing slicing processing to remove the slices without the focus, wherein the threshold range of the image is (-1000, 500);

s2, processing the data, including randomly rotating the image, horizontally and vertically turning the image, histogram equalization technology and image normalization processing;

s3, constructing an N-Net network structure, wherein the sampling sequence is opposite to that of U-Net, and the sampling is performed after up-sampling, and the detail information of the input picture is amplified;

s4, optimizing blocks of two 3x3 convolutions of each layer in N-Net, and simultaneously optimizing blocks of two 3x3 convolutions of each layer in U-Net;

and S5, combining the U-Net and the N-Net network structures into a final network structure NU-Net.

Preferably, in step S3, the N-Net encoder performs upsampling to convert the input to a higher dimensionality, and then the decoder performs downsampling, wherein the upsampling is bilinear interpolation and the downsampling is maximal pooling.

Preferably, in step S4, the Block optimization process for each layer of N-Net with two 3x3 convolutions is as follows, where each Block1 has three branches, and the first branch is upsampled, then convolved by one 3x3, and then downsampled; the second branch is a convolution of two 3x 3; the third branch is downsampled and then upsampled after being convolved by 3x 3.

Preferably, in step S5, the two network structures are merged into a new network structure NU-Net, the upper branch is N-Net, the lower branch is U-Net, the original image is cut into a plurality of images with the same size from top to bottom and from left to right, the images are input to the end of the N-Net branch, the images are spliced into the original image according to the original sequence, the input and output sizes of the lower branch U-Net are matched with the original image, the results of the upper and lower branches are merged, and the merged image is classified by a softmax classifier through convolution with 1x 1.

Therefore, the present invention adopts the above-mentioned novel coronary pneumonia disease detection and segmentation method, and the method for automatically detecting and quantifying the breast covi-19 lesion may play an important role in the monitoring and management of the disease. Infected areas can be automatically identified from CT slices of the chest, and infected people can be screened, so that the infected people can be treated and nursed, and isolated, and virus transmission is reduced.

The specific technical effects are as follows:

(1) a new Block is added to the basic network, and the method can be used for better extracting features.

(2) An over-complete network structure is explored, the under-complete network structure and the over-complete network structure are combined, a new Block is added, a new network structure NU-Net is provided, and edge and detail characteristics are captured better than those of the U-Net obviously.

(3) Faster convergence rate, better performance and better generalization are achieved in the segmentation field.

The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.

Drawings

FIG. 1 is a schematic diagram of the upsampling of an N-Net encoder;

FIG. 2 is a schematic diagram of down-sampling by an N-Net encoder;

FIG. 3 is a schematic diagram of an optimized Block;

FIG. 4 is a schematic diagram of optimized U-Net and N-Net;

FIG. 5 is a schematic diagram of the NU-Net model structure.

Detailed Description

The technical solution of the present invention is further illustrated by the accompanying drawings and examples.

Unless defined otherwise, technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention belongs.

It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein, and any reference signs in the claims are not intended to be construed as limiting the claim concerned.

Furthermore, it should be understood that although the specification describes embodiments, not every embodiment includes only a single embodiment, and such description is for clarity purposes only, and it will be understood by those skilled in the art that the specification as a whole and the embodiments may be combined as appropriate to form other embodiments understood by those skilled in the art. These other embodiments are also covered by the scope of the present invention.

It should be understood that the above-mentioned embodiments are only for explaining the present invention, and the protection scope of the present invention is not limited thereto, and any person skilled in the art should be able to cover the technical scope of the present invention and the equivalent replacement or change of the technical solution and the inventive concept thereof in the technical scope of the present invention.

The use of the word "comprising" or "comprises" and the like in the present invention means that the element preceding the word covers the element listed after the word and does not exclude the possibility of also covering other elements. The terms "inner", "outer", "upper", "lower", and the like indicate orientations or positional relationships based on orientations or positional relationships shown in the drawings, merely for convenience in describing the present invention and simplifying the description, but do not indicate or imply that the referred devices or elements must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as limiting the present invention, and when the absolute position of the described object is changed, the relative positional relationships may be changed accordingly. In the present invention, unless otherwise expressly stated or limited, the terms "attached" and the like are to be construed broadly, e.g., as meaning a fixed connection, a removable connection, or an integral part; either directly or indirectly through intervening media, either internally or in any other relationship. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations. The term "about" as used herein has the meaning well known to those skilled in the art, and preferably means that the term modifies a value within the range of ± 50%, ± 40%, ± 30%, ± 20%, ± 10%, ± 5% or ± 1% thereof.

All terms (including technical or scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs unless specifically defined otherwise. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.

The disclosures of the prior art documents cited in the present description are incorporated by reference in their entirety and are, therefore, part of the present disclosure.

Example one

Data set Using the data set of COVID-19Lung CT Segmentation Challenge-20, COVID-19Lung CT Lesion Segmentation Challenge (COVID-19-20) in 2020 created a common platform to evaluate emerging artificial intelligence methods to segment and quantify Lung lesions caused by SARS-CoV-2 infection from CT images.

Lung lesions caused by SARS-CoV-2 infection were segmented and quantified from CT images from multiple institutions, multiple countries, patients of different ages, sexes and different disease severity. The data set included 199 training data and 50 validation data.

The invention discloses a novel coronary pneumonia disease detection and segmentation method, which comprises the following specific steps:

step (1): a data set of COVID-19Lung CT Segmentation Challenge-20 Challenge was obtained and pre-processed with a threshold cut of (-1000, 500) for the image, slices at the beginning and end of the Lung were found, slices were dilated outward and processed to remove slices without lesions.

Step (2): in order to improve the quality of model training, data enhancement is performed to improve the generalization effect and robustness of the model, and the data enhancement used comprises techniques of randomly rotating images, horizontally and vertically turning images, histogram equalization (for increasing image contrast) and image normalization.

And (3): the traditional U-Net network is up-sampling after down-sampling, the network shape and letter U are similar to the U-Net, so the structure is called U-Net, the N-Net network structure constructed by the invention is opposite to the U-Net, and the up-sampling and the down-sampling are performed before the up-sampling, so that the detail information of an input picture can be amplified to extract the detail information.

In the step (3), an N-Net encoder is formed, upsampling is adopted, input is converted into higher dimensionality (the dimensionality is a space meaning and is not a channel meaning), then a decoder carries out downsampling, the adopted upsampling is bilinear interpolation (shown in figure 1), and the downsampling is maximum pooling (shown in figure 2), so that fine details and edge characteristics can be accurately captured.

And (4): the Block of two 3x3 convolutions of each layer in N-Net is optimized, which is the improved N-Net. The same optimization was done for two 3x3 convolved blocks for each layer in U-Net at the same time.

In the step (4), two blocks of 3x3 in the original network are optimized, the new Block is shown as a Block1 in fig. 3, each Block1 has three branches, and the first branch is upsampled, then convolved by 3x3 and then downsampled, so that large target and overall information can be better extracted. The second branch is convolution of two 3x3, and the third branch is downsampling and then upsampling after convolution of 3x3, so that detail and edge information can be better extracted. After the three branches are merged, extraction of different targets is facilitated, element-wise add is performed on the three branches and the original input of the three branches, so that the captured information is more comprehensive, gradient disappearance can be reduced, Block2 in the graph serves as a substitute of Block1, and Block2 is used on a large-size characteristic diagram to replace Block1 so as to reduce memory consumption. And optimizing the U-Net and N-Net network structures of Block, as shown in FIG. 4, except that the up-down sampling sequence is different.

And (5): the U-Net and N-Net network structures are combined into a final network structure NU-Net, as shown in FIG. 5, which can not only ensure the original segmentation effect of U-Net, but also segment the fine organization structure and the boundary region.

In the step (5), two network structures are combined into a new network structure NU-Net, the upper branch is N-Net, the lower branch is U-Net, the size of the original image is (512 ), the image is cut into 4 images with the size of (256 ) in consideration of overlarge image size, the images are input into the network structure, and then the network structure is convoluted by 3x 3.

Due to the fact that upsampling has a large demand on a display memory, in order to enable a model to run smoothly, the size of an image input into an upper N-Net branch is cut into 16 pictures with the size of (64, 64) from top to bottom from left to right as an input, (256 ) the pictures are spliced into the pictures with the size of (256 ) at the tail end of the upper N-Net branch according to the original sequence, the input and output sizes of a lower branch U-Net are (256 ), then the results of the upper branch and the lower branch are fused by using a catate, and the upper branch and the lower branch are classified by using a softmax classifier through convolution of 1x 1.

Four modules in the network structure replace Block1 with Block2 because of video memory limitation, the rest positions still use Block1, all used convolutions of 3x3 do not change the image size, and in the final network structure, the convolution kernel number in the upper branch N-Net is set to be 80, 40, 20, 10 and 5 respectively, and the convolution kernel number in the lower branch U-Net is set to be 32, 64, 128, 256 and 512 respectively.

The overcomplete network structure N-Net of the invention, combined with the U-Net network structure, can obtain advanced features and deep semantic information, and can better capture details and segment, thus obtaining better performance in the aspect of segmenting the focus.

Test of

Using Dice, Jaccrad, Volume Similarity, Hausdorff 95, Hausdorff 100, Surface Dice AT1mm, Average Surface Distance GT to Pred, and Average Surface Distance Pred to GT as evaluation indexes, and looking AT Dice and Jaccard, the calculation method is as follows:

the results of comparing four network structures, i.e., U-Net B (FIG. 4), Sym-Unet (a dual-branch structure consisting of Unet and a network symmetrical to it, one branch using U-Net, the other using up-sampling and down-sampling codec, and common 3x3 convolution for each Block) and NU-Net, are shown in Table 1.

Table 1 comparison of four sets of network structures

From Table 1, it can be seen that the Dice coefficient and the Jaccard coefficient of NU-Net are 0.6918 and 0.5556 respectively, while the Dice coefficient and the Jaccard coefficient of U-Net are 0.6829 and 0.5347 respectively, and the Dice coefficient and the Jaccard coefficient of NU-Net are improved by 0.0089 and 0.0209 compared with U-Net, which are obviously improved, indicating that the segmentation effect is better.

Hausdorff 95 and Hausdorff 100 are obviously reduced, the segmentation edge and fine detail effects are improved, the surface Dice AT1mm is improved, the Dice coefficient of the surface volume is improved, and the segmentation result is more accurate. Through the comparison of U-Net and U-Net B and the comparison of Sym-Unet and NU-Net, the addition of Block can really improve the segmentation effect, and the module is proved to be effective.

Therefore, the invention adopts the novel coronary pneumonia disease detection and segmentation method to automatically identify the infected area from the chest CT slice and screen the infected people, so that the infected people can be treated, nursed and isolated, and the virus transmission is reduced.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting the same, and although the present invention is described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that: modifications and equivalents may be made to the invention without departing from the spirit and scope of the invention.

Claims

1. A novel coronary pneumonia disease detection and segmentation method is characterized by comprising the following steps:

s3, constructing an N-Net network structure, wherein the sampling sequence is opposite to that of U-Net, and the sampling is performed after the up-sampling;

2. The novel coronary pneumonia disease detection and segmentation method according to claim 1, characterized in that: in step S3, the process of constructing N-Net is as follows, where the N-Net encoder performs upsampling to convert the input to a higher dimensionality, and then the decoder performs downsampling, where the upsampling is bilinear interpolation and the downsampling is maximum pooling.

3. The method of claim 1, wherein the disease detection and segmentation method for coronary pneumonia comprises: in step S4, the Block optimization process of each layer of two 3x3 convolutions in N-Net is as follows, each Block1 has three branches, the first branch is upsampled, then is convolved by one 3x3, and then is downsampled; the second branch is a convolution of two 3x 3; the third branch is downsampled and then upsampled after being convolved by 3x 3.

4. The method of claim 1, wherein the disease detection and segmentation method for coronary pneumonia comprises: in step S5, two network structures are merged into a new network structure NU-Net, the upper branch is N-Net, the lower branch is U-Net, the original image is cut into a plurality of images with the same size from top to bottom and from left to right, and the images are input to the end of the N-Net branch, and then the images are spliced into the original image according to the original sequence, the input and output sizes of the lower branch U-Net are matched with the original image, then the results of the upper and lower branches are merged, and the merged images are classified by a 1x1 convolution using a softmax classifier.