CN113643308A - Lung image segmentation method and device, storage medium and computer equipment - Google Patents

Lung image segmentation method and device, storage medium and computer equipment Download PDF

Info

Publication number
CN113643308A
CN113643308A CN202111015975.8A CN202111015975A CN113643308A CN 113643308 A CN113643308 A CN 113643308A CN 202111015975 A CN202111015975 A CN 202111015975A CN 113643308 A CN113643308 A CN 113643308A
Authority
CN
China
Prior art keywords
lung
image
dimensional
parenchyma
segmentation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111015975.8A
Other languages
Chinese (zh)
Inventor
张俊杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Ping An Medical Health Technology Service Co Ltd
Original Assignee
Ping An Medical and Healthcare Management Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Medical and Healthcare Management Co Ltd filed Critical Ping An Medical and Healthcare Management Co Ltd
Priority to CN202111015975.8A priority Critical patent/CN113643308A/en
Publication of CN113643308A publication Critical patent/CN113643308A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/11Region-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/23Clustering techniques
    • G06F18/232Non-hierarchical techniques
    • G06F18/2321Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/187Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00ICT specially adapted for the handling or processing of medical images
    • G16H30/20ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10028Range image; Depth image; 3D point clouds
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30061Lung
    • G06T2207/30064Lung nodule

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Molecular Biology (AREA)
  • Software Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Probability & Statistics with Applications (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Epidemiology (AREA)
  • Medical Informatics (AREA)
  • Primary Health Care (AREA)
  • Public Health (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to the technical field of artificial intelligence and digital medical treatment, and provides a lung image segmentation method, a lung image segmentation device, a storage medium and computer equipment. The method comprises the following steps: constructing a lung nodule segmentation model through a densely connected hybrid expansion convolution module and a hierarchy attention mechanism based on a full convolution neural network model; taking the lung parenchyma image sample as input, taking the lung nodule segmentation image of the lung parenchyma image sample as output, and training the lung nodule segmentation model to obtain a trained lung nodule segmentation model; acquiring a lung image to be processed, and segmenting the lung image through a lung parenchyma segmentation algorithm to obtain a lung parenchyma image; and obtaining a lung nodule segmentation result of the lung image through the trained lung nodule segmentation model according to the lung parenchyma image. The method can effectively improve the segmentation precision of the lung nodule.

Description

Lung image segmentation method and device, storage medium and computer equipment
Technical Field
The invention relates to the technical field of artificial intelligence and digital medical treatment, in particular to a lung image segmentation method, a lung image segmentation device, a storage medium and computer equipment.
Background
In recent years, the incidence of lung diseases has gradually become highly prevalent due to various causes such as deterioration of air quality, increase of second-hand smoke harm, and influence of occupational factors, and among them, lung nodules are one of early signs of lung cancer, and the division method thereof has become one of hot spots of extensive research in the industry.
In the prior art, a full convolution neural network model is generally adopted as a basic model for building a lung nodule segmentation model. However, when the lung image is downsampled by using the full convolution neural network model, a lot of context information is often lost, so that it is difficult to completely recover the detail information of the target to be segmented and the corresponding spatial dimension in the upsampling process, the result obtained by upsampling is not clear, and the segmentation accuracy of the lung nodule is low. In addition, the expansion convolution used in the conventional full convolution neural network model is prone to grid problems, and the grid problems may cause that the sampling result cannot cover all image features, and the continuity of the image features may also be lost, which further reduces the accuracy of the lung nodule segmentation result.
Disclosure of Invention
In view of this, the present application provides a lung image segmentation method, device, storage medium and computer device, and mainly aims to solve the technical problem of low accuracy of lung nodule segmentation.
According to a first aspect of the present invention, there is provided a lung image segmentation method, comprising:
constructing a lung nodule segmentation model through a densely connected hybrid expansion convolution module and a hierarchy attention mechanism based on a full convolution neural network model;
taking the lung parenchyma image sample as input, taking the lung nodule segmentation image of the lung parenchyma image sample as output, and training the lung nodule segmentation model to obtain a trained lung nodule segmentation model;
acquiring a lung image to be processed, and segmenting the lung image through a lung parenchyma segmentation algorithm to obtain a lung parenchyma image;
and obtaining a lung nodule segmentation result of the lung image through the trained lung nodule segmentation model according to the lung parenchyma image.
According to a second aspect of the present invention, there is provided a lung image segmentation apparatus, comprising:
the model construction module is used for constructing a lung nodule segmentation model through a densely connected hybrid expansion convolution module and a hierarchy attention mechanism based on a full convolution neural network model;
the model training module is used for training the lung nodule segmentation model by taking the lung parenchyma image sample as input and taking the lung nodule segmentation image of the lung parenchyma image sample as output to obtain the trained lung nodule segmentation model;
the image segmentation module is used for acquiring a lung image to be processed, and segmenting the lung image through a lung parenchyma segmentation algorithm to obtain a lung parenchyma image;
and the image processing module is used for obtaining a lung nodule segmentation result of the lung image through the trained lung nodule segmentation model according to the lung parenchyma image.
According to a third aspect of the present invention, a storage medium is provided, on which a computer program is stored which, when being executed by a processor, carries out the above-mentioned lung image segmentation method.
According to a fourth aspect of the present invention, there is provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the above-mentioned lung image segmentation method when executing the program.
The invention provides a lung image segmentation method, a device, a storage medium and computer equipment. According to the method, by the aid of the up-down sampling structure and the hierarchy attention mechanism of the full-convolution neural network model, a large amount of context information in the lung image is obtained, meanwhile, valuable characteristic channels are selectively amplified, useless characteristic channels are restrained, the lung nodule segmentation model can better capture key characteristic information of lung nodules, and segmentation accuracy of the lung nodules is improved. In addition, according to the method, the densely connected hybrid expansion convolution module is added in the full convolution neural network model, the characteristics of the multi-level receptive field in the lung image can be extracted, the network grid problem can be avoided, and the segmentation precision of the lung nodule is further improved.
The foregoing description is only an overview of the technical solutions of the present application, and the present application can be implemented according to the content of the description in order to make the technical means of the present application more clearly understood, and the following detailed description of the present application is given in order to make the above and other objects, features, and advantages of the present application more clearly understandable.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
fig. 1 is a flowchart illustrating a lung image segmentation method according to an embodiment of the present invention;
fig. 2 is a schematic view of a scene for performing binarization processing on a lung image according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a scene for three-dimensional connected reconstruction of a lung image according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating a scene of clustering lung images according to an embodiment of the present invention;
FIG. 5 is a diagram illustrating a scene of a dilation operation performed on an image of a lung according to an embodiment of the present invention;
FIG. 6 is a diagram illustrating a scene of a mask operation performed on a lung image according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram illustrating a lung image segmentation apparatus according to an embodiment of the present invention.
Detailed Description
The invention will be described in detail hereinafter with reference to the accompanying drawings in conjunction with embodiments. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
In an embodiment, as shown in fig. 1, a lung image segmentation method is provided, which is described by taking an example that the method is applied to a computer device such as a server, where the server may be an independent server, or a cloud server that provides basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, a middleware service, a domain name service, a security service, a Content Delivery Network (CDN), and a big data and artificial intelligence platform. The method comprises the following steps:
101. and constructing a lung nodule segmentation model through a densely connected hybrid expansion convolution module and a hierarchy attention mechanism based on a full convolution neural network model.
In particular, a Convolutional Neural Network (CNN) is often used as a basic network for model building in image processing, and generally speaking, the Convolutional Neural network is mainly used for processing 2D images. In this embodiment, a full convolution neural network (e.g., a V-Net network or a U-Net network) is mainly used as a building basis for the lung nodule model, where the full convolution neural network is a variant of the convolutional neural network, and a short-circuit connection manner of ResNet is used to replace a pooling layer for up-sampling and down-sampling in the convolutional neural network with convolution layers. In the structure of the full convolutional neural network, one side is a gradually compressed path formed by an encoder, and the other side is a gradually decompressed path formed by a decoder. In the embodiment, the full convolution neural network has a better effect on feature extraction of lung images than a common convolution neural network.
Further, on the basis of the full convolution neural network, a densely connected hybrid expansion convolution network module and a hierarchical attention mechanism can be added, and an initial lung nodule segmentation model is constructed according to the densely connected hybrid expansion convolution network module and the hierarchical attention mechanism. The hierarchy attention mechanism is to perform weight distribution on each hierarchy of an encoder and a decoder of a full convolution neural network, so that key features of a target to be detected are highlighted; the expansion convolution network refers to a network for expanding the convolution kernel of the convolution network according to the expansion coefficient, namely, the expanded convolution kernel is obtained by zero filling the area not occupied by the original convolution kernel. In this embodiment, the expansion convolution network can expand the receptive field of the convolution kernel without increasing the calculation amount, and can also avoid the loss of detail features in the downsampling process, but a common expansion convolution network easily causes the grid problem, so that in this embodiment, the mixed expansion convolution is adopted to replace the expansion convolution and is added to the full convolution neural network, the grid problem existing in the expansion convolution can be avoided through the expansion rate features of the mixed expansion convolution, and the training time of network parameters can be reduced through the structure of the mixed expansion convolution intensive connection.
In this embodiment, the encoder and decoder of the full convolution neural network can extract the characteristics of the multi-level receptive field in the lung image through the densely connected hybrid expansion convolution network, and can avoid the grid problem, while the hierarchical attention mechanism can reduce unnecessary characteristics and highlight key characteristics, thereby capturing the key information in the original image better. Therefore, based on the full convolution neural network model, the performance of the lung nodule segmentation model can be effectively improved through the densely connected hybrid expansion convolution module and the model constructed by the hierarchy attention mechanism, and therefore the lung nodule segmentation precision is improved.
102. And training the lung nodule segmentation model by taking the lung parenchymal image sample as input and taking the lung nodule segmentation image of the lung parenchymal image sample as output to obtain the trained lung nodule segmentation model.
The lung parenchyma image sample refers to an image sample which is obtained by segmenting a lung image and contains gray information, and the lung nodule segmentation image of the lung parenchyma image sample refers to an image sample obtained by artificially labeling the lung parenchyma image sample. In this embodiment, each lung parenchyma image sample corresponds to one labeled lung nodule segmented image, and in addition, the lung parenchyma image sample and the lung nodule segmented image sample corresponding to the lung parenchyma image sample may be acquired from a database or from other approaches, which is not specifically limited herein.
Specifically, the computer device may first obtain a large number of lung parenchymal image samples and a lung nodule segmentation image corresponding to each lung parenchymal image sample, then perform continuous iterative training on the model constructed in step 101 by using each lung parenchymal image sample as an input and using the lung nodule segmentation image corresponding to the sample as an output, and finally obtain the trained lung nodule segmentation model.
103. And acquiring a lung image to be processed, and segmenting the lung image through a lung parenchyma segmentation algorithm to obtain a lung parenchyma image.
The lung image is an image obtained by performing computed tomography on the lung. In this embodiment, the lung image may be processed sequentially through a binarization algorithm, a three-dimensional communication algorithm, a clustering algorithm, an expansion algorithm, and a mask algorithm to obtain a lung parenchyma image. The binarization algorithm refers to performing binarization operation on pixels in an image, namely setting the gray value of pixel points on the image to be 0 or 255, so that the whole image presents an obvious black-and-white effect, and common binarization algorithms mainly comprise a gray average value method, a double peak method, an OTSU method, a Niblack method and the like; the three-dimensional connected algorithm is a three-dimensional image reconstruction method based on a connected domain algorithm, namely an algorithm for marking binarized image pixels and performing connected reconstruction in a three-dimensional direction to obtain a three-dimensional image; the clustering algorithm is an algorithm for dividing an image into a plurality of target areas with different meanings, and mainly comprises a fuzzy clustering algorithm, a K-means clustering algorithm, a C-means clustering algorithm and the like; the dilation algorithm is a morphological algorithm, and refers to an algorithm that merges all background points in contact with an object into the object and expands the boundary to the outside; the mask algorithm is an algorithm for multiplying the processed image by the pixels of the original image. In this embodiment, the names of the algorithms used in the binarization algorithm, the three-dimensional connectivity algorithm, the clustering algorithm, the dilation algorithm, and the mask algorithm may be selected according to actual situations, and this embodiment is not specifically limited herein.
104. And obtaining a lung nodule segmentation result of the lung image through the trained lung nodule segmentation model according to the lung parenchyma image.
Specifically, after the lung parenchymal image is obtained through a series of algorithms, the lung parenchymal image can be input into a trained lung nodule segmentation model, and lung nodules in the lung parenchymal image are segmented through the lung nodule model, so that feature expression of a lung nodule segmentation result is obtained. In this embodiment, the feature information output by the pulmonary nodule model may be input into a nonlinear activation function, and then a voxel value in the feature information is subjected to binarization processing by the nonlinear activation function, so as to obtain a final pulmonary nodule segmentation result.
The lung image segmentation method provided by this embodiment is to construct a lung nodule segmentation model based on a full convolution neural network model through a densely connected hybrid expansion convolution module and a hierarchical attention mechanism, train the lung nodule segmentation model with a lung parenchyma image sample as an input and an artificially labeled lung nodule image sample as an output, further segment a lung image to be processed to obtain a lung parenchyma image, and process the lung image through the trained lung nodule segmentation model to obtain a lung nodule segmentation result. According to the method, by the aid of the up-down sampling structure and the hierarchy attention mechanism of the full-convolution neural network model, a large amount of context information in the lung image is obtained, meanwhile, valuable characteristic channels are selectively amplified, useless characteristic channels are restrained, the lung nodule segmentation model can better capture key characteristic information of lung nodules, and segmentation accuracy of the lung nodules is improved. In addition, according to the method, the densely connected hybrid expansion convolution module is added in the full convolution neural network model, the characteristics of the multi-level receptive field in the lung image can be extracted, the network grid problem can be avoided, and the segmentation precision of the lung nodule is further improved.
In an embodiment, optionally, the step 103 may be implemented by: firstly, carrying out binarization processing on a lung image through a binarization algorithm to obtain a binarized lung image, then carrying out connected reconstruction on the binarized lung image through a three-dimensional connected algorithm to obtain a three-dimensional lung image, further carrying out clustering processing on the three-dimensional lung image through a mean value clustering algorithm to obtain a binarized three-dimensional connected lung parenchyma image, further carrying out expansion operation on the binarized three-dimensional connected lung parenchyma image to obtain a three-dimensional lung parenchyma outline, and finally carrying out mask operation on the three-dimensional lung parenchyma outline and the lung image to obtain a lung parenchyma image. In the embodiment, the lung image is processed through a series of algorithms such as a binarization algorithm, a three-dimensional communication algorithm, a clustering algorithm, an expansion operation and a mask operation, compared with a single lung parenchyma segmentation algorithm in the prior art, the method can effectively remove the chest contour in the lung image and retain the lung nodules with pleura adhesion, thereby effectively reducing the probability of missing detection of the lung nodules.
In one embodiment, the method for performing connected reconstruction on the binarized lung image may be implemented by the following steps: firstly, through a three-dimensional connectivity algorithm, image pixels in a binarized lung image are connected in six neighborhoods (namely an upper neighborhood, a lower neighborhood, a left neighborhood, a right neighborhood, a front neighborhood and a rear neighborhood) in a three-dimensional direction to obtain a plurality of connected regions of the binarized lung image, and then the plurality of connected regions of the binarized lung image are reconstructed according to a preset sequence to obtain the lung three-dimensional image.
In one embodiment, optionally, the clustering process for the lung three-dimensional image can be implemented by: firstly, taking the distance between the central points of a plurality of connected regions of the lung three-dimensional image and the central point of the lung three-dimensional image as a clustering characteristic, clustering the lung three-dimensional image by using a mean value clustering algorithm to obtain a plurality of three-dimensional connected regions of the lung three-dimensional image, then reserving the binarized three-dimensional connected regions of the lung three-dimensional image, and setting the voxel value of the non-binarized three-dimensional connected regions of the lung three-dimensional image to be 1 to obtain the binarized lung parenchyma three-dimensional connected image. In the present embodiment, the plurality of three-dimensional connected regions include a binarized three-dimensional connected region which refers to a three-dimensional connected region composed of binarized lung parenchyma, and a non-binarized three-dimensional connected region which refers to other three-dimensional connected regions than the binarized three-dimensional connected region.
In an embodiment, in order to specifically describe the implementation process of step 103, this embodiment describes, with reference to a specific example, the implementation processes of a series of algorithms such as a binarization algorithm, a three-dimensional connectivity algorithm, a clustering algorithm, an inflation operation, a mask operation, and the like. In this embodiment, an OTSU binarization algorithm is first used to perform binarization processing on the lung image, so as to obtain a binarized lung CT sequence slice image. The OTSU binarization algorithm is a binarization algorithm based on a maximum inter-class variance method, and can divide an image into a background part and an object part according to the gray level characteristics of the image, wherein the larger the inter-class variance between the background and the object is, the larger the difference between the two parts forming the image is, and the smaller the difference between the two parts is caused when part of the object is mistaken for the background or part of the background is mistaken for the object. Therefore, the segmentation accuracy for maximizing the inter-class variance is higher, wherein the image binarization schematic diagram is shown in FIG. 1. Then, the image pixels in the binarized lung CT sequence slice image can be connected in six fields of up, down, left, right, front and back in the three-dimensional direction by a three-dimensional connection algorithm to obtain a plurality of connected regions, and finally the connected regions are reconstructed according to a certain sequence to obtain a lung three-dimensional image, wherein a schematic diagram of the connected reconstruction is shown in fig. 2.
Further, after the lung three-dimensional image is obtained, the lung three-dimensional image can be clustered by using a C-means clustering algorithm, wherein the C-means clustering algorithm is also called as a K-means clustering algorithm and is a distance-based clustering algorithm. In the embodiment, the distance between the center point of the connected region and the center point of the three-dimensional lung image can be used as a clustering feature, the three-dimensional lung image is processed based on a C-means clustering algorithm, in the C-means clustering algorithm, the closer the distance between two objects is, the greater the similarity is, further, the objects with similar clustering features can be compacted into clusters, and finally the compact and independent clusters are used as different clustering targets. In the embodiment, the lung parenchyma, the background, the chest contour, the noise and other parts in the three-dimensional lung image can be distinguished conveniently and accurately by using a C-means clustering algorithm, and other areas are removed by setting the voxel values of the other areas to be 1, and only the three-dimensional connected area formed by the binarized lung parenchyma is reserved, wherein a clustering segmentation schematic diagram is shown in fig. 3.
Further, after obtaining the binarized lung parenchyma, the three-dimensional connected region constituted by the binarized lung parenchyma may be subjected to expansion processing by an expansion operation, wherein the structural element size of the expansion operation is 3. In this embodiment, the lung parenchymal region may be expanded through an expansion operation, so that pleural adhesive nodules that may be mistakenly deleted due to deletion of the sternum portion in the clustering segmentation process are retained. Finally, mask operation can be performed on the expanded lung parenchyma image and the lung CT image, that is, pixels of the expanded lung parenchyma image and pixels of the lung CT image are correspondingly multiplied, so that the lung parenchyma image containing gray information can be obtained. The expansion budget and the mask operation are shown in fig. 4 and 5, respectively.
In an embodiment, optionally, the step 104 may be implemented by: firstly, inputting a lung parenchymal image into a trained lung nodule segmentation model to obtain characteristic information of a lung image, then inputting the characteristic information of the lung image into a nonlinear activation function, and carrying out binarization processing on the characteristic information of the lung image through the nonlinear activation function to obtain a lung nodule segmentation result of the lung image. The nonlinear activation function may be a sigmoid function. In this embodiment, the information output by the model is further input into the nonlinear activation function for binarization processing, so that the segmentation accuracy of the lung nodule can be further improved.
In an embodiment, optionally, the step 101 may be implemented by: firstly, an encoder and a decoder of a full convolutional neural network model are constructed, wherein the encoder is composed of a mixed expansion convolutional module, a first normalization layer and a first activation layer which are connected densely, the decoder is composed of an deconvolution module, a second normalization layer and a second activation layer which are connected densely, then, a hierarchical attention mechanism is introduced between the encoder and the decoder to construct a lung nodule segmentation model, the feature layers of the encoder and the decoder under the hierarchical attention mechanism are opposite pairwise, and feature pixels in the feature layers opposite pairwise are multiplied correspondingly respectively.
In one embodiment, the densely connected hybrid dilation convolution module in the lung nodule segmentation model may optionally include a plurality of first three-dimensional convolution layers, wherein a dense structural connection is employed between every two first three-dimensional convolution layers of the plurality of first three-dimensional convolution layers. Further, the densely connected deconvolution module in the lung nodule segmentation model may include a plurality of second three-dimensional convolution layers, wherein a dense structure is connected between every two second three-dimensional convolution layers of the plurality of second three-dimensional convolution layers. In this embodiment, the number of the first three-dimensional convolution layers and the number of the second three-dimensional convolution layers are the same, and the number of the first three-dimensional convolution layers and the number of the second three-dimensional convolution layers may be set to about 3 to 5 in order to achieve both the accuracy of the model and the processing efficiency.
In one embodiment, to facilitate the detailed description of the implementation process of step 101, the present embodiment combines a specific example model creation process. In the embodiment, the lung nodule segmentation model is constructed based on a V-Net network and is composed of an encoder and a decoder. Wherein the encoder portion comprises a dense hybrid dilation convolution module, which may be composed of a plurality of 3D convolutional layers, a batch normalization layer (BN), and an activation layer (ReLU), and in the present embodiment, the dense hybrid dilation convolution module is composed of 5 3D convolutional layers. Moreover, the size of the convolution kernel is 3 multiplied by 3, the convolution layers are connected in a dense structure, and the expansion rate can be set to be 1, 2, 5, 2, 1. The decoder part comprises a deconvolution module, a batch normalization layer and an activation layer, wherein the deconvolution module can be composed of a plurality of 3D convolution layers, and the deconvolution module is composed of 5 3D convolution layers in the embodiment. Moreover, the size of the convolution kernel is also 3 multiplied by 3, and the convolution layers are connected with each other by adopting a dense structure. The encoder can continuously extract key features of the image and compress important information extracted from a lower layer to a higher layer, and the decoder can reconstruct a final output feature map from feature maps compressed by the encoder.
Further, the depth of the encoder and the decoder may be set to 7 layers, wherein the dense hybrid dilation convolution module of the encoder may be disposed between the batch normalization layer and the active layer, and the deconvolution module of the decoder may be disposed between the batch normalization layer and the active layer. Through the arrangement mode, the encoder and the decoder can extract the characteristics of the image multi-level receptive field through the dense connection structure and the expansion convolution network, the grid problem is avoided, the accuracy of the model is improved, and meanwhile, the network parameters and the training time are reduced.
Further, in this embodiment, a hierarchy attention mechanism is further introduced between the encoder and the decoder, specifically, in this embodiment, downsampling is performed between the encoder and the decoder in each hierarchy of the lung nodule model according to the hierarchy attention mechanism, and then pixels output by the encoder and feature pixels output by the decoder of the corresponding hierarchy are multiplied one by one to highlight the key features. In this embodiment, a specific process of applying the hierarchical attention mechanism to the lung nodule segmentation model is as follows:
down-sampling the first layer characteristic of the encoder to make the first layer characteristic size of the encoder consistent with the seventh layer characteristic size, and multiplying the first layer characteristic pixel and the seventh layer characteristic pixel of the encoder one by one; down-sampling the second layer characteristic of the encoder to make the second layer characteristic size of the encoder consistent with the sixth layer characteristic size, and multiplying the second layer characteristic pixels of the encoder by the sixth layer characteristic pixels one by one; and performing down-sampling on the third-layer characteristic of the encoder to enable the third-layer characteristic size of the encoder to be consistent with the fifth-layer characteristic size, and multiplying the third-layer characteristic pixels and the fifth-layer characteristic pixels of the encoder one by one. Down-sampling the seventh layer characteristic of the decoder to make the seventh layer characteristic size of the decoder consistent with the first layer characteristic size, and multiplying the seventh layer characteristic pixel of the decoder with the first layer characteristic pixel one by one; down-sampling the sixth layer characteristic of the decoder to make the sixth layer characteristic size of the decoder consistent with the second layer characteristic size, and multiplying the sixth layer characteristic pixel of the decoder and the second layer characteristic pixel one by one; and performing down-sampling on the fifth-layer characteristic of the decoder to enable the fifth-layer characteristic size of the decoder to be consistent with the third-layer characteristic size, and multiplying the fifth-layer characteristic pixels and the third-layer characteristic pixels of the decoder one by one. The embodiment introduces a hierarchical attention mechanism to the V-Net network, so that unnecessary features can be reduced, key features can be highlighted, and key information of an original image can be captured better.
Further, as a refinement and expansion of the specific implementation of the above embodiment, for fully explaining the implementation process of the present embodiment, a lung image segmentation method is provided, as shown in fig. 2, and the method includes the following steps:
further, as a specific implementation of the method shown in fig. 1, the present embodiment provides a lung image segmentation apparatus, as shown in fig. 3, the apparatus includes: a model construction module 21, a model training module 22, an image segmentation module 23 and an image processing module 24.
The model construction module 21 is used for constructing a lung nodule segmentation model based on a full convolution neural network model through a densely connected hybrid expansion convolution module and a hierarchy attention mechanism;
the model training module 22 is configured to train the lung nodule segmentation model by using the lung parenchyma image sample as an input and using the lung nodule segmentation image of the lung parenchyma image sample as an output, so as to obtain a trained lung nodule segmentation model;
the image segmentation module 23 is configured to obtain a lung image to be processed, and segment the lung image by using a lung parenchyma segmentation algorithm to obtain a lung parenchyma image;
and the image processing module 24 is configured to obtain a lung nodule segmentation result of the lung image through the trained lung nodule segmentation model according to the lung parenchyma image.
In a specific application scenario, the image segmentation module 23 is specifically configured to perform binarization processing on the lung image through a binarization algorithm to obtain a binarized lung image; performing connected reconstruction on the binary lung image through a three-dimensional connected algorithm to obtain a three-dimensional lung image; clustering the lung three-dimensional images through a mean value clustering algorithm to obtain a binary lung parenchyma three-dimensional connected image; performing expansion operation on the three-dimensional connected image of the binary lung parenchyma to obtain a three-dimensional lung parenchyma outline; and carrying out mask operation on the three-dimensional lung parenchyma outline and the lung image to obtain a lung parenchyma image.
In a specific application scenario, the image segmentation module 23 is specifically configured to communicate image pixels in the binarized lung image in six neighborhoods in a three-dimensional direction by using a three-dimensional communication algorithm, so as to obtain a plurality of communication regions of the binarized lung image; and reconstructing a plurality of connected regions of the binarized lung image according to a preset sequence to obtain a lung three-dimensional image.
In a specific application scenario, the image segmentation module 23 is specifically configured to perform clustering processing on the three-dimensional lung image by using distances between center points of a plurality of connected regions of the three-dimensional lung image and center points of the three-dimensional lung image as clustering characteristics through a mean value clustering algorithm to obtain a plurality of three-dimensional connected regions of the three-dimensional lung image, where the plurality of three-dimensional connected regions include a binarized three-dimensional connected region and a non-binarized three-dimensional connected region; and reserving the binarized three-dimensional connected region of the lung three-dimensional image, and setting the voxel value of the non-binarized three-dimensional connected region of the lung three-dimensional image to be 1 to obtain the binarized lung parenchyma three-dimensional connected image.
In a specific application scenario, the image processing module 24 may be specifically configured to input the lung parenchymal image into a trained lung nodule segmentation model to obtain feature information of the lung image; and inputting the characteristic information of the lung image into a nonlinear activation function, and performing binarization processing on the characteristic information of the lung image to obtain a lung nodule segmentation result of the lung image.
In a specific application scenario, the model building module 21 may be specifically configured to build an encoder and a decoder of a full convolution neural network model, where the encoder is composed of a densely connected hybrid expanded convolution module, a first batch of normalization layers, and a first activation layer, and the decoder is composed of a densely connected deconvolution module, a second batch of normalization layers, and a second activation layer; and introducing a hierarchical attention mechanism between the encoder and the decoder to construct a lung nodule segmentation model, wherein feature layers of the encoder and the decoder under the hierarchical attention mechanism are opposite to each other, and feature pixels in the feature layers opposite to each other are correspondingly multiplied respectively.
In a specific application scenario, the densely connected hybrid expansion convolution module comprises a plurality of first three-dimensional convolution layers, and every two first three-dimensional convolution layers of the plurality of first three-dimensional convolution layers are connected through a dense structure; the densely connected deconvolution module comprises a plurality of second three-dimensional convolution layers, and every two second three-dimensional convolution layers of the plurality of second three-dimensional convolution layers are connected by a dense structure.
It should be noted that other corresponding descriptions of the functional units related to the lung image segmentation apparatus provided in this embodiment may refer to the corresponding descriptions in fig. 1, and are not repeated herein.
Based on the method shown in fig. 1, correspondingly, the present embodiment further provides a storage medium, on which a computer program is stored, which when executed by a processor implements the lung image segmentation method shown in fig. 1.
Based on such understanding, the technical solution of the present application may be embodied in the form of a software product, and the software product to be identified may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, or the like), and include several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the method according to the implementation scenarios of the present application.
Based on the method shown in fig. 1 and the embodiment of the lung image segmentation apparatus shown in fig. 7, in order to achieve the above object, the present embodiment further provides a physical device for lung image segmentation, which may be a personal computer, a server, a smart phone, a tablet computer, a smart watch, or other network devices, and the physical device includes a storage medium and a processor; a storage medium for storing a computer program; a processor for executing a computer program for implementing the above-described method as shown in fig. 1.
Optionally, the entity device may further include a user interface, a network interface, a camera, a Radio Frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, and the like. The user interface may include a Display screen (Display), an input unit such as a keypad (Keyboard), etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), etc.
It will be understood by those skilled in the art that the present embodiment provides a solid device structure for lung image segmentation, which does not constitute a limitation of the solid device structure, and may include more or less components, or combine some components, or arrange different components.
The storage medium may further include an operating system and a network communication module. The operating system is a program for managing the hardware of the above-mentioned entity device and the software resources to be identified, and supports the operation of the information processing program and other software and/or programs to be identified. The network communication module is used for realizing communication among components in the storage medium and communication with other hardware and software in the information processing entity device.
Through the above description of the embodiments, those skilled in the art will clearly understand that the present application can be implemented by software plus a necessary general hardware platform, and can also be implemented by hardware. According to the technical scheme, a lung nodule segmentation model is constructed on the basis of a full convolution neural network model through a densely connected hybrid expansion convolution module and a hierarchy attention mechanism, then a lung parenchyma image sample is used as input, an artificially labeled lung nodule image sample is used as output, the lung nodule segmentation model is trained, a lung image to be processed is segmented to obtain a lung parenchyma image, and finally the lung image is processed through the trained lung nodule segmentation model to obtain a lung nodule segmentation result. Compared with the prior art, the method obtains a lot of context information in the lung image through the up-down sampling structure and the hierarchy attention mechanism of the full convolution neural network model, selectively amplifies valuable characteristic channels, and inhibits useless characteristic channels, so that the lung nodule segmentation model can better capture the key characteristic information of the lung nodule, and the segmentation precision of the lung nodule is improved. In addition, according to the method, the densely connected hybrid expansion convolution module is added in the full convolution neural network model, the characteristics of the multi-level receptive field in the lung image can be extracted, the network grid problem can be avoided, and the segmentation precision of the lung nodule is further improved.
Those skilled in the art will appreciate that the figures are merely schematic representations of one preferred implementation scenario and that the blocks or flow diagrams in the figures are not necessarily required to practice the present application. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario may be distributed in the devices in the implementation scenario according to the description of the implementation scenario, or may be located in one or more devices different from the present implementation scenario with corresponding changes. The modules of the implementation scenario may be combined into one module, or may be further split into a plurality of sub-modules.
The above application serial numbers are for description purposes only and do not represent the superiority or inferiority of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of the present application, but the present application is not limited thereto, and any variations that can be made by those skilled in the art are intended to fall within the scope of the present application.

Claims (10)

1. A method of lung image segmentation, the method comprising:
constructing a lung nodule segmentation model through a densely connected hybrid expansion convolution module and a hierarchy attention mechanism based on a full convolution neural network model;
taking a lung parenchymal image sample as input, taking a lung nodule segmentation image of the lung parenchymal image sample as output, and training the lung nodule segmentation model to obtain a trained lung nodule segmentation model;
acquiring a lung image to be processed, and segmenting the lung image through a lung parenchyma segmentation algorithm to obtain a lung parenchyma image;
and obtaining a lung nodule segmentation result of the lung image through the trained lung nodule segmentation model according to the lung parenchyma image.
2. The method of claim 1, wherein the segmenting the lung image by a lung parenchymal segmentation algorithm to obtain a lung parenchymal image comprises:
carrying out binarization processing on the lung image through a binarization algorithm to obtain a binarized lung image;
performing connected reconstruction on the binarized lung image through a three-dimensional connected algorithm to obtain a lung three-dimensional image;
clustering the lung three-dimensional images through a mean value clustering algorithm to obtain a binary lung parenchyma three-dimensional connected image;
performing expansion operation on the binarized three-dimensional lung parenchyma connected image to obtain a three-dimensional lung parenchyma outline;
and carrying out mask operation on the three-dimensional lung parenchyma outline and the lung image to obtain a lung parenchyma image.
3. The method according to claim 2, wherein the performing connected reconstruction on the binarized lung image through a three-dimensional connected algorithm to obtain a three-dimensional lung image comprises:
communicating image pixels in the binarized lung image on six neighborhoods in the three-dimensional direction through a three-dimensional communication algorithm to obtain a plurality of communication areas of the binarized lung image;
and reconstructing a plurality of connected regions of the binarized lung image according to a preset sequence to obtain the lung three-dimensional image.
4. The method according to claim 3, wherein the clustering the three-dimensional lung image by a mean value clustering algorithm to obtain a binarized three-dimensional lung parenchyma connected image comprises:
taking the distance between the central point of a plurality of connected regions of the lung three-dimensional image and the central point of the lung three-dimensional image as a clustering characteristic, and clustering the lung three-dimensional image by using a mean value clustering algorithm to obtain a plurality of three-dimensional connected regions of the lung three-dimensional image, wherein the plurality of three-dimensional connected regions comprise a binarized three-dimensional connected region and a non-binarized three-dimensional connected region;
and reserving the binarized three-dimensional connected region of the lung three-dimensional image, and setting the voxel value of the non-binarized three-dimensional connected region of the lung three-dimensional image to be 1 to obtain the binarized lung parenchyma three-dimensional connected image.
5. The method according to claim 1, wherein obtaining a lung nodule segmentation result of the lung image through the trained lung nodule segmentation model according to the lung parenchymal image comprises:
inputting the lung parenchyma image into the trained lung nodule segmentation model to obtain characteristic information of the lung image;
and inputting the characteristic information of the lung image into a nonlinear activation function, and performing binarization processing on the characteristic information of the lung image to obtain a lung nodule segmentation result of the lung image.
6. The method according to any one of claims 1 to 5, wherein the constructing of the lung nodule segmentation model through a densely connected hybrid expansion convolution module and a hierarchical attention mechanism based on the full convolution neural network model comprises:
the encoder and the decoder are used for constructing a full convolution neural network model, wherein the encoder is composed of a densely connected hybrid expansion convolution module, a first batch of normalization layers and a first activation layer, and the decoder is composed of a densely connected deconvolution module, a second batch of normalization layers and a second activation layer;
introducing a hierarchical attention mechanism between the encoder and the decoder to construct the lung nodule segmentation model, wherein feature layers of the encoder and the decoder under the hierarchical attention mechanism are opposite in pairs, and feature pixels in the opposite feature layers are correspondingly multiplied respectively.
7. The method of claim 6, wherein the densely-connected hybrid convolutional module comprises a plurality of first three-dimensional convolutional layers, and each two first three-dimensional convolutional layers of the plurality of first three-dimensional convolutional layers are connected by a dense structure; the densely connected deconvolution module comprises a plurality of second three-dimensional convolution layers, and every two second three-dimensional convolution layers of the plurality of second three-dimensional convolution layers are connected through a dense structure.
8. A lung image segmentation apparatus, characterized in that the apparatus comprises:
the model construction module is used for constructing a lung nodule segmentation model through a densely connected hybrid expansion convolution module and a hierarchy attention mechanism based on a full convolution neural network model;
the model training module is used for training the lung nodule segmentation model by taking a lung parenchyma image sample as input and taking a lung nodule segmentation image of the lung parenchyma image sample as output to obtain a trained lung nodule segmentation model;
the image segmentation module is used for acquiring a lung image to be processed, and segmenting the lung image through a lung parenchyma segmentation algorithm to obtain a lung parenchyma image;
and the image processing module is used for obtaining a lung nodule segmentation result of the lung image through the trained lung nodule segmentation model according to the lung parenchyma image.
9. A storage medium having a computer program stored thereon, the computer program, when being executed by a processor, realizing the steps of the method of any one of claims 1 to 7.
10. A computer arrangement comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the computer program realizes the steps of the method of any one of claims 1 to 7 when executed by the processor.
CN202111015975.8A 2021-08-31 2021-08-31 Lung image segmentation method and device, storage medium and computer equipment Pending CN113643308A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111015975.8A CN113643308A (en) 2021-08-31 2021-08-31 Lung image segmentation method and device, storage medium and computer equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111015975.8A CN113643308A (en) 2021-08-31 2021-08-31 Lung image segmentation method and device, storage medium and computer equipment

Publications (1)

Publication Number Publication Date
CN113643308A true CN113643308A (en) 2021-11-12

Family

ID=78424690

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111015975.8A Pending CN113643308A (en) 2021-08-31 2021-08-31 Lung image segmentation method and device, storage medium and computer equipment

Country Status (1)

Country Link
CN (1) CN113643308A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117496044A (en) * 2023-09-19 2024-02-02 上海睿触科技有限公司 Lung CT image reconstruction method based on deep learning

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108765369A (en) * 2018-04-20 2018-11-06 平安科技(深圳)有限公司 Detection method, device, computer equipment and the storage medium of Lung neoplasm
CN112967279A (en) * 2021-04-02 2021-06-15 慧影医疗科技(北京)有限公司 Method, device, storage medium and electronic equipment for detecting pulmonary nodules

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108765369A (en) * 2018-04-20 2018-11-06 平安科技(深圳)有限公司 Detection method, device, computer equipment and the storage medium of Lung neoplasm
CN112967279A (en) * 2021-04-02 2021-06-15 慧影医疗科技(北京)有限公司 Method, device, storage medium and electronic equipment for detecting pulmonary nodules

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
CHAO YI ETAL: "Attentional Dilated Convolution Neural Network for Nuclei Segmentation in Histopathology Images", 2020 CHINESE AUTOMATION CONGRESS (CAC), 29 January 2021 (2021-01-29), pages 1 - 4 *
NABIL IBTEHAZ ETAL: "MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation", COMPUTER VISION AND PATTERN RECOGNITION, 12 February 2019 (2019-02-12), pages 1 - 25 *
李岩: "图像分割方法介绍及应用", pages 25 - 25, Retrieved from the Internet <URL:https://mp.weixin.qq.com/s/_mGOfvJsHxRZ-nVhcySA6Q> *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117496044A (en) * 2023-09-19 2024-02-02 上海睿触科技有限公司 Lung CT image reconstruction method based on deep learning

Similar Documents

Publication Publication Date Title
CN109598728B (en) Image segmentation method, image segmentation device, diagnostic system, and storage medium
CN108446730B (en) CT pulmonary nodule detection device based on deep learning
CN109410219B (en) Image segmentation method and device based on pyramid fusion learning and computer readable storage medium
CN109685060B (en) Image processing method and device
CN109522908B (en) Image significance detection method based on region label fusion
CN110298383B (en) Multi-mode deep learning-based pathology classification method and system
CN112150428A (en) Medical image segmentation method based on deep learning
CN111640121A (en) Rectum CT image tumor segmentation method based on improved U-net
WO2019184851A1 (en) Image processing method and apparatus, and training method for neural network model
Bandyopadhyay Pre-processing of mammogram images
CN111414923A (en) Indoor scene three-dimensional reconstruction method and system based on single RGB image
WO2023082453A1 (en) Image processing method and device
CN115965750B (en) Vascular reconstruction method, vascular reconstruction device, vascular reconstruction computer device, and vascular reconstruction program
CN112116562A (en) Method, device, equipment and medium for detecting focus based on lung image data
CN113782181A (en) CT image-based lung nodule benign and malignant diagnosis method and device
CN107480673B (en) Method and device for determining interest region in medical image and image editing system
CN116681667A (en) Small sample medical image segmentation method, system, equipment and medium based on self-supervision learning
CN111814832A (en) Target detection method, device and storage medium
CN113643308A (en) Lung image segmentation method and device, storage medium and computer equipment
Major et al. Interpreting medical image classifiers by optimization based counterfactual impact analysis
CN114973057A (en) Video image detection method based on artificial intelligence and related equipment
CN110992310A (en) Method and device for determining partition where mediastinal lymph node is located
Kamencay et al. 3D image reconstruction from 2D CT slices
CN111815591B (en) Lung nodule detection method based on CT image
Rezaei Abkenar et al. Salient region detection using feature extraction in the non‐subsampled contourlet domain

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right

Effective date of registration: 20220525

Address after: 518000 China Aviation Center 2901, No. 1018, Huafu Road, Huahang community, Huaqiang North Street, Futian District, Shenzhen, Guangdong Province

Applicant after: Shenzhen Ping An medical and Health Technology Service Co.,Ltd.

Address before: Room 12G, Area H, 666 Beijing East Road, Huangpu District, Shanghai 200001

Applicant before: PING AN MEDICAL AND HEALTHCARE MANAGEMENT Co.,Ltd.

TA01 Transfer of patent application right