CN112950582A - 3D lung lesion segmentation method and device based on deep learning - Google Patents
3D lung lesion segmentation method and device based on deep learning Download PDFInfo
- Publication number
- CN112950582A CN112950582A CN202110223645.1A CN202110223645A CN112950582A CN 112950582 A CN112950582 A CN 112950582A CN 202110223645 A CN202110223645 A CN 202110223645A CN 112950582 A CN112950582 A CN 112950582A
- Authority
- CN
- China
- Prior art keywords
- image
- regression
- segmentation
- dicom
- spherical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000011218 segmentation Effects 0.000 title claims abstract description 71
- 238000000034 method Methods 0.000 title claims abstract description 50
- 230000003902 lesion Effects 0.000 title claims abstract description 27
- 238000013135 deep learning Methods 0.000 title claims abstract description 23
- 210000004072 lung Anatomy 0.000 title claims abstract description 19
- 238000007781 pre-processing Methods 0.000 claims abstract description 20
- 206010056342 Pulmonary mass Diseases 0.000 claims abstract description 14
- 238000000605 extraction Methods 0.000 claims abstract description 9
- 238000010586 diagram Methods 0.000 claims description 18
- 230000006870 function Effects 0.000 claims description 18
- 238000002372 labelling Methods 0.000 claims description 17
- 238000012549 training Methods 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 13
- 238000004364 calculation method Methods 0.000 claims description 8
- 125000004122 cyclic group Chemical group 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 6
- 238000004891 communication Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 239000000654 additive Substances 0.000 description 2
- 230000000996 additive effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 210000005036 nerve Anatomy 0.000 description 2
- 230000002685 pulmonary effect Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 206010028980 Neoplasm Diseases 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 201000011510 cancer Diseases 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20132—Image cropping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30061—Lung
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
- Apparatus For Radiation Diagnosis (AREA)
Abstract
The invention discloses a 3D lung lesion segmentation method based on deep learning, a device, electronic equipment and a storage medium, wherein the method comprises the steps of acquiring a dicom image of a lung nodule, and preprocessing the dicom image; three-dimensionally stacking the preprocessed dicom images to obtain 3D image blocks, and cutting the 3D image blocks; performing feature extraction on the cut 3D image through a pre-trained spherical segmentation model to obtain a regression subgraph; compared with the conventional mainstream 3D segmentation method, the method is simpler, more convenient and faster, meets the shape characteristics of the nodular sphere, achieves the precision close to that of the mainstream segmentation method, and provides a new idea.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a 3D lung lesion segmentation method and device based on deep learning, electronic equipment and a storage medium.
Background
In medicine, in the medical field, medical images become a step of accurate diagnosis and inseparable lesion location, but some lesions are small and difficult to find, so that doctors often neglect the lesions to cause misdiagnosis, and the shapes and sizes of the lesions are inconsistent, so that the doctors are relatively difficult to measure the lesions.
The deep learning method is proposed from the end of the 20 th century, has been well developed so far, simulates the human nerve composition, and completes the simulation of nerve connection by using a large amount of data, thereby achieving the effect of simulating the human brain and completing the work which can not even be completed by some professionals.
The LIDC-IDRI is a data set composed of chest medical image files (e.g., CT, X-ray film) and corresponding diagnostic lesion labels. This data was collected by the National Cancer Institute (National Cancer Institute) initiative for the purpose of studying early Cancer detection in high risk groups. The data set contains CT images of 1018 patients, and lung nodule labeling at a pixel-by-pixel level is completed, and certain labeling is also performed on classification of lung nodules.
The current method for detecting and segmenting the LIDC-IDRI by using the deep learning scheme mainly detects lung nodules first, and performs pixel-by-pixel (2D) or voxel-by-voxel (3D) segmentation according to the detection result, so as to obtain a description of the volume of the lung nodules and other corresponding descriptions. Current methods, however, involve the use of a two-stage model, or one-stage detection and segmentation. However, such a method results in a correspondingly high time consumption or a correspondingly high memory usage.
Disclosure of Invention
The invention aims to provide a 3D lung lesion segmentation method, a device, electronic equipment and a storage medium based on deep learning, which do not need to detect and segment again firstly, and can directly segment through a 3D image to obtain a corresponding result.
In a first aspect, an embodiment of the present invention provides a deep learning 3D lung lesion segmentation method, including the following steps:
acquiring a dicom image of a lung nodule, and preprocessing the dicom image;
three-dimensionally stacking the preprocessed dicom images to obtain 3D image blocks, and cutting the 3D image blocks;
performing feature extraction on the cut 3D image through a pre-trained spherical segmentation model to obtain a regression subgraph;
and calculating the product of the centrality and the probability of the regression subgraph to obtain a plurality of central point coordinates, and obtaining coordinates of the regression point through the central point coordinates to obtain a segmentation result.
Optionally, the training process of the spherical segmentation model includes:
preprocessing the labeling data of the dicom image to obtain spherical coordinate data;
extracting a feature map based on the constructed network;
constructing a feature map pyramid network FPN and a cyclic feature pyramid network RFP based on ResNeSt-34;
and dividing the characteristic diagram into three parts through the FPN and the RFP, calculating a loss function of the spherical coordinate data of the characteristic diagram, and solving an optimized model.
Optionally, pre-processing the dicom image comprises;
image normalization and image enhancement;
the image normalization comprises the steps of adjusting the image value of the dicom image through the window width and window level to obtain an image normalized to the pixel value of 0-255 by using the window width and window level values of 500-1500 respectively.
Optionally, the pre-processing the labeling data of the dicom image to obtain the ball coordinate data includes:
obtaining coordinate information and segmentation contours of data according to the labeling data of the dicom image;
carrying out consistent interpolation on the labeling data of the dicom image according to the space distance, and setting the unit value of each dimension of the voxel as the same value in an interpolation mode of 3 times of linear interpolation;
after interpolation of the labeled data of the dicom image is completed, calculating the spherical coordinate space position of the regression pixel; wherein the spatial expression of the spherical coordinates is as follows:
z=rcosθ
wherein,n is taken to be the value 36 in the present invention,m is 36 in the invention, and r is the sphere center coordinate from each corresponding theta andthe angle reaches the length of the edge of the object.
Further, dividing the feature map into three parts through the FPN and RFP to calculate a loss function for the spherical coordinate data of the feature map comprises:
the first part regresses the corresponding classification of the characteristic map, its size is D H W k, DHW corresponds to the length of the characteristic map separately and is tall and wide three dimensions, k is the classification number, set up as 1, activate the function as sigmoid, namely judge whether it is a nodule through the probability value, the loss function used is Focal loss;
the second part is the centrality of the spherical coordinates corresponding to the feature map, the size of the centrality is D H W1, DHW is three dimensions of the length, the width and the height of the feature map, and the centrality is calculated by the following formula:
wherein d isiIs the length of the first ray in which the spherical coordinates would regress n rays, where n in the present invention is taken as 72 (representing 36+36), in terms of the sum of θ and θ of the spherical coordinatesUniformly taking values, namely taking 36 values of theta from 0-2 pi at equal intervals,taking 36 values from 0-2 pi at equal intervals to obtain 72 regression lines which represent regression distances from a central point to 72 directions;
the third part is the distance from the central point on n angles of the spherical coordinates corresponding to the regression feature map, the size of the distance is D H W n, DHW is three dimensions of length, width and height, n is n angles, and as shown in the second part, n is 72 in the invention.
In a second aspect, an embodiment of the present invention provides a deep learning-based 3D lung lesion segmentation apparatus, where the segmentation apparatus includes:
the image processing module is used for acquiring a lung nodule dicom image and preprocessing the dicom image;
the 3D image acquisition module is used for three-dimensionally stacking the preprocessed dicom images to obtain 3D image blocks and cutting the 3D image blocks;
the regression subgraph acquisition module is used for extracting the characteristics of the cut 3D image through a pre-trained spherical segmentation model to obtain a regression subgraph;
and the calculation module is used for calculating the product of the centrality and the probability of the regression subgraph to obtain a plurality of central point coordinates, and obtaining the coordinates of the regression points through the central point coordinates to obtain a segmentation result.
In a third aspect, the present invention provides an electronic device, comprising:
a processor; a memory for storing processor-executable instructions;
wherein the processor implements the above method by executing the executable instructions.
In a fourth aspect, the present invention provides a computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the steps of the above-described method.
Advantageous effects
The invention provides a 3D lung lesion segmentation method based on deep learning, a device, electronic equipment and a storage medium, wherein the method comprises the steps of acquiring a dicom image of a lung nodule, and preprocessing the dicom image; three-dimensionally stacking the preprocessed dicom images to obtain 3D image blocks, and cutting the 3D image blocks; performing feature extraction on the cut 3D image through a pre-trained spherical segmentation model to obtain a regression subgraph; the regression subgraph is subjected to product calculation of centrality and probability to obtain a plurality of central point coordinates, coordinates of regression points are obtained through the central point coordinates, and segmentation results are obtained.
Drawings
Fig. 1 is a flowchart of a deep learning-based 3D lung lesion segmentation method according to an embodiment of the present invention;
FIG. 2 is a training process of a spherical segmentation model according to an embodiment of the present invention;
fig. 3 is a flowchart illustrating preprocessing of annotation data of a dicom image to obtain spherical coordinate data according to an embodiment of the present invention;
fig. 4 is a diagram illustrating coordinate information and a segmentation contour of data obtained by performing uniform interpolation according to spatial distance through labeled data of a dicom image according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of the calculation of the loss function of the spherical coordinate data of the feature map by dividing the feature map into three parts through the FPN and RFP according to the embodiment of the present invention;
fig. 6 is a block diagram illustrating a 3D lung lesion segmentation apparatus based on deep learning according to an embodiment of the present invention;
fig. 7 is a block diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the following embodiments, and it should be understood that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The present invention aims to provide a method, an apparatus, an electronic device and a storage medium for segmenting a 3D lung lesion based on deep learning, which do not need to detect and then segment first, and segment directly through a 3D image to obtain a corresponding result, and the following description and specific embodiments are further described with reference to the accompanying drawings:
fig. 1 shows a flowchart of a deep learning-based 3D lung lesion segmentation method according to an embodiment of the present invention, as shown in fig. 1, the segmentation method includes the following steps:
s20, acquiring a lung nodule dicom image, and preprocessing the dicom image;
s40, carrying out three-dimensional stacking on the preprocessed dicom images to obtain 3D image blocks, and cutting the 3D image blocks;
s60, performing feature extraction on the cut 3D image through a pre-trained spherical segmentation model to obtain a regression subgraph;
s80, calculating the product of the centrality and the probability of the regression subgraph to obtain a plurality of central point coordinates, and obtaining coordinates of the regression points through the central point coordinates to obtain a segmentation result.
In the method of the embodiment, a lung nodule dicom image is obtained, and the dicom image is preprocessed; three-dimensionally stacking the preprocessed dicom images to obtain 3D image blocks, and cutting the 3D image blocks; performing feature extraction on the cut 3D image through a pre-trained spherical segmentation model to obtain a regression subgraph; the regression subgraph is subjected to product calculation of centrality and probability to obtain a plurality of central point coordinates, coordinates of regression points are obtained through the central point coordinates, and segmentation results are obtained.
Specifically, as shown in fig. 2, the training process of the spherical segmentation model includes:
s601, preprocessing the labeling data of the dicom image to obtain spherical coordinate data;
s602, extracting a feature graph based on the constructed network;
s603, constructing a feature map pyramid network FPN and a cyclic feature pyramid network RFP based on ResNeSt-34;
s604, dividing the characteristic diagram into three parts through the FPN and the RFP, calculating a loss function of the spherical coordinate data of the characteristic diagram, and solving an optimized model.
Specifically, as shown in fig. 3 to 4, the pre-processing the labeling data of the dicom image to obtain the spherical coordinate data includes:
s6011, obtaining coordinate information and a segmentation contour of data according to the labeling data of the dicom image;
s6012, carrying out consistent interpolation on the labeling data of the dicom image according to the spatial distance, and setting the unit value of each dimension of the voxel as the same value in an interpolation mode of 3 times of linear interpolation;
s6013, after interpolation of the labeled data of the dicom image is completed, calculating a spherical coordinate space position of a regression pixel; wherein the spatial expression of the spherical coordinates is as follows:
wherein, among others,n is taken to be the value 36 in the present invention,m is 36 in the invention, and r is the sphere center coordinate from each corresponding theta andthe angle reaches the length of the edge of the object.
In the data processing process, the coordinate information and the segmentation contour of the data are obtained through the labeled data of the dicom image, consistent interpolation is performed according to the spatial distance, and the size of the voxel (unit value of each dimension of the voxel) is set to be the same value in an interpolation mode, that is, the unit size of each dimension is the same millimeter number, for example, one voxel corresponds to a spatial size of 0.6 × 0.6 mm. By processing the corresponding regression radius and angle through the method, the model can more easily perform radius and additive regression on the same pixel value, and the model can easily complete better training. After the interpolation of the original data is finished, calculating the spherical coordinate space position of the regression pixel, wherein in the data processing process, the distance from the central point of each pixel to the node segmentation edge is calculated by calculating the distance from the central point of each pixel to the node segmentation edge as shown in the formula (1), n values are equally spaced from 0-2 pi according to theta, wherein n is 36 in the invention, and phi is also equally spaced from 0-2 pi similarly, and the number of the values is also n; phi and n are respectively used as submodules of two 36 channels after sampling based on the output of the feature map pyramid network FPN and the cyclic feature pyramid network RFP of ResNeSt-34 to carry out regression training, and the regression target is the distance from the central point to the outer edge of the nodule at each angle corresponding to phi and n. And regressing the distances from the center point to the 36 phi and 36 n different directions on each pixel, thereby obtaining the segmentation result of the spherical coordinates.
Specifically, as shown in fig. 5, dividing the feature map into three parts through the FPN and RFP to calculate the loss function of the spherical coordinate data of the feature map includes:
the first part regresses the corresponding classification of the characteristic map, its size is D H W k, DHW corresponds to the length of the characteristic map separately and is tall and wide three dimensions, k is the classification number, set up as 1, activate the function as sigmoid, namely judge whether it is a nodule through the probability value, the loss function used is Focal loss; the probability values are calculated by the neural networks FPN and RFP, and the neural networks FPN and RFP learn the probability calculation result on each feature map corresponding to the classification part in fig. 4 by learning the label pattern.
The second part is the centrality of the spherical coordinates corresponding to the feature map, the size of the centrality is D H W1, DHW is three dimensions of the length, the width and the height of the feature map, and the centrality is calculated by the following formula:
wherein d isiIs the length of the first ray in which the spherical coordinates would regress n rays, where n in the present invention is taken as 72 (representing 36+36), in terms of the sum of θ and θ of the spherical coordinatesUniformly taking values, namely taking 36 values of theta from 0-2 pi at equal intervals,taking 36 values from 0-2 pi at equal intervals to obtain 72 regression lines which represent regression distances from a central point to 72 directions;
the third part is the distance from the central point on n angles of the spherical coordinates corresponding to the regression feature map, the size of the distance is D H W n, DHW is three dimensions of length, width and height, n is n angles, and as shown in the second part, n is 72 in the invention.
In some embodiments, pre-processing the dicom image comprises;
image normalization and image enhancement;
the image normalization comprises the steps of adjusting the image value of the dicom image through the window width and window level to obtain an image normalized to the pixel value of 0-255 by using the window width and window level values of 500-1500 respectively. Specifically, first xc,i,jThe pixel values of the ith row and the jth column in the c channel of a picture are represented, window width window level values are respectively 500-1500 in the invention, namely, an image obtained from a dicom image is normalized to an image with the pixel value of 0-255 by adjusting the image value through the window width window level. In the present invention, c is {1},and (3) training parameter configuration:
in the training, the initial learning rate is set to 0.01, the training passage number is 50 epochs, the learning rate updating mode is WarmUpCosinaleleringrate, namely, in the first 5 epochs, the learning rate is increased to 0.01 from 0.002 by the fluctuation of 0.2 per epoch, and in the subsequent 45 epochs, according to the formulaAnd calculating, wherein n is the total number of epochs for training, 150 is taken in the experiment, and e is the current epoch number. The optimizer used in the training was an Adam optimizer and the loss function was MSE.
In the actual data processing process, limited by the storage of the display card in the training process, the invention performs 128 × 128 pixel block taking on n × 512 images, and due to the limitation of the storage of the display card, the value taking can simultaneously see the image details, and can ensure that the program runs smoothly without errors, and the result of each block is spliced back to the final result, wherein n is the number of images, and as each dicom sequence has a plurality of dicoms stacked to form a 3D structure. The final result is obtained by multiplying the centrality and the probability value of each feature map pixel, and the results are sorted according to the product of the centrality and the probability value.
In the application of an actual scene, firstly, reading a dicom image through pydicom, carrying out fixed window width window level conversion on the dicom image, carrying out three-dimensional stacking on a plurality of dicom images to obtain a 3D image block, cutting the 3D image block, wherein the size of the cut image is 128 × 128, carrying out feature extraction on the cut image by using a statistical Segmentation model to obtain a regression subgraph, then carrying out product calculation of the central degree and the probability value to obtain a plurality of central point coordinates, and obtaining the coordinates of the regression point through the central point coordinates to obtain the final Segmentation result.
Based on the same inventive concept, the embodiment of the present invention further provides a 3D lung lesion segmentation apparatus based on deep learning, which can be used to implement the method described in the above embodiments, as described in the following embodiments. Because the principle of solving the problem of the 3D lung lesion segmentation device based on deep learning is similar to that of a 3D lung lesion segmentation method based on deep learning, the implementation of the 3D lung lesion segmentation device based on deep learning can refer to the implementation of the 3D lung lesion segmentation method based on deep learning, and repeated points are not repeated. As used hereinafter, the term "unit" or "module" may be a combination of software and/or hardware that implements a predetermined function. While the system described in the embodiments below is preferably implemented in software, implementations in hardware, or a combination of software and hardware are also possible and contemplated.
As shown in fig. 6, the segmenting apparatus provided in the embodiment of the present invention includes:
the image processing module 20 is configured to acquire a dicom image of a lung nodule and perform preprocessing on the dicom image;
the 3D image acquisition module 40 is used for three-dimensionally stacking the preprocessed dicom images to obtain 3D image blocks and cutting the 3D image blocks;
the regression subgraph obtaining module 60 is used for performing feature extraction on the cut 3D image through a pre-trained spherical segmentation model to obtain a regression subgraph;
and the calculating module 80 is configured to calculate a product of centrality and probability of the regression subgraph to obtain a plurality of central point coordinates, and obtain coordinates of a regression point through the central point coordinates to obtain a segmentation result.
The segmentation device of the embodiment acquires a dicom image of a lung nodule through an image processing module 20, and pre-processes the dicom image; the 3D image acquisition module 40 three-dimensionally stacks the preprocessed dicom images to obtain 3D image blocks, and cuts the 3D image blocks; the regression subgraph obtaining module 60 performs feature extraction on the cut 3D image through a spherical segmentation model trained in advance to obtain a regression subgraph; the calculation module 80 calculates the product of the centrality and the probability of the regression subgraph to obtain a plurality of central point coordinates, and obtains the coordinates of the regression points through the central point coordinates to obtain the segmentation results.
Specifically, as shown in fig. 2, the training process of the spherical segmentation model includes:
s601, preprocessing the labeling data of the dicom image to obtain spherical coordinate data;
s602, extracting a characteristic diagram based on the network ResNeSt-34;
s603, constructing a feature map pyramid network FPN and a cyclic feature pyramid network RFP based on ResNeSt-34;
s604, dividing the characteristic diagram into three parts through the FPN and the RFP, calculating a loss function of the spherical coordinate data of the characteristic diagram, and solving an optimized model.
Specifically, as shown in fig. 3 to 4, the pre-processing the labeling data of the dicom image to obtain the spherical coordinate data includes:
s6011, obtaining coordinate information and a segmentation contour of data according to the labeling data of the dicom image;
s6012, carrying out consistent interpolation on the labeling data of the dicom image according to the spatial distance, and setting the unit value of each dimension of the voxel as the same value in an interpolation mode of 3 times of linear interpolation;
s6013, after interpolation of the labeled data of the dicom image is completed, calculating a spherical coordinate space position of a regression pixel; wherein the spatial expression of the spherical coordinates is as follows:
wherein, among others,n is taken to be the value 36 in the present invention,m is 36 in the invention, and r is the sphere center coordinate from each corresponding theta andthe angle reaches the length of the edge of the object.
In the data processing process, the coordinate information and the segmentation contour of the data are obtained through the labeled data of the dicom image, consistent interpolation is performed according to the spatial distance, and the size of the voxel (unit value of each dimension of the voxel) is set to be the same value in an interpolation mode, that is, the unit size of each dimension is the same millimeter number, for example, one voxel corresponds to a spatial size of 0.6 × 0.6 mm. By processing the corresponding regression radius and angle through the method, the model can more easily perform radius and additive regression on the same pixel value, and the model can easily complete better training. After the interpolation of the original data is finished, calculating the spherical coordinate space position of the regression pixel, wherein in the data processing process, the distance from the central point of each pixel to the node segmentation edge is calculated by calculating the distance from the central point of each pixel to the node segmentation edge as shown in the formula (1), n values are equally spaced from 0-2 pi according to theta, wherein n is 36 in the invention, and phi is also equally spaced from 0-2 pi similarly, and the number of the values is also n; phi and n are respectively used as submodules of two 36 channels after sampling based on the output of the feature map pyramid network FPN and the cyclic feature pyramid network RFP of ResNeSt-34 to carry out regression training, and the regression target is the distance from the central point to the outer edge of the nodule at each angle corresponding to phi and n. And regressing the distances from the center point to the 36 phi and 36 n different directions on each pixel, thereby obtaining the segmentation result of the spherical coordinates.
An electronic device is further provided in an embodiment of the present invention, and fig. 7 shows a schematic structural diagram of an electronic device to which an embodiment of the present invention can be applied, and as shown in fig. 7, the computer electronic device includes a Central Processing Unit (CPU)701 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data necessary for system operation are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
The following components are connected to the I/O interface 705: an input portion 706 including a keyboard, a mouse, and the like; an output section 707 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker and the like; a storage section 708 including a hard disk and the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. The driver 310 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read out therefrom is mounted into the storage section 708 as necessary.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The present invention also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the apparatus for deep learning based 3D pulmonary lesion segmentation in the above embodiments; or it may be a computer-readable storage medium that exists separately and is not built into the electronic device. The computer readable storage medium stores one or more programs for use by one or more processors in performing the deep learning based 3D pulmonary lesion segmentation method described in the present invention.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (8)
1. A3D lung lesion segmentation method based on deep learning is characterized by comprising the following steps:
acquiring a dicom image of a lung nodule, and preprocessing the dicom image;
three-dimensionally stacking the preprocessed dicom images to obtain 3D image blocks, and cutting the 3D image blocks;
performing feature extraction on the cut 3D image through a pre-trained spherical segmentation model to obtain a regression subgraph;
and calculating the product of the centrality and the probability of the regression subgraph to obtain a plurality of central point coordinates, and obtaining coordinates of the regression point through the central point coordinates to obtain a segmentation result.
2. The segmentation method according to claim 1, wherein the training process of the spherical segmentation model comprises:
preprocessing the labeling data of the dicom image to obtain spherical coordinate data;
constructing a feature map pyramid network FPN and a cyclic feature pyramid network RFP based on ResNeSt-34;
extracting a feature map based on the constructed network;
and dividing the characteristic diagram into three parts through the FPN and the RFP, calculating a loss function of the spherical coordinate data of the characteristic diagram, and solving an optimized model.
3. The segmentation method according to claim 1, wherein pre-processing the dicom image comprises;
image normalization and image enhancement;
the image normalization comprises the steps of adjusting the image value of the dicom image through the window width and window level to obtain an image normalized to the pixel value of 0-255 by using the window width and window level values of 500-1500 respectively.
4. The segmentation method as claimed in claim 2, wherein the pre-processing of the annotation data of the dicom image to obtain the spherical coordinate data comprises:
obtaining coordinate information and segmentation contours of data according to the labeling data of the dicom image;
carrying out consistent interpolation on the labeling data of the dicom image according to the space distance, and setting the unit value of each dimension of the voxel as the same value in an interpolation mode of 3 times of linear interpolation;
after interpolation of the labeled data of the dicom image is completed, calculating the spherical coordinate space position of the regression pixel; wherein the spatial expression of the spherical coordinates is as follows:
z=rcosθ
5. The segmentation method as claimed in claim 4, wherein the step of dividing the feature map into three parts through the FPN and RFP comprises the steps of:
the first part regresses the corresponding classification of the characteristic map, its size is D H W k, DHW corresponds to the length of the characteristic map separately, k is the classification number, set up as 1, activate the function as sigmoid, namely judge whether it is a node through the probability value, the loss function used is Focalloss;
the second part is the centrality of the spherical coordinates corresponding to the feature map, the size of the centrality is D H W1, DHW is three dimensions of the length, the width and the height of the feature map, and the centrality is calculated by the following formula:
wherein d isiIs the length of the first ray in which the spherical coordinates would regress n rays, where n in the present invention is taken as 72 (representing 36+36), in terms of the sum of θ and θ of the spherical coordinatesUniformly taking values, namely taking 36 values of theta from 0-2 pi at equal intervals,taking 36 values from 0-2 pi at equal intervals to obtain 72 regression lines which represent regression distances from a central point to 72 directions;
the third part is the distance from the center point on n angles of the spherical coordinates corresponding to the regression feature map, the size of the distance is D H W n, DHW is three dimensions of length, width and height, n is n angles, and as shown in the second part, n is 72 in the invention.
6. A deep learning-based 3D lung lesion segmentation apparatus, the segmentation apparatus comprising:
the image processing module is used for acquiring a lung nodule dicom image and preprocessing the dicom image;
the 3D image acquisition module is used for three-dimensionally stacking the preprocessed dicom images to obtain 3D image blocks and cutting the 3D image blocks;
the regression subgraph acquisition module is used for extracting the characteristics of the cut 3D image through a pre-trained spherical segmentation model to obtain a regression subgraph;
and the calculation module is used for calculating the product of the centrality and the probability of the regression subgraph to obtain a plurality of central point coordinates, and obtaining the coordinates of the regression points through the central point coordinates to obtain the segmentation results.
7. An electronic device, comprising:
a processor, a memory for storing processor-executable instructions;
wherein the processor implements the method of any one of claims 1-5 by executing the executable instructions.
8. A computer readable storage medium having stored thereon computer instructions which, when executed by a processor, carry out the steps of the method according to any one of claims 1 to 5.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110223645.1A CN112950582B (en) | 2021-03-01 | 2021-03-01 | 3D lung focus segmentation method and device based on deep learning |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110223645.1A CN112950582B (en) | 2021-03-01 | 2021-03-01 | 3D lung focus segmentation method and device based on deep learning |
Publications (2)
Publication Number | Publication Date |
---|---|
CN112950582A true CN112950582A (en) | 2021-06-11 |
CN112950582B CN112950582B (en) | 2023-11-24 |
Family
ID=76246858
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202110223645.1A Active CN112950582B (en) | 2021-03-01 | 2021-03-01 | 3D lung focus segmentation method and device based on deep learning |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN112950582B (en) |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180122082A1 (en) * | 2016-11-02 | 2018-05-03 | General Electric Company | Automated segmentation using deep learned priors |
CN110689547A (en) * | 2019-09-25 | 2020-01-14 | 重庆大学 | Pulmonary nodule segmentation method based on three-dimensional CT image |
CN110807764A (en) * | 2019-09-20 | 2020-02-18 | 成都智能迭迦科技合伙企业(有限合伙) | Lung cancer screening method based on neural network |
US20200160997A1 (en) * | 2018-11-02 | 2020-05-21 | University Of Central Florida Research Foundation, Inc. | Method for detection and diagnosis of lung and pancreatic cancers from imaging scans |
CN111553892A (en) * | 2020-04-23 | 2020-08-18 | 北京小白世纪网络科技有限公司 | Lung nodule segmentation calculation method, device and system based on deep learning |
CN111932540A (en) * | 2020-10-14 | 2020-11-13 | 北京信诺卫康科技有限公司 | CT image contrast characteristic learning method for clinical typing of new coronary pneumonia |
-
2021
- 2021-03-01 CN CN202110223645.1A patent/CN112950582B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180122082A1 (en) * | 2016-11-02 | 2018-05-03 | General Electric Company | Automated segmentation using deep learned priors |
US20200160997A1 (en) * | 2018-11-02 | 2020-05-21 | University Of Central Florida Research Foundation, Inc. | Method for detection and diagnosis of lung and pancreatic cancers from imaging scans |
CN110807764A (en) * | 2019-09-20 | 2020-02-18 | 成都智能迭迦科技合伙企业(有限合伙) | Lung cancer screening method based on neural network |
CN110689547A (en) * | 2019-09-25 | 2020-01-14 | 重庆大学 | Pulmonary nodule segmentation method based on three-dimensional CT image |
CN111553892A (en) * | 2020-04-23 | 2020-08-18 | 北京小白世纪网络科技有限公司 | Lung nodule segmentation calculation method, device and system based on deep learning |
CN111932540A (en) * | 2020-10-14 | 2020-11-13 | 北京信诺卫康科技有限公司 | CT image contrast characteristic learning method for clinical typing of new coronary pneumonia |
Non-Patent Citations (2)
Title |
---|
YEGANEH JALALI, MANSOOR FATEH, MOHSEN REZVANI, VAHID ABOLGHASEMI, MOHAMMAD HOSSEIN ANISI: "ResBCDU-Net: A Deep Learning Framework for Lung CT Image Segmentation", Retrieved from the Internet <URL:https://pubmed.ncbi.nlm.nih.gov/33401581/> * |
陈万顺: "基于深度学习的肺结节检测方法", 中国优秀硕士学位论文全文数据库医药卫生科技辑 * |
Also Published As
Publication number | Publication date |
---|---|
CN112950582B (en) | 2023-11-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3979198A1 (en) | Image segmentation model training method and apparatus, computer device, and storage medium | |
US11288808B2 (en) | System and method for n-dimensional image segmentation using convolutional neural networks | |
US11887311B2 (en) | Method and apparatus for segmenting a medical image, and storage medium | |
US20210390706A1 (en) | Detection model training method and apparatus, computer device and storage medium | |
CN110599528A (en) | Unsupervised three-dimensional medical image registration method and system based on neural network | |
US10853409B2 (en) | Systems and methods for image search | |
CN111932529B (en) | Image classification and segmentation method, device and system | |
Li et al. | Automated measurement network for accurate segmentation and parameter modification in fetal head ultrasound images | |
CN111488872B (en) | Image detection method, image detection device, computer equipment and storage medium | |
CN111932552B (en) | Aorta modeling method and device | |
WO2020234349A1 (en) | Sampling latent variables to generate multiple segmentations of an image | |
CN112750137B (en) | Liver tumor segmentation method and system based on deep learning | |
CN113240661B (en) | Deep learning-based lumbar vertebra bone analysis method, device, equipment and storage medium | |
CN112597847A (en) | Face pose estimation method and device, electronic equipment and storage medium | |
CN114792326A (en) | Surgical navigation point cloud segmentation and registration method based on structured light | |
CN117237322A (en) | Organ segmentation modeling method and terminal based on medical image | |
CN115546270A (en) | Image registration method, model training method and equipment for multi-scale feature fusion | |
CN108597589B (en) | Model generation method, target detection method and medical imaging system | |
CN113850796A (en) | Lung disease identification method and device based on CT data, medium and electronic equipment | |
CN116309551B (en) | Method, device, electronic equipment and readable medium for determining focus sampling area | |
CN112950582B (en) | 3D lung focus segmentation method and device based on deep learning | |
CN113408595B (en) | Pathological image processing method and device, electronic equipment and readable storage medium | |
CN115439478A (en) | Lung lobe perfusion strength evaluation method, system, equipment and medium based on lung perfusion | |
CN111598904B (en) | Image segmentation method, device, equipment and storage medium | |
Ye et al. | Active contour image segmentation method for training talents of computer graphics and image processing technology |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |