CN112950582A - 3D lung lesion segmentation method and device based on deep learning - Google Patents

3D lung lesion segmentation method and device based on deep learning Download PDF

Info

Publication number
CN112950582A
CN112950582A CN202110223645.1A CN202110223645A CN112950582A CN 112950582 A CN112950582 A CN 112950582A CN 202110223645 A CN202110223645 A CN 202110223645A CN 112950582 A CN112950582 A CN 112950582A
Authority
CN
China
Prior art keywords
image
regression
segmentation
dicom
spherical
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110223645.1A
Other languages
Chinese (zh)
Other versions
CN112950582B (en
Inventor
杜强
陈相儒
郭雨晨
聂方兴
唐超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Xbentury Network Technology Co ltd
Original Assignee
Beijing Xbentury Network Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Xbentury Network Technology Co ltd filed Critical Beijing Xbentury Network Technology Co ltd
Priority to CN202110223645.1A priority Critical patent/CN112950582B/en
Publication of CN112950582A publication Critical patent/CN112950582A/en
Application granted granted Critical
Publication of CN112950582B publication Critical patent/CN112950582B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G06T2207/10012—Stereo images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20132—Image cropping
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30061—Lung

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

The invention discloses a 3D lung lesion segmentation method based on deep learning, a device, electronic equipment and a storage medium, wherein the method comprises the steps of acquiring a dicom image of a lung nodule, and preprocessing the dicom image; three-dimensionally stacking the preprocessed dicom images to obtain 3D image blocks, and cutting the 3D image blocks; performing feature extraction on the cut 3D image through a pre-trained spherical segmentation model to obtain a regression subgraph; compared with the conventional mainstream 3D segmentation method, the method is simpler, more convenient and faster, meets the shape characteristics of the nodular sphere, achieves the precision close to that of the mainstream segmentation method, and provides a new idea.

Description

3D lung lesion segmentation method and device based on deep learning
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a 3D lung lesion segmentation method and device based on deep learning, electronic equipment and a storage medium.
Background
In medicine, in the medical field, medical images become a step of accurate diagnosis and inseparable lesion location, but some lesions are small and difficult to find, so that doctors often neglect the lesions to cause misdiagnosis, and the shapes and sizes of the lesions are inconsistent, so that the doctors are relatively difficult to measure the lesions.
The deep learning method is proposed from the end of the 20 th century, has been well developed so far, simulates the human nerve composition, and completes the simulation of nerve connection by using a large amount of data, thereby achieving the effect of simulating the human brain and completing the work which can not even be completed by some professionals.
The LIDC-IDRI is a data set composed of chest medical image files (e.g., CT, X-ray film) and corresponding diagnostic lesion labels. This data was collected by the National Cancer Institute (National Cancer Institute) initiative for the purpose of studying early Cancer detection in high risk groups. The data set contains CT images of 1018 patients, and lung nodule labeling at a pixel-by-pixel level is completed, and certain labeling is also performed on classification of lung nodules.
The current method for detecting and segmenting the LIDC-IDRI by using the deep learning scheme mainly detects lung nodules first, and performs pixel-by-pixel (2D) or voxel-by-voxel (3D) segmentation according to the detection result, so as to obtain a description of the volume of the lung nodules and other corresponding descriptions. Current methods, however, involve the use of a two-stage model, or one-stage detection and segmentation. However, such a method results in a correspondingly high time consumption or a correspondingly high memory usage.
Disclosure of Invention
The invention aims to provide a 3D lung lesion segmentation method, a device, electronic equipment and a storage medium based on deep learning, which do not need to detect and segment again firstly, and can directly segment through a 3D image to obtain a corresponding result.
In a first aspect, an embodiment of the present invention provides a deep learning 3D lung lesion segmentation method, including the following steps:
acquiring a dicom image of a lung nodule, and preprocessing the dicom image;
three-dimensionally stacking the preprocessed dicom images to obtain 3D image blocks, and cutting the 3D image blocks;
performing feature extraction on the cut 3D image through a pre-trained spherical segmentation model to obtain a regression subgraph;
and calculating the product of the centrality and the probability of the regression subgraph to obtain a plurality of central point coordinates, and obtaining coordinates of the regression point through the central point coordinates to obtain a segmentation result.
Optionally, the training process of the spherical segmentation model includes:
preprocessing the labeling data of the dicom image to obtain spherical coordinate data;
extracting a feature map based on the constructed network;
constructing a feature map pyramid network FPN and a cyclic feature pyramid network RFP based on ResNeSt-34;
and dividing the characteristic diagram into three parts through the FPN and the RFP, calculating a loss function of the spherical coordinate data of the characteristic diagram, and solving an optimized model.
Optionally, pre-processing the dicom image comprises;
image normalization and image enhancement;
the image normalization comprises the steps of adjusting the image value of the dicom image through the window width and window level to obtain an image normalized to the pixel value of 0-255 by using the window width and window level values of 500-1500 respectively.
Optionally, the pre-processing the labeling data of the dicom image to obtain the ball coordinate data includes:
obtaining coordinate information and segmentation contours of data according to the labeling data of the dicom image;
carrying out consistent interpolation on the labeling data of the dicom image according to the space distance, and setting the unit value of each dimension of the voxel as the same value in an interpolation mode of 3 times of linear interpolation;
after interpolation of the labeled data of the dicom image is completed, calculating the spherical coordinate space position of the regression pixel; wherein the spatial expression of the spherical coordinates is as follows:
Figure BDA0002955907200000031
Figure BDA0002955907200000032
z＝rcosθ
wherein,
Figure BDA0002955907200000033
n is taken to be the value 36 in the present invention,
Figure BDA0002955907200000034
m is 36 in the invention, and r is the sphere center coordinate from each corresponding theta and
Figure BDA0002955907200000035
the angle reaches the length of the edge of the object.
Further, dividing the feature map into three parts through the FPN and RFP to calculate a loss function for the spherical coordinate data of the feature map comprises:
the first part regresses the corresponding classification of the characteristic map, its size is D H W k, DHW corresponds to the length of the characteristic map separately and is tall and wide three dimensions, k is the classification number, set up as 1, activate the function as sigmoid, namely judge whether it is a nodule through the probability value, the loss function used is Focal loss;
the second part is the centrality of the spherical coordinates corresponding to the feature map, the size of the centrality is D H W1, DHW is three dimensions of the length, the width and the height of the feature map, and the centrality is calculated by the following formula:
Figure BDA0002955907200000036
wherein d isiIs the length of the first ray in which the spherical coordinates would regress n rays, where n in the present invention is taken as 72 (representing 36+36), in terms of the sum of θ and θ of the spherical coordinates
Figure BDA0002955907200000037
Uniformly taking values, namely taking 36 values of theta from 0-2 pi at equal intervals,
Figure BDA0002955907200000038
taking 36 values from 0-2 pi at equal intervals to obtain 72 regression lines which represent regression distances from a central point to 72 directions;
the third part is the distance from the central point on n angles of the spherical coordinates corresponding to the regression feature map, the size of the distance is D H W n, DHW is three dimensions of length, width and height, n is n angles, and as shown in the second part, n is 72 in the invention.
In a second aspect, an embodiment of the present invention provides a deep learning-based 3D lung lesion segmentation apparatus, where the segmentation apparatus includes:
the image processing module is used for acquiring a lung nodule dicom image and preprocessing the dicom image;
the 3D image acquisition module is used for three-dimensionally stacking the preprocessed dicom images to obtain 3D image blocks and cutting the 3D image blocks;
the regression subgraph acquisition module is used for extracting the characteristics of the cut 3D image through a pre-trained spherical segmentation model to obtain a regression subgraph;
and the calculation module is used for calculating the product of the centrality and the probability of the regression subgraph to obtain a plurality of central point coordinates, and obtaining the coordinates of the regression points through the central point coordinates to obtain a segmentation result.
In a third aspect, the present invention provides an electronic device, comprising:
a processor; a memory for storing processor-executable instructions;
wherein the processor implements the above method by executing the executable instructions.
In a fourth aspect, the present invention provides a computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the steps of the above-described method.
Advantageous effects
The invention provides a 3D lung lesion segmentation method based on deep learning, a device, electronic equipment and a storage medium, wherein the method comprises the steps of acquiring a dicom image of a lung nodule, and preprocessing the dicom image; three-dimensionally stacking the preprocessed dicom images to obtain 3D image blocks, and cutting the 3D image blocks; performing feature extraction on the cut 3D image through a pre-trained spherical segmentation model to obtain a regression subgraph; the regression subgraph is subjected to product calculation of centrality and probability to obtain a plurality of central point coordinates, coordinates of regression points are obtained through the central point coordinates, and segmentation results are obtained.
Drawings
Fig. 1 is a flowchart of a deep learning-based 3D lung lesion segmentation method according to an embodiment of the present invention;
FIG. 2 is a training process of a spherical segmentation model according to an embodiment of the present invention;
fig. 3 is a flowchart illustrating preprocessing of annotation data of a dicom image to obtain spherical coordinate data according to an embodiment of the present invention;
fig. 4 is a diagram illustrating coordinate information and a segmentation contour of data obtained by performing uniform interpolation according to spatial distance through labeled data of a dicom image according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of the calculation of the loss function of the spherical coordinate data of the feature map by dividing the feature map into three parts through the FPN and RFP according to the embodiment of the present invention;
fig. 6 is a block diagram illustrating a 3D lung lesion segmentation apparatus based on deep learning according to an embodiment of the present invention;
fig. 7 is a block diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the following embodiments, and it should be understood that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The present invention aims to provide a method, an apparatus, an electronic device and a storage medium for segmenting a 3D lung lesion based on deep learning, which do not need to detect and then segment first, and segment directly through a 3D image to obtain a corresponding result, and the following description and specific embodiments are further described with reference to the accompanying drawings:
fig. 1 shows a flowchart of a deep learning-based 3D lung lesion segmentation method according to an embodiment of the present invention, as shown in fig. 1, the segmentation method includes the following steps:
s20, acquiring a lung nodule dicom image, and preprocessing the dicom image;
s40, carrying out three-dimensional stacking on the preprocessed dicom images to obtain 3D image blocks, and cutting the 3D image blocks;
s60, performing feature extraction on the cut 3D image through a pre-trained spherical segmentation model to obtain a regression subgraph;
s80, calculating the product of the centrality and the probability of the regression subgraph to obtain a plurality of central point coordinates, and obtaining coordinates of the regression points through the central point coordinates to obtain a segmentation result.
In the method of the embodiment, a lung nodule dicom image is obtained, and the dicom image is preprocessed; three-dimensionally stacking the preprocessed dicom images to obtain 3D image blocks, and cutting the 3D image blocks; performing feature extraction on the cut 3D image through a pre-trained spherical segmentation model to obtain a regression subgraph; the regression subgraph is subjected to product calculation of centrality and probability to obtain a plurality of central point coordinates, coordinates of regression points are obtained through the central point coordinates, and segmentation results are obtained.
Specifically, as shown in fig. 2, the training process of the spherical segmentation model includes:
s601, preprocessing the labeling data of the dicom image to obtain spherical coordinate data;
s602, extracting a feature graph based on the constructed network;
s603, constructing a feature map pyramid network FPN and a cyclic feature pyramid network RFP based on ResNeSt-34;
s604, dividing the characteristic diagram into three parts through the FPN and the RFP, calculating a loss function of the spherical coordinate data of the characteristic diagram, and solving an optimized model.
Specifically, as shown in fig. 3 to 4, the pre-processing the labeling data of the dicom image to obtain the spherical coordinate data includes:
s6011, obtaining coordinate information and a segmentation contour of data according to the labeling data of the dicom image;
s6012, carrying out consistent interpolation on the labeling data of the dicom image according to the spatial distance, and setting the unit value of each dimension of the voxel as the same value in an interpolation mode of 3 times of linear interpolation;
s6013, after interpolation of the labeled data of the dicom image is completed, calculating a spherical coordinate space position of a regression pixel; wherein the spatial expression of the spherical coordinates is as follows:
Figure BDA0002955907200000071
wherein, among others,
Figure BDA0002955907200000072
n is taken to be the value 36 in the present invention,
Figure BDA0002955907200000073
m is 36 in the invention, and r is the sphere center coordinate from each corresponding theta and
Figure BDA0002955907200000074
the angle reaches the length of the edge of the object.
In the data processing process, the coordinate information and the segmentation contour of the data are obtained through the labeled data of the dicom image, consistent interpolation is performed according to the spatial distance, and the size of the voxel (unit value of each dimension of the voxel) is set to be the same value in an interpolation mode, that is, the unit size of each dimension is the same millimeter number, for example, one voxel corresponds to a spatial size of 0.6 × 0.6 mm. By processing the corresponding regression radius and angle through the method, the model can more easily perform radius and additive regression on the same pixel value, and the model can easily complete better training. After the interpolation of the original data is finished, calculating the spherical coordinate space position of the regression pixel, wherein in the data processing process, the distance from the central point of each pixel to the node segmentation edge is calculated by calculating the distance from the central point of each pixel to the node segmentation edge as shown in the formula (1), n values are equally spaced from 0-2 pi according to theta, wherein n is 36 in the invention, and phi is also equally spaced from 0-2 pi similarly, and the number of the values is also n; phi and n are respectively used as submodules of two 36 channels after sampling based on the output of the feature map pyramid network FPN and the cyclic feature pyramid network RFP of ResNeSt-34 to carry out regression training, and the regression target is the distance from the central point to the outer edge of the nodule at each angle corresponding to phi and n. And regressing the distances from the center point to the 36 phi and 36 n different directions on each pixel, thereby obtaining the segmentation result of the spherical coordinates.
Specifically, as shown in fig. 5, dividing the feature map into three parts through the FPN and RFP to calculate the loss function of the spherical coordinate data of the feature map includes:
the first part regresses the corresponding classification of the characteristic map, its size is D H W k, DHW corresponds to the length of the characteristic map separately and is tall and wide three dimensions, k is the classification number, set up as 1, activate the function as sigmoid, namely judge whether it is a nodule through the probability value, the loss function used is Focal loss; the probability values are calculated by the neural networks FPN and RFP, and the neural networks FPN and RFP learn the probability calculation result on each feature map corresponding to the classification part in fig. 4 by learning the label pattern.
The second part is the centrality of the spherical coordinates corresponding to the feature map, the size of the centrality is D H W1, DHW is three dimensions of the length, the width and the height of the feature map, and the centrality is calculated by the following formula:
Figure BDA0002955907200000081
wherein d isiIs the length of the first ray in which the spherical coordinates would regress n rays, where n in the present invention is taken as 72 (representing 36+36), in terms of the sum of θ and θ of the spherical coordinates
Figure BDA0002955907200000082
Uniformly taking values, namely taking 36 values of theta from 0-2 pi at equal intervals,
Figure BDA0002955907200000083
taking 36 values from 0-2 pi at equal intervals to obtain 72 regression lines which represent regression distances from a central point to 72 directions;
the third part is the distance from the central point on n angles of the spherical coordinates corresponding to the regression feature map, the size of the distance is D H W n, DHW is three dimensions of length, width and height, n is n angles, and as shown in the second part, n is 72 in the invention.
In some embodiments, pre-processing the dicom image comprises;
image normalization and image enhancement;
the image normalization comprises the steps of adjusting the image value of the dicom image through the window width and window level to obtain an image normalized to the pixel value of 0-255 by using the window width and window level values of 500-1500 respectively. Specifically, first xc,i,jThe pixel values of the ith row and the jth column in the c channel of a picture are represented, window width window level values are respectively 500-1500 in the invention, namely, an image obtained from a dicom image is normalized to an image with the pixel value of 0-255 by adjusting the image value through the window width window level. In the present invention, c is {1},
Figure BDA0002955907200000084
and (3) training parameter configuration:
in the training, the initial learning rate is set to 0.01, the training passage number is 50 epochs, the learning rate updating mode is WarmUpCosinaleleringrate, namely, in the first 5 epochs, the learning rate is increased to 0.01 from 0.002 by the fluctuation of 0.2 per epoch, and in the subsequent 45 epochs, according to the formula
Figure BDA0002955907200000085
And calculating, wherein n is the total number of epochs for training, 150 is taken in the experiment, and e is the current epoch number. The optimizer used in the training was an Adam optimizer and the loss function was MSE.
In the actual data processing process, limited by the storage of the display card in the training process, the invention performs 128 × 128 pixel block taking on n × 512 images, and due to the limitation of the storage of the display card, the value taking can simultaneously see the image details, and can ensure that the program runs smoothly without errors, and the result of each block is spliced back to the final result, wherein n is the number of images, and as each dicom sequence has a plurality of dicoms stacked to form a 3D structure. The final result is obtained by multiplying the centrality and the probability value of each feature map pixel, and the results are sorted according to the product of the centrality and the probability value.
In the application of an actual scene, firstly, reading a dicom image through pydicom, carrying out fixed window width window level conversion on the dicom image, carrying out three-dimensional stacking on a plurality of dicom images to obtain a 3D image block, cutting the 3D image block, wherein the size of the cut image is 128 × 128, carrying out feature extraction on the cut image by using a statistical Segmentation model to obtain a regression subgraph, then carrying out product calculation of the central degree and the probability value to obtain a plurality of central point coordinates, and obtaining the coordinates of the regression point through the central point coordinates to obtain the final Segmentation result.
Based on the same inventive concept, the embodiment of the present invention further provides a 3D lung lesion segmentation apparatus based on deep learning, which can be used to implement the method described in the above embodiments, as described in the following embodiments. Because the principle of solving the problem of the 3D lung lesion segmentation device based on deep learning is similar to that of a 3D lung lesion segmentation method based on deep learning, the implementation of the 3D lung lesion segmentation device based on deep learning can refer to the implementation of the 3D lung lesion segmentation method based on deep learning, and repeated points are not repeated. As used hereinafter, the term "unit" or "module" may be a combination of software and/or hardware that implements a predetermined function. While the system described in the embodiments below is preferably implemented in software, implementations in hardware, or a combination of software and hardware are also possible and contemplated.
As shown in fig. 6, the segmenting apparatus provided in the embodiment of the present invention includes:
the image processing module 20 is configured to acquire a dicom image of a lung nodule and perform preprocessing on the dicom image;
the 3D image acquisition module 40 is used for three-dimensionally stacking the preprocessed dicom images to obtain 3D image blocks and cutting the 3D image blocks;
the regression subgraph obtaining module 60 is used for performing feature extraction on the cut 3D image through a pre-trained spherical segmentation model to obtain a regression subgraph;
and the calculating module 80 is configured to calculate a product of centrality and probability of the regression subgraph to obtain a plurality of central point coordinates, and obtain coordinates of a regression point through the central point coordinates to obtain a segmentation result.
The segmentation device of the embodiment acquires a dicom image of a lung nodule through an image processing module 20, and pre-processes the dicom image; the 3D image acquisition module 40 three-dimensionally stacks the preprocessed dicom images to obtain 3D image blocks, and cuts the 3D image blocks; the regression subgraph obtaining module 60 performs feature extraction on the cut 3D image through a spherical segmentation model trained in advance to obtain a regression subgraph; the calculation module 80 calculates the product of the centrality and the probability of the regression subgraph to obtain a plurality of central point coordinates, and obtains the coordinates of the regression points through the central point coordinates to obtain the segmentation results.
Specifically, as shown in fig. 2, the training process of the spherical segmentation model includes:
s601, preprocessing the labeling data of the dicom image to obtain spherical coordinate data;
s602, extracting a characteristic diagram based on the network ResNeSt-34;
s603, constructing a feature map pyramid network FPN and a cyclic feature pyramid network RFP based on ResNeSt-34;
s604, dividing the characteristic diagram into three parts through the FPN and the RFP, calculating a loss function of the spherical coordinate data of the characteristic diagram, and solving an optimized model.
Specifically, as shown in fig. 3 to 4, the pre-processing the labeling data of the dicom image to obtain the spherical coordinate data includes:
s6011, obtaining coordinate information and a segmentation contour of data according to the labeling data of the dicom image;
s6012, carrying out consistent interpolation on the labeling data of the dicom image according to the spatial distance, and setting the unit value of each dimension of the voxel as the same value in an interpolation mode of 3 times of linear interpolation;
s6013, after interpolation of the labeled data of the dicom image is completed, calculating a spherical coordinate space position of a regression pixel; wherein the spatial expression of the spherical coordinates is as follows:
Figure BDA0002955907200000111
wherein, among others,
Figure BDA0002955907200000112
n is taken to be the value 36 in the present invention,
Figure BDA0002955907200000113
m is 36 in the invention, and r is the sphere center coordinate from each corresponding theta and
Figure BDA0002955907200000114
the angle reaches the length of the edge of the object.
In the data processing process, the coordinate information and the segmentation contour of the data are obtained through the labeled data of the dicom image, consistent interpolation is performed according to the spatial distance, and the size of the voxel (unit value of each dimension of the voxel) is set to be the same value in an interpolation mode, that is, the unit size of each dimension is the same millimeter number, for example, one voxel corresponds to a spatial size of 0.6 × 0.6 mm. By processing the corresponding regression radius and angle through the method, the model can more easily perform radius and additive regression on the same pixel value, and the model can easily complete better training. After the interpolation of the original data is finished, calculating the spherical coordinate space position of the regression pixel, wherein in the data processing process, the distance from the central point of each pixel to the node segmentation edge is calculated by calculating the distance from the central point of each pixel to the node segmentation edge as shown in the formula (1), n values are equally spaced from 0-2 pi according to theta, wherein n is 36 in the invention, and phi is also equally spaced from 0-2 pi similarly, and the number of the values is also n; phi and n are respectively used as submodules of two 36 channels after sampling based on the output of the feature map pyramid network FPN and the cyclic feature pyramid network RFP of ResNeSt-34 to carry out regression training, and the regression target is the distance from the central point to the outer edge of the nodule at each angle corresponding to phi and n. And regressing the distances from the center point to the 36 phi and 36 n different directions on each pixel, thereby obtaining the segmentation result of the spherical coordinates.
An electronic device is further provided in an embodiment of the present invention, and fig. 7 shows a schematic structural diagram of an electronic device to which an embodiment of the present invention can be applied, and as shown in fig. 7, the computer electronic device includes a Central Processing Unit (CPU)701 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data necessary for system operation are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
The following components are connected to the I/O interface 705: an input portion 706 including a keyboard, a mouse, and the like; an output section 707 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker and the like; a storage section 708 including a hard disk and the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. The driver 310 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read out therefrom is mounted into the storage section 708 as necessary.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The present invention also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the apparatus for deep learning based 3D pulmonary lesion segmentation in the above embodiments; or it may be a computer-readable storage medium that exists separately and is not built into the electronic device. The computer readable storage medium stores one or more programs for use by one or more processors in performing the deep learning based 3D pulmonary lesion segmentation method described in the present invention.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.

Claims (8)

1. A3D lung lesion segmentation method based on deep learning is characterized by comprising the following steps:
acquiring a dicom image of a lung nodule, and preprocessing the dicom image;
three-dimensionally stacking the preprocessed dicom images to obtain 3D image blocks, and cutting the 3D image blocks;
performing feature extraction on the cut 3D image through a pre-trained spherical segmentation model to obtain a regression subgraph;
and calculating the product of the centrality and the probability of the regression subgraph to obtain a plurality of central point coordinates, and obtaining coordinates of the regression point through the central point coordinates to obtain a segmentation result.
2. The segmentation method according to claim 1, wherein the training process of the spherical segmentation model comprises:
preprocessing the labeling data of the dicom image to obtain spherical coordinate data;
constructing a feature map pyramid network FPN and a cyclic feature pyramid network RFP based on ResNeSt-34;
extracting a feature map based on the constructed network;
and dividing the characteristic diagram into three parts through the FPN and the RFP, calculating a loss function of the spherical coordinate data of the characteristic diagram, and solving an optimized model.
3. The segmentation method according to claim 1, wherein pre-processing the dicom image comprises;
image normalization and image enhancement;
the image normalization comprises the steps of adjusting the image value of the dicom image through the window width and window level to obtain an image normalized to the pixel value of 0-255 by using the window width and window level values of 500-1500 respectively.
4. The segmentation method as claimed in claim 2, wherein the pre-processing of the annotation data of the dicom image to obtain the spherical coordinate data comprises:
obtaining coordinate information and segmentation contours of data according to the labeling data of the dicom image;
carrying out consistent interpolation on the labeling data of the dicom image according to the space distance, and setting the unit value of each dimension of the voxel as the same value in an interpolation mode of 3 times of linear interpolation;
after interpolation of the labeled data of the dicom image is completed, calculating the spherical coordinate space position of the regression pixel; wherein the spatial expression of the spherical coordinates is as follows:
Figure FDA0002955907190000021
Figure FDA0002955907190000022
z＝rcosθ
wherein,
Figure FDA0002955907190000023
n is taken to be the value 36 in the present invention,
Figure FDA0002955907190000024
m is 36 in the invention, and r is the sphere center coordinate from each corresponding theta and
Figure FDA0002955907190000025
the angle reaches the length of the edge of the object.
5. The segmentation method as claimed in claim 4, wherein the step of dividing the feature map into three parts through the FPN and RFP comprises the steps of:
the first part regresses the corresponding classification of the characteristic map, its size is D H W k, DHW corresponds to the length of the characteristic map separately, k is the classification number, set up as 1, activate the function as sigmoid, namely judge whether it is a node through the probability value, the loss function used is Focalloss;
the second part is the centrality of the spherical coordinates corresponding to the feature map, the size of the centrality is D H W1, DHW is three dimensions of the length, the width and the height of the feature map, and the centrality is calculated by the following formula:
Figure FDA0002955907190000026
wherein d isiIs the length of the first ray in which the spherical coordinates would regress n rays, where n in the present invention is taken as 72 (representing 36+36), in terms of the sum of θ and θ of the spherical coordinates
Figure FDA0002955907190000027
Uniformly taking values, namely taking 36 values of theta from 0-2 pi at equal intervals,
Figure FDA0002955907190000028
taking 36 values from 0-2 pi at equal intervals to obtain 72 regression lines which represent regression distances from a central point to 72 directions;
the third part is the distance from the center point on n angles of the spherical coordinates corresponding to the regression feature map, the size of the distance is D H W n, DHW is three dimensions of length, width and height, n is n angles, and as shown in the second part, n is 72 in the invention.
6. A deep learning-based 3D lung lesion segmentation apparatus, the segmentation apparatus comprising:
the image processing module is used for acquiring a lung nodule dicom image and preprocessing the dicom image;
the 3D image acquisition module is used for three-dimensionally stacking the preprocessed dicom images to obtain 3D image blocks and cutting the 3D image blocks;
the regression subgraph acquisition module is used for extracting the characteristics of the cut 3D image through a pre-trained spherical segmentation model to obtain a regression subgraph;
and the calculation module is used for calculating the product of the centrality and the probability of the regression subgraph to obtain a plurality of central point coordinates, and obtaining the coordinates of the regression points through the central point coordinates to obtain the segmentation results.
7. An electronic device, comprising:
a processor, a memory for storing processor-executable instructions;
wherein the processor implements the method of any one of claims 1-5 by executing the executable instructions.
8. A computer readable storage medium having stored thereon computer instructions which, when executed by a processor, carry out the steps of the method according to any one of claims 1 to 5.
CN202110223645.1A 2021-03-01 2021-03-01 3D lung focus segmentation method and device based on deep learning Active CN112950582B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110223645.1A CN112950582B (en) 2021-03-01 2021-03-01 3D lung focus segmentation method and device based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110223645.1A CN112950582B (en) 2021-03-01 2021-03-01 3D lung focus segmentation method and device based on deep learning

Publications (2)

Publication Number Publication Date
CN112950582A true CN112950582A (en) 2021-06-11
CN112950582B CN112950582B (en) 2023-11-24

Family

ID=76246858

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110223645.1A Active CN112950582B (en) 2021-03-01 2021-03-01 3D lung focus segmentation method and device based on deep learning

Country Status (1)

Country Link
CN (1) CN112950582B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180122082A1 (en) * 2016-11-02 2018-05-03 General Electric Company Automated segmentation using deep learned priors
CN110689547A (en) * 2019-09-25 2020-01-14 重庆大学 Pulmonary nodule segmentation method based on three-dimensional CT image
CN110807764A (en) * 2019-09-20 2020-02-18 成都智能迭迦科技合伙企业(有限合伙) Lung cancer screening method based on neural network
US20200160997A1 (en) * 2018-11-02 2020-05-21 University Of Central Florida Research Foundation, Inc. Method for detection and diagnosis of lung and pancreatic cancers from imaging scans
CN111553892A (en) * 2020-04-23 2020-08-18 北京小白世纪网络科技有限公司 Lung nodule segmentation calculation method, device and system based on deep learning
CN111932540A (en) * 2020-10-14 2020-11-13 北京信诺卫康科技有限公司 CT image contrast characteristic learning method for clinical typing of new coronary pneumonia

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180122082A1 (en) * 2016-11-02 2018-05-03 General Electric Company Automated segmentation using deep learned priors
US20200160997A1 (en) * 2018-11-02 2020-05-21 University Of Central Florida Research Foundation, Inc. Method for detection and diagnosis of lung and pancreatic cancers from imaging scans
CN110807764A (en) * 2019-09-20 2020-02-18 成都智能迭迦科技合伙企业(有限合伙) Lung cancer screening method based on neural network
CN110689547A (en) * 2019-09-25 2020-01-14 重庆大学 Pulmonary nodule segmentation method based on three-dimensional CT image
CN111553892A (en) * 2020-04-23 2020-08-18 北京小白世纪网络科技有限公司 Lung nodule segmentation calculation method, device and system based on deep learning
CN111932540A (en) * 2020-10-14 2020-11-13 北京信诺卫康科技有限公司 CT image contrast characteristic learning method for clinical typing of new coronary pneumonia

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
YEGANEH JALALI, MANSOOR FATEH, MOHSEN REZVANI, VAHID ABOLGHASEMI, MOHAMMAD HOSSEIN ANISI: "ResBCDU-Net: A Deep Learning Framework for Lung CT Image Segmentation", Retrieved from the Internet <URL:https://pubmed.ncbi.nlm.nih.gov/33401581/> *
陈万顺: "基于深度学习的肺结节检测方法", 中国优秀硕士学位论文全文数据库医药卫生科技辑 *

Also Published As

Publication number Publication date
CN112950582B (en) 2023-11-24

Similar Documents

Publication Publication Date Title
EP3979198A1 (en) Image segmentation model training method and apparatus, computer device, and storage medium
US11288808B2 (en) System and method for n-dimensional image segmentation using convolutional neural networks
US11887311B2 (en) Method and apparatus for segmenting a medical image, and storage medium
US20210390706A1 (en) Detection model training method and apparatus, computer device and storage medium
CN110599528A (en) Unsupervised three-dimensional medical image registration method and system based on neural network
US10853409B2 (en) Systems and methods for image search
CN111932529B (en) Image classification and segmentation method, device and system
Li et al. Automated measurement network for accurate segmentation and parameter modification in fetal head ultrasound images
CN111488872B (en) Image detection method, image detection device, computer equipment and storage medium
CN111932552B (en) Aorta modeling method and device
WO2020234349A1 (en) Sampling latent variables to generate multiple segmentations of an image
CN112750137B (en) Liver tumor segmentation method and system based on deep learning
CN113240661B (en) Deep learning-based lumbar vertebra bone analysis method, device, equipment and storage medium
CN112597847A (en) Face pose estimation method and device, electronic equipment and storage medium
CN114792326A (en) Surgical navigation point cloud segmentation and registration method based on structured light
CN117237322A (en) Organ segmentation modeling method and terminal based on medical image
CN115546270A (en) Image registration method, model training method and equipment for multi-scale feature fusion
CN108597589B (en) Model generation method, target detection method and medical imaging system
CN113850796A (en) Lung disease identification method and device based on CT data, medium and electronic equipment
CN116309551B (en) Method, device, electronic equipment and readable medium for determining focus sampling area
CN112950582B (en) 3D lung focus segmentation method and device based on deep learning
CN113408595B (en) Pathological image processing method and device, electronic equipment and readable storage medium
CN115439478A (en) Lung lobe perfusion strength evaluation method, system, equipment and medium based on lung perfusion
CN111598904B (en) Image segmentation method, device, equipment and storage medium
Ye et al. Active contour image segmentation method for training talents of computer graphics and image processing technology

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant