CN111080592A - Rib extraction method and device based on deep learning - Google Patents
Rib extraction method and device based on deep learning Download PDFInfo
- Publication number
- CN111080592A CN111080592A CN201911243888.0A CN201911243888A CN111080592A CN 111080592 A CN111080592 A CN 111080592A CN 201911243888 A CN201911243888 A CN 201911243888A CN 111080592 A CN111080592 A CN 111080592A
- Authority
- CN
- China
- Prior art keywords
- neural network
- convolutional neural
- rib
- deep
- medical image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30008—Bone
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention provides a rib extraction method based on deep learning, which comprises the following steps: preprocessing a training sample; establishing a first deep convolutional neural network for rib extraction, and training the first deep convolutional neural network through a training sample; establishing a second deep convolutional neural network for performing discrimination optimization on the first deep convolutional neural network, and optimizing the first deep convolutional neural network; and extracting a rib image in the medical image through the trained first deep convolutional neural network. The embodiment of the invention adopts a sample augmentation mode to improve the training effect of the model; secondly, a deep convolutional neural network is adopted, so that higher recognition rate can be obtained; thirdly, when feature fusion is carried out, boundary loss and regional loss are considered at the same time, and network convergence is facilitated; fourthly, in the training part of the network, the attention of the model to difficult samples is increased, and the network can better converge.
Description
Technical Field
The invention belongs to the technical field of medical image processing, and particularly relates to a rib extraction method and device based on deep learning.
Background
Medical image analysis is a method commonly used by physicians in diagnosing lung diseases. In actual work, on one hand, the ribs often form a shield on part of lung tissues, so that accurate diagnosis of diseases by doctors is influenced. To overcome this problem, the ribs in the medical image may be separated in advance, so that the ribs are removed from the medical image, thereby forming a medical image showing only the tissue structure of the lung, and reducing the adverse effect of the ribs on the disease diagnosis. On the other hand, the extracted rib image can provide a basis for analyzing the rib anatomical structure and diagnosing various related diseases. However, the existing rib extraction work usually adopts a manual marking method, which mainly occupies a lot of time of doctors, and the marked rib region has low precision, so that once the marking is wrong, the rib region is difficult to find and correct in time.
Disclosure of Invention
Aiming at the problems in the prior art, the embodiment of the invention provides a rib extraction method and device based on deep learning.
In a first aspect, an embodiment of the present invention provides a rib extraction method based on deep learning, including the following steps:
s1: preprocessing a training sample;
s2: establishing a first deep convolutional neural network for rib extraction, and training the first deep convolutional neural network through a training sample;
s3: establishing a second deep convolutional neural network for performing discrimination optimization on the first deep convolutional neural network; optimizing the first deep convolutional neural network according to the output result of the training sample and the first deep convolutional neural network;
s4: and extracting a rib image in the medical image through the trained first deep convolutional neural network.
Further, the method for training sample preprocessing in step S1 includes:
s11: performing rotation, horizontal turning, vertical turning and/or elastic deformation processing on the medical image sample with the rib region marked in advance to realize sample augmentation;
s12: calculating horizontal gradient and vertical gradient images of the medical image obtained in step S11;
s13: and adding the gray value of the medical image and the horizontal gradient and the vertical gradient obtained in the step S12 to the medical image data to obtain a preprocessed training sample.
Further, the loss function of the first deep convolutional neural network is:
wherein y' is a probability value output by each pixel point in the medical image, y is a label (value is 1 or 0) of the corresponding pixel point on the pre-marked medical image, α and λ are 0.25 and 2 respectively, m is a medical image m1, m2 and m3 obtained by sampling a feature map of the preprocessed sample in an interlaced manner by 2 times, 4 times and 8 times, e is a medical image feature map e1 obtained in step S2, f is a feature map and a probability map finally output through the first deep neural network processing, and ω (m, e) is 0.8 when y belongs to a point m1, 0.6 when y belongs to a point m2, 0.5 when y belongs to a point m3, and 1.0 when y belongs to a point e 1.
Further, the method for designing the loss function of the second deep convolutional neural network comprises the following steps:
preprocessing a training sample, inputting the training sample into a first deep convolution neural network, outputting a label of each pixel point as True, and calculating a loss function based on a cross entropy method;
and inputting the rib probability map output by the first deep convolutional neural network into a second deep convolutional neural network, wherein the label of each output pixel point is False, and calculating a loss function based on a cross entropy method.
Further, when the first deep convolutional neural network is trained in step S2, the difficult samples are extracted separately for the same batch input, and the training is repeated.
In a second aspect, an embodiment of the present invention provides a deep learning based rib extraction device, which operates based on the method in the first aspect, and includes:
the medical image preprocessing module is used for preprocessing a training sample;
the rib extraction module adopts a first depth convolution neural network structure and is used for obtaining the probability of whether each pixel point in the medical image belongs to a rib;
and the discrimination optimization module adopts a first deep convolution neural network structure and is used for optimizing the first deep convolution neural network adopted by the rib extraction module.
Further, the loss function of the first deep convolutional neural network adopted by the rib extraction module is as follows:
wherein y' is a probability value output by each pixel point in the medical image, y is a label (value is 1 or 0) of the corresponding pixel point on the pre-marked medical image, α and λ are 0.25 and 2 respectively, m is a medical image m1, m2 and m3 obtained by sampling a feature map of the preprocessed sample in an interlaced manner by 2 times, 4 times and 8 times, e is a medical image feature map e1 obtained in step S2, f is a feature map and a probability map finally output through the first deep neural network processing, and ω (m, e) is 0.8 when y belongs to a point m1, 0.6 when y belongs to a point m2, 0.5 when y belongs to a point m3, and 1.0 when y belongs to a point e 1.
Further, the method for designing the loss function of the second deep convolutional neural network adopted by the discriminant optimization module is as follows:
preprocessing a training sample and inputting the training sample into a first deep convolution neural network, wherein the label of each pixel point output by the network is True, and a loss function is calculated based on a cross entropy method;
and inputting the extracted rib probability map into a second deep convolution neural network, wherein the label of each pixel point output by the network is False, and calculating a loss function based on a cross entropy method.
In a third aspect, an embodiment of the present invention provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method provided in the first aspect when executing the program.
In a fourth aspect, an embodiment of the present invention provides a non-transitory computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the steps of the method as provided in the first aspect.
According to the rib extraction method and device based on deep learning, provided by the embodiment of the invention, firstly, aiming at the problem that manual calibration of a medical image is time-consuming and labor-consuming, a sample augmentation mode is adopted, and the training effect of a model is improved; secondly, a deep convolutional neural network is adopted, and a high recognition rate can be obtained aiming at the problem that the difference between the rib and the background of the medical image is small; thirdly, when feature fusion is carried out, the influence of feature layers of two adjacent scales is considered at the same time, namely, the loss of the boundary and the loss of the area are considered at the same scale in the design of the loss function, and the loss function is designed at different scales, so that the convergence of the network is facilitated; and fourthly, in the training part of the network, the difficult samples are independently extracted for the input of the same batch, and the secondary training is carried out, so that the attention of the model to the difficult samples is increased, the network can better converge, and a better effect is obtained.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
Fig. 1 is a schematic overall flow chart of a rib extraction method based on deep learning according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a residual error module of a rib extraction method and apparatus based on deep learning according to an embodiment of the present invention;
fig. 3 is a schematic diagram of an overall structure of an attention network according to an embodiment of the present invention;
fig. 4 is a schematic view of an overall structure of a rib extraction module according to an embodiment of the present invention;
fig. 5 is a schematic view of an overall structure of a discrimination optimization module according to an embodiment of the present invention;
FIG. 6 is a diagram illustrating the actual effect of rib extraction according to an embodiment of the present invention;
fig. 7 is a schematic physical structure diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic overall flow chart of a rib extraction method based on deep learning according to an embodiment of the present invention, as shown in fig. 1, the method includes the following steps:
s1: preprocessing a training sample;
s2: establishing a first deep convolutional neural network for rib extraction, and training the first deep convolutional neural network through a training sample;
s3: establishing a second deep convolutional neural network for performing discrimination optimization on the first deep convolutional neural network; optimizing the first deep convolutional neural network according to the output result of the training sample and the first deep convolutional neural network;
s4: and extracting a rib image in the medical image through the trained first deep convolutional neural network.
Specifically, the method provided by the embodiment of the invention is applied to a rib extraction device based on deep learning, so that the automatic extraction of the rib image in the medical image is realized. The rib extraction device based on deep learning comprises:
the medical image preprocessing module is used for preprocessing a training sample;
the rib extraction module adopts a first depth convolution neural network structure and is used for obtaining the probability of whether each pixel point in the medical image belongs to a rib;
and the discrimination optimization module adopts a first deep convolution neural network structure and is used for optimizing the first deep convolution neural network adopted by the rib extraction module.
The method and apparatus for rib extraction based on deep learning are described in detail below.
The medical image preprocessing module is used for receiving the training samples, carrying out preprocessing such as amplification and normalization on the training samples, and outputting results to the rib extraction module and the discrimination optimization module at the rear end. Specifically, the working process of the medical image preprocessing module is as follows:
s11: and (3) performing rotation, horizontal turning, vertical turning and/or elastic deformation processing on the medical image sample with the rib region marked in advance, so as to realize sample augmentation. For the medical image sample with the rib region marked in advance, the marking is performed pixel by pixel, for example, the rib region is marked as 1, and other regions are marked as 0. Different samples can be selected to carry out different treatments in a random mode; for the rotation operation, the random rotation can be limited within the range of-30 ° to + 30 °, so as to obtain medical image samples of different angles; thereby realizing sample data amplification. Preferably, a histogram normalization operation is performed on the processed medical image. To balance output resolution and processing speed, the augmented image may be scaled to a uniform size, such as 512x512 pixels.
S12: calculating horizontal gradient and vertical gradient images of the medical image obtained in step S11;
s13: and (4) adding the gray value (normalized to the [0, 1] interval) of the medical image and the horizontal gradient and the vertical gradient obtained in the step S12 to the medical image data to obtain a training sample of 512x512x3 after preprocessing.
In the embodiment of the invention, two deep convolutional network models are established and are respectively applied to a rib extraction module and a discrimination optimization module. The rib extraction module is used for obtaining the probability of whether each pixel point in the medical image belongs to a rib; and the discrimination optimization module is used for optimizing the first deep convolution neural network adopted by the rib extraction module. These two modules will be described separately below.
One specific processing procedure of the rib extraction module is as follows:
1: and (3) performing the expansion convolution module operation on the preprocessed feature map m of 512x512x3 to obtain a feature map of 512x512x64, wherein the feature map is recorded as e 1. Specifically, the operation method of the dilation convolution module is as follows: the expansion convolution with kernel of 3x3, stride of 1, expansion coefficient of 2, padding of 2 is performed on the feature map, and then BN and ReLU are performed.
2: and e1 is subjected to residual module 1 to obtain a feature map e2 of 256x256x 256. And interleaving m to obtain a 256x256x3 feature map m1, and connecting m1 and e2 according to channels to obtain a 256x256x259 feature map. And performing convolution module operation of 1x1x256x1 on the feature map to obtain a 256x256x256 feature map, and updating the feature map to e 2. The structure of the residual module is shown in fig. 2.
3: residual module 2 is executed on e2 to obtain a 128x128x512 feature map, which is recorded as e 3. And (3) interleaving m1 to obtain m2, and connecting m2 and e3 according to channels to obtain a 128x128x515 feature map. The 1x1x512x1 operation is performed on the feature map to obtain a 128x128x512 feature map, and the feature map is updated to e 3. And (3) performing deconvolution operation on e3 with kernel of 2, step size of 2 and output channel of 256 to obtain a 128x128x256 feature map. And connecting the characteristic diagram with e2 according to the channel to obtain a characteristic diagram of 256x256x 512. And performing convolution module operation of 1x1x256x1 on the feature map to obtain a 256x256x256 feature map, and updating the feature map to e 2.
4: and (3) executing a residual module 3 on e3 to obtain a 64x64x1024 feature map, recording the feature map as e4, performing interleaved sampling on m2 to obtain m3, connecting m3 and e4 according to channels to obtain a 64x64x1027 feature map, executing an operation of 1x1x1024x1 on the feature map to obtain a 64x64x1024 feature map, and updating the feature map to be e 4. And e4 is subjected to deconvolution operation with kernel of 2, step length of 2 and output channel of 512 to obtain a 128x128x512 feature map, the feature map is connected with e3 according to channels to obtain a 128x128x1024 feature map, 1x1x512x1 convolution module operation is carried out on the feature map to obtain a 128x128x512 feature map, and the feature map is updated to e 3.
5: and e4 is executed with the residual module 4 to obtain a feature map of 32x32x2048, which is recorded as e 5. And performing interleaved sampling on m3 to obtain m4, connecting m4 and e5 according to channels to obtain a feature map of 32x32x2051, performing operation of 1x1x2048x1 on the feature map to obtain a feature map of 32x32x2048, and updating the feature map to be e 5. Performing deconvolution operation on e5 with kernel of 2, step length of 2 and output channel of 1024 to obtain a feature map of 64x64x1024, connecting the feature map with e4 according to the channel to obtain a feature map of 128x128x2048, performing convolution module operation of 1x1x1024x1 on the feature map to obtain a feature map of 128x128x1024, and updating the feature map to e 4.
6: and e5 is subjected to convolution module operation of 3x3x2048x1, and a feature map d5 of 32x32x2048 is obtained.
7: deconvolution of d5 with kernel of 2, step size of 2 and output channel of 1024 yields a feature map d4 of 64x64x 1024.
8: and (4) obtaining a 64x64x1024 feature map by using the attention network with e4 and d4 as parameters, and updating e4 into the feature map. e4 and d4 are connected in the channel direction to obtain a 64x64x2048 feature map, a convolution module operation of 3x3x1024x1 is executed to obtain a 64x64x1024 feature map, and d4 is updated to the feature map.
The structure of the attention network is shown in fig. 3, and includes:
(1) e performs a We operation (first performs a convolution operation of 3x3xcex1(ce is the number of channels of e), and then performs a BN operation) to obtain a feature map e1, d performs a Wd operation (first performs a 3x3xcd (cd is the number of channels of d), and then performs a BN operation) to obtain a feature map d1, and adds e1 and d1 by channels to obtain a new feature map, and updates e1 with the feature map.
(2) E1 is used to perform a convolution operation of ReLU and 1x1x1x1, then BN is performed, and finally an update e1 is output by the probability map through a Sigmoid output.
(3) E1 is multiplied by the characteristic of each channel in the channel and e, the obtained characteristic diagram and e are added according to the channel to obtain a new characteristic diagram, and e is updated by the characteristic diagram.
9: deconvolution of d4 with kernel of 2, step size of 2, and output channel of 512 yields a 64 × 512 feature map. This signature is recorded as d 3.
10: obtaining a 64x64x512 feature map by taking e3 and d3 as parameters and utilizing an attention network, updating e3 to the feature map, connecting e3 and d3 according to channels to obtain a 64x64x1024 feature map, then performing convolution module operation of 3x3x512x1 on the feature map module to obtain a 64x64x512 feature map, and updating d3 to the feature map.
11: deconvolution of d3 with kernel of 2, step size of 2 and output channel of 256 was performed to obtain a feature map of 256 × 256, which is recorded as d 2.
12: using e2 and d2 as parameters, utilizing an attention network to obtain a 256x256x256 feature map, updating e2 to the feature map, connecting e3 and d3 according to channels to obtain a 256x256x512 feature map, performing a convolution module operation of 3x3x256x1 on the feature map to obtain a 256x256x256 feature map, and updating d2 to the feature map.
13: deconvolution of d2 with kernel of 2, step size of 2, and output channel of 128 was performed to obtain a feature map of 512 × 128, which is recorded as d 1.
14: and e1 is subjected to a 3x3x128 expansion convolution module operation to obtain a feature map of 512x512x128, and e1 is updated to the feature map. And e1 and d1 are taken as parameters, a characteristic diagram of 512x512x128 is obtained by utilizing the attention network, and e1 is updated into the characteristic diagram. Connecting e1 and d1 according to a channel to obtain a 512x512x256 feature map, performing convolution module operation of 3x3x256x1 on the channel to obtain a 512x512x128 feature map, and updating d1 into the feature map.
15: performing operation T3 on d3 to obtain a feature map f3 of 128x128x 1; performing operation T2 on d2 to obtain a 256x256x1 probability map f 2; d1 is executed in parallel to perform operation T1, resulting in two 512x512x1 probability images f1, fe, respectively.
The structure of the rib extraction model is shown in fig. 3.
The distinguishing and optimizing module has the function of optimizing and training the first deep convolutional neural network adopted by the rib extracting module, so that the result output by the rib extracting module is consistent with the rib area of an input precise marking image (a medical image sample for marking the rib area in advance), the rib edge is smoother, and a better recognition effect is obtained. Specifically, a specific processing procedure of the discrimination optimization module is as follows:
1: the convolution module operation of 3x3x32x1 is performed on the medical image (generation probability map of ribs or fine-scale image of ribs) m input as 512x152x1, and a feature map e1 of 512x512x32 is obtained.
2: the convolution module operation of 3x3x64x2 is performed on e1, resulting in a feature map e2 of 256x256x 64.
3: and performing convolution module operations of 1x1x32x1, 3x3x32x2 and 1x1x128x1 on e2 to obtain a 128x128x128 feature map e 3.
4: and performing convolution module operations of 1x1x64x1, 3x3x64x2 and 1x1x256x1 on e3 to obtain a feature map e4 of 64x64x 256.
5: and performing convolution module operations of 1x1x128x1, 3x3x128x2 and 1x1x512x1 on e4 to obtain a feature map e5 of 32x32x 512.
6: deconvolution of e5 with kernel of 2, step size of 2 and output channel of 256 is performed to obtain a feature map d4 of 64x64x256, d4 and e4 are connected in the channel direction to obtain a feature map of 64x64x512, convolution module operation of 3x3x256x2 is performed to the feature map to obtain a feature map of 64x64x256, and the feature map is updated to d 4.
7: deconvolution of d4 with kernel of 2 and step size of 2 and output channel of 128 is performed to obtain a 128x128x128 feature map d3, d3 and e3 are connected in the channel direction to obtain a 128x128x256 feature map, then a convolution module of 3x3x128x1 is performed to obtain a 128x128x128 feature map, and the feature map is updated to d 3.
8: deconvolution of d3 with kernel of 2, step size of 2 and output channel of 64 is performed to obtain a feature map d2 of 256x256x64, d2 and e2 are connected in the channel direction to obtain a feature map of 256x256x128, then a convolution module operation of 3x3x64x1 is performed to obtain a feature map of 256x256x64, and the feature map is updated to d 2.
9: deconvolution of d2 with kernel of 2, step length of 2 and output channel of 32 is performed to obtain a feature map of 512x512x32, the feature map is connected with e1 in the channel direction to obtain a feature map of 512x512x64, then convolution module operation of 1x1x1 is performed to obtain a feature map of 512x512x1, and a probability map f of 512x512 is obtained after a Sigmoid function is output.
The overall structure of the discriminant optimization module is shown in fig. 5.
Specific ways of designing the rib extraction module loss function are given below. The rib extraction module loss function comprises a boundary-based loss function and a regional loss function, and the specific method comprises the following steps:
1: the seminal standard of the rib is scaled to the same size as f1, f2 and f3, and template images m1, m2 and m3 of the rib are obtained respectively. Extracting the boundaries of the ribs for m1 to obtain a boundary map e1 of the ribs, and binarizing the four maps.
2: the Loss function adopts a Focal local function of cross entropy, and specifically comprises the following steps:
wherein y' is a probability value output by each pixel point in the medical image, y is a label (value is 1 or 0) of the corresponding pixel point on the pre-marked medical image, the values of α and lambda are 0.25 and 2 respectively, m is m1, m2 and m3, namely the medical image obtained by sampling the feature map of the preprocessed sample by 2 times, 4 times and 8 times in an interlaced manner, e is e1, namely the 512x512x64 feature map obtained in the step 1 of the rib extraction module, and f is the feature map and the probability maps f1, f2, f3 and fe finally output after the first deep neural network processing, namely the feature map and the probability map obtained in the step 15 of the rib extraction module are obtained.
The values of ω (m, e) are: when y belongs to the point m1, ω (m, e) is 0.8, when y belongs to the point m2, ω (m, e) is 0.6, when y belongs to the point m3, ω (m, e) is 0.5, and when y belongs to the point e1, ω (m, e) is 1.0. The value is taken to make the network focus more on the boundary (the boundary of the rib is often difficult to be divided), and the weight given by f1 as the final required output is a little greater.
The specific method for designing the loss function of the discriminant optimization module is given as follows:
(1) preprocessing a training sample, inputting the training sample into a first deep convolution neural network, outputting a label of each pixel point as True, and calculating a loss function based on a cross entropy method;
(2) and inputting the rib probability map output by the first deep convolutional neural network into a second deep convolutional neural network, wherein the label of each output pixel point is False, and calculating a loss function based on a cross entropy method.
One specific implementation of the training process is given below:
(1) firstly, training a network for rib extraction for N rounds (in the embodiment, N is set to 10), calculating a loss function by a rib extraction network and a discrimination optimization network respectively, and only updating parameters of a rib extraction module.
(2) And fixing the model weight of the rib extraction module, training the discrimination network for N/2 times, and updating the model weight in the discrimination optimization module.
One specific embodiment of rib extraction module training is given below:
(1) the batch select 50 first calculates a loss function based on the batch and updates the network weight parameters. The loss of each of the batchs is recorded at the same time, and the 30 input data with the largest loss are selected.
(2) These 30 input data are input into the network and the weights of the network are updated.
(3) Alternating exercises 1, 2 until the loss in both 1 and 2 converges to a relatively small level.
As shown in fig. 6, the method and apparatus for rib extraction based on deep learning described above are an actual effect diagram for rib extraction.
Fig. 7 is a schematic entity structure diagram of an electronic device according to an embodiment of the present invention, and as shown in fig. 7, the electronic device may include: a processor (processor)701, a communication Interface (Communications Interface)702, a memory (memory)703 and a communication bus 704, wherein the processor 701, the communication Interface 702 and the memory 703 complete communication with each other through the communication bus 704. The processor 701 may invoke a computer program stored on the memory 703 and executable on the processor 701 to perform the methods provided by the embodiments described above, including for example: preprocessing a training sample; establishing a first deep convolutional neural network for rib extraction, and training the first deep convolutional neural network through a training sample; establishing a second deep convolutional neural network for performing discrimination optimization on the first deep convolutional neural network; optimizing the first deep convolutional neural network according to the output result of the training sample and the first deep convolutional neural network; and extracting a rib image in the medical image through the trained first deep convolutional neural network.
In addition, the logic instructions in the memory 703 can be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solutions of the embodiments of the present invention may be essentially implemented or make a contribution to the prior art, or may be implemented in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Embodiments of the present invention further provide a non-transitory computer-readable storage medium, on which a computer program is stored, where the computer program is implemented to perform the method provided in the foregoing embodiments when executed by a processor, and the method includes: preprocessing a training sample; establishing a first deep convolutional neural network for rib extraction, and training the first deep convolutional neural network through a training sample; establishing a second deep convolutional neural network for performing discrimination optimization on the first deep convolutional neural network; optimizing the first deep convolutional neural network according to the output result of the training sample and the first deep convolutional neural network; and extracting a rib image in the medical image through the trained first deep convolutional neural network.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A rib extraction method based on deep learning is characterized in that: the method comprises the following steps:
s1: preprocessing a training sample;
s2: establishing a first deep convolutional neural network for rib extraction, and training the first deep convolutional neural network through a training sample;
s3: establishing a second deep convolutional neural network for performing discrimination optimization on the first deep convolutional neural network; optimizing the first deep convolutional neural network according to the output result of the training sample and the first deep convolutional neural network;
s4: and extracting a rib image in the medical image through the trained first deep convolutional neural network.
2. The rib extraction method based on deep learning of claim 1, wherein: the method for training sample preprocessing in step S1 includes:
s11: performing rotation, horizontal turning, vertical turning and/or elastic deformation processing on the medical image sample with the rib region marked in advance to realize sample augmentation;
s12: calculating horizontal gradient and vertical gradient images of the medical image obtained in step S11;
s13: and adding the gray value of the medical image and the horizontal gradient and the vertical gradient obtained in the step S12 to the medical image data to obtain a preprocessed training sample.
3. The rib extraction method based on deep learning of claim 1, wherein: the loss function of the first deep convolutional neural network is:
wherein y' is a probability value output by each pixel point in the medical image, y is a label (value is 1 or 0) of the corresponding pixel point on the pre-marked medical image, α and λ are 0.25 and 2 respectively, m is a medical image m1, m2 and m3 obtained by sampling a feature map of the preprocessed sample in an interlaced manner by 2 times, 4 times and 8 times, e is a medical image feature map e1 obtained in step S2, f is a feature map and a probability map finally output through the first deep neural network processing, and ω (m, e) is 0.8 when y belongs to a point m1, 0.6 when y belongs to a point m2, 0.5 when y belongs to a point m3, and 1.0 when y belongs to a point e 1.
4. The rib extraction method based on deep learning of claim 3, wherein: the design method of the loss function of the second deep convolutional neural network comprises the following steps:
preprocessing a training sample, inputting the training sample into a first deep convolution neural network, outputting a label of each pixel point as True, and calculating a loss function based on a cross entropy method;
and inputting the rib probability map output by the first deep convolutional neural network into a second deep convolutional neural network, wherein the label of each output pixel point is False, and calculating a loss function based on a cross entropy method.
5. The rib extraction method based on deep learning of claim 4, wherein: when the first deep convolutional neural network is trained in step S2, the difficult samples are extracted separately for the same batch input, and the training is repeated.
6. A rib extraction device based on deep learning is characterized in that: the method according to any one of claims 1 to 5, comprising:
the medical image preprocessing module is used for preprocessing a training sample;
the rib extraction module adopts a first depth convolution neural network structure and is used for obtaining the probability of whether each pixel point in the medical image belongs to a rib;
and the discrimination optimization module adopts a first deep convolution neural network structure and is used for optimizing the first deep convolution neural network adopted by the rib extraction module.
7. The deep learning based rib extraction device according to claim 6, wherein: the loss function of the first deep convolutional neural network adopted by the rib extraction module is as follows:
wherein y' is a probability value output by each pixel point in the medical image, y is a label (value is 1 or 0) of the corresponding pixel point on the pre-marked medical image, α and λ are 0.25 and 2 respectively, m is a medical image m1, m2 and m3 obtained by sampling a feature map of the preprocessed sample in an interlaced manner by 2 times, 4 times and 8 times, e is a medical image feature map e1 obtained in step S2, f is a feature map and a probability map finally output through the first deep neural network processing, and ω (m, e) is 0.8 when y belongs to a point m1, 0.6 when y belongs to a point m2, 0.5 when y belongs to a point m3, and 1.0 when y belongs to a point e 1.
8. The deep learning based rib extraction device according to claim 7, wherein: the design method of the loss function of the second deep convolutional neural network adopted by the discrimination optimization module comprises the following steps:
preprocessing a training sample and inputting the training sample into a first deep convolution neural network, wherein the label of each pixel point output by the network is True, and a loss function is calculated based on a cross entropy method;
and inputting the extracted rib probability map into a second deep convolution neural network, wherein the label of each pixel point output by the network is False, and calculating a loss function based on a cross entropy method.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method according to any of claims 1 to 5 are implemented when the processor executes the program.
10. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911243888.0A CN111080592B (en) | 2019-12-06 | 2019-12-06 | Rib extraction method and device based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911243888.0A CN111080592B (en) | 2019-12-06 | 2019-12-06 | Rib extraction method and device based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111080592A true CN111080592A (en) | 2020-04-28 |
| CN111080592B CN111080592B (en) | 2021-06-01 |
Family
ID=70313194
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911243888.0A Active CN111080592B (en) | 2019-12-06 | 2019-12-06 | Rib extraction method and device based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111080592B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111739008A (en) * | 2020-06-23 | 2020-10-02 | 北京百度网讯科技有限公司 | Image processing method, device, equipment and readable storage medium |
| CN112734755A (en) * | 2021-03-15 | 2021-04-30 | 四川大学 | Lung lobe segmentation method based on 3D full convolution neural network and multitask learning |
| CN112836820A (en) * | 2021-01-31 | 2021-05-25 | 云知声智能科技股份有限公司 | Deep convolutional network training method, device and system for image classification task |
| CN114049358A (en) * | 2021-11-17 | 2022-02-15 | 苏州体素信息科技有限公司 | Method and system for rib case segmentation, counting and positioning |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106845522A (en) * | 2016-12-26 | 2017-06-13 | 华北理工大学 | A kind of discriminant classification system in metallurgical globulation |
| CN107945204A (en) * | 2017-10-27 | 2018-04-20 | 西安电子科技大学 | A kind of Pixel-level portrait based on generation confrontation network scratches drawing method |
| CN107958271A (en) * | 2017-12-06 | 2018-04-24 | 电子科技大学 | The cutaneous lesions deep learning identifying system of Analysis On Multi-scale Features based on expansion convolution |
| CN108197628A (en) * | 2017-12-07 | 2018-06-22 | 维森软件技术（上海）有限公司 | The joint judgment method of characteristics of image based on deep neural network |
| CN109086768A (en) * | 2018-07-13 | 2018-12-25 | 南京邮电大学 | The semantic image dividing method of convolutional neural networks |
| CN109829894A (en) * | 2019-01-09 | 2019-05-31 | 平安科技（深圳）有限公司 | Parted pattern training method, OCT image dividing method, device, equipment and medium |
| US20190304067A1 (en) * | 2018-03-29 | 2019-10-03 | Pixar | Temporal techniques of denoising monte carlo renderings using neural networks |
| CN110309849A (en) * | 2019-05-10 | 2019-10-08 | 腾讯医疗健康(深圳)有限公司 | Blood-vessel image processing method, device, equipment and storage medium |
| CN110490224A (en) * | 2019-07-09 | 2019-11-22 | 北京迈格威科技有限公司 | Method, apparatus, system and the storage medium of training neural network |
-
2019
- 2019-12-06 CN CN201911243888.0A patent/CN111080592B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106845522A (en) * | 2016-12-26 | 2017-06-13 | 华北理工大学 | A kind of discriminant classification system in metallurgical globulation |
| CN107945204A (en) * | 2017-10-27 | 2018-04-20 | 西安电子科技大学 | A kind of Pixel-level portrait based on generation confrontation network scratches drawing method |
| CN107958271A (en) * | 2017-12-06 | 2018-04-24 | 电子科技大学 | The cutaneous lesions deep learning identifying system of Analysis On Multi-scale Features based on expansion convolution |
| CN108197628A (en) * | 2017-12-07 | 2018-06-22 | 维森软件技术（上海）有限公司 | The joint judgment method of characteristics of image based on deep neural network |
| US20190304067A1 (en) * | 2018-03-29 | 2019-10-03 | Pixar | Temporal techniques of denoising monte carlo renderings using neural networks |
| CN109086768A (en) * | 2018-07-13 | 2018-12-25 | 南京邮电大学 | The semantic image dividing method of convolutional neural networks |
| CN109829894A (en) * | 2019-01-09 | 2019-05-31 | 平安科技（深圳）有限公司 | Parted pattern training method, OCT image dividing method, device, equipment and medium |
| CN110309849A (en) * | 2019-05-10 | 2019-10-08 | 腾讯医疗健康(深圳)有限公司 | Blood-vessel image processing method, device, equipment and storage medium |
| CN110490224A (en) * | 2019-07-09 | 2019-11-22 | 北京迈格威科技有限公司 | Method, apparatus, system and the storage medium of training neural network |
Non-Patent Citations (3)
| Title |
|---|
| JIAN-XUN PENG等: "A New Jacobian Matrix for Optimal Learning of Single-Layer Neural Networks", 《IEEE TRANSACTIONS ON NEURAL NETWORKS》 * |
| 李鸣等: "基于卷积神经网络迭代优化的图像分类算法", 《计算机工程与设计》 * |
| 苏赋等: "基于深度学习的图像分类研究综述", 《电信科学》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111739008A (en) * | 2020-06-23 | 2020-10-02 | 北京百度网讯科技有限公司 | Image processing method, device, equipment and readable storage medium |
| CN111739008B (en) * | 2020-06-23 | 2024-04-12 | 北京百度网讯科技有限公司 | Image processing method, device, equipment and readable storage medium |
| CN112836820A (en) * | 2021-01-31 | 2021-05-25 | 云知声智能科技股份有限公司 | Deep convolutional network training method, device and system for image classification task |
| CN112836820B (en) * | 2021-01-31 | 2023-12-19 | 云知声智能科技股份有限公司 | Deep convolution network training method, device and system for image classification task |
| CN112734755A (en) * | 2021-03-15 | 2021-04-30 | 四川大学 | Lung lobe segmentation method based on 3D full convolution neural network and multitask learning |
| CN114049358A (en) * | 2021-11-17 | 2022-02-15 | 苏州体素信息科技有限公司 | Method and system for rib case segmentation, counting and positioning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111080592B (en) | 2021-06-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111080592B (en) | Rib extraction method and device based on deep learning | |
| CN110084173B (en) | Human head detection method and device | |
| CN110232383B (en) | Focus image recognition method and focus image recognition system based on deep learning model | |
| CN109242844B (en) | Pancreatic cancer tumor automatic identification system based on deep learning, computer equipment and storage medium | |
| CN107895367B (en) | Bone age identification method and system and electronic equipment | |
| CN107492099B (en) | Medical image analysis method, medical image analysis system, and storage medium | |
| US9697602B1 (en) | System and method for auto-contouring in adaptive radiotherapy | |
| US20210057069A1 (en) | Method and device for generating medical report | |
| CN110276745B (en) | Pathological image detection algorithm based on generation countermeasure network | |
| CN107563434B (en) | Brain MRI image classification method and device based on three-dimensional convolutional neural network | |
| WO2021136368A1 (en) | Method and apparatus for automatically detecting pectoralis major region in molybdenum target image | |
| CN113826143A (en) | Feature point detection | |
| US20220335600A1 (en) | Method, device, and storage medium for lesion segmentation and recist diameter prediction via click-driven attention and dual-path connection | |
| CN111028923B (en) | Digital pathological image staining normalization method, electronic device and storage medium | |
| CN110570394B (en) | Medical image segmentation method, device, equipment and storage medium | |
| CN112614573A (en) | Deep learning model training method and device based on pathological image labeling tool | |
| CN109685830A (en) | Method for tracking target, device and equipment and computer storage medium | |
| CN114549861A (en) | Target matching method based on feature point and convolution optimization calculation and storage medium | |
| CN114708264B (en) | Light spot quality judging method, device, equipment and storage medium | |
| CN115880358A (en) | Construction method of positioning model, positioning method of image mark points and electronic equipment | |
| CN112634224B (en) | Focus detection method and device based on target image | |
| CN111598144B (en) | Training method and device for image recognition model | |
| CN111476775B (en) | DR symptom identification device and method | |
| CN116342466A (en) | Image matting method and related device | |
| CN112906690A (en) | License plate segmentation model training method, license plate segmentation method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |