CN112258456B - Three-dimensional image segmentation method based on convolutional neural network supervision - Google Patents

Three-dimensional image segmentation method based on convolutional neural network supervision Download PDF

Info

Publication number
CN112258456B
CN112258456B CN202011040012.9A CN202011040012A CN112258456B CN 112258456 B CN112258456 B CN 112258456B CN 202011040012 A CN202011040012 A CN 202011040012A CN 112258456 B CN112258456 B CN 112258456B
Authority
CN
China
Prior art keywords
neural network
convolutional neural
lstm
layer
convolution
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011040012.9A
Other languages
Chinese (zh)
Other versions
CN112258456A (en
Inventor
庄树昕
庄哲民
陈贵清
袁野
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shantou University
Original Assignee
Shantou University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shantou University filed Critical Shantou University
Priority to CN202011040012.9A priority Critical patent/CN112258456B/en
Publication of CN112258456A publication Critical patent/CN112258456A/en
Application granted granted Critical
Publication of CN112258456B publication Critical patent/CN112258456B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G06T2207/10012—Stereo images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10132—Ultrasound image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30068—Mammography; Breast

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)
  • Image Processing (AREA)

Abstract

The application discloses a three-dimensional image segmentation method based on convolutional neural network supervision, which combines 3D U-Net based on convolutional neural network supervision with a bidirectional convolutional recurrent neural network to construct a totally new ABVUS three-dimensional image segmentation network model architecture, and segments a three-dimensional tumor, a gland layer, a fat layer and a subcutaneous tissue layer from a three-dimensional mammary gland ultrasonic image; the encoder for the convolutional neural network may have the problem of gradient disappearance and performance degradation in the training process. Therefore, by adding the depth boundary supervision into the encoder part of the convolutional neural network to serve as the convolutional neural network and by means of boundary prompt of the depth boundary supervision convolutional neural network, the shallow layer network is trained more fully, gradient disappearance is avoided, and meanwhile learning capacity of the convolutional neural network can be improved, so that noise of an ultrasonic image is resisted better.

Description

Three-dimensional image segmentation method based on convolutional neural network supervision
Technical Field
The application relates to the field of ultrasonic imaging methods, in particular to a three-dimensional image segmentation method based on convolutional neural network supervision.
Background
The application discloses a full-automatic breast volume ultrasonic imaging system (Automated Breast Volume Ultrasound System, ABVUS) as an emerging ultrasonic technology, and only three companies of Siemens, general American and Shanshan Korea ultrasonic instruments research all of the company (SIUI) in Germany are currently used for providing own breast volume ultrasonic imaging products, and the application provides an artificial intelligent segmentation scheme for the breast volume ultrasonic imaging products attached to the company (SIUI) of Shanshan Korea ultrasonic instruments research all of China.
The method has important significance in artificial intelligence aided diagnosis due to high-quality breast ultrasound medical image segmentation. The three-dimensional breast ultrasonic image is segmented into a three-dimensional tumor form, so that doctors can better judge focuses; accurate segmentation of the tumor-imaged region also aids the clinician in assessing the surgical region; meanwhile, the breast density is considered as one of biomarkers of breast cancer risk, and the results of the three-dimensional breast image segmentation and layering are divided into glandular layers, fat layers, subcutaneous tissue layers and the like, so that the results can be used for breast density calculation and breast assessment, and the segmentation work has important clinical significance.
Disclosure of Invention
The application aims to provide a three-dimensional image segmentation method based on convolutional neural network supervision, which aims to solve one or more technical problems in the prior art and at least provides a beneficial selection or creation condition.
The application provides a method for constructing a totally new ABVUS three-dimensional image segmentation network model architecture by combining a 3D U-Net (3D Auxiliary Supervision U-Net, convolutional neural network) based on convolutional neural network supervision with a bidirectional convolutional neural network, and segmenting a three-dimensional tumor, a gland layer, a fat layer and a subcutaneous tissue layer from a three-dimensional mammary gland ultrasonic image.
In order to achieve the above object, the present application provides a three-dimensional image segmentation method based on convolutional neural network supervision, the method comprising the steps of:
preliminary segmentation is carried out on the breast three-dimensional ultrasonic image through a convolutional neural network to obtain a first segmentation atlas;
dividing the first divided atlas by the cyclic neural network to obtain a second divided atlas;
the first segmentation atlas and the second segmentation atlas are three-dimensional feature images of the breast three-dimensional ultrasound image.
Further, the breast three-dimensional ultrasound image is obtained by scanning the entire breast in a fixed trajectory by a full-automatic breast volume ultrasound imaging system (Automated Breast Volume Ultrasound System, ABVUS) of all companies (SIUI) under the study of the ultrasonic instruments in siemens, usa and china, germany; or from an open medical database: LIDC-NLST (The Lung Image Database Consortium, lung CT dataset), ADNI (Alzheimer's Disease Neuroimaging Initiative, brain Alzheimer's disease MR three-dimensional image dataset), braTS (Brain Tumor Segmentation, brain MR three-dimensional image dataset), and a CT, MR data source provided by the affiliated Hospital of the university of Shandong university of medicine.
Further, the convolutional neural network is 3D U-Net, and is composed of an encoder and a decoder, wherein the encoder is used for analyzing and learning characteristics from input data, and the decoder is used for generating a separation result according to the output characteristics of the encoder; wherein all the operation procedures are transformed into 3D operations such as 3D convolution, 3D max-pooling layer, 3D up-convolution layer, etc. The breast tumor segmentation and layering difficulty is increased by considering the characteristics of high noise, blurred edges, uneven gray values, complex tumor shape and tissue hierarchy and the like of the breast ultrasonic image.
Further, the convolutional neural network is formed by coupling a single convolutional layer and a deconvolution layer, and the convolutional neural network is connected to the tail end of an encoder of the convolutional neural network in an opposite mode to realize depth boundary supervision; auxiliary because edge pixels and non-edge pixels of the breast ultrasound image are not distributed evenlyThe auxiliary boundary network adopts a class balance cross entropy function and is used for calculating a loss function corresponding to a convolution layer of the convolution neural networkOptimizing the loss function as:
wherein ,W,w(m) For the convolution layer of the convolution neural network, W represents the weight parameter of the convolution neural network neuron, and W (m) An mth layer representing a convolution layer; wherein,a loss function of a convolution layer which is an mth connected convolution neural network, wherein E is an edge mapping set of a layering boundary; /> and />|E -| and |E+ The I represents the real edge label sets of the edge and the non-edge surface respectively; />Is the activation value calculated using the sigmoid function σ (·) at pixel i; obtaining edge mapping prediction ++after each convolution neural network output>I.e. < ->
wherein ,is output by the convolutional neural network each time;
finally, the function of the convolutional neural network can be realized by minimizing the loss function corresponding to each layer, and the total loss function of the convolutional neural network is as follows:wherein p is a network parameter of the convolutional neural network, < ->Is the weight matrix parameter of the convolutional neural network, p is more than or equal to 1 and less than or equal to Q, and Q is the total number of convolutional layers in the encoder, and is +.>Label prediction loss of the mth layer of the convolution layer is represented, i i·i 2 The L2 norm of the expression; by minimizing the total loss function L (E, X; W e ) And boundary regularization of the convolutional neural network, the shallow network of the encoder portion of the convolutional neural network can learn more useful features.
Further, the method for segmenting the first segmentation atlas to obtain the second segmentation atlas through the cyclic neural network comprises the following steps: the accurate segmentation of the second stage is performed, namely, a 3D convolution long-short-term memory network (3 DConvolume-LSTM, 3D C-LSTM) is constructed based on a full-connection long-short-term memory (FC-LSTM) bidirectional coding mechanism, so that abundant context information on the breast ultrasound 3D data is extracted for prediction enhancement, boundary information is further perfected, and the final segmentation edge and texture are clear and continuous.
The FC-LSTM combines the long-short-term memory neural network LSTM and the fully-connected neural network FC, and the probability volume of the primary segmentation result and the original ultrasonic volume data can be fused, so that the problem of boundary incompleteness can be effectively solved, and the method is used for prediction enhancement. Therefore, the FC-LSTM can further refine the segmentation result based on acquiring more detailed boundary information. Although the FC-LSTM layer is powerful in terms of processing time dependencies, it uses a full connection in input to state and state-to-state transitions, without encoding three-dimensional spatial information. Therefore, we extend LSTM through 3D convolution to obtain convolution long-short-term memory network 3D C-LSTM, which contains 3D convolution structure in both input to state and state-to-state transition.
The construction method of the 3D C-LSTM comprises the following steps:
all inputs, neurons and various gate operations of the full-connection long-short-time memory FC-LSTM framework are expanded into 3D tensors, 3D data are directly adopted as inputs, vector multiplication is replaced by convolution operation, and the framework is defined as follows: i.e t ＝σ(W xi *x t +H t-1 *W hi +W ci oc t-1 +b i )，f t ＝σ(x t *W xf +H t-1 *W hf +W cf oc t-1 +b f )，c t ＝c t-1 of t +i t otanh(x t *W xc +H t-1 *W hc +b c )，o t ＝σ(x t *W xo +H t-1 *W ho +W co oc t +b o )，H t ＝o t otanh(c t ) The method comprises the steps of carrying out a first treatment on the surface of the Here, σ (·) and tanh (·) are logical Sigmoid and hyperbolic tangent functions; i.e t 、f t 、o t 、c 1 ，…，c t The parameters are 3D tensors, and the 3D data input of the network at the t moment is x t ,H t To conceal the layer state, H t-1 In the state of the hidden layer at the previous moment, W xi 、W hi 、W ci 、W xf 、W cf 、W xc 、W hc 、W xo 、W ho 、W co All are weight matrixes in the network; b i 、b f 、b c 、b o Are both bias vectors.
Although the current time output can be inferred by 3D C-LSTM using the current input and the history information accumulated in the hidden state, to better recover breast ultrasound image boundary information, 3D C-LSTM is extended to a Bi-Directional convolutional neural network (BDC-LSTM) that works in two opposite directions by stacking two layers of 3D C-LSTM. Since the context information carried in the two layers is more detailed, the two layers of context information are connected as output, so that the structure can extract finer nonlinear characteristics in the three-dimensional breast ultrasound image.
Further, due to the problems of over-tuning of BDC-LSTM in tuning early time steps and parameters, a suitable depth supervision strategy is advantageous for shortening the gradient back propagation path while preserving the potential context information in the sequence. Therefore, a deep auxiliary supervision mechanism is added in the network structure to improve the training efficiency and generalization capability of BDC-LSTM; each time an auxiliary loss function is added through p BDC-LSTM sequences, each additional auxiliary loss function is l in turn 0 ，l 1 ，...，l k K is the number of auxiliary loss functions that need to be appended, and the final loss function is: wherein ,LN Is the main loss function of BDC-LSTM, L n Is auxiliary loss function, X, Y are three-dimensional input and output respectively, N is the number of BDC-LSTM sequences used, N is the number of auxiliary loss functions used, beta n And the final loss association rate. Through the auxiliary loss function, the BDC-LSTM can acquire faster convergence speed and lower training error, and meanwhile, the generalization capability of the BDC-LSTM is improved.
The beneficial effects of the application are as follows: the application provides a three-dimensional image segmentation method based on convolutional neural network supervision, which aims at solving the problems that gradient disappearance and performance degradation can occur in the training process of an encoder of a convolutional neural network. Therefore, by adding the depth boundary supervision into the encoder part of the convolutional neural network to serve as the convolutional neural network and by means of boundary prompt of the depth boundary supervision convolutional neural network, the shallow layer network is trained more fully, gradient disappearance is avoided, and meanwhile learning capacity of the convolutional neural network can be improved, so that noise of an ultrasonic image is resisted better.
Drawings
The above and other features of the present application will become more apparent from the detailed description of the embodiments thereof given in conjunction with the accompanying drawings, in which like reference characters designate like or similar elements, and it is apparent that the drawings in the following description are merely some examples of the present application, and other drawings may be obtained from these drawings without inventive effort to those of ordinary skill in the art, in which:
FIG. 1 is a flow chart of image segmentation by an image segmentation model;
FIG. 2 is a flow chart for constructing an image segmentation model.
Detailed Description
The conception, specific structure, and technical effects produced by the present application will be clearly and completely described below with reference to the embodiments and the drawings to fully understand the objects, aspects, and effects of the present application. It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other.
The application provides a three-dimensional image segmentation method based on convolutional neural network supervision, which can accurately obtain wind load of each part in a rigid body section model force measurement test by utilizing a plurality of force sensors and can ensure that the wind load of each part of a rigid body member is within a safe range, and the method specifically comprises the following steps:
as shown in fig. 1, fig. 1 is a flowchart for segmenting an image by an image segmentation model; the convolution neural network and the circulation neural network are image segmentation models;
preliminary segmentation is carried out on the breast three-dimensional ultrasonic image through a convolutional neural network to obtain a first segmentation atlas;
dividing the first divided atlas by the cyclic neural network to obtain a second divided atlas;
the first segmentation atlas and the second segmentation atlas are three-dimensional feature images of the breast three-dimensional ultrasound image.
As shown in fig. 2, fig. 2 is a flowchart of constructing an image segmentation model; the following is a flow of constructing a model structure of the image segmentation model convolutional neural network and the cyclic neural network;
further, the convolutional neural network is 3D U-Net, and is composed of an encoder and a decoder, wherein the encoder is used for analyzing and learning characteristics from input data, and the decoder is used for generating a separation result according to the output characteristics of the encoder; wherein all the operation procedures are transformed into 3D operations such as 3D convolution, 3D max-pooling layer, 3D up-convolution layer, etc. The breast tumor segmentation and layering difficulty is increased by considering the characteristics of high noise, blurred edges, uneven gray values, complex tumor shape and tissue hierarchy and the like of the breast ultrasonic image.
Further, the convolutional neural network is formed by coupling a single convolutional layer and a deconvolution layer, and the convolutional neural network is connected to the tail end of an encoder of the convolutional neural network in an opposite mode to realize depth boundary supervision; because the edge pixels and the non-edge pixels of the mammary gland ultrasonic image are unevenly distributed, a similar balance cross entropy function is adopted for the auxiliary boundary network and is used for calculating a loss function corresponding to a convolution layer of the convolution neural networkOptimizing the loss function as:
wherein ,W,w(m) For the convolution layer of the convolution neural network, W represents the weight parameter of the convolution neural network neuron, and W (m) An mth layer representing a convolution layer; wherein,a loss function of a convolution layer which is an mth connected convolution neural network, wherein E is an edge mapping set of a layering boundary; /> and />|E -| and |E+ The I represents the real edge label sets of the edge and the non-edge surface respectively; />Is the activation value calculated using the sigmoid function σ (·) at pixel i; obtaining edge mapping prediction ++after each convolution neural network output>I.e. < ->
wherein ,is output by the convolutional neural network each time;
finally, the function of the convolutional neural network can be realized by minimizing the loss function corresponding to each layer, and the total loss function of the convolutional neural network is as follows:wherein p is a network parameter of the convolutional neural network, < ->Is the weight matrix parameter of the convolutional neural network, p is more than or equal to 1 and less than or equal to Q, and Q is the total number of convolutional layers in the encoder, and is +.>Label prediction loss of the mth layer of the convolution layer is represented, i i·i 2 The L2 norm of the expression; by minimizing the total loss function L (E, X; W e ) And boundary regularization of the convolutional neural network, the shallow network of the encoder portion of the convolutional neural network can learn more useful features.
Further, the method for segmenting the first segmentation atlas to obtain the second segmentation atlas through the cyclic neural network comprises the following steps: the accurate segmentation of the second stage is performed, namely, a 3D convolution long-short-term memory network (3 DConvolume-LSTM, 3D C-LSTM) is constructed based on a full-connection long-short-term memory (FC-LSTM) bidirectional coding mechanism, so that abundant context information on the breast ultrasound 3D data is extracted for prediction enhancement, boundary information is further perfected, and the final segmentation edge and texture are clear and continuous.
The FC-LSTM combines the long-short-term memory neural network LSTM and the fully-connected neural network FC, and the probability volume of the primary segmentation result and the original ultrasonic volume data can be fused, so that the problem of boundary incompleteness can be effectively solved, and the method is used for prediction enhancement. Therefore, the FC-LSTM can further refine the segmentation result based on acquiring more detailed boundary information. Although the FC-LSTM layer is powerful in terms of processing time dependencies, it uses a full connection in input to state and state-to-state transitions, without encoding three-dimensional spatial information. Therefore, we extend LSTM through 3D convolution to obtain convolution long-short-term memory network 3D C-LSTM, which contains 3D convolution structure in both input to state and state-to-state transition.
The construction method of the 3D C-LSTM comprises the following steps:
all inputs, neurons and various gate operations of the full-connection long-short-time memory FC-LSTM framework are expanded into 3D tensors, 3D data are directly adopted as inputs, vector multiplication is replaced by convolution operation, and the framework is defined as follows: i.e t ＝σ(W xi *x t +H t-1 *W hi +W ci oc t-1 +b i )，f t ＝σ(x t *W xf +H t-1 *W hf +W cf oc t-1 +b f )，c t ＝c t-1 of t +i t otanh(x t *W xc +H t-1 *W hc +b c )，o t ＝σ(x t *W xo +H t-1 *W ho +W co oc t +b o )，H t ＝o t otanh(c t ) The method comprises the steps of carrying out a first treatment on the surface of the Here, σ (·) and tanh (·) are logical Sigmoid and hyperbolic tangent functions; i.e t 、f t 、o t 、c 1 ，…，c t The parameters are 3D tensors, and the 3D data input of the network at the t moment is x t ,H t To conceal the layer state, H t-1 In the state of the hidden layer at the previous moment, W xi 、W hi 、W ci 、W xf 、W cf 、W xc 、W hc 、W xo 、W ho 、W co All are weight matrixes in the network; b i 、b f 、b c 、b o Are both bias vectors.
Although the current time output can be inferred by 3D C-LSTM using the current input and the history information accumulated in the hidden state, to better recover breast ultrasound image boundary information, 3D C-LSTM is extended to a Bi-Directional convolutional neural network (BDC-LSTM) that works in two opposite directions by stacking two layers of 3D C-LSTM. Since the context information carried in the two layers is more detailed, the two layers of context information are connected as output, so that the structure can extract finer nonlinear characteristics in the three-dimensional breast ultrasound image.
Further, due to the problems of over-tuning of BDC-LSTM in tuning early time steps and parameters, a suitable depth supervision strategy is advantageous for shortening the gradient back propagation path while preserving the potential context information in the sequence. Therefore, a deep auxiliary supervision mechanism is added in the network structure to improve the training efficiency and generalization capability of BDC-LSTM; each time an auxiliary loss function is added through p BDC-LSTM sequences, each additional auxiliary loss function is l in turn 0 ，l 1 ，...，l k K is the number of auxiliary loss functions that need to be appended, and the final loss function is: wherein ,LN Is the main loss function of BDC-LSTM, L n Is auxiliary loss function, X, Y are three-dimensional input and output respectively, N is the number of BDC-LSTM sequences used, N is the number of auxiliary loss functions used, beta n And the final loss association rate. Through the auxiliary loss function, the BDC-LSTM can acquire faster convergence speed and lower training error, and meanwhile, the generalization capability of the BDC-LSTM is improved.
Although the present application has been described in considerable detail and with particularity with respect to several described embodiments, it is not intended to be limited to any such detail or embodiment or any particular embodiment so as to effectively cover the intended scope of the application. Furthermore, the foregoing description of the application has been presented in its embodiments contemplated by the inventors for the purpose of providing a useful description, and for the purposes of providing a non-essential modification of the application that may not be presently contemplated, may represent an equivalent modification of the application.

Claims (4)

1. A three-dimensional image segmentation method based on convolutional neural network supervision, which is characterized by comprising the following steps:
preliminary segmentation is carried out on the breast three-dimensional ultrasonic image through a convolutional neural network to obtain a first segmentation atlas;
dividing the first divided atlas by the cyclic neural network to obtain a second divided atlas;
the convolutional neural network is 3D U-Net, and is composed of an encoder and a decoder, wherein the encoder is used for analyzing and learning characteristics from input data, and the decoder is used for generating a separation result according to the output characteristics of the encoder;
the convolutional neural network is formed by coupling a single convolutional layer and an deconvolution layer, and a quasi-balanced cross entropy function is adopted for an auxiliary boundary network and is used for calculating a loss function corresponding to the convolutional layer of the convolutional neural networkOptimizing the loss function as:
wherein ,W,w(m) For the convolution layer of the convolution neural network, W represents the weight parameter of the convolution neural network neuron, and W (m) An mth layer representing a convolution layer; wherein,a loss function of a convolution layer which is an mth connected convolution neural network, wherein E is an edge mapping set of a layering boundary; /> and />|E -| and |E+ The I represents the real edge label sets of the edge and the non-edge surface respectively; />Is the activation value calculated using the sigmoid function σ (·) at pixel i; obtaining edge mapping prediction ++after each convolution neural network output>I.e. < -> wherein ,is output by the convolutional neural network each time; finally, the total loss function of the convolutional neural network is:wherein p is a network parameter of the convolutional neural network, W e p Is the weight matrix parameter of the convolutional neural network, p is more than or equal to 1 and less than or equal to Q, Q is the total number of convolutional layers in the encoder, and l (W, W (m) ；W e p ) Label prediction loss of the mth layer of the convolution layer is represented, i i·i 2 The L2 norm of the expression; by minimizing the total loss function L (E, X; W e ) And boundary regularization of the convolutional neural network, the shallow network of the encoder portion of the convolutional neural network learning more useful features.
2. The three-dimensional image segmentation method based on convolutional neural network supervision according to claim 1, wherein the breast three-dimensional ultrasonic image is obtained by scanning the whole breast according to a fixed track through a full-automatic breast volume ultrasonic imaging system; or from an open medical database: one of the LIDC-NLST, ADNI or BraTS CT, MR data sources are read.
3. The three-dimensional image segmentation method based on convolutional neural network supervision according to claim 1, wherein the method for segmenting the first segmentation atlas by the convolutional neural network to obtain the second segmentation atlas comprises the following steps: and (3) performing accurate segmentation in the second stage, namely constructing a 3D convolution long-short-term memory network 3DC-LSTM based on a full-connection long-short-term memory bidirectional coding mechanism:
the construction method of the 3D C-LSTM comprises the following steps:
all inputs, neurons and various gate operations of the full-connection long-short-time memory FC-LSTM framework are expanded into 3D tensors, 3D data are directly adopted as inputs, vector multiplication is replaced by convolution operation, and the framework is defined as follows: i.e t ＝σ(W xi *x t +H t-1 *W hi +W ci oc t-1 +b i )，f t ＝σ(x t *W xf +H t-1 *W hf +W cf oc t-1 +b f )，c t ＝c t-1 of t +i t otanh(x t *W xc +H t-1 *W hc +b c )，o t ＝σ(x t *W xo +H t-1 *W ho +W co oc t +b o )，H t ＝o t otanh(c t ) The method comprises the steps of carrying out a first treatment on the surface of the Here, σ (·) and tanh (·) are logical Sigmoid and hyperbolic tangent functions; i.e t 、f t 、o t 、c 1 ，…，c t The parameters are 3D tensors, and the 3D data input of the network at the t moment is x t ,H t To conceal the layer state, H t-1 In the state of the hidden layer at the previous moment, W xi 、W hi 、W ci 、W xf 、W cf 、W xc 、W hc 、W xo 、W ho 、W co All are weight matrixes in the network; b i 、b f 、b c 、b o Are both bias vectors.
4. A three-dimensional image segmentation method based on convolutional neural network supervision according to claim 3, wherein 3D C-LSTM is extended into a two-way convolutional neural network BDC-LSTM, by stacking two layers of 3D C-LSTM so that they operate in two opposite directions, an auxiliary loss function is added once per p BDC-LSTM sequences, each additional auxiliary loss function being in turn l 0 ，l 1 ，...，l k K is the number of auxiliary loss functions that need to be appended, and the final loss function is: wherein ,LN Is the main loss function of BDC-LSTM, L n Is auxiliary loss function, X, Y are three-dimensional input and output respectively, N is the number of BDC-LSTM sequences used, N is the number of auxiliary loss functions used, beta n And the final loss association rate.
CN202011040012.9A 2020-09-28 2020-09-28 Three-dimensional image segmentation method based on convolutional neural network supervision Active CN112258456B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011040012.9A CN112258456B (en) 2020-09-28 2020-09-28 Three-dimensional image segmentation method based on convolutional neural network supervision

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011040012.9A CN112258456B (en) 2020-09-28 2020-09-28 Three-dimensional image segmentation method based on convolutional neural network supervision

Publications (2)

Publication Number Publication Date
CN112258456A CN112258456A (en) 2021-01-22
CN112258456B true CN112258456B (en) 2023-10-17

Family

ID=74234169

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011040012.9A Active CN112258456B (en) 2020-09-28 2020-09-28 Three-dimensional image segmentation method based on convolutional neural network supervision

Country Status (1)

Country Link
CN (1) CN112258456B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112801282A (en) * 2021-03-24 2021-05-14 东莞中国科学院云计算产业技术创新与育成中心 Three-dimensional image processing method, three-dimensional image processing device, computer equipment and storage medium
CN113269691B (en) * 2021-05-27 2022-10-21 北京卫星信息工程研究所 SAR image denoising method for noise affine fitting based on convolution sparsity
CN113538693B (en) * 2021-07-06 2022-10-14 太原理工大学 Microwave mammary gland image reconstruction method based on deep learning
CN113837931A (en) * 2021-09-27 2021-12-24 海南长光卫星信息技术有限公司 Method and device for detecting transformation of remote sensing image, electronic equipment and storage medium
CN114283164B (en) * 2022-03-02 2022-06-14 华南理工大学 Breast cancer pathological section image segmentation prediction system based on UNet3+

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109410220A (en) * 2018-10-16 2019-03-01 腾讯科技（深圳）有限公司 Image partition method, device, computer equipment and storage medium
CN109690554A (en) * 2016-07-21 2019-04-26 西门子保健有限责任公司 Method and system for the medical image segmentation based on artificial intelligence
CN110414481A (en) * 2019-08-09 2019-11-05 华东师范大学 A kind of identification of 3D medical image and dividing method based on Unet and LSTM
CN110516740A (en) * 2019-08-28 2019-11-29 电子科技大学 A kind of fault recognizing method based on Unet++ convolutional neural networks
CN111179237A (en) * 2019-12-23 2020-05-19 北京理工大学 Image segmentation method and device for liver and liver tumor
CN111242940A (en) * 2020-01-19 2020-06-05 复旦大学 Tongue image segmentation method based on weak supervised learning

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10922816B2 (en) * 2018-08-27 2021-02-16 Siemens Healthcare Gmbh Medical image segmentation from raw data using a deep attention neural network

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109690554A (en) * 2016-07-21 2019-04-26 西门子保健有限责任公司 Method and system for the medical image segmentation based on artificial intelligence
CN109410220A (en) * 2018-10-16 2019-03-01 腾讯科技（深圳）有限公司 Image partition method, device, computer equipment and storage medium
CN110414481A (en) * 2019-08-09 2019-11-05 华东师范大学 A kind of identification of 3D medical image and dividing method based on Unet and LSTM
CN110516740A (en) * 2019-08-28 2019-11-29 电子科技大学 A kind of fault recognizing method based on Unet++ convolutional neural networks
CN111179237A (en) * 2019-12-23 2020-05-19 北京理工大学 Image segmentation method and device for liver and liver tumor
CN111242940A (en) * 2020-01-19 2020-06-05 复旦大学 Tongue image segmentation method based on weak supervised learning

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
A deep-learning based automatic pulmonary nodule detection system;Zhao, Yiyuan 等;《Proceedings of the SPIE》;1-8 *
An RDAU-NET model for lesion segmentation in breast ultrasound images;Zhemin Zhuang 等;《PLOS ONE》;1-23 *
Boundary Loss for Remote Sensing Imagery Semantic Segmentation;Alexey Bokhovkin 等;《arXiv:1905.07852》;1- *
Combining Fully Convolutional and Recurrent Neural Networks for 3D Biomedical Image Segmentation;Jianxu Chen 等;《arXiv:1609.01006v2》;1-9 *
Deep Image-to-Image Recurrent Network with Shape Basis Learning for Automatic Vertebra Labeling in Large-Scale 3D CT Volumes;Dong Yang 等;《Medical Image Computing and Computer Assisted Intervention − MICCAI 2017》;第10435卷;498-506 *
基于深度学习的左心房分割算法研究;代洋洋;《中国优秀硕士学位论文全文数据库 医药卫生科技辑》(第2期);E062-257 *

Also Published As

Publication number Publication date
CN112258456A (en) 2021-01-22

Similar Documents

Publication Publication Date Title
CN112258456B (en) Three-dimensional image segmentation method based on convolutional neural network supervision
Asgari Taghanaki et al. Deep semantic segmentation of natural and medical images: a review
CN111192245B (en) Brain tumor segmentation network and method based on U-Net network
CN112465827B (en) Contour perception multi-organ segmentation network construction method based on class-by-class convolution operation
Biffi et al. Explainable anatomical shape analysis through deep hierarchical generative models
CN110475505A (en) Utilize the automatic segmentation of full convolutional network
Xing et al. Lesion segmentation in ultrasound using semi-pixel-wise cycle generative adversarial nets
CN116309650B (en) Medical image segmentation method and system based on double-branch embedded attention mechanism
Wang et al. MMNet: A multi-scale deep learning network for the left ventricular segmentation of cardiac MRI images
Chen et al. Generative adversarial U-Net for domain-free medical image augmentation
CN114972362A (en) Medical image automatic segmentation method and system based on RMAU-Net network
Suganthi Review of medical image synthesis using GAN techniques
CN115908463A (en) 3D coronary artery image segmentation method based on semi-supervised consistency learning
CN117058307A (en) Method, system, equipment and storage medium for generating heart three-dimensional nuclear magnetic resonance image
CN114387282A (en) Accurate automatic segmentation method and system for medical image organs
Zhang et al. Multi-modality reconstruction attention and difference enhancement network for brain mri image segmentation
Qiu et al. A deep learning approach for segmentation, classification, and visualization of 3-D high-frequency ultrasound images of mouse embryos
Shoaib et al. YOLO Object Detector and Inception-V3 Convolutional Neural Network for Improved Brain Tumor Segmentation.
Ruan et al. An efficient tongue segmentation model based on u-net framework
Poonkodi et al. 3D-MedTranCSGAN: 3D medical image transformation using CSGAN
CN110570416B (en) Method for visualization and 3D printing of multi-modal cardiac images
CN115731444A (en) Medical image fusion method based on artificial intelligence and superpixel segmentation
Mansouri Musolu et al. Deep learning and its applications in medical imaging
Wang et al. Ultrasonic breast tumor extraction based on adversarial mechanism and active contour
Song et al. A survey of deep learning based methods in medical image processing

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant