CN110555836A - Automatic identification method and system for standard fetal section in ultrasonic image - Google Patents
Automatic identification method and system for standard fetal section in ultrasonic image Download PDFInfo
- Publication number
- CN110555836A CN110555836A CN201910835283.4A CN201910835283A CN110555836A CN 110555836 A CN110555836 A CN 110555836A CN 201910835283 A CN201910835283 A CN 201910835283A CN 110555836 A CN110555836 A CN 110555836A
- Authority
- CN
- China
- Prior art keywords
- layer
- size
- data set
- image
- output matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10132—Ultrasound image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30044—Fetus; Embryo
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Computational Linguistics (AREA)
- Medical Informatics (AREA)
- Epidemiology (AREA)
- Public Health (AREA)
- Quality & Reliability (AREA)
- Primary Health Care (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Ultra Sonic Daignosis Equipment (AREA)
Abstract
The invention belongs to the technical field of prenatal ultrasonic examination, and discloses an automatic identification method of a standard fetal section in an ultrasonic image, which comprises the following steps: and acquiring a data set, preprocessing the acquired data set to obtain a preprocessed data set, and inputting the acquired preprocessed data set into a trained deep convolutional neural network to obtain the category and the position of a target contained in each fetal ultrasound sectional plane image. The deep convolutional neural network comprises a backbone network ResNet-50, a characteristic pyramid network, a classification subnet and a positioning subnet which are connected in sequence. The invention can solve the technical problems of relatively high technical requirements on doctors, low accuracy and consistency caused by the difference of the examination results of different doctors and overlong examination time in the existing standard section identification method.
Description
Technical Field
the invention belongs to the technical field of prenatal ultrasonic examination, and particularly relates to an automatic identification method and system for a standard fetal section in an ultrasonic image.
background
ultrasound imaging has been widely used in pregnancy diagnosis at present due to its advantages of low consumption, real-time imaging and no radiation. In the process of performing ultrasonic diagnosis of fetal diseases by using ultrasonic imaging, an sonographer firstly needs to acquire each standard section of a fetus, then identifies the standard sections, inspects main anatomical structures through the standard sections, then performs further diagnosis and inspection, and performs analysis and diagnosis according to the growth condition of the fetus and whether the anatomical structures of the fetus are abnormal.
in the current ultrasonic diagnosis process, the identification of the standard section is mainly obtained by manually marking points and lines on the basis of manually obtaining the standard section or three-dimensional ultrasonic volume data based on two-dimensional ultrasonic scanning. However, this identification method has some non-negligible drawbacks: first, the acquisition of a standard cut plane is quite demanding for the sonographer as it is overly dependent on the sonographer's clinical experience and anatomical knowledge; secondly, the accuracy and consistency of the examination result are low due to the difference of the recognition results of different sonographers on the standard section; finally, the existing identification method has the disadvantages of overlong examination time for pregnant women, frequent operation and high labor intensity in the examination process of sonographers.
Disclosure of Invention
aiming at the defects or the improvement requirements of the prior art, the invention provides an automatic identification method and an automatic identification system for a standard fetal section in an ultrasonic image, and aims to solve the technical problems of relatively high technical requirements on sonographers, low accuracy and consistency caused by difference of examination results of different sonographers, overlong examination time, frequent operation in the examination process of the sonographers and high labor intensity in the existing identification method for the standard fetal section.
In order to achieve the above object, according to an aspect of the present invention, there is provided a method for automatically identifying a standard fetal tangent plane in an ultrasound image, comprising the steps of:
(1) Acquiring a data set;
(2) And (2) carrying out preprocessing operation on the data set acquired in the step (1) to obtain a preprocessed data set.
(3) And (3) inputting the preprocessed data set obtained in the step (2) into a trained deep convolutional neural network to obtain the category and the position of the target contained in each fetal ultrasound sectional plane image.
Preferably, the data set is a plurality of frames of fetal ultrasound sectional images acquired from an ultrasound device.
Preferably, the preprocessing operation performed on the acquired data set in step (2) includes the sub-steps of:
(2-1) deleting redundant information related to ultrasonic equipment parameters in each image in the acquired data set, zooming the image to 800x600 pixels, and performing normalization processing on the zoomed image by using a linear function to obtain a normalized image;
(2-2) carrying out random enhancement operation on each image normalized in the step (2-1) to obtain a randomly enhanced image;
and (2-3) graying each image randomly enhanced in the step (2-2) to obtain a grayscale image, performing histogram equalization on the obtained grayscale image to obtain an equalized image, processing the grayscale image by using a Gaussian difference function to obtain a Gaussian difference image, and synthesizing each obtained grayscale image, the equalized image corresponding to the grayscale image and the Gaussian difference image into a three-channel image serving as a preprocessed image, wherein all preprocessed images form a preprocessed data set.
preferably, the deep convolutional neural network comprises a backbone network ResNet-50, a feature pyramid network, a classification subnet, and a positioning subnet, which are connected in sequence.
preferably, for the backbone network ResNet-50, the network structure is as follows:
the first layer is an input layer, which is a matrix of 600 × 800 × 3 pixels;
the second layer is a feature extraction layer, which uses the public feature extraction network Resnet-50, and takes the output matrices of three layers, i.e., the conv3.x layer, the conv4.x layer and the conv5.x layer, in the feature extraction network Resnet-50 as extracted features C3, C4 and C5, whose sizes are 75 × 100 × 512, 38 × 50 × 1024 and 19 × 25 × 2048, respectively.
preferably, the feature pyramid layer performs feature fusion on features C3, C4 and C5 input by the backbone network ResNet-50, and outputs fused 5-scale features P3, P4, P5, P6 and P7, and the network structure of the feature pyramid layer is as follows:
The first layer is a convolution layer based on feature C5 with convolution kernel size 1 x 256 and step size 1, this layer is filled using SAME pattern, the output matrix is 19 x 25 x 256;
The second layer is a convolution layer with convolution kernel size 3 x 256 and step size 1, this layer is filled using SAME pattern, its output matrix P5 size is 19 x 25 x 256;
the third layer is a convolution layer based on feature C4, with convolution kernel size 1 x 256 and step size 1, this layer is filled using SAME pattern, and its output matrix P4_ size 38 x 50 x 256 is noted;
The fourth layer is an upsampling layer, which upsamples the output matrix P5 into an output matrix P5_ upsamplie, which has a size of 38 × 50 × 1024;
the fifth layer is an add layer that adds an output matrix P5_ update and an output matrix P4_ with an output matrix size of 38 x 50 x 1024;
the sixth layer is a convolution layer with convolution kernel size 3 x 256 and step size 1, filled using SAME pattern, and output matrix P4 size 38 x 50 x 256;
The seventh layer is a convolutional layer based on feature C3 with convolutional kernel size 1 x 256 and step size 1, filled using SAME pattern, with output matrix P3_ size 75 x 100 x 256;
the eighth layer is an upsampling layer, which upsamples P4 to size 75 × 100, with an output matrix P4_ upsampled size of 75 × 100 × 512;
The ninth layer is an Add layer that adds P4_ update and P3_ with an output matrix size of 75 × 100 × 512;
the tenth layer is a convolution layer with convolution kernel size 3 x 256 and step size 1, this layer is filled using SAME pattern, output matrix P3 size 75 x 100 x 512;
the eleventh layer is a convolution layer on C5 with convolution kernel size 3 x 256 and step size 2, this layer is filled with SAME pattern, and its output matrix P6 is 19 x 25 x 256 in size;
the twelfth layer is a convolution layer with convolution kernel size 3 x 256 and step size 2, filled using SAME pattern, and output matrix P7 size 19 x 25 x 256.
Preferably, the input matrices for the sorting and localization subnets are identical to the output matrices P3, P4, P5, P6, P7 of the aforementioned characteristic pyramid layers, the first to fourth layers of the sorting and localization subnets are all sequentially connected, identical convolution layers, the convolution kernel size is 3 × 3 256, the step size is 1, the first to fourth layers are all filled using SAME pattern, the output matrices of the layers are the SAME size, are all 75 × 100 × 256, 38 × 50 × 256, 19 × 25 × 256, for the localization subnets, the fifth layer is a convolution, the convolution kernel size is 3 × 36, the step size is 1, the layers are filled using SAME pattern, the output matrices are 75 × 100 × 36, 38 × 50, 19 × 36, 25 × 36 for the localization subnets, the convolution kernel size was 3 x 80, the step size was 1, the layers were filled using SAME patterns, and the output matrix sizes were 75 x 100 x 80, 38 x 50 x 80, 19 x 25 x 80, respectively.
preferably, the deep convolutional neural network is trained by the following steps:
(a1) Acquiring a data set, sending the data set to an ultrasonic expert, and acquiring the data set labeled by the ultrasonic expert;
(a2) Preprocessing the labeled data set to obtain a preprocessed data set;
(a3) counting the data set labeled in the step (a1) by using a K-means clustering algorithm to obtain 3 proportional values which can represent the length and the width of the key target in the data set most and are used as the proportion of the anchor points in the deep convolutional neural network;
(a4) inputting a batch of data in the training set part in the preprocessed data set obtained in the step (a2) into a deep convolutional neural network to obtain an inference output, and inputting the inference output and the data set labeled by the ultrasonic expert in the step (a1) into a loss function Lfl of the deep convolutional neural network to obtain a loss value.
(a5) optimizing a loss function L fl of the deep convolutional neural network according to an Adam algorithm by using the loss value obtained in the step (a 4);
(a6) And (d) repeating the steps (a4) and (a5) in sequence for the rest batches of data in the training set part in the preprocessed data set obtained in the step (a2) until the number of iterations is reached, so as to obtain the trained deep convolutional neural network.
preferably, the loss values used in the deep convolutional neural network are calculated by the following loss function L fl:
wherein y' represents the inference output of the deep convolutional neural network on the input image, y represents whether the evaluation object is a foreground, γ represents the rate of adjusting the weight reduction of the simple samples, γ belongs to [0,5], α is a weight factor, and α belongs to [0,1 ].
according to another aspect of the present invention, there is provided a system for automatically identifying a standard fetal tangent plane in an ultrasound image, comprising:
A first module for obtaining a data set;
and the second module is used for carrying out preprocessing operation on the data set acquired by the first module to obtain a preprocessed data set.
And the third module is used for inputting the preprocessing data set obtained by the second module into the trained deep convolutional neural network so as to obtain the category and the position of the target contained in each fetal ultrasonic sectional image.
In general, compared with the prior art, the above technical solution contemplated by the present invention can achieve the following beneficial effects:
(1) since the data sets used in the learning process of the invention are all selected and precisely labeled by the sonographer according to clinical experience, the invention can obtain the knowledge of the most experienced sonographer through machine learning, so that the sonographer can be programmed and automated in the whole process of patient examination, thereby solving the technical problem that the prior standard section identification method excessively depends on the clinical experience of the sonographer and the knowledge of the anatomical structure, and therefore the technical requirement of the acquisition of the standard section on the sonographer is quite high.
(2) The selection standard of the data set adopted in the training process is finally and uniquely determined by the sonographer, so that the concept of the standard tangent plane of the data set for training is clearly defined, namely, the standard tangent planes of various types contained in the data set are deterministic and regular, and the technical problem of low accuracy and consistency caused by the difference of the examination results of different sonographers in the existing standard tangent plane identification method can be solved.
(3) the invention is full-automatic and programmed, the sonographer does not need to stop to manually intercept the section in the detection process, the accuracy of the intercepted section does not need to be checked, and the standard section is identified in real time, so the technical problems of overlong examination time, frequent operation in the examination process of the sonographer and high labor intensity in the existing standard section identification method can be solved.
drawings
FIG. 1 is a flow chart of the method for automatically identifying a standard fetal tangent plane in an ultrasound image according to the present invention.
FIG. 2 is an architectural diagram of a deep self-convolution neural network used in step (2) of the method of the present invention;
fig. 3 is a schematic diagram of the recognition results obtained after different sections of the fetus are input into the deep convolutional neural network of the present invention.
Detailed Description
in order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
the invention aims to provide an automatic identification method of a fetal standard section in an ultrasonic image, which enables a computer to learn autonomously and artificially guide the computer to learn clinical prior knowledge so as to achieve the automatic identification of more than 40 standard sections required clinically. And (4) using a deep learning network to train and learn the ultrasonic section of the fetus in all categories. Then, the newly acquired ultrasonic images are inferred, and the categories of various sections and the probability of the sections meeting the standard are judged.
as shown in fig. 1, the present invention provides a method for automatically identifying a standard fetal tangent plane in an ultrasound image, comprising the following steps:
(1) acquiring a data set;
in particular, the data set is a multi-frame fetal ultrasound sectional image taken from ultrasound equipment manufactured by mainstream manufacturers on the market (including samsung, siemens, kelly, etc.).
(2) and (2) carrying out preprocessing operation on the data set acquired in the step (1) to obtain a preprocessed data set.
The preprocessing operation on the acquired data set in the step comprises the following substeps:
(2-1) deleting redundant information related to ultrasonic equipment parameters in each image in the acquired data set, zooming the image to 800x600 pixels, and performing normalization processing on the zoomed image by using a linear function to obtain a normalized image;
(2-2) carrying out random enhancement operation on each image normalized in the step (2-1) to obtain a randomly enhanced image;
specifically, the enhancement operation may be one of an angular rotation, a horizontal-vertical flipping, a fill-mode scaling of edge mirroring, a small-scale degree of brightness random perturbation, or any combination thereof.
(2-3) carrying out graying processing on each image randomly enhanced in the step (2-2) to obtain a grayscale image, carrying out histogram equalization processing on the obtained grayscale image to obtain an equalized image, processing the grayscale image by utilizing a Gaussian difference function to obtain a Gaussian difference image, synthesizing each obtained grayscale image, the equalized image corresponding to the grayscale image and the Gaussian difference image into a three-channel image serving as a preprocessed image, and forming a preprocessed data set by all preprocessed images;
(3) and (3) inputting the preprocessed data set obtained in the step (2) into a trained deep convolutional neural network to obtain the category and the position of the target contained in each fetal ultrasound sectional plane image.
as shown in fig. 2, the deep convolutional neural network used in the present invention includes a backbone network ResNet-50, a Feature Pyramid Network (FPN), a classification subnet, and a positioning subnet, which are connected in sequence.
For the backbone network ResNet-50, the network structure is as follows:
The first layer is an input layer, which is a matrix of 600 × 800 × 3 pixels;
the second layer is a feature extraction layer, which adopts the public feature extraction network Resnet-50, and takes the output matrixes of three layers, namely, the conv3.x layer, the conv4.x layer and the conv5.x layer in the feature extraction network Resnet-50 as extracted features C3, C4 and C5, wherein the sizes of the extracted features are 75 × 100 × 512, 38 × 50 × 1024 and 19 × 25 × 2048 respectively;
for the feature pyramid layer, feature fusion is performed on features C3, C4 and C5 input by a backbone network ResNet-50, and fused 5-scale features P3, P4, P5, P6 and P7 are output, and the feature pyramid layer can be further subdivided into specific 12 layers, and the network structure is as follows:
the first layer is a convolution layer based on feature C5 with convolution kernel size 1 x 256 and step size 1, this layer is filled using SAME pattern, the output matrix is 19 x 25 x 256;
the second layer is a convolution layer with convolution kernel size 3 x 256 and step size 1, this layer is filled using SAME pattern, its output matrix P5 size is 19 x 25 x 256;
the third layer is a convolution layer based on feature C4, with convolution kernel size 1 x 256 and step size 1, this layer is filled using SAME pattern, and its output matrix P4_ size 38 x 50 x 256 is noted;
the fourth layer is an upsampling layer, which upsamples the output matrix P5 into an output matrix P5_ upsamplie, which has a size of 38 × 50 × 1024;
the fifth layer is an Add (Add) layer that adds output matrix P5_ upsample and output matrix P4_ with an output matrix size of 38 x 50 x 1024;
the sixth layer is a convolution layer with convolution kernel size 3 x 256 and step size 1, filled using SAME pattern, and output matrix P4 size 38 x 50 x 256;
the seventh layer is a convolutional layer based on feature C3 with convolutional kernel size 1 x 256 and step size 1, filled using SAME pattern, with output matrix P3_ size 75 x 100 x 256;
The eighth layer is an upsampling layer, which upsamples P4 to size 75 × 100, with an output matrix P4_ upsampled size of 75 × 100 × 512;
The ninth layer is an Add layer that adds P4_ update and P3_ with an output matrix size of 75 × 100 × 512;
The tenth layer is a convolution layer with convolution kernel size 3 x 256 and step size 1, this layer is filled using SAME pattern, output matrix P3 size 75 x 100 x 512;
The eleventh layer is a convolution layer on C5 with convolution kernel size 3 x 256 and step size 2, this layer is filled with SAME pattern, and its output matrix P6 is 19 x 25 x 256 in size;
the twelfth layer is a convolution layer with convolution kernel size 3 x 256 and step size 2, filled using SAME pattern, and output matrix P7 size 19 x 25 x 256;
for the classification subnet and the positioning subnet, both are 5 layers, and the input matrices of both are the same, and the output matrices P3, P4, P5, P6, P7, which are all feature pyramid layers, are also the same, and the specific structures of the first 4-layer network structures of both are the same as follows:
the first to fourth layers are sequentially connected, completely identical convolution layers, the convolution kernel size is 3 x 256, the step size is 1, the first to fourth layers are all filled by using the SAME mode, and the output matrixes of the layers are the SAME in size and are all 75 x 100 x 256, 38 x 50 x 256, 19 x 25 x 256 and 19 x 25 x 256;
The fifth layer is a convolution layer with convolution kernel size 3 x 36, step size 1, filled using SAME pattern, output matrix size 75 x 100 x 36, 38 x 50 x 36, 19 x 25 x 36, respectively; for the class subnet, the convolution kernel size is 3 x 80, the step size is 1, the layer is filled using SAME pattern, and the output matrix size is 75 x 100 x 80, 38 x 50 x 80, 19 x 25 x 80, respectively.
The loss value used in the present deep convolutional neural network is calculated by the following loss function L fl:
Where y' represents the inference output of the deep convolutional neural network on the input image, y represents whether the evaluation object is foreground, γ represents the rate of adjusting the weight reduction of the simple samples, γ is 0,5, preferably 2, α is a weighting factor for balancing the importance of the positive/negative number samples, and α is 0, 1.
specifically, the deep convolutional neural network used in this step is obtained by training through the following steps:
(1) acquiring a data set, sending the data set to an ultrasonic expert, and acquiring the data set labeled by the ultrasonic expert;
Specifically, the data set is 30000 fetal ultrasound sectional images obtained from ultrasound devices manufactured by major manufacturers on the market (including samsung, siemens, kelly, etc.), and these fetal ultrasound sectional images are randomly divided into 3 parts, of which 80% is a training set (Train set), 10% is a verification set (Validation set), and 10% is a test set (Testset).
(2) preprocessing the labeled data set to obtain a preprocessed data set;
specifically, the preprocessing process in this step is completely the same as the preprocessing process described above, and is not described herein again.
(3) Counting the data set labeled in the step (1) by using a K-means clustering algorithm to obtain 3 proportional values which can represent the length and the width of a key target in the data set most and are used as the proportion of anchor points (anchors) in the deep convolutional neural network;
(4) and (3) inputting a batch of secondary data in the training set part in the preprocessed data set obtained in the step (2) into the deep convolutional neural network to obtain inference output, and inputting the inference output and the data set labeled by the ultrasonic expert in the step (1) into a loss function L fl of the deep convolutional neural network to obtain a loss value.
specifically, one batch of data is 4 images;
(5) And (4) optimizing a loss function L fl of the deep convolutional neural network according to the Adam algorithm and by using the loss value obtained in the step (4) so as to achieve the purpose of gradually updating the parameters in the deep convolutional neural network.
specifically, in the optimization process, the learning rate lr is 0.001, the impulse ξ is 0.9, and the weight attenuation ψ is 0.004.
(6) Repeating the step (4) and the step (5) in sequence aiming at the residual batch of data in the training set part in the preprocessed data set obtained in the step (2) until the iteration number is reached, thereby obtaining a trained deep convolutional neural network;
Specifically, the training process in this step includes 120 cycles, and the number of iterations in each cycle is 6000.
(7) And (3) verifying the trained deep convolutional neural network by using the test set part in the preprocessed data set obtained in the step (2).
Test results
Inputting a new non-standard section, an approximate standard section and a standard section of each part of the fetus in prenatal ultrasonic examination into a trained model, automatically identifying the type of the input section by the model and giving an identification result, as shown in fig. 3.
The average accuracy, the standard tangent plane detection rate and the standard tangent plane false detection rate of the trained model on a new test set are as follows, and it can be seen that the accuracy and the standard tangent plane detection rate of the invention are quite high, and the standard tangent plane false detection rate is very low.
| Average rate of accuracy | Standard section detection rate | false detection rate of standard tangent plane |
| 98.61％ | 96.88％ | 0.001％ |
it will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. An automatic identification method for a standard fetal tangent plane in an ultrasonic image is characterized by comprising the following steps:
(1) acquiring a data set;
(2) and (2) carrying out preprocessing operation on the data set acquired in the step (1) to obtain a preprocessed data set.
(3) And (3) inputting the preprocessed data set obtained in the step (2) into a trained deep convolutional neural network to obtain the category and the position of the target contained in each fetal ultrasound sectional plane image.
2. The automatic identification method of claim 1, wherein the data set is a plurality of frames of fetal ultrasound sectional images acquired from an ultrasound device.
3. the automatic identification method according to claim 1, wherein the preprocessing operation performed on the acquired data set in step (2) comprises the sub-steps of:
(2-1) deleting redundant information related to ultrasonic equipment parameters in each image in the acquired data set, zooming the image to 800x600 pixels, and performing normalization processing on the zoomed image by using a linear function to obtain a normalized image;
(2-2) carrying out random enhancement operation on each image normalized in the step (2-1) to obtain a randomly enhanced image;
and (2-3) graying each image randomly enhanced in the step (2-2) to obtain a grayscale image, performing histogram equalization on the obtained grayscale image to obtain an equalized image, processing the grayscale image by using a Gaussian difference function to obtain a Gaussian difference image, and synthesizing each obtained grayscale image, the equalized image corresponding to the grayscale image and the Gaussian difference image into a three-channel image serving as a preprocessed image, wherein all preprocessed images form a preprocessed data set.
4. the automatic identification method of claim 1, wherein the deep convolutional neural network comprises a backbone network ResNet-50, a feature pyramid network, a classification subnet, and a positioning subnet connected in sequence.
5. the automatic identification method according to claim/4, wherein for the backbone network ResNet-50, the network structure is as follows:
The first layer is an input layer, which is a matrix of 600 × 800 × 3 pixels;
The second layer is a feature extraction layer, which uses the public feature extraction network Resnet-50, and takes the output matrices of three layers, i.e., the conv3.x layer, the conv4.x layer and the conv5.x layer, in the feature extraction network Resnet-50 as extracted features C3, C4 and C5, whose sizes are 75 × 100 × 512, 38 × 50 × 1024 and 19 × 25 × 2048, respectively.
6. the automatic identification method of claim 5, wherein the feature pyramid layer performs feature fusion on features C3, C4 and C5 input by a backbone network ResNet-50 and outputs fused 5-scale features P3, P4, P5, P6 and P7, and the network structure of the feature pyramid layer is as follows:
the first layer is a convolution layer based on feature C5 with convolution kernel size 1 x 256 and step size 1, this layer is filled using SAME pattern, the output matrix is 19 x 25 x 256;
The second layer is a convolution layer with convolution kernel size 3 x 256 and step size 1, this layer is filled using SAME pattern, its output matrix P5 size is 19 x 25 x 256;
The third layer is a convolution layer based on feature C4, with convolution kernel size 1 x 256 and step size 1, this layer is filled using SAME pattern, and its output matrix P4_ size 38 x 50 x 256 is noted;
The fourth layer is an upsampling layer, which upsamples the output matrix P5 into an output matrix P5_ upsamplie, which has a size of 38 × 50 × 1024;
The fifth layer is an add layer that adds an output matrix P5_ update and an output matrix P4_ with an output matrix size of 38 x 50 x 1024;
the sixth layer is a convolution layer with convolution kernel size 3 x 256 and step size 1, filled using SAME pattern, and output matrix P4 size 38 x 50 x 256;
The seventh layer is a convolutional layer based on feature C3 with convolutional kernel size 1 x 256 and step size 1, filled using SAME pattern, with output matrix P3_ size 75 x 100 x 256;
The eighth layer is an upsampling layer, which upsamples P4 to size 75 × 100, with an output matrix P4_ upsampled size of 75 × 100 × 512;
the ninth layer is an Add layer that adds P4_ update and P3_ with an output matrix size of 75 × 100 × 512;
the tenth layer is a convolution layer with convolution kernel size 3 x 256 and step size 1, this layer is filled using SAME pattern, output matrix P3 size 75 x 100 x 512;
The eleventh layer is a convolution layer on C5 with convolution kernel size 3 x 256 and step size 2, this layer is filled with SAME pattern, and its output matrix P6 is 19 x 25 x 256 in size;
The twelfth layer is a convolution layer with convolution kernel size 3 x 256 and step size 2, filled using SAME pattern, and output matrix P7 size 19 x 25 x 256.
7. The automatic recognition method according to claim 6,
Input matrixes for classifying and positioning subnets, and output matrixes P3, P4, P5, P6 and P7 of the characteristic pyramid layer;
The first layer to the fourth layer of the classifying subnets and the positioning subnets are completely identical, are sequentially connected and completely identical convolution layers, the convolution kernel size is 3 × 256, the step size is 1, the first layer to the fourth layer are filled by using an SAME mode, and the output matrixes of the layers are the SAME in size and are 75 × 100 × 256, 38 × 50 × 256, 19 × 25 × 256 and 19 × 25 × 256;
for the localization subnetwork, its fifth layer is a convolution layer, the convolution kernel size is 3 x 36, the step size is 1, the layer is filled using SAME pattern, the output matrix size is 75 x 100 x 36, 38 x 50 x 36, 19 x 25 x 36, respectively;
for the sorting subnetwork, its fifth layer is the convolutional layer, with convolutional kernel size 3 x 80, step size 1, this layer is filled with SAME pattern, output matrix size 75 x 100 x 80, 38 x 50 x 80, 19 x 25 x 80, respectively.
8. the automatic identification method of claim 1, wherein the deep convolutional neural network is trained by the following steps:
(a1) Acquiring a data set, sending the data set to an ultrasonic expert, and acquiring the data set labeled by the ultrasonic expert;
(a2) Preprocessing the labeled data set to obtain a preprocessed data set;
(a3) counting the data set labeled in the step (a1) by using a K-means clustering algorithm to obtain 3 proportional values which can represent the length and the width of the key target in the data set most and are used as the proportion of the anchor points in the deep convolutional neural network;
(a4) Inputting a batch of data in the training set part in the preprocessed data set obtained in the step (a2) into a deep convolutional neural network to obtain an inference output, and inputting the inference output and the data set labeled by the ultrasonic expert in the step (a1) into a loss function Lfl of the deep convolutional neural network to obtain a loss value.
(a5) optimizing a loss function L fl of the deep convolutional neural network according to an Adam algorithm by using the loss value obtained in the step (a 4);
(a6) and (d) repeating the steps (a4) and (a5) in sequence for the rest batches of data in the training set part in the preprocessed data set obtained in the step (a2) until the number of iterations is reached, so as to obtain the trained deep convolutional neural network.
9. The automatic identification method of claim 8, wherein the loss value used in the deep convolutional neural network is calculated by the following loss function L fl:
Wherein y' represents the inference output of the deep convolutional neural network on the input image, y represents whether the evaluation object is a foreground, γ represents the rate of adjusting the weight reduction of the simple samples, γ belongs to [0,5], α is a weight factor, and α belongs to [0,1 ].
10. An automatic identification system for standard fetal tangent plane in ultrasonic image, comprising:
a first module for obtaining a data set;
and the second module is used for carrying out preprocessing operation on the data set acquired by the first module to obtain a preprocessed data set.
And the third module is used for inputting the preprocessing data set obtained by the second module into the trained deep convolutional neural network so as to obtain the category and the position of the target contained in each fetal ultrasonic sectional image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910835283.4A CN110555836A (en) | 2019-09-05 | 2019-09-05 | Automatic identification method and system for standard fetal section in ultrasonic image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910835283.4A CN110555836A (en) | 2019-09-05 | 2019-09-05 | Automatic identification method and system for standard fetal section in ultrasonic image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110555836A true CN110555836A (en) | 2019-12-10 |
Family
ID=68739056
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910835283.4A Pending CN110555836A (en) | 2019-09-05 | 2019-09-05 | Automatic identification method and system for standard fetal section in ultrasonic image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110555836A (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110974299A (en) * | 2019-12-31 | 2020-04-10 | 上海杏脉信息科技有限公司 | Ultrasonic scanning robot system, ultrasonic scanning method and medium |
| CN111223092A (en) * | 2020-02-28 | 2020-06-02 | 长沙大端信息科技有限公司 | Automatic quality control system and detection method for ultrasonic sectional images of fetus |
| CN111462059A (en) * | 2020-03-24 | 2020-07-28 | 湖南大学 | Parallel processing method and device for intelligent target detection of fetal ultrasound image |
| CN111915562A (en) * | 2020-07-02 | 2020-11-10 | 杭州深睿博联科技有限公司 | Deep learning children echocardiogram standard tangent plane identification method and device |
| CN111932513A (en) * | 2020-08-07 | 2020-11-13 | 深圳市妇幼保健院 | Method and system for imaging three-dimensional image of fetal sulcus gyrus in ultrasonic image |
| CN112102230A (en) * | 2020-07-24 | 2020-12-18 | 湖南大学 | Ultrasonic tangent plane identification method, system, computer equipment and storage medium |
| CN112155602A (en) * | 2020-09-24 | 2021-01-01 | 广州爱孕记信息科技有限公司 | Method and device for determining optimal standard section of fetus |
| CN113487614A (en) * | 2021-09-08 | 2021-10-08 | 四川大学 | Training method and device for fetus ultrasonic standard section image recognition network model |
| CN113688874A (en) * | 2021-07-29 | 2021-11-23 | 天津中科智能识别产业技术研究院有限公司 | Method and system for automatically segmenting iris region in human eye iris image |
| CN114334095A (en) * | 2021-12-31 | 2022-04-12 | 深圳度影医疗科技有限公司 | Intelligent identification method and system for ultrasonic examination and terminal equipment |
| CN114782407A (en) * | 2022-05-23 | 2022-07-22 | 中山大学附属第一医院 | Fetal ultrasonic key section identification method and device based on convolutional neural network |
| CN115482190A (en) * | 2021-11-10 | 2022-12-16 | 中山大学附属第七医院(深圳) | Fetal heart structure segmentation measurement method and device and computer storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109034268A (en) * | 2018-08-20 | 2018-12-18 | 北京林业大学 | A kind of red turpentine beetle detector optimization method of Information element trapper |
| CN109166095A (en) * | 2018-07-11 | 2019-01-08 | 广东技术师范学院 | A kind of ophthalmoscopic image cup disk dividing method based on generation confrontation mechanism |
| WO2019034546A1 (en) * | 2017-08-17 | 2019-02-21 | Koninklijke Philips N.V. | Ultrasound system with extraction of image planes from volume data using touch interaction with an image |
| US20190244357A1 (en) * | 2018-02-07 | 2019-08-08 | International Business Machines Corporation | System for Segmentation of Anatomical Structures in Cardiac CTA Using Fully Convolutional Neural Networks |
-
2019
- 2019-09-05 CN CN201910835283.4A patent/CN110555836A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019034546A1 (en) * | 2017-08-17 | 2019-02-21 | Koninklijke Philips N.V. | Ultrasound system with extraction of image planes from volume data using touch interaction with an image |
| US20190244357A1 (en) * | 2018-02-07 | 2019-08-08 | International Business Machines Corporation | System for Segmentation of Anatomical Structures in Cardiac CTA Using Fully Convolutional Neural Networks |
| CN109166095A (en) * | 2018-07-11 | 2019-01-08 | 广东技术师范学院 | A kind of ophthalmoscopic image cup disk dividing method based on generation confrontation mechanism |
| CN109034268A (en) * | 2018-08-20 | 2018-12-18 | 北京林业大学 | A kind of red turpentine beetle detector optimization method of Information element trapper |
Non-Patent Citations (6)
| Title |
|---|
| 余良凯: "基于深度学习的机场场面目标检测", 《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑(月刊)》 * |
| 刘国华: "HALCON数字图像处理", 《西安电子科技大学出版社》 * |
| 张明月: "考虑产品特征的个性化推荐及应用", 《企业管理出版社》 * |
| 熊欣: "人脸识别技术与应用", 《黄河水利出版社》 * |
| 胡鹏辉: "胎儿超声图像中标准切面自动定位方法研究", 《中国优秀博硕士学位论文全文数据库(硕士)医药卫生科技辑(月刊)》 * |
| 裴明涛等: "视频事件的分析与理解", 《北京理工大学出版社》 * |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110974299A (en) * | 2019-12-31 | 2020-04-10 | 上海杏脉信息科技有限公司 | Ultrasonic scanning robot system, ultrasonic scanning method and medium |
| CN111223092A (en) * | 2020-02-28 | 2020-06-02 | 长沙大端信息科技有限公司 | Automatic quality control system and detection method for ultrasonic sectional images of fetus |
| CN111462059A (en) * | 2020-03-24 | 2020-07-28 | 湖南大学 | Parallel processing method and device for intelligent target detection of fetal ultrasound image |
| CN111462059B (en) * | 2020-03-24 | 2023-09-29 | 湖南大学 | Parallel processing method and device for intelligent target detection of fetal ultrasonic image |
| CN111915562A (en) * | 2020-07-02 | 2020-11-10 | 杭州深睿博联科技有限公司 | Deep learning children echocardiogram standard tangent plane identification method and device |
| CN112102230A (en) * | 2020-07-24 | 2020-12-18 | 湖南大学 | Ultrasonic tangent plane identification method, system, computer equipment and storage medium |
| CN111932513A (en) * | 2020-08-07 | 2020-11-13 | 深圳市妇幼保健院 | Method and system for imaging three-dimensional image of fetal sulcus gyrus in ultrasonic image |
| WO2022062458A1 (en) * | 2020-09-24 | 2022-03-31 | 广州爱孕记信息科技有限公司 | Method and apparatus for determining optimal fetal standard view |
| CN112155602A (en) * | 2020-09-24 | 2021-01-01 | 广州爱孕记信息科技有限公司 | Method and device for determining optimal standard section of fetus |
| CN113688874A (en) * | 2021-07-29 | 2021-11-23 | 天津中科智能识别产业技术研究院有限公司 | Method and system for automatically segmenting iris region in human eye iris image |
| CN113688874B (en) * | 2021-07-29 | 2024-05-31 | 天津中科智能识别产业技术研究院有限公司 | Automatic iris region segmentation method and system in human eye iris image |
| CN113487614B (en) * | 2021-09-08 | 2021-11-30 | 四川大学 | Training method and device for fetus ultrasonic standard section image recognition network model |
| CN113487614A (en) * | 2021-09-08 | 2021-10-08 | 四川大学 | Training method and device for fetus ultrasonic standard section image recognition network model |
| CN115482190A (en) * | 2021-11-10 | 2022-12-16 | 中山大学附属第七医院(深圳) | Fetal heart structure segmentation measurement method and device and computer storage medium |
| CN114334095A (en) * | 2021-12-31 | 2022-04-12 | 深圳度影医疗科技有限公司 | Intelligent identification method and system for ultrasonic examination and terminal equipment |
| CN114782407A (en) * | 2022-05-23 | 2022-07-22 | 中山大学附属第一医院 | Fetal ultrasonic key section identification method and device based on convolutional neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110555836A (en) | Automatic identification method and system for standard fetal section in ultrasonic image | |
| CN112101451B (en) | Breast cancer tissue pathological type classification method based on generation of antagonism network screening image block | |
| CN107644420B (en) | Blood vessel image segmentation method based on centerline extraction and nuclear magnetic resonance imaging system | |
| CN112116605B (en) | Pancreas CT image segmentation method based on integrated depth convolution neural network | |
| CN106056595B (en) | Based on the pernicious assistant diagnosis system of depth convolutional neural networks automatic identification Benign Thyroid Nodules | |
| CN110232383A (en) | A kind of lesion image recognition methods and lesion image identifying system based on deep learning model | |
| CN111488921B (en) | Intelligent analysis system and method for panoramic digital pathological image | |
| Omonigho et al. | Breast cancer: tumor detection in mammogram images using modified alexnet deep convolution neural network | |
| CN107247971B (en) | Intelligent analysis method and system for ultrasonic thyroid nodule risk index | |
| CN108257135A (en) | The assistant diagnosis system of medical image features is understood based on deep learning method | |
| CN111291825B (en) | Focus classification model training method, apparatus, computer device and storage medium | |
| CN108364294A (en) | Abdominal CT images multiple organ dividing method based on super-pixel | |
| CN107766874B (en) | Measuring method and measuring system for ultrasonic volume biological parameters | |
| CN105640577A (en) | Method and system automatically detecting local lesion in radiographic image | |
| CN112381164B (en) | Ultrasound image classification method and device based on multi-branch attention mechanism | |
| CN112614133B (en) | Three-dimensional pulmonary nodule detection model training method and device without anchor point frame | |
| CN110910377B (en) | Cerebral infarction MRI image identification method based on neural network | |
| CN112819821B (en) | Cell nucleus image detection method | |
| CN109215040B (en) | Breast tumor segmentation method based on multi-scale weighted learning | |
| CN111524144A (en) | Intelligent pulmonary nodule diagnosis method based on GAN and Unet network | |
| CN112508953B (en) | Meningioma rapid segmentation qualitative method based on deep neural network | |
| CN112581450B (en) | Pollen detection method based on expansion convolution pyramid and multi-scale pyramid | |
| CN111027590A (en) | Breast cancer data classification method combining deep network features and machine learning model | |
| CN112365973A (en) | Pulmonary nodule auxiliary diagnosis system based on countermeasure network and fast R-CNN | |
| CN112102230A (en) | Ultrasonic tangent plane identification method, system, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200514 Address after: Room 515a266, block BCD, Lugu business center, 199 Lulong Road, Changsha hi tech Development Zone, Hunan 410205 Applicant after: Changsha Datang Information Technology Co.,Ltd. Address before: Yuelu District City, Hunan province 410082 Changsha Lushan Lushan Road South Gate Applicant before: Li Kenli |
|
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20211125 Address after: No.1023-1063, shatai South Road, Guangzhou, Guangdong 510515 Applicant after: SOUTHERN MEDICAL University Applicant after: Hunan University Address before: 410205 room 515a266, block BCD, Lugu business center, No. 199 Lulong Road, high tech Development Zone, Changsha, Hunan Applicant before: Changsha Datang Information Technology Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191210 |
|
| RJ01 | Rejection of invention patent application after publication |