Summary of the invention
The present invention provides a kind of, and method for distinguishing is divided and known to the heart coronary artery based on deep learning, solves heart
The segmentation and identification problem of coronarogram picture.It can be divided according to the technical solution of the present invention by higher accuracy rate
And identify the heart coronary artery in contrastographic picture, lesion, which is analyzed, for doctor provides auxiliary material at the same time as blood vessel three-dimensional
The basis of reconstruction.
In order to achieve the above object, a kind of heart coronary artery segmentation and identification based on deep learning provided by the invention
Method, including:
It chooses any one frame picture in segmentation heart radiography Dicom video and training sample is inputted into mind as training sample
Through in network;The neural network is made of convolutional neural networks module, pyramid model and warp lamination;
Convolutional neural networks module in neural network receives the training sample, by the method for deep learning to training
Picture in sample carries out the segmentation and identification of blood vessel, and the cardiovascular characteristic pattern for being used to divide and identify is exported to pyramid
Module;
Pyramid module receives the cardiovascular characteristic pattern for dividing and identifying, defeated using the method that pyramid merges
The cardiovascular characteristic pattern of different scale is to warp lamination out;
Warp lamination receives the cardiovascular characteristic pattern of different scale, obtains heart coronaries by the method for bilinear interpolation
Artery segmentation and identification vessel graph.
Further, the acquisition methods of the segmentation heart radiography Dicom video include:
Receive the whole section of heart radiography Dicom video corresponding with lesion information stored in medical integrated database;
Based on lesion information, the key feature occurred in whole section of heart radiography Dicom video of SSN Cooperative Analysis is used
Information.
Based on key feature information and position information is combined, whole section of Dicom video is segmented, and the iteration step,
Meet the segmenting video of setting until eventually finding.
Further, the convolutional neural networks module is repeatedly stacked by multilayer same unit, and the unit is from upper
Convolutional layer, batch standardization layer, quick articulamentum, activation primitive layer are followed successively by under.
Further, the convolutional layer receives training sample, to the block of pixels of each fixed size in training sample data
2D convolution algorithm is carried out, extracts the characteristic pattern for dividing and identifying contained in training sample data, and characteristic pattern is exported
To batch standardization layer.
Further, described batch of standardization layer receives the characteristic pattern of convolutional layer output, subtract to feature diagram data
It is worth the operation divided by variance, so that feature diagram data univesral distribution, and batch standardization characteristic pattern is output to quick articulamentum.
Further, the quick articulamentum receives batch output of standardization layer, by the input of convolutional layer and batch standardization
The output of layer is added to obtain characteristic pattern by weight, and characteristic pattern is output to activation primitive layer.
Further, the activation primitive receives the output of quick articulamentum, carries out non-linear place to the data received
Reason carries out relu operation to these characteristic patterns;By the convolutional layer of treated data input next unit;Until nerve net
All feature extraction layers of convolutional network have been calculated in all units in network structure, obtain the cardiovascular for dividing and identifying
Characteristic pattern, and the cardiovascular characteristic pattern for being used to divide and identify is inputted into pyramid module.
Further, the pyramid module receives the cardiovascular characteristic pattern for dividing and identifying, using pyramid
The method of fusion first carries out convolution operation to characteristic pattern, exports the cardiovascular characteristic pattern of different scale;By the heart of different scale
Dirty blood vessel characteristic pattern inputs warp lamination;
Warp lamination receives the cardiovascular characteristic pattern of different scale, by the method for bilinear interpolation by different scale
Cardiovascular characteristic pattern is amplified to same size, is finally merged together along a dimension, obtains heart coronary artery segmentation
And identification vessel graph.
Further, further include the steps that parameter updates, which includes:
Compare output heart coronary artery segmentation and identification vessel graph and doctor's essence mark heart coronary artery segmentation and
The difference of identification vessel graph obtains penalty values, and the parameter by gradient descent method to each layer of neural network is updated;Iteration
All steps are run, are lower than until by penalty values between neural network segmentation and the vessel graph identified and doctor's essence mark
Preset threshold value.
It further, further include testing procedure, which includes:
Step 1:The heart radiography Dicom video file of the patient taken is read, key frame is extracted, inputs nerve net
Network;And read the corresponding model parameter of the position.
Step 2:Neural network is initialized, and multilayer neural network structure is established, and reads trained corresponding position
Model parameter.
Step 3:Neural network receives the heart radiography Dicom video image of patient, by the method for deep learning to defeated
Enter segmentation and detection that picture carries out blood vessel, exports the blood vessel segmentation and identification picture of different position key frames;
Step 4:Step 1 is repeated to step 3 to different positions, is disposed until by the key frame of all positions.
A kind of heart coronary artery segmentation and knowledge method for distinguishing based on deep learning provided by the invention, passes through to choose and divide
Any one frame picture inputs training sample in neural network as training sample in section heart radiography Dicom video;The mind
It is made of through network convolutional neural networks module, pyramid model and warp lamination;Convolutional neural networks mould in neural network
Block receives the training sample, carries out the segmentation and identification of blood vessel to the picture in training sample by the method for deep learning,
The cardiovascular characteristic pattern for being used to divide and identify is exported to pyramid module;Pyramid module is received for dividing and identifying
Cardiovascular characteristic pattern export the cardiovascular characteristic pattern of different scale to warp lamination using the method that pyramid merges;
Warp lamination receives the cardiovascular characteristic pattern of different scale, obtains heart coronary artery segmentation by the method for bilinear interpolation
And the technical solution of identification vessel graph, the image Segmentation Technology based on deep learning is applied in coronary artery segmentation.It can be with
It is automatically performed the segmentation and identification mission of heart contrastographic picture end to end.Divide positioning heart radiography figure with higher accuracy rate
Coronary artery as in.It can label to each pixel in picture, to identify the type of different blood vessel in picture.It answers
With the method for deep learning, on the one hand eliminates and classify not caused by being differed greatly due to background pixel and blood vessel pixel ratio
On the other hand equalization problem also effectively avoids in image background the interference that the texture of similar vessel-like introduces, improve point
Cut accuracy.
Embodiment one
Referring to Fig.1, Fig. 1 is shown, a kind of heart coronary artery segmentation and knowledge based on deep learning provided by the invention
Method for distinguishing, including step S110- step S140:
In step s 110, choosing any one frame picture in segmentation heart radiography Dicom video will instruct as training sample
Practice in sample input neural network;
The neural network is made of convolutional neural networks module, pyramid model and warp lamination.
In the step s 120, the convolutional neural networks module in neural network receives the training sample, by depth
The method of habit carries out the segmentation and identification of blood vessel, the cardiovascular feature that will be used to divide and identify to the picture in training sample
Figure is exported to pyramid module.
In step s 130, the pyramid module in neural network receives the cardiovascular feature for dividing and identifying
Figure exports the cardiovascular characteristic pattern of different scale to warp lamination using the method that pyramid merges.
In step S140, the warp lamination in neural network receives the cardiovascular characteristic pattern of different scale, by double
The method of linear interpolation obtains heart coronary artery segmentation and identification vessel graph.
Segmentation heart radiography Dicom video acquisition methods include:
Receive the whole section of heart radiography Dicom video corresponding with lesion information stored in medical integrated database;
Based on lesion information, the key feature information occurred in whole section of heart radiography Dicom video of SSN Cooperative Analysis is used;Base
In key feature information and position information is combined, whole section of Dicom video is segmented, and the iteration step, until finally looking for
To the segmenting video for meeting setting.The relatively clear Digital Subtraction heart radiography of vessel profile is intercepted out from dicom file
Image, and picture is handled as single pass gray scale picture, input convolutional neural networks.
Wherein, heart radiography Dicom sets of video data is by the Dicom coronary digital of about 100 patients with coronary heart disease
Subtractive angiography (Digital imaging in medicine communication) file composition.Every patient has multiple Dicom files of different positions, often
A Dicom file all includes the coronarography of several frames, and each frame has different types of blood vessel, including Left main artery, a left side
Circumflex branch, left anterior descending branch, collateral, left branches of interventricular septum, right hat etc..It needs to divide in the present invention and the blood vessel identified is exactly these blood
Pipe.For each frame image in video, doctor can carry out fine Pixel-level mark to the blood vessel in figure.Using these this
A little data train network model, carry out the segmentation and identification of blood vessel using the model trained later.
The convolutional neural networks module is repeatedly stacked by multilayer same unit, and the unit is followed successively by from top to bottom
Convolutional layer, batch standardization layer, quick articulamentum, activation primitive layer.
Further, the convolutional layer receives training sample, to the block of pixels of each fixed size in training sample data
2D convolution algorithm is carried out, extracts the characteristic pattern for dividing and identifying contained in training sample data, and characteristic pattern is exported
To batch standardization layer.
High-dimensional feature in picture is extracted to reach by carrying out convolution algorithm again and again to input picture,
Exactly these features contain all information used in segmentation and identification process.
Further, described batch of standardization layer receives the characteristic pattern of convolutional layer output, subtract to feature diagram data
It is worth the operation divided by variance, so that feature diagram data univesral distribution, and batch standardization characteristic pattern is output to quick articulamentum.
The feature diagram data of univesral distribution as shown in Figure 2, under normal circumstances, training neural network generally require to arrive for three days
One week time, along with experiment results are done, time cost often needs an important factor for considering.Criticize standardization layer just
It is that can be substantially reduced a kind of method of time cost with acceleration model training speed.Batch standardization is by Identity Planization to same
The lower convergence rate for accelerating network of suitable distribution, the first step of concrete operations is standardized to the feature of input, will be defeated
The feature entered subtracts after its mean value divided by its variance, and detailed process is represented by:
WhereinFeature after indicating standardization, x indicate the feature of input, E [x(k)] indicate input feature vector mean value,Indicate the variance of input feature vector.
During extracting cardiovascular feature, time cost is also reduced.So after convolutional layer, to volume
The image of the heart coronaries radiography of lamination output does one batch of standardization processing.One is carried out to the feature of convolutional layer output to subtract
Mean value while needing to do each layer of mean value and variance one storage divided by the operation of variance so as to during the test can be with
It directly uses, the heart contrastographic picture after convolution can be made to have unified data distribution, in this way so as to accelerate to mention
Take the task of blood vessel feature.
Second step is to carry out Pan and Zoom to the feature of standardization, it is therefore an objective to network oneself be allowed to learn to be suitble to the defeated of network
Out, detailed process is represented by:
Wherein γ(k)For the scaling parameter that can learn, β(k)For the translation parameters that can learn.
Further, the quick articulamentum receives batch output of standardization layer, by the input of convolutional layer and batch standardization
The output of layer is added to obtain characteristic pattern by weight, and characteristic pattern is output to activation primitive layer.If entire neural network be exactly by
Dry quick articulamentum, which is connected, to be formed.
The number of plies of neural network is deeper, and the dimension for the feature that can be acquired is also higher, therefore the number of plies is to neural network
Have very big influence.But when the number of plies of neural network becomes deeper and deeper, deeper model is difficult table
Up to the feature of low dimensional, so the problems such as just will appear gradient explosion, gradient disappearance.Quick connection unit is to solve this to ask
The method of topic.Remember H (X)=F (X)+X, in extreme circumstances F (X) there is nothing study arrive, i.e. F (X)=0, at this time H (X)=
X.This ensures that shallow-layer feature is transmitted backward, and the feature that whole network learns will not be too poor, and quick connection unit is used
In the extracting the feature extraction of heart coronary artery contrastographic picture of the task.Model oneself is allowed to determine that it wants to extract characteristic dimension
Just, accomplish the cardiovascular feature for retaining useful low dimensional as far as possible.It is asked to solve gradient explosion with what gradient disappeared
Topic.
Entire quick connection procedure can be expressed as:
Y=F (x, { wi})+x
Wherein y indicates the feature of output, and x indicates the feature of input, and F (X, { Wi }) indicates to need the residual error of training to map letter
Number, Wi indicate the weight of this layer.
Further, the activation primitive receives the output of quick articulamentum, carries out non-linear place to the data received
Reason carries out relu operation to these characteristic patterns;By the convolutional layer of treated data input next unit;Until nerve net
All feature extraction layers of convolutional network have been calculated in all units in network structure, obtain the cardiovascular for dividing and identifying
Characteristic pattern, and the cardiovascular characteristic pattern for being used to divide and identify is inputted into pyramid module.
If it is simple these linear convolutions are connected to the network, then final effect is only single with one
Convolution unit is the same.So needing to introduce activation primitive layer in actual use, the image that such as figure is activation primitive, tool
Body process is represented by:
Y=G (X)
Wherein y is output feature, and x is input feature vector, and G is activation primitive.
During the test, also the heart coronary artery blood-vessel image that process of convolution is crossed subtract mean value divided by variance
Operation guarantees that test is consistent with the distribution of training process cardiac coronary artery characteristics of image.
Further, the pyramid module receives the cardiovascular characteristic pattern for dividing and identifying, using pyramid
The method of fusion first carries out convolution operation to characteristic pattern, exports the cardiovascular characteristic pattern of different scale;By the heart of different scale
Dirty blood vessel characteristic pattern inputs warp lamination.
Pyramid module has merged the feature of 4 kinds of different scales of the heart coronary artery image extracted.I.e. by four kinds
The fusion of different size of heart features, if figure the first row red is most coarse heart coronary artery characteristics of image, behind three rows
It is the heart coronary artery image pond feature of different scale.In order to guarantee the weight of global characteristics, if pyramid share it is N number of
Rank then will be reduced to rank channel using the convolution of 1x1 after each rank the 1/N of script.Pass through bilinear interpolation again
Size before obtaining non-pond, is finally merged together along a dimension.
Warp lamination receives the cardiovascular characteristic pattern of different scale, by the method for bilinear interpolation by different scale
Cardiovascular characteristic pattern is amplified to same size, is finally merged together along a dimension, obtains heart coronary artery segmentation
And identification vessel graph.
Further, further include the steps that parameter updates, which includes:
Compare output heart coronary artery segmentation and identification vessel graph and doctor's essence mark heart coronary artery segmentation and
The difference of identification vessel graph obtains penalty values, and the parameter by gradient descent method to each layer of neural network is updated;Iteration
All steps are run, are lower than until by penalty values between neural network segmentation and the vessel graph identified and doctor's essence mark
Preset threshold value.
It further, further include testing procedure, which includes:
Step 1:The heart radiography Dicom video file of the patient taken is read, key frame is extracted, inputs nerve net
Network;And read the corresponding model parameter of the position.
Step 2:Neural network is initialized, and multilayer neural network structure is established, and reads trained corresponding position
Model parameter.
Step 3:Neural network receives the heart radiography Dicom video image of patient, by the method for deep learning to defeated
Enter segmentation and detection that picture carries out blood vessel, exports the blood vessel segmentation and identification picture of different position key frames;
Step 4:Step 1 is repeated to step 3 to different positions, is disposed until by the key frame of all positions.
One preferred embodiment, laboratory hardware:Intel Xeon CPU E5-2630 v4 CPU and NVIDIA GTX
1080 Ti GPU carry out Collaborative Control.
One, reading data
Step 1:The whole section of heart corresponding with the lesion information for receiving to store from medical integrated database is made
Shadow Dicom video.
Step 2:Based on disease information, the key occurred in whole section of heart radiography Dicom video of SSN Cooperative Analysis is used
Characteristic information.
Step 3:Based on key feature information and position information is combined, whole section of Dicom video is segmented, and iteration
The step meets the segmenting video of setting until eventually finding.
Step 4:Any one frame in selecting video segmentation is inputted in neural network module as training sample.
Two, training network is split and detects to blood vessel
Step 1:Neural network is initialized, and multilayer neural network structure is established, repeatedly stacked by similar units and
At, be followed successively by from top to bottom in a unit convolutional layer, batch standardization layer, quick articulamentum, activation primitive layer.It reads simultaneously pre-
Training pattern parameter.
Step 2:Neural network receives digital subtraction angiography image, by the method for deep learning to input picture
Carry out the segmentation and detection of blood vessel.
Step 3:Convolutional layer receives digital subtraction angiography image, to the pixel of each fixed size in data
Block carries out 2D convolution algorithm, extracts the main information that can be used for dividing and identifying contained in data, and these information are defeated
Out to batch standardization layer.
Step 4:The characteristic pattern that standardization layer receives convolutional layer output is criticized, subtract mean value divided by variance to data
Operation, so that data be allowed to have a unified distribution, and is output to quick articulamentum for processed characteristic pattern.
Step 5:Quick articulamentum receives batch output of standardization layer, and inputting for convolutional layer is defeated with batch standardization layer
It is added to obtain characteristic pattern by weight out, and is output to activation primitive layer.
Step 6:Pair activation primitive receives the output of quick articulamentum, carries out Nonlinear Processing to the data that receive, i.e.,
These characteristic patterns carry out relu operation.By the convolutional layer of treated data input next unit.
Step 7:All feature extraction layers of three to six steps until convolutional network has been calculated are repeated, final spy is obtained
Sign figure.All main informations for blood vessel segmentation and identification are needed here it is us.By these information input pyramid modules.
Step 8:Pyramid module receives these cardiovascular characteristic patterns for dividing and identifying, melts using pyramid
The method of conjunction first carries out convolution operation to characteristic pattern, exports the cardiovascular characteristic pattern of four kinds of different scales.By four kinds of different rulers
The cardiovascular characteristic pattern of degree inputs warp lamination.
Step 9:Warp lamination receives the cardiovascular characteristic pattern of four kinds of different scales, passes through the method for bilinear interpolation
The cardiovascular characteristic pattern of four kinds of different scales is amplified to same size, is finally merged together along a dimension.This is just
Last segmentation identification vessel graph is obtained.
Step 10:The difference of the segmentation identification vessel graph and doctor's essence mark picture that more finally export obtains penalty values,
The parameter by gradient descent method to each layer of neural network is updated later.
Step 11:Iteration operating procedure two to step 10 until divided by neural network and the vessel graph that identifies with
Penalty values are lower than preset threshold value between doctor's essence mark.
Step 12:Model parameter after training and Artificial Neural Network Structures are stored, so as in later test process
It uses.
Step 13:Train the model parameter of different position data and storage.
Test network is split and detects to blood vessel
Step 1:The Dicom file of the patient taken is read, key frame is extracted, inputs neural network.And read the body
The corresponding model parameter in position.
Step 2:Neural network is initialized, and multilayer neural network structure, and trained correspondence before reading are established
The model parameter of position.
Step 3:Neural network receives digital subtraction angiography image, by the method for deep learning to input picture
The segmentation and detection for carrying out blood vessel export the blood vessel segmentation and identification picture of different position key frames.
Step 4:Above-mentioned one to three step is repeated to different positions, is disposed until by the key frame of all positions.
A kind of heart coronary artery segmentation and knowledge method for distinguishing based on deep learning provided by the invention, passes through to choose and divide
Any one frame picture inputs training sample in neural network as training sample in section heart radiography Dicom video;The mind
It is made of through network convolutional neural networks module, pyramid model and warp lamination;Convolutional neural networks mould in neural network
Block receives the training sample, carries out the segmentation and identification of blood vessel to the picture in training sample by the method for deep learning,
The cardiovascular characteristic pattern for being used to divide and identify is exported to pyramid module;Pyramid module is received for dividing and identifying
Cardiovascular characteristic pattern export the cardiovascular characteristic pattern of different scale to warp lamination using the method that pyramid merges;
Warp lamination receives the cardiovascular characteristic pattern of different scale, obtains heart coronary artery segmentation by the method for bilinear interpolation
And the technical solution of identification vessel graph, the image Segmentation Technology based on deep learning is applied in coronary artery segmentation.It can be with
It is automatically performed the segmentation and identification mission of heart contrastographic picture end to end.Divide positioning heart radiography figure with higher accuracy rate
Coronary artery as in.It can label to each pixel in picture, to identify the type of different blood vessel in picture.It answers
With the method for deep learning, on the one hand eliminates and classify not caused by being differed greatly due to background pixel and blood vessel pixel ratio
On the other hand equalization problem also effectively avoids in image background the interference that the texture of similar vessel-like introduces, improve point
Cut accuracy.
The serial number of the above embodiments of the invention is only for description, does not represent the advantages or disadvantages of the embodiments.
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can easily think of the change or the replacement, and should all contain
Lid is within protection scope of the present invention.Therefore, protection scope of the present invention should be based on the protection scope of the described claims.