CN116051826A - Coronary vessel segmentation method based on DSA image continuous frame sequence, computer equipment and readable storage medium - Google Patents
Coronary vessel segmentation method based on DSA image continuous frame sequence, computer equipment and readable storage medium Download PDFInfo
- Publication number
- CN116051826A CN116051826A CN202211476674.XA CN202211476674A CN116051826A CN 116051826 A CN116051826 A CN 116051826A CN 202211476674 A CN202211476674 A CN 202211476674A CN 116051826 A CN116051826 A CN 116051826A
- Authority
- CN
- China
- Prior art keywords
- continuous frame
- data set
- coronary
- feature fusion
- deep learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30101—Blood vessel; Artery; Vein; Vascular
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/03—Recognition of patterns in medical or anatomical images
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The application relates to a coronary vessel segmentation method, computer equipment and readable storage medium based on a DSA image continuous frame sequence, wherein the method comprises the following steps: obtaining a training data set, wherein the training data set comprises an input data set and an output data set, the input data set is a first continuous frame with the sequence length of N in a coronary DSA image continuous frame sequence, and the output data set is an intermediate frame which is marked with a target blood vessel in advance and is positioned in the intermediate position of the first continuous frame; training a deep learning network model by using the training data set: the deep learning network model comprises a feature fusion device, an encoder and a decoder, wherein the feature fusion device performs feature fusion on a first continuous frame to obtain a three-dimensional image, and the three-dimensional image sequentially passes through the encoder and the decoder, and the output of the decoder is connected with an intermediate frame; and continuously training until a trained deep learning network model is obtained, detecting an input coronary DSA image continuous frame sequence by using the trained deep learning network model, and outputting a segmentation prediction graph of the target blood vessel.
Description
Technical Field
The application relates to the technical field of coronary artery physiology, in particular to a coronary artery segmentation method based on a DSA image continuous frame sequence, computer equipment and a readable storage medium.
Background
Coronary heart disease, collectively referred to as coronary atherosclerotic heart disease, sometimes referred to as ischemic heart disease, refers to heart disease caused by myocardial ischemia and hypoxia due to coronary atherosclerosis. Coronary arteries are the only blood vessels that supply the heart's blood and are shaped like crowns, and are therefore called coronary arteries. Coronary blood vessels can also be hardened as other blood vessels of the whole body, and are changed in atherosclerosis, so that the blood circulation of the heart is impaired, and myocardial ischemia and hypoxia are caused, namely coronary heart disease. Coronary heart disease is a common disease of middle-aged and elderly people, frequently occurring, and seriously endangers lives of people.
Conventional means for diagnosing coronary heart disease include simple non-invasive electrocardiography, coronary CTA that can only acquire still images, invasive means including coronary intravascular ultrasound (IVUS) and dynamic coronary angiography. In these several ways, coronary angiography is considered as the "gold standard" for coronary heart disease diagnosis. As a major imaging technique for diagnosing coronary artery disease, the morphology of the coronary arteries is obtained by real-time visualization of the catheter lumen during coronary angiography, while Quantitative Coronary Angiography (QCA) can also be used to provide objective morphological quantitative measurements.
Since the coronary DSA is a projection of a three-dimensional coronary artery on a two-dimensional plane, a projection error is easily generated in coronary angiography (QCA), and blood vessels overlapped with each other need to be deeply understood about the structure of a coronary artery tree and considerable training to be accurately identified, and although a computer-aided tool such as an edge detection method is used, manual correction is necessary for accurate segmentation of the coronary artery. Although new image processing methods have been proposed to automatically detect the entire vessel region, the processing time required to apply multiple filters is not practical nor does it take into account the identification of a particular vessel. Recently, a deep learning model was introduced for DSA segmentation. However, the deep learning method of the main vessel segmentation has not yet reached the prediction accuracy of clinical application.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a coronary vessel segmentation method based on a DSA image continuous frame sequence.
The coronary vessel segmentation method based on the DSA image continuous frame sequence comprises the following steps:
the method comprises the steps of obtaining a training data set, wherein the training data set comprises an input data set and an output data set, the input data set is a first continuous frame with a sequence length of N in a coronary DSA image continuous frame sequence, and the output data set is an intermediate frame which is marked with a target blood vessel in advance and is positioned in the middle of the first continuous frame;
training a deep learning network model by using the training data set: the deep learning network model comprises a feature fusion device, an encoder and a decoder, wherein the feature fusion device carries out feature fusion on the first continuous frame to obtain a three-dimensional image, the three-dimensional image sequentially passes through the encoder and the decoder, and the output of the decoder is connected with the intermediate frame;
and continuously training until a trained deep learning network model is obtained, detecting an input coronary DSA image continuous frame sequence by using the trained deep learning network model, and outputting a segmentation prediction graph of the target blood vessel.
Optionally, the feature fusion device includes a first feature fusion device and a second feature fusion device, and the feature fusion device performs feature fusion on the first continuous frame to obtain a three-dimensional image, which specifically includes:
dividing the first continuous frame into (N-M+1) second continuous frames with the sequence length of M according to the time sequence;
the first feature fusion device performs feature fusion on each second continuous frame to obtain (N-M+1) fusion frames;
and the second feature fusion device performs feature fusion on each fusion frame to obtain the three-dimensional image.
Optionally, N is an odd number and M is an odd number less than N.
Optionally, the first feature fusion device selects a bidirectional long-short-time memory network, and the second feature fusion device selects a depth residual error neural network;
the value of N is five, and the value of M is three.
Optionally, the encoder includes a shallow feature extractor in a relatively shallow layer and a deep feature extractor in a relatively deep layer, wherein the shallow feature extractor uses a depth residual neural network, and the deep feature extractor uses a transducer model.
Optionally, the three-dimensional image is downsampled by the encoder and upsampled by the decoder sequentially, and the encoder sequentially includes, from a shallow layer to a deep layer:
the first feature extractor selects a depth residual neural network;
the second feature extractor selects a depth residual neural network;
the third feature extractor selects a transducer model;
and the fourth feature extractor adopts a transducer model.
Optionally, the target blood vessel is any one of the following: anterior descending branch, circumflex branch, right coronary artery, posterior descending branch belonging to right coronary artery, and posterolateral artery belonging to right coronary artery;
the segmentation prediction graph is a binary graph of the target blood vessel.
Optionally, detecting the continuous frame sequence of the input coronary DSA image by using the trained deep learning network model, and outputting a segmentation prediction graph of the target blood vessel, which specifically comprises:
obtaining a test data set, wherein the test data set is a first continuous frame with the sequence length of N in a coronary DSA image continuous frame sequence;
and detecting the first continuous frame by using the trained deep learning network model, and outputting a segmentation prediction graph of the intermediate frame at the intermediate position of the first continuous frame.
The present application also provides a computer device comprising a memory, a processor and a computer program stored on the memory, the processor executing the computer program to implement the steps of the DSA image continuous frame sequence based coronary vessel segmentation method described herein.
The present application also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the DSA image continuous frame sequence based coronary vessel segmentation method described herein.
The coronary vessel segmentation method based on the DSA image continuous frame sequence has at least the following effects:
the method inputs a plurality of continuous image frames in the DSA image, namely a first continuous frame, to a deep learning model for training and use, and outputs information of an intermediate frame during use. Compared with the technical scheme that an input single Zhang Ying frame is used during semantic segmentation model training, the method and the device solve the problem that the segmentation result difference between frames is large due to the fact that information between image frames is not considered, avoid affecting the application of the segmentation result in a subsequent process, improve the segmentation consistency of continuous image frames, and can adapt to clinical requirements.
Drawings
FIG. 1 is a flowchart of a method for segmenting coronary vessels based on a DSA image continuous frame sequence according to an embodiment of the present application;
FIG. 2 is a flow chart of a method for segmenting coronary vessels based on a DSA image continuous frame sequence in an embodiment of the present application;
FIG. 3 is a schematic diagram of a deep learning network model according to an embodiment of the present application;
FIG. 4 is a schematic diagram of an original image of a DSA image according to an embodiment of the present disclosure;
FIG. 5 is a DSA image of a pre-labeled target vessel provided in an embodiment of the present application;
FIG. 6 is a schematic diagram illustrating verification of output results of a deep learning network model according to an embodiment of the present application;
FIG. 7 is an intermediate frame of a first continuous frame input by a trained deep learning network model in an embodiment of the present application;
FIG. 8 is a segmentation prediction graph for the output of FIG. 7;
fig. 9 is an internal structural diagram of a computer device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
Referring to fig. 1 and 2, in an embodiment of the present application, a coronary vessel segmentation method based on a DSA image continuous frame sequence is provided, which includes steps S100 to S300. Wherein:
step S100, a training data set is obtained, wherein the training data set comprises an input data set and an output data set, the input data set is a first continuous frame with the sequence length of N in a coronary DSA image continuous frame sequence, and the output data set is an intermediate frame which is marked with a target blood vessel in advance and is positioned in the middle of the first continuous frame.
The first consecutive frame means several consecutive image frames in time sequence, and the sequence length of N means that there are N consecutive image frames. The target blood vessel is any one of the following: anterior descending LAD, circumflex LCX, right coronary RCA, posterior descending RPDA belonging to right coronary artery, and posterolateral artery RPLV belonging to right coronary artery.
Step S200, training the deep learning network model by using the training data set: the deep learning network model comprises a feature fusion device, an encoder and a decoder, wherein the feature fusion device performs feature fusion on the first continuous frame to obtain a three-dimensional image, the three-dimensional image sequentially passes through the encoder and the decoder, and the output of the decoder is connected with an intermediate frame.
Step S300, training is carried out continuously until a trained deep learning network model is obtained, the trained deep learning network model is utilized to detect an input coronary DSA image continuous frame sequence, and a segmentation prediction graph of a target blood vessel is output.
The segmentation prediction map is specifically a binary map of the target blood vessel. The output of the decoder is connected with the intermediate frame when training is carried out, and the decoder outputs a segmentation prediction graph of the intermediate frame after training is finished.
In the prior art, a mode of inputting a single image frame to train and using a deep learning network model exists, and in the mode, if the image quality of the single image is low, the binary image output by the trained deep learning network model is finally distorted. In this embodiment, a plurality of continuous image frames in the DSA image, that is, the first continuous frame, is input to the deep learning model, compared with the technical scheme of using the input single Zhang Ying frame in model training, the problem that the difference of the segmentation results between frames is large due to the fact that information between the image frames is not considered is improved, the application of the segmentation results in the subsequent flow is prevented from being influenced, the segmentation consistency of the continuous image frames is improved, and the clinical requirement can be met. The coronary vessel segmentation method based on the DSA image continuous frame sequence in the embodiment is an automatic image segmentation algorithm which is constructed, and can efficiently and accurately complete coronary vessel segmentation aiming at the coronary DSA image.
It should be understood that, although the steps in the flowchart of fig. 1 are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in fig. 1 may include multiple sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, nor do the order in which the sub-steps or stages are performed necessarily performed in sequence, but may be performed alternately or alternately with at least a portion of other steps or sub-steps of other steps.
In step S100, a DSA image is first acquired, and then a DICOM image is labeled, that is, a target blood vessel is labeled in advance (the acquired coronary DSA image and the data label in fig. 2) to obtain an output data set.
Acquiring DSA images includes: the DICOM image is generated by imaging through a femoral or radial catheter using a standard catheter and digitally recording the coronary angiography. The personal patient information in the DICOM format image is desensitized by using an anonymization tool, and the personal information of the DICOM image is removed.
The manner of pre-labeling (labeling DICOM images) is as follows: a main vessel region on the angiography is labeled by a clinician expert group to generate a label mask. Specifically, on the end-diastole image frame and the image frame of the filling of the main vessel region in the first 5 frames of the frame, an initial mask of the main vessel boundary is generated using a semi-automatic edge detection tool, and then manually corrected. For each main vessel, the segmented region is arranged from the opening to the far end, and the target vessel to be marked has a descending anterior branch LAD, a circumflex branch LCX and a right coronary artery RCA. For the right coronary RCA, the distal end of the segmental region is distinguished from the bifurcation point between the posterior descending branch (PDA) and the posterolateral artery (PL), and the continued vessel ends are labeled separately, and distinguished with RPDA and RPLV labels, respectively. Pixel information is captured and extracted over the boundary using custom labeling software and a label mask is created separately.
In step S100, the feature fusion device performs feature fusion on the first continuous frame to obtain a three-dimensional image, which can be understood as channel reconstruction of the input image frame image (image frame image channel reconstruction in fig. 2). Taking N as 5, for example, the size of the input data (DICOM image) is n×5×512×512, where N is the batch size of one training data and 5 is the channel number. Thus requiring channel reconstruction of the original input image frame image. The channel reconstruction uses a pre-marked (marking the DICOM image) image frame image as an intermediate frame, and takes m-2, m-1, m+1 and m+2 as data of the other four channels again if the frame number of the intermediate frame is m, so that a first continuous frame with the sequence length of N is integrally formed, and the first continuous frames are sequentially stacked from left to right according to the sequence. The method comprises the steps that 5 frames of continuous gray images are adopted before channel reconstruction, and 5×512×512 numpy array data in nii format are adopted after channel reconstruction, namely, a feature fusion device performs feature fusion on a first continuous frame to obtain a three-dimensional image.
In step S100, for the detailed process of channel reconstruction, the feature fusion device includes a first feature fusion device and a second feature fusion device, and the feature fusion device performs feature fusion on the first continuous frame to obtain a three-dimensional image, which specifically includes steps S110 to S130. Wherein:
step S110, dividing the first continuous frame into (N-M+1) second continuous frames with the sequence length of M according to the time sequence;
step S120, the first feature fusion device performs feature fusion on each second continuous frame to obtain (N-M+1) fusion frames;
and step S130, the second feature fusion device performs feature fusion on each fusion frame to obtain a three-dimensional image.
In step S100, N is an odd number, and M is an odd number smaller than N. The value of N may be five, for example, and the value of M may be three, for example. The first feature fusion device selects a bidirectional long-short-time memory network BiLSTM, and the second feature fusion device selects a depth residual error neural network ResNet50.
Referring to fig. 3, specifically, a first continuous frame having a sequence length of 5 is input, the apirt is divided into 3 second continuous frames having a sequence length of 3, and the number of frames of the intermediate frame is m. The 3 second consecutive frames are the first three frames (m-2, m-1, m), the middle three frames (m-1, m, m+1), and the last three frames (m, m+1, m+2), respectively. In the process of feature fusion, semantic information between continuous frames is extracted through a first feature fusion device bidirectional long-short-time memory network BiLSTM, and (N-M+1) fusion frames, such as 3 fusion frames, corresponding to (N-M+1) second continuous frames respectively are obtained. And the second feature fusion device depth residual neural network ResNet50 performs stack, performs feature fusion to obtain a three-dimensional image, and is used for inputting an encoder.
In combination with the above, the first continuous frame is adopted as input, and the continuous information of different frames in the first continuous frame is utilized to optimize the input and output training process of the model, so that the distortion of the output binary image caused by taking a single image frame as an input set is avoided. However, if the continuous information of the first continuous frame is too much, image information with poor continuity (poor continuity compared with the intermediate frame) is introduced in the input and output process of the model, and finally, the output single image is distorted.
In the steps S110 to S130 of this embodiment, not only the technical advantage of taking the first continuous frame as input is obtained, but also the problem of poorer continuity compared with the prior art is avoided by the division of the first continuous frame and the step fusion of the first feature fusion device and the second feature fusion device.
In step S200, the encoder includes a shallow feature extractor in a relatively shallow layer and a deep feature extractor in a relatively deep layer, wherein the shallow feature extractor uses a depth residual neural network, and the deep feature extractor uses a transducer model.
In the step, the data in the relative deep layer is larger, the data in the relative shallow layer is smaller, the shallow layer feature extractor selects a depth residual neural network, the deep layer feature extractor selects a transducer model, and meanwhile, the precision of the transducer model and the efficiency of the depth residual neural network are both considered.
In step S200, the three-dimensional image is downsampled sequentially by an encoder and upsampled sequentially by a decoder, and the encoder sequentially includes, from a shallow layer to a deep layer: the method comprises the steps of selecting a first feature extractor of a depth residual neural network, selecting a second feature extractor of the depth residual neural network, selecting a third feature extractor of a transducer model, and selecting a fourth feature extractor of the transducer model.
Referring to fig. 3, the deep learning network model trained and used by the embodiments of the present application is a BiLSTM (bi-directional long and short term memory network) -Res (deep residual neural network Res net) -Trans (transducer model) -UNet semantic segmentation network model.
The BiLSTM-Res-Trans-UNet semantic segmentation network model modifies the coding strategy of the Encoder Encoder module to carry out model innovation and result optimization on the basis of the U-Net semantic segmentation architecture, builds a binary semantic segmentation model with a coding and decoding structure, and completes the automatic segmentation of the coronary DSA image, so that the precision and the segmentation consistency among frames meet the clinical requirements.
The model takes a three-dimensional image output by a feature fusion device as an original input, and the encoder stage finishes downsampling through 3×3 convolution with the step length of 2 for 4 times, so as to achieve the purpose of deep extraction of picture features (a first feature extractor at a first layer, a second feature extractor at a second layer, a third feature extractor at a third layer and a fourth feature extractor at a fourth layer from a relatively shallow layer to a relatively deep layer in sequence).
The decoder stage selects deconvolution operation to complete up-sampling, gradually restores the feature map size to 512×512, and finally outputs 1×512×512 divided prediction map (binary map of target blood vessel). The encoder adds long-range jump connection with a transducer structure (transducer model) in the last two layers (third layer and fourth layer) of downsampling, extracts deep semantic features, and realizes feature fusion in the same stage in the subsequent upsampling stage of the decoder. And the feature map detail is restored, and the global information is further extracted, so that the aim of improving the overall performance of the model is fulfilled.
The BiLSTM-Res-Trans-UNet semantic segmentation network model is improved on the basis of a traditional UNet framework and is described in detail below. The BiLSTM-Res-Trans-UNet model includes an encoder, a decoder, a feature fusion engine (BiLSTM that extracts semantic information between frames), a long-range jump connection with a transform structure, and a ResNet feature extractor with a jump connection. The model of choice for the depth residual neural network in embodiments may be ResNet50.
The encoder includes a total of 4 coding blocks and 4 downsampling operations. Each coding block consists of two 3×3 convolution blocks, wherein each convolution block comprises a 3×3 convolution layer, and a BN batch normalization layer and a ReLU activation layer are added after the convolution layers; downsampling selects a convolution implementation with a step size of 2.
The decoder includes a total of 4 decoding blocks and 4 upsampling operations. Each decoding block consists of two 3×3 convolution blocks, wherein each convolution block comprises a 3×3 convolution layer, and a BN batch normalization layer and a ReLU activation layer are added after the convolution layers; up-sampling selects a deconvolution implementation with a step size of 2.
The long-range jump connection comprises 2 paths (a third layer and a fourth layer), and a transducer structure with the same repetition number is inserted into each path to serve as a feature extractor of deep semantics. The hop connection also contains 2 paths (first and second layers), each with ResNet inserted as a feature extractor for low-level semantics. The transducer takes feature codes and position codes as integral input and sequentially passes through two core algorithm blocks. The first algorithm block consists of a multi-head attention mechanism and a fusion normalization layer, and the second algorithm block consists of a multi-layer perceptron and a fusion normalization layer, and short-range jump connection is introduced between the input and the output of the module.
Step S300 specifically includes steps S310 to S320. Wherein:
step S310, a test data set is obtained, wherein the test data set is a first continuous frame with the sequence length of N in a coronary DSA image continuous frame sequence;
step S320, detecting the first continuous frame by using the trained deep learning network model, and outputting a segmentation prediction graph of the intermediate frame at the intermediate position of the first continuous frame.
In step S300, the trained deep learning network model is used to automatically segment the first continuous frame of the DSA image, the blood vessel region is marked with white, and the obtained accurate consistent segmentation result can be used for subsequent three-dimensional reconstruction so as to facilitate the analysis of blood flow dynamics.
The DSA coronary region extraction method based on the image continuous frame sequence in the embodiments of the application has the following advantages: (1) When the deep learning network model is used, the segmentation prediction graph of the output target blood vessel is highly automated, and the addition of the starting point position and the modification of the boundary position are not needed to be performed by human intervention. (2) The segmentation result consistency of the deep learning network model is higher than that of a common deep learning algorithm by adopting a first continuous frame input method. (3) The accuracy of the transducer model and the efficiency of the depth residual neural network resnet can be considered, the diagnosis speed of a patient can be improved, and the method has high clinical application value.
In one embodiment, a computer device is provided, which may be a terminal, and the internal structure thereof may be as shown in fig. 9. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program, when executed by a processor, implements a method for coronary vessel segmentation based on a sequence of consecutive frames of DSA images. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
In one embodiment, a computer device is provided comprising a memory and a processor, the memory having stored therein a computer program, the processor when executing the computer program performing the steps of:
step S100, a training data set is obtained, wherein the training data set comprises an input data set and an output data set, the input data set is a first continuous frame with the sequence length of N in a coronary DSA image continuous frame sequence, and the output data set is an intermediate frame which is marked with a target blood vessel in advance and is positioned in the intermediate position of the first continuous frame;
step S200, training the deep learning network model by using the training data set: the deep learning network model comprises a feature fusion device, an encoder and a decoder, wherein the feature fusion device performs feature fusion on a first continuous frame to obtain a three-dimensional image, and the three-dimensional image sequentially passes through the encoder and the decoder, and the output of the decoder is connected with an intermediate frame;
step S300, training is carried out continuously until a trained deep learning network model is obtained, the trained deep learning network model is utilized to detect an input coronary DSA image continuous frame sequence, and a segmentation prediction graph of a target blood vessel is output.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:
step S100, a training data set is obtained, wherein the training data set comprises an input data set and an output data set, the input data set is a first continuous frame with the sequence length of N in a coronary DSA image continuous frame sequence, and the output data set is an intermediate frame which is marked with a target blood vessel in advance and is positioned in the intermediate position of the first continuous frame;
step S200, training the deep learning network model by using the training data set: the deep learning network model comprises a feature fusion device, an encoder and a decoder, wherein the feature fusion device performs feature fusion on a first continuous frame to obtain a three-dimensional image, and the three-dimensional image sequentially passes through the encoder and the decoder, and the output of the decoder is connected with an intermediate frame;
step S300, training is carried out continuously until a trained deep learning network model is obtained, the trained deep learning network model is utilized to detect an input coronary DSA image continuous frame sequence, and a segmentation prediction graph of a target blood vessel is output.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the various embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description. When technical features of different embodiments are embodied in the same drawing, the drawing can be regarded as a combination of the embodiments concerned also being disclosed at the same time.
The above examples merely represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the invention. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.
Claims (10)
1. The coronary vessel segmentation method based on the DSA image continuous frame sequence is characterized by comprising the following steps of:
the method comprises the steps of obtaining a training data set, wherein the training data set comprises an input data set and an output data set, the input data set is a first continuous frame with a sequence length of N in a coronary DSA image continuous frame sequence, and the output data set is an intermediate frame which is marked with a target blood vessel in advance and is positioned in the middle of the first continuous frame;
training a deep learning network model by using the training data set: the deep learning network model comprises a feature fusion device, an encoder and a decoder, wherein the feature fusion device carries out feature fusion on the first continuous frame to obtain a three-dimensional image, the three-dimensional image sequentially passes through the encoder and the decoder, and the output of the decoder is connected with the intermediate frame;
and continuously training until a trained deep learning network model is obtained, detecting an input coronary DSA image continuous frame sequence by using the trained deep learning network model, and outputting a segmentation prediction graph of the target blood vessel.
2. The method for segmenting coronary vessels based on a DSA image continuous frame sequence according to claim 1, wherein the feature fusion device comprises a first feature fusion device and a second feature fusion device, and the feature fusion device performs feature fusion on the first continuous frame to obtain a three-dimensional image, and specifically comprises:
dividing the first continuous frame into (N-M+1) second continuous frames with the sequence length of M according to the time sequence;
the first feature fusion device performs feature fusion on each second continuous frame to obtain (N-M+1) fusion frames;
and the second feature fusion device performs feature fusion on each fusion frame to obtain the three-dimensional image.
3. The method of claim 2, wherein N is an odd number and M is an odd number less than N.
4. The method for segmenting coronary vessels based on a DSA image continuous frame sequence according to claim 3 wherein the first feature fusion device is a bidirectional long-short-time memory network, and the second feature fusion device is a depth residual neural network;
the value of N is five, and the value of M is three.
5. The method of claim 1, wherein the encoder comprises a shallow feature extractor in a relatively shallow layer and a deep feature extractor in a relatively deep layer, the shallow feature extractor selecting a deep residual neural network, and the deep feature extractor selecting a transducer model.
6. The method for segmenting coronary vessels based on a continuous frame sequence of DSA images according to claim 5 wherein the three-dimensional image is downsampled sequentially by the encoder and upsampled sequentially by the decoder, the encoder comprising, in order from shallow to deep:
the first feature extractor selects a depth residual neural network;
the second feature extractor selects a depth residual neural network;
the third feature extractor selects a transducer model;
and the fourth feature extractor adopts a transducer model.
7. The method of claim 1, wherein the target vessel is any one of the following: anterior descending branch, circumflex branch, right coronary artery, posterior descending branch belonging to right coronary artery, and posterolateral artery belonging to right coronary artery;
the segmentation prediction graph is a binary graph of the target blood vessel.
8. The method for segmenting coronary vessels based on DSA image continuous frame sequences according to any one of claims 1 to 7, wherein the training-completed deep learning network model is used for detecting the input coronary DSA image continuous frame sequences and outputting a segmentation prediction map of the target vessels, and specifically comprises the following steps:
obtaining a test data set, wherein the test data set is a first continuous frame with the sequence length of N in a coronary DSA image continuous frame sequence;
and detecting the first continuous frame by using the trained deep learning network model, and outputting a segmentation prediction graph of the intermediate frame at the intermediate position of the first continuous frame.
9. Computer device comprising a memory, a processor and a computer program stored on the memory, characterized in that the processor executes the computer program to carry out the steps of the DSA image continuous frame sequence based coronary vessel segmentation method according to any one of claims 1 to 7.
10. A computer readable storage medium having stored thereon a computer program, which when executed by a processor performs the steps of the DSA image continuous frame sequence based coronary vessel segmentation method according to any one of claims 1 to 7.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211476674.XA CN116051826A (en) | 2022-11-23 | 2022-11-23 | Coronary vessel segmentation method based on DSA image continuous frame sequence, computer equipment and readable storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211476674.XA CN116051826A (en) | 2022-11-23 | 2022-11-23 | Coronary vessel segmentation method based on DSA image continuous frame sequence, computer equipment and readable storage medium |
Publications (1)
Publication Number | Publication Date |
---|---|
CN116051826A true CN116051826A (en) | 2023-05-02 |
Family
ID=86130252
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202211476674.XA Pending CN116051826A (en) | 2022-11-23 | 2022-11-23 | Coronary vessel segmentation method based on DSA image continuous frame sequence, computer equipment and readable storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN116051826A (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116703948A (en) * | 2023-08-03 | 2023-09-05 | 杭州脉流科技有限公司 | Intracranial vessel tree segmentation method and device based on deep neural network |
CN116912851A (en) * | 2023-07-25 | 2023-10-20 | 京东方科技集团股份有限公司 | Image processing method, device, electronic equipment and readable storage medium |
CN117974654A (en) * | 2024-03-29 | 2024-05-03 | 杭州脉流科技有限公司 | Coronary image segmentation method, device, computer equipment and storage medium |
CN117974720A (en) * | 2024-03-29 | 2024-05-03 | 杭州脉流科技有限公司 | Guide wire real-time tracking method, computer equipment and readable storage medium |
-
2022
- 2022-11-23 CN CN202211476674.XA patent/CN116051826A/en active Pending
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116912851A (en) * | 2023-07-25 | 2023-10-20 | 京东方科技集团股份有限公司 | Image processing method, device, electronic equipment and readable storage medium |
CN116703948A (en) * | 2023-08-03 | 2023-09-05 | 杭州脉流科技有限公司 | Intracranial vessel tree segmentation method and device based on deep neural network |
CN116703948B (en) * | 2023-08-03 | 2023-11-14 | 杭州脉流科技有限公司 | Intracranial vessel tree segmentation method and device based on deep neural network |
CN117974654A (en) * | 2024-03-29 | 2024-05-03 | 杭州脉流科技有限公司 | Coronary image segmentation method, device, computer equipment and storage medium |
CN117974720A (en) * | 2024-03-29 | 2024-05-03 | 杭州脉流科技有限公司 | Guide wire real-time tracking method, computer equipment and readable storage medium |
CN117974654B (en) * | 2024-03-29 | 2024-06-11 | 杭州脉流科技有限公司 | Coronary image segmentation method, device, computer equipment and storage medium |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110934606B (en) | Cerebral apoplexy early-stage flat-scan CT image evaluation system and method and readable storage medium | |
CN116051826A (en) | Coronary vessel segmentation method based on DSA image continuous frame sequence, computer equipment and readable storage medium | |
CN111161290B (en) | Image segmentation model construction method, image segmentation method and image segmentation system | |
CN111667478B (en) | Method and system for identifying carotid plaque through CTA-MRA cross-modal prediction | |
CN110934608B (en) | Cerebral apoplexy early-stage CTA image evaluation system and method and readable storage medium | |
CN112435212A (en) | Brain focus region volume obtaining method and device based on deep learning, computer equipment and storage medium | |
CN111584066B (en) | Brain medical image diagnosis method based on convolutional neural network and symmetric information | |
KR20190136577A (en) | Method for classifying images using deep neural network and apparatus using the same | |
CN116503607B (en) | CT image segmentation method and system based on deep learning | |
CN115546570A (en) | Blood vessel image segmentation method and system based on three-dimensional depth network | |
CN113935943A (en) | Method, device, computer equipment and storage medium for intracranial aneurysm identification detection | |
CN115409859A (en) | Coronary artery blood vessel image segmentation method and device, storage medium and terminal | |
CN113706451B (en) | Method, apparatus, system and computer readable storage medium for intracranial aneurysm identification detection | |
CN116681716B (en) | Method, device, equipment and storage medium for dividing intracranial vascular region of interest | |
CN115861612A (en) | Method for automatically segmenting coronary vessels based on DSA (digital Signal amplification) images | |
CN112950555A (en) | Deep learning-based type 2 diabetes cardiovascular disease image classification method | |
CN116612090A (en) | System and method for determining pulmonary artery embolism index | |
CN116109872A (en) | Blood vessel naming method and device, electronic equipment and storage medium | |
CN115294023A (en) | Liver tumor automatic segmentation method and device | |
CN115115657A (en) | Focus segmentation method and device, electronic device and storage medium | |
CN117078711A (en) | Medical image segmentation method, system, electronic device and storage medium | |
CN117974654B (en) | Coronary image segmentation method, device, computer equipment and storage medium | |
WO2022096867A1 (en) | Image processing of intravascular ultrasound images | |
CN114119688B (en) | Depth learning-based coronary angiography front-back single-mode medical image registration method | |
CN116649995B (en) | Method and device for acquiring hemodynamic parameters based on intracranial medical image |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |