WO2022257408A1 - Medical image segmentation method based on u-shaped network - Google Patents

Medical image segmentation method based on u-shaped network Download PDF

Info

Publication number
WO2022257408A1
WO2022257408A1 PCT/CN2021/138825 CN2021138825W WO2022257408A1 WO 2022257408 A1 WO2022257408 A1 WO 2022257408A1 CN 2021138825 W CN2021138825 W CN 2021138825W WO 2022257408 A1 WO2022257408 A1 WO 2022257408A1
Authority
WO
WIPO (PCT)
Prior art keywords
segmentation
network
constant scaling
weight
medical image
Prior art date
Application number
PCT/CN2021/138825
Other languages
French (fr)
Chinese (zh)
Inventor
张登银
严伟丹
赵蓉
朱虹
杨硕
杜群荐
孙俊杰
Original Assignee
南京邮电大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 南京邮电大学 filed Critical 南京邮电大学
Priority to US17/697,884 priority Critical patent/US20220398737A1/en
Publication of WO2022257408A1 publication Critical patent/WO2022257408A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30088—Skin; Dermal
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Definitions

  • the invention relates to a U-shaped network-based medical image segmentation method, which belongs to the technical field of image processing.
  • image segmentation methods can be divided into traditional image segmentation methods and image segmentation based on deep learning.
  • Traditional image segmentation methods mainly use digital image processing, mathematics, physics, optics, topological maps and other knowledge for image processing, including image processing based on Threshold segmentation methods, region-based segmentation methods, boundary-based segmentation methods, wavelet-based analysis and transformation, and genetic algorithms.
  • Traditional image segmentation methods mainly use edge and algorithm characteristics for segmentation, and it is easy to ignore the deep semantic category information of the image.
  • deep learning has made great progress in the field of computer vision, some of which are gradually approaching human judgment. Therefore, image segmentation using deep learning methods has gradually become the mainstream. This method has no prior knowledge restrictions and can be obtained when the network is well trained. better effect.
  • U-Net is improved based on FCN (fully convolutional network), including encoder, bottleneck (bottleneck) module, and decoder. Due to its U-shaped structure combined with context information and fast training speed, the amount of data used is small, it meets The appeal of medical image segmentation is widely used in medical image segmentation. Since the publication of U-Net, its encoder-decoder-jump network structure has inspired a large number of improved medical image segmentation methods based on the U-Net structure.
  • the purpose of the present invention is to overcome the deficiencies in the prior art, to provide a medical image segmentation method based on a U-shaped network, to replace the original codec module with a constant scaling residual network and a cyclic neural network, and to improve the U-shaped network.
  • the possible semantic gap between the shallow and deep network connections and the loss of shallow information during the maximum pooling downsampling process can fully capture target areas of different sizes, improve the segmentation accuracy of different scales, and make it More image feature information can be obtained while training a deeper network.
  • the invention provides a method for segmenting medical images based on a U-shaped network, comprising the following steps:
  • Step 1 Obtain a medical image dataset
  • Step 2 Obtain a pair of original pictures and the real segmentation map of the target area in the original picture from the medical image data set, and use them together as the input data set of the pre-built constant scaling segmentation network, wherein the input data set is divided into are the training set, validation set and test set;
  • Step 3 use the training set to train the constant scaling segmentation network to obtain a trained segmentation network model, and use the verification set to verify the constant scaling segmentation network, wherein the constant scaling segmentation network includes a feature extraction module and A resolution upscaling module, where each decoder layer is concatenated with the corresponding cropped feature maps from the same layer from the encoder during decoding;
  • Step 4 Input the original picture to be segmented into the segmentation network model for segmentation to obtain a real segmented image.
  • the ratio of the training set, verification set and test set in the input data set is 6:2:2.
  • the feature extraction module includes 5 first constant scaling modules and 4 downsampling modules, the constant scaling modules are connected through the downsampling module, and the resolution increasing module includes 4 There are four upsampling modules and four second constant scaling modules, and the second constant scaling modules are connected through upsampling modules.
  • the constant scaling module is composed of a constant scaling residual network structure and a cyclic neural network, and the output of the constant scaling residual network structure is composed of two parts: the input feature map is multiplied by the weight a ; and the input feature map is multiplied by weight b after two weight layers; weight a and weight b should satisfy the following relationship:
  • the cyclic neural network enters the convolution layer from the input feature map, and then performs a cyclic operation, so that each convolution operation can obtain the feature information obtained by the previous convolution operation, and finally obtains the output through the ReLu activation function.
  • the cyclic neural network is used to replace the weight layer in the constant scaling residual network structure of the constant scaling model, thereby forming a constant scaling module, the output of which is composed of the addition of two parts: the input feature map is multiplied by the weight a; And the input feature map is multiplied by the weight b after two circular convolution blocks including the convolution block and the ReLU activation function; the weight a and the weight b should satisfy the formula (1).
  • the loss function is set as a set similarity measurement function, and the specific formula is:
  • represents the common elements between set A and set B
  • represents the number of elements in A
  • represents the number of elements in B
  • the elements in set A are input
  • the data set is segmented by the constant zoom segmentation network to obtain the real segmented image
  • the elements in set B are the real segmented image of the target area in the original image;
  • step 3 when the loss function is minimized, the training is stopped, and the trained segmentation network model is obtained, including the following steps:
  • the weight parameters of the constant scaling segmentation network are initialized at each stage, and the weight parameters are randomly initialized using a Gaussian distribution with an average value of 0;
  • the sample picture includes the synthetic image and the original picture, first use the forward propagation to find the total error between the real segmentation image and the real segmentation map of the target area in the original picture, and then Use backpropagation to find the partial derivatives of each weight parameter, and finally update the weight parameters according to the gradient descent method;
  • the present invention makes up for the loss of the traditional U-shaped network’s ability to capture detailed information and the incomplete capture of deep and shallow information features, and improves the understanding of different Adaptability to segmentation tasks improves the final segmentation results.
  • the constant scaling residual network proposed by the present invention uses weight a and weight b of different numerical combinations before outputting feature information.
  • Weight a controls the feature information of the original input image
  • weight b controls the semantic information extracted by the weight layer.
  • different combination methods can be selected, and the proportion of shallow feature information transmitted to the next layer can be flexibly improved, which enhances the usability and sensitivity of the model algorithm.
  • the constant scaling segmentation network used in the present invention fuses the constant scaling residual network and the cyclic neural network, wherein the residual network uses a skip structure to associate shallow spatial features with deep semantics through weight values, and the cyclic neural network further Mining the deep semantic information of the input image improves the semantic gap caused by the direct connection in the traditional U-shaped network, enhances the extraction of detailed information, and improves the fusion effect of feature maps at different levels.
  • Fig. 1 is the flow chart of the method provided by the embodiment of the present invention.
  • FIG. 2 is a schematic diagram of a constant zoom segmentation network structure provided by an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of a constant scaling residual network structure provided by an embodiment of the present invention.
  • FIG. 4 is a schematic structural diagram of a recurrent neural network provided by an embodiment of the present invention.
  • Fig. 5 is a schematic diagram of a constant scaling module provided by an embodiment of the present invention.
  • the present invention provides a kind of medical image segmentation method based on U-shaped network, comprises the following steps:
  • Step 1 Obtain a medical image dataset
  • Step 2 Obtain a pair of original pictures and the real segmentation map of the target area in the original picture from the medical image data set, and use them together as the input data set of the pre-built constant scaling segmentation network, wherein the input data set is divided into are training set, verification set and test set; the ratio of training set, verification set and test set in the input data set is 6:2:2;
  • Step 3 use the training set to train the constant scaling segmentation network to obtain a trained segmentation network model, and use the verification set to verify the constant scaling segmentation network, wherein the constant scaling segmentation network includes a feature extraction module and The resolution upscaling module, each decoder layer is concatenated with the corresponding cropped feature maps from the same layer of the encoder during decoding;
  • the feature extraction module includes 5 first constant scaling modules, 4 downsampling modules, the The constant scaling modules are connected through a downsampling module, and the resolution increasing module includes 4 upsampling modules and 4 second constant scaling modules, and the second constant scaling modules are connected through an upsampling module;
  • the constant scaling module is composed of a constant scaling residual network structure and a cyclic neural network, and the output of the constant scaling residual network structure is composed of two parts: the input feature map is multiplied by the weight a; and the input The feature map is multiplied by weight b after two weight layers; weight a and weight b should satisfy the following relationship:
  • the cyclic neural network enters the convolution layer from the input feature map, and then performs a cyclic operation, so that each convolution operation can obtain the feature information obtained by the previous convolution operation, and finally the output is obtained through the ReLu activation function.
  • the recurrent neural network is used to replace the weight layer in the constant scaling residual network structure of the constant scaling model, thereby forming a constant scaling module whose output consists of the addition of two parts: the input feature map multiplied by the weight a; and the input feature
  • the graph is multiplied by weight b after two circular convolution blocks containing convolution blocks and ReLU activation functions; weight a and weight b should satisfy formula (1).
  • the loss function is set as a set similarity measurement function, and the specific formula is:
  • represents the common elements between set A and set B
  • represents the number of elements in A
  • represents the number of elements in B
  • the elements in set A are input data Set-to-constant scaling segmentation network segmentation to obtain the real segmented image
  • the elements in set B are the real segmented image of the target area in the original image
  • the training is stopped, and the trained segmentation network model is obtained, including the following steps:
  • the weight parameters of the constant scaling segmentation network are initialized at each stage, and the weight parameters are randomly initialized using a Gaussian distribution with an average value of 0;
  • the sample picture includes the synthetic image and the original picture, first use the forward propagation to find the total error between the real segmentation image and the real segmentation map of the target area in the original picture, and then Use backpropagation to find the partial derivatives of each weight parameter, and finally update the weight parameters according to the gradient descent method;
  • Step 4 Input the original picture to be segmented into the segmentation network model for segmentation to obtain a real segmented image.
  • the constant-scaling segmentation network used in the present invention fuses the constant-scaling residual network and the cyclic neural network, wherein the residual network uses a jump structure to associate shallow spatial features with deep semantics through weight values, and the cyclic neural network further mines the input image
  • the deep semantic information improves the semantic gap caused by the direct connection in the traditional U-shaped network, enhances the extraction of detailed information, and improves the fusion effect of feature maps at different levels.
  • Step 1 Obtain a medical image dataset.
  • the medical image dataset is the 2018 ISIC challenge: a skin disease dataset for melanoma detection;
  • the medical image dataset is downloaded and called from the existing medical image database.
  • Step 2 Take out the paired melanoma skin pathogenic map and its real segmentation label from the skin disease data set for melanoma detection, and divide the data set into training set, verification set and test set according to the ratio of 6:2:2.
  • Step 3 Use the input dataset from step 2 to train a constant scaling segmentation network including feature extraction and increasing resolution, with each decoder layer being the same layer from the encoder during decoding
  • the test set and verification set in step 2 are input into the constant scaling segmentation network of the present invention (as shown in Figure 2), and the feature extraction includes 5 constant scaling modules, 4 downsampling, constant scaling Downsampling connections are used between modules; increasing resolution includes 4 upsampling and 4 constant scaling modules, upsampling is used before the constant scaling module, and the output of the constant scaling module (as shown in Figure 5) consists of two parts added : (1) The input feature map is multiplied by the weight a; (2) The input feature map is multiplied by the weight b after two circular convolution blocks containing the convolution block and the ReLU activation function,
  • the pixel coordinates of the k-th feature map of the circular convolution layer in x l are (i, j), and at time t, the circular convolution layer outputs It can be expressed as:
  • Step 4 The specific implementation is as follows, setting the segmentation network loss function with constant scaling.
  • the Loss function is set to the dice coefficient commonly used in medicine, and the specific formula is
  • represents the common elements between set A and set B
  • represents the number of elements in A
  • represents the number of elements in B
  • the elements in set A are real
  • the input data set is segmented by the multi-feature fusion segmentation network to obtain the real segmented image
  • the elements in set B are the real segmented image of the target area in the original image
  • Step 5 train segmentation network
  • step 5 use the Adam optimizer to initialize the weight parameters of the network at each stage, and use a Gaussian distribution with an average value of 0 to randomly initialize the weight parameters;
  • Step 6 Input the melanotic skin disease image to be segmented into the segmentation network of the present invention to obtain the segmented melanotic skin disease image.
  • the present invention improves the original U-shaped network structure to improve the loss of shallow detail information during the downsampling process, and further improves The fusion of deep and shallow semantics reduces the semantic gap and improves the segmentation accuracy of medical image foreground and background.
  • different weight combinations can be selected for application. For multiple scenarios, this The method proposed by the invention has good usability.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

A medical image segmentation method based on a U-shaped network, which method belongs to the technical field of image processing. The method comprises the following steps: first, according to a certain proportion, dividing original images and actual segmented images of a medical image data set into a training set, a validation set and a test set; and sending the training set and the validation set to an improved constant-scaling segmentation network for training. In the method, an encoder and a decoder in a traditional U-shaped network are improved, and a constant-scaling residual network and a recurrent neural network being combined to replace an original codec module is proposed, thereby enhancing the flexible fusion of shallow layer information and deep layer semantics, more deeply extracting feature information of an image, and also improving the accuracy of medical image segmentation. By means of the method, the problems of the capability of a traditional U-shaped network capturing detailed information being lost and a feature fusion proportion of deep and shallow layer information being fixed are made up for from the perspective of improving a module structure, thereby improving a final segmentation result.

Description

一种基于U型网络的医学图像分割方法A Medical Image Segmentation Method Based on U-shaped Network 技术领域technical field
本发明涉及一种基于U型网络的医学图像分割方法，属于图像处理技术领域。The invention relates to a U-shaped network-based medical image segmentation method, which belongs to the technical field of image processing.
背景技术Background technique
医学图像分割技术的发展，是从手工分割到人机式的半自动分割，再逐步发展到全自动分割的过程。手工分割指的是由具有丰富经验的临床医生在原始胶片上直接勾勒出组织的边界或者感兴趣的区域，手工分割对人的先验知识有很高的要求，且标注时间长、成本较高。随着深度学习在计算机视觉领域的发展出现了半自动分割技术，该分割技术是将计算机的数据存储和计算功能以及医学专家的经验和知识结合起来，运用人机交互的方法来完成图像的分割。全自动分割则是计算机根据事先编好的算法运行独立自动完成图像的分割全过程。但大部分全自动分割算法的实现复杂，分割结果不理想，且分割的速度和性能也需要提高，目前临床医学上，研究实用的全自动分割方法来取代繁琐的人工分割或者半自动分割一直是人们追求的目标，全自动分割方法是近年来医学图像的研究重点和关键技术。为使机器自动分割医学图像，减少人工繁琐工作，为之后的肿瘤识别和病理判断打下坚实基础，研究如何使得分割边缘结果更精确至关重要。The development of medical image segmentation technology is a process from manual segmentation to man-machine semi-automatic segmentation, and then gradually to fully automatic segmentation. Manual segmentation refers to the direct outline of tissue boundaries or regions of interest on the original film by experienced clinicians. Manual segmentation has high requirements for human prior knowledge, and the labeling time is long and the cost is high. . With the development of deep learning in the field of computer vision, semi-automatic segmentation technology has emerged. This segmentation technology combines the data storage and computing functions of computers and the experience and knowledge of medical experts, and uses human-computer interaction methods to complete image segmentation. Fully automatic segmentation means that the computer runs independently and automatically completes the whole process of image segmentation according to the pre-programmed algorithm. However, the implementation of most fully automatic segmentation algorithms is complicated, the segmentation results are not ideal, and the speed and performance of the segmentation also need to be improved. At present, in clinical medicine, research on practical automatic segmentation methods to replace cumbersome manual segmentation or semi-automatic segmentation has always been a priority. The pursuit of the goal, fully automatic segmentation method is the focus and key technology of medical image research in recent years. In order to enable the machine to automatically segment medical images, reduce manual tedious work, and lay a solid foundation for subsequent tumor recognition and pathological judgment, it is very important to study how to make the segmentation edge results more accurate.
目前图像分割方法可以分为传统的图像分割方法及基于深度学习的图像分割两种，传统的图像分割方法主要利用数字图像处理、数学、物理、光学、拓扑图等知识来进行图像处理，包括基于阈值的分割方法、基于区域的分割方法、 基于边界的分割方法、基于小波分析和变换以及基于遗传算法。传统的图像分割方法主要利用边缘及算法特性进行分割，易忽略图像深层的语义类别信息等。近年来，深度学习在计算机视觉领域取得很大进展，有些逐渐逼近人为判断，因此使用深度学习方法做图像分割逐渐成为主流，该方法没有先验知识的限制，在网络训练好的情况下可以取得较好的效果。At present, image segmentation methods can be divided into traditional image segmentation methods and image segmentation based on deep learning. Traditional image segmentation methods mainly use digital image processing, mathematics, physics, optics, topological maps and other knowledge for image processing, including image processing based on Threshold segmentation methods, region-based segmentation methods, boundary-based segmentation methods, wavelet-based analysis and transformation, and genetic algorithms. Traditional image segmentation methods mainly use edge and algorithm characteristics for segmentation, and it is easy to ignore the deep semantic category information of the image. In recent years, deep learning has made great progress in the field of computer vision, some of which are gradually approaching human judgment. Therefore, image segmentation using deep learning methods has gradually become the mainstream. This method has no prior knowledge restrictions and can be obtained when the network is well trained. better effect.
2015年，Ronneberger等人在MICCAI会议发表U-Net，是深度学习在医学影像分割中的突破性的进展。U-Net是基于FCN(fully convolutional network)改进而成，包括编码器、瓶颈(bottleneck)模块、解码器几部分组成,由于其U型结构结合上下文信息和训练速度快、使用数据量小，满足医学影像分割的诉求，而在医学影像分割中广泛应用。U-Net自发表以来，其编码器-解码器-跳连的网络结构启发了大量基于U-Net结构改进的医学影像分割方法。随着深度学习技术的发展，包括注意力机制、稠密模块、特征增强、评价函数改进等基于U-Net的基础结构，将这些深度神经网络发展的最新技术引入到医学影像分割应用中，成为被广泛采取的改进方法。这些相关工作或者面向不同的优化目标，或者通过结构改进、添加新模块等手段,提高医学影像分割的准确性、运算效率、适用范围等。In 2015, Ronneberger et al. published U-Net at the MICCAI conference, which is a breakthrough in deep learning in medical image segmentation. U-Net is improved based on FCN (fully convolutional network), including encoder, bottleneck (bottleneck) module, and decoder. Due to its U-shaped structure combined with context information and fast training speed, the amount of data used is small, it meets The appeal of medical image segmentation is widely used in medical image segmentation. Since the publication of U-Net, its encoder-decoder-jump network structure has inspired a large number of improved medical image segmentation methods based on the U-Net structure. With the development of deep learning technology, including attention mechanism, dense module, feature enhancement, evaluation function improvement and other basic structures based on U-Net, the latest technology developed by these deep neural networks is introduced into the application of medical image segmentation, and has become the most popular Widely adopted improvement methods. These related works are either aimed at different optimization goals, or through structural improvements, adding new modules, etc., to improve the accuracy, computing efficiency, and scope of application of medical image segmentation.
但在目前的医学分割领域，由于神经网络中不断的“卷积-池化”操作丢失了许多浅层的空间及细节信息，出现的梯度消失现象易导致小病灶或器官边缘的分割效果不好，在所需分割的目标(器官或病灶)大小差异较大时总体分割效果不佳，如何灵活地确定深浅层信息特征融合比例，以及提取更深层次的特征信息也是目前学者正在努力的方向，其中对U型网络的提升优化是在医学图像分割领域的研究热点。However, in the current field of medical segmentation, due to the continuous "convolution-pooling" operation in the neural network, many shallow spatial and detailed information are lost, and the phenomenon of gradient disappearance can easily lead to poor segmentation of small lesions or organ edges. , when the size of the target (organ or lesion) to be segmented varies greatly, the overall segmentation effect is not good. How to flexibly determine the fusion ratio of deep and shallow information features and extract deeper feature information is also the direction that scholars are currently working on. Among them The improvement and optimization of U-shaped network is a research hotspot in the field of medical image segmentation.
公开于该背景技术部分的信息仅仅旨在增加对本发明的总体背景的理解，而不应当被视为承认或以任何形式暗示该信息构成已为本领域普通技术人员所公知的现有技术。The information disclosed in this Background section is only for enhancement of understanding of the general background of the present invention and should not be taken as an acknowledgment or any form of suggestion that the information constitutes the prior art that is already known to those of ordinary skill in the art.
发明内容Contents of the invention
本发明的目的在于克服现有技术中的不足，提供一种基于U型网络的医学图像分割方法，将恒定缩放的残差网络与循环神经网络结合代替原有的编解码器模块，改善U型网络浅层及深层网络连接时可能产生的语义差距以及最大池化下采样过程中的浅层信息丢失问题，充分捕捉到大小不一的目标区域，提升不同尺度分割目标的分割准确度，使其在训练更深的网络的同时可以获得到更多的图像特征信息。The purpose of the present invention is to overcome the deficiencies in the prior art, to provide a medical image segmentation method based on a U-shaped network, to replace the original codec module with a constant scaling residual network and a cyclic neural network, and to improve the U-shaped network. The possible semantic gap between the shallow and deep network connections and the loss of shallow information during the maximum pooling downsampling process can fully capture target areas of different sizes, improve the segmentation accuracy of different scales, and make it More image feature information can be obtained while training a deeper network.
为达到上述目的，本发明是采用下述技术方案实现的：In order to achieve the above object, the present invention is achieved by adopting the following technical solutions:
本发明提供了一种基于U型网络的医学图像分割方法，包括以下步骤：The invention provides a method for segmenting medical images based on a U-shaped network, comprising the following steps:
步骤1：获取医学图像数据集；Step 1: Obtain a medical image dataset;
步骤2：从所述医学图像数据集中获取成对的原始图片及对原始图片中目标区域的真实分割图，一起作为预先构建的恒定缩放分割网络的输入数据集，其中，所述输入数据集分为训练集、验证集和测试集；Step 2: Obtain a pair of original pictures and the real segmentation map of the target area in the original picture from the medical image data set, and use them together as the input data set of the pre-built constant scaling segmentation network, wherein the input data set is divided into are the training set, validation set and test set;
步骤3：利用所述训练集对恒定缩放分割网络进行训练，得到训练好的分割网络模型，利用所述验证集对恒定缩放分割网络进行验证，其中，所述恒定缩放分割网络包括特征提取模块和分辨率增大模块，在解码过程中每个解码器层都与来自编码器的相同层次相应剪裁的特征图连接；Step 3: use the training set to train the constant scaling segmentation network to obtain a trained segmentation network model, and use the verification set to verify the constant scaling segmentation network, wherein the constant scaling segmentation network includes a feature extraction module and A resolution upscaling module, where each decoder layer is concatenated with the corresponding cropped feature maps from the same layer from the encoder during decoding;
步骤4：将待分割的原始图片输入到所述分割网络模型中进行分割，得到真实分割图。Step 4: Input the original picture to be segmented into the segmentation network model for segmentation to obtain a real segmented image.
进一步的，所述输入数据集中训练集、验证集和测试集的比例为6：2：2。Further, the ratio of the training set, verification set and test set in the input data set is 6:2:2.
进一步的，所述步骤3中，特征提取模块包括5个第一恒定缩放模块，4个下采样模块，所述恒定缩放模块之间通过下采样模块相连接，所述分辨率增大模块包括4个上采样模块和4个第二恒定缩放模块，所述第二恒定缩放模块之间通过上采样模块相连接。Further, in the step 3, the feature extraction module includes 5 first constant scaling modules and 4 downsampling modules, the constant scaling modules are connected through the downsampling module, and the resolution increasing module includes 4 There are four upsampling modules and four second constant scaling modules, and the second constant scaling modules are connected through upsampling modules.
进一步的，所述恒定缩放模块由恒定缩放的残差网络结构与循环神经网络结合而成，所述恒定缩放的残差网络结构的输出由两部分相加组成：输入特征图乘以权值a；以及输入特征图经过两次权重层后乘以权值b；权值a与权值b应满足如下关系：Further, the constant scaling module is composed of a constant scaling residual network structure and a cyclic neural network, and the output of the constant scaling residual network structure is composed of two parts: the input feature map is multiplied by the weight a ; and the input feature map is multiplied by weight b after two weight layers; weight a and weight b should satisfy the following relationship:
a+b＝1  (1)。a+b=1 (1).
进一步的，所述循环神经网络由输入特征图进入卷积层，后进行循环操作，使得每一次卷积操作都能获取上一次卷积操作得到的特征信息，最后经过ReLu激活函数得到输出。Further, the cyclic neural network enters the convolution layer from the input feature map, and then performs a cyclic operation, so that each convolution operation can obtain the feature information obtained by the previous convolution operation, and finally obtains the output through the ReLu activation function.
进一步的，使用所述循环神经网络替换恒定缩放模型的恒定缩放的残差网络结构中的权重层，从而形成恒定缩放模块，其输出由两部分相加组成：输入特征图乘以权值a；以及输入特征图经过两次包含卷积块以及ReLU激活函数的循环卷积块后乘以权值b；权值a与权值b应满足公式(1)。Further, the cyclic neural network is used to replace the weight layer in the constant scaling residual network structure of the constant scaling model, thereby forming a constant scaling module, the output of which is composed of the addition of two parts: the input feature map is multiplied by the weight a; And the input feature map is multiplied by the weight b after two circular convolution blocks including the convolution block and the ReLU activation function; the weight a and the weight b should satisfy the formula (1).
进一步的，所述步骤3中，在所述恒定缩放分割网络中，设置损失函数为集合相似度度量函数，具体公式为：Further, in the step 3, in the constant scaling segmentation network, the loss function is set as a set similarity measurement function, and the specific formula is:
Figure PCTCN2021138825-appb-000001
Figure PCTCN2021138825-appb-000001
其中,|A∩B|表示集合A和集合B之间的共同元素，|A|表示A|中的元素的个数，|B|表示B中的元素的个数，集合A中元素为输入数据集对恒定缩放分 割网络分割得到真实的分割图像，集合B中元素为原始图片中目标区域的真实分割图；Among them, |A∩B| represents the common elements between set A and set B, |A| represents the number of elements in A|, |B| represents the number of elements in B, and the elements in set A are input The data set is segmented by the constant zoom segmentation network to obtain the real segmented image, and the elements in set B are the real segmented image of the target area in the original image;
为了计算预测的真实分割图的集合相似度度量函数，将|A|+|B|近似为实际分割得到的图像和真实分割图像之间的点乘，并将集合A和集合B中每个像素点的值相加；当损失函数最小时停止训练，得到训练好的分割网络模型。In order to calculate the set similarity measure function of the predicted real segmentation map, |A|+|B| is approximated as the point product between the actual segmented image and the real segmented image, and each pixel in set A and set B The values of the points are added; when the loss function is minimized, the training is stopped, and the trained segmentation network model is obtained.
进一步的，所述步骤3中，当损失函数最小时停止训练，得到训练好的分割网络模型，包括以下步骤：Further, in the step 3, when the loss function is minimized, the training is stopped, and the trained segmentation network model is obtained, including the following steps:
基于Adam优化器对每一阶段恒定缩放分割网络的权重参数进行初始化，使用平均值为0的高斯分布随机初始化权重参数；Based on the Adam optimizer, the weight parameters of the constant scaling segmentation network are initialized at each stage, and the weight parameters are randomly initialized using a Gaussian distribution with an average value of 0;
对每个输入分割网络模型的训练集中的样本图片，样本图片包括合成图像和原始图片，先利用前向传播求出真实的分割图像与原始图片中目标区域的真实分割图间的总误差，再利用反向传播求出各个权重参数的偏导数，最后根据梯度下降法对权重参数进行更新；For each sample picture in the training set of the input segmentation network model, the sample picture includes the synthetic image and the original picture, first use the forward propagation to find the total error between the real segmentation image and the real segmentation map of the target area in the original picture, and then Use backpropagation to find the partial derivatives of each weight parameter, and finally update the weight parameters according to the gradient descent method;
重复上述步骤直至损失函数达到最小，得到训练好的分割网络模型。Repeat the above steps until the loss function reaches the minimum, and the trained segmentation network model is obtained.
与现有技术相比，本发明所达到的有益效果：Compared with the prior art, the beneficial effects achieved by the present invention are as follows:
第一、本发明从改进编码器与解码器的角度，结合其他具有特定功能的网络模型，弥补了传统U型网络捕捉细节信息能力丢失、深浅层信息特征捕捉不完整的问题，提升了对不同分割任务的适应性，改善了最终分割结果。First, from the perspective of improving the encoder and decoder, the present invention, combined with other network models with specific functions, makes up for the loss of the traditional U-shaped network’s ability to capture detailed information and the incomplete capture of deep and shallow information features, and improves the understanding of different Adaptability to segmentation tasks improves the final segmentation results.
第二、本发明提出的恒定缩放的残差网络，在输出特征信息前使用不同数值组合的权重a和权重b，权重a控制原始输入图像的特征信息，权重b控制权重层提取的语义信息，使其在不同的医学影像场景下，能够选择不同的组合方式，灵活改善浅层特征信息传入到下一层的比例，增强了模型算法的可用性与 灵敏性。Second, the constant scaling residual network proposed by the present invention uses weight a and weight b of different numerical combinations before outputting feature information. Weight a controls the feature information of the original input image, and weight b controls the semantic information extracted by the weight layer. In different medical imaging scenarios, different combination methods can be selected, and the proportion of shallow feature information transmitted to the next layer can be flexibly improved, which enhances the usability and sensitivity of the model algorithm.
第三、本发明所使用的恒定缩放的分割网络融合恒定缩放的残差网络与循环神经网络，其中残差网络使用跳跃结构将浅层空间特征与深层语义通过权重值关联起来，循环神经网络进一步挖掘输入图像的深层次语义信息，改善了传统U型网络中直接相连造成的语义鸿沟问题，增强了对细节信息的提取，改善了不同层级特征图的融合效果。Third, the constant scaling segmentation network used in the present invention fuses the constant scaling residual network and the cyclic neural network, wherein the residual network uses a skip structure to associate shallow spatial features with deep semantics through weight values, and the cyclic neural network further Mining the deep semantic information of the input image improves the semantic gap caused by the direct connection in the traditional U-shaped network, enhances the extraction of detailed information, and improves the fusion effect of feature maps at different levels.
附图说明Description of drawings
图1是本发明实施例提供的方法流程图；Fig. 1 is the flow chart of the method provided by the embodiment of the present invention;
图2是本发明实施例提供的恒定缩放分割网络结构的示意图；FIG. 2 is a schematic diagram of a constant zoom segmentation network structure provided by an embodiment of the present invention;
图3是本发明实施例提供的恒定缩放的残差网络结构示意图；FIG. 3 is a schematic diagram of a constant scaling residual network structure provided by an embodiment of the present invention;
图4是本发明实施例提供的循环神经网络结构示意图；FIG. 4 is a schematic structural diagram of a recurrent neural network provided by an embodiment of the present invention;
图5是本发明实施例提供的恒定缩放模块的示意图。Fig. 5 is a schematic diagram of a constant scaling module provided by an embodiment of the present invention.
具体实施方式Detailed ways
下面结合附图对本发明作进一步描述。以下实施例仅用于更加清楚地说明本发明的技术方案，而不能以此来限制本发明的保护范围。The present invention will be further described below in conjunction with the accompanying drawings. The following examples are only used to illustrate the technical solution of the present invention more clearly, but not to limit the protection scope of the present invention.
如图1至图5所示，本发明提供了一种基于U型网络的医学图像分割方法，包括以下步骤：As shown in Fig. 1 to Fig. 5, the present invention provides a kind of medical image segmentation method based on U-shaped network, comprises the following steps:
步骤1：获取医学图像数据集；Step 1: Obtain a medical image dataset;
步骤2：从所述医学图像数据集中获取成对的原始图片及对原始图片中目标区域的真实分割图，一起作为预先构建的恒定缩放分割网络的输入数据集，其中，所述输入数据集分为训练集、验证集和测试集；所述输入数据集中训练集、验证集和测试集的比例为6：2：2；Step 2: Obtain a pair of original pictures and the real segmentation map of the target area in the original picture from the medical image data set, and use them together as the input data set of the pre-built constant scaling segmentation network, wherein the input data set is divided into are training set, verification set and test set; the ratio of training set, verification set and test set in the input data set is 6:2:2;
步骤3：利用所述训练集对恒定缩放分割网络进行训练，得到训练好的分割网络模型，利用所述验证集对恒定缩放分割网络进行验证，其中，所述恒定缩放分割网络包括特征提取模块和分辨率增大模块，在解码过程中每个解码器层都与来自编码器的相同层次相应剪裁的特征图连接；特征提取模块包括5个第一恒定缩放模块，4个下采样模块，所述恒定缩放模块之间通过下采样模块相连接，所述分辨率增大模块包括4个上采样模块和4个第二恒定缩放模块，所述第二恒定缩放模块之间通过上采样模块相连接；Step 3: use the training set to train the constant scaling segmentation network to obtain a trained segmentation network model, and use the verification set to verify the constant scaling segmentation network, wherein the constant scaling segmentation network includes a feature extraction module and The resolution upscaling module, each decoder layer is concatenated with the corresponding cropped feature maps from the same layer of the encoder during decoding; the feature extraction module includes 5 first constant scaling modules, 4 downsampling modules, the The constant scaling modules are connected through a downsampling module, and the resolution increasing module includes 4 upsampling modules and 4 second constant scaling modules, and the second constant scaling modules are connected through an upsampling module;
所述恒定缩放模块由恒定缩放的残差网络结构与循环神经网络结合而成，所述恒定缩放的残差网络结构的输出由两部分相加组成：输入特征图乘以权值a；以及输入特征图经过两次权重层后乘以权值b；权值a与权值b应满足如下关系：The constant scaling module is composed of a constant scaling residual network structure and a cyclic neural network, and the output of the constant scaling residual network structure is composed of two parts: the input feature map is multiplied by the weight a; and the input The feature map is multiplied by weight b after two weight layers; weight a and weight b should satisfy the following relationship:
a+b＝1   (1)。a+b=1 (1).
所述循环神经网络由输入特征图进入卷积层，后进行循环操作，使得每一次卷积操作都能获取上一次卷积操作得到的特征信息，最后经过ReLu激活函数得到输出。The cyclic neural network enters the convolution layer from the input feature map, and then performs a cyclic operation, so that each convolution operation can obtain the feature information obtained by the previous convolution operation, and finally the output is obtained through the ReLu activation function.
使用所述循环神经网络替换恒定缩放模型的恒定缩放的残差网络结构中的权重层，从而形成恒定缩放模块，其输出由两部分相加组成：输入特征图乘以权值a；以及输入特征图经过两次包含卷积块以及ReLU激活函数的循环卷积块后乘以权值b；权值a与权值b应满足公式(1)。The recurrent neural network is used to replace the weight layer in the constant scaling residual network structure of the constant scaling model, thereby forming a constant scaling module whose output consists of the addition of two parts: the input feature map multiplied by the weight a; and the input feature The graph is multiplied by weight b after two circular convolution blocks containing convolution blocks and ReLU activation functions; weight a and weight b should satisfy formula (1).
在所述恒定缩放分割网络中，设置损失函数为集合相似度度量函数，具体公式为：In the constant scaling segmentation network, the loss function is set as a set similarity measurement function, and the specific formula is:
Figure PCTCN2021138825-appb-000002
Figure PCTCN2021138825-appb-000002
其中,|A∩B|表示集合A和集合B之间的共同元素，|A|表示A中的元素的个数，|B|表示B中的元素的个数，集合A中元素为输入数据集对恒定缩放分割网络分割得到真实的分割图像，集合B中元素为原始图片中目标区域的真实分割图；Among them, |A∩B| represents the common elements between set A and set B, |A| represents the number of elements in A, |B| represents the number of elements in B, and the elements in set A are input data Set-to-constant scaling segmentation network segmentation to obtain the real segmented image, the elements in set B are the real segmented image of the target area in the original image;
为了计算预测的真实分割图的集合相似度度量函数，将|A|+|B|近似为实际分割得到的图像和真实分割图像之间的点乘，并将集合A和集合B中每个像素点的值相加；当损失函数最小时停止训练，得到训练好的分割网络模型。In order to calculate the set similarity measure function of the predicted real segmentation map, |A|+|B| is approximated as the point product between the actual segmented image and the real segmented image, and each pixel in set A and set B The values of the points are added; when the loss function is minimized, the training is stopped, and the trained segmentation network model is obtained.
当损失函数最小时停止训练，得到训练好的分割网络模型，包括以下步骤：When the loss function is minimized, the training is stopped, and the trained segmentation network model is obtained, including the following steps:
基于Adam优化器对每一阶段恒定缩放分割网络的权重参数进行初始化，使用平均值为0的高斯分布随机初始化权重参数；Based on the Adam optimizer, the weight parameters of the constant scaling segmentation network are initialized at each stage, and the weight parameters are randomly initialized using a Gaussian distribution with an average value of 0;
对每个输入分割网络模型的训练集中的样本图片，样本图片包括合成图像和原始图片，先利用前向传播求出真实的分割图像与原始图片中目标区域的真实分割图间的总误差，再利用反向传播求出各个权重参数的偏导数，最后根据梯度下降法对权重参数进行更新；For each sample picture in the training set of the input segmentation network model, the sample picture includes the synthetic image and the original picture, first use the forward propagation to find the total error between the real segmentation image and the real segmentation map of the target area in the original picture, and then Use backpropagation to find the partial derivatives of each weight parameter, and finally update the weight parameters according to the gradient descent method;
重复上述步骤直至损失函数达到最小，得到训练好的分割网络模型；Repeat the above steps until the loss function reaches the minimum, and the trained segmentation network model is obtained;
步骤4：将待分割的原始图片输入到所述分割网络模型中进行分割，得到真实分割图。Step 4: Input the original picture to be segmented into the segmentation network model for segmentation to obtain a real segmented image.
本发明所使用的恒定缩放的分割网络融合恒定缩放的残差网络与循环神经网络，其中残差网络使用跳跃结构将浅层空间特征与深层语义通过权重值关联起来，循环神经网络进一步挖掘输入图像的深层次语义信息，改善了传统U型网络中直接相连造成的语义鸿沟问题，增强了对细节信息的提取，改善了不同层级特征图的融合效果。The constant-scaling segmentation network used in the present invention fuses the constant-scaling residual network and the cyclic neural network, wherein the residual network uses a jump structure to associate shallow spatial features with deep semantics through weight values, and the cyclic neural network further mines the input image The deep semantic information improves the semantic gap caused by the direct connection in the traditional U-shaped network, enhances the extraction of detailed information, and improves the fusion effect of feature maps at different levels.
下面结合一个优选实施例，对上述实施例中设计到的内容进行说明。The content designed in the above embodiment will be described below in conjunction with a preferred embodiment.
步骤1：获取医学图像数据集，本实施例中医学图像数据集为2018年ISIC挑战：黑色素瘤检测的皮肤病数据集；Step 1: Obtain a medical image dataset. In this embodiment, the medical image dataset is the 2018 ISIC challenge: a skin disease dataset for melanoma detection;
医学图像数据集是从现有医学图像数据库中下载调用。The medical image dataset is downloaded and called from the existing medical image database.
步骤2：从黑色素瘤检测的皮肤病数据集中取出成对的黑色素皮肤病原图及其真实的分割标签，并将其数据集按照6：2：2的比例分成训练集、验证集以及测试集。Step 2: Take out the paired melanoma skin pathogenic map and its real segmentation label from the skin disease data set for melanoma detection, and divide the data set into training set, verification set and test set according to the ratio of 6:2:2.
步骤3：利用步骤2中输入数据集对恒定缩放分割网络进行训练，所述恒定缩放分割网络包括特征提取和增大分辨率，在解码过程中每个解码器层都与来自编码器的相同层次相应剪裁的特征图连接，将步骤2中的测试集以及验证集输入本发明所述恒定缩放分割网络(如图2所示)，特征提取包括5个恒定缩放模块，4个下采样，恒定缩放模块之间采用下采样连接；增大分辨率包括4个上采样和4个恒定缩放模块，恒定缩放模块之前采用上采样，恒定缩放模块(如图5所示)的输出由两部分相加组成：(1)输入特征图乘以权值a；(2)输入特征图经过两次包含卷积块以及ReLU激活函数的循环卷积块后乘以权值b，Step 3: Use the input dataset from step 2 to train a constant scaling segmentation network including feature extraction and increasing resolution, with each decoder layer being the same layer from the encoder during decoding Correspondingly clipped feature maps are connected, and the test set and verification set in step 2 are input into the constant scaling segmentation network of the present invention (as shown in Figure 2), and the feature extraction includes 5 constant scaling modules, 4 downsampling, constant scaling Downsampling connections are used between modules; increasing resolution includes 4 upsampling and 4 constant scaling modules, upsampling is used before the constant scaling module, and the output of the constant scaling module (as shown in Figure 5) consists of two parts added : (1) The input feature map is multiplied by the weight a; (2) The input feature map is multiplied by the weight b after two circular convolution blocks containing the convolution block and the ReLU activation function,
假设x l为第l层循环卷积块的输入，x l中位于循环卷积层第k个特征图的像素点坐标为(i，j)，在t时，循环卷积层输出
Figure PCTCN2021138825-appb-000003
可以表示为： Assuming that x l is the input of the circular convolution block of the l-th layer, the pixel coordinates of the k-th feature map of the circular convolution layer in x l are (i, j), and at time t, the circular convolution layer outputs
Figure PCTCN2021138825-appb-000003
It can be expressed as:
Figure PCTCN2021138825-appb-000004
Figure PCTCN2021138825-appb-000004
其中
Figure PCTCN2021138825-appb-000005
和
Figure PCTCN2021138825-appb-000006
分别表示第l层循环卷积层的两个标准卷积层的输入，
Figure PCTCN2021138825-appb-000007
和
Figure PCTCN2021138825-appb-000008
分别表示第l层循环卷积层的两个标准卷积层第k个特征图的权重向量，b k则为偏移量。循环卷积层的输出经过标准ReLU即函数f(·)处理，得到： in
Figure PCTCN2021138825-appb-000005
and
Figure PCTCN2021138825-appb-000006
respectively represent the input of the two standard convolutional layers of the l-th circular convolutional layer,
Figure PCTCN2021138825-appb-000007
and
Figure PCTCN2021138825-appb-000008
Respectively represent the weight vectors of the k-th feature map of the two standard convolution layers of the l-th circular convolution layer, and b k is the offset. The output of the circular convolutional layer is processed by the standard ReLU function f( ) to obtain:
Figure PCTCN2021138825-appb-000009
Figure PCTCN2021138825-appb-000009
其中
Figure PCTCN2021138825-appb-000010
表示第l层循环卷积网络的输出，而第l层残卷循环网络的输出x l+1表示为： in
Figure PCTCN2021138825-appb-000010
Represents the output of the l-layer circular convolutional network, and the output x l+1 of the l-layer residual volume circular network is expressed as:
Figure PCTCN2021138825-appb-000011
Figure PCTCN2021138825-appb-000011
步骤4：具体实施如下，设置恒定缩放的分割网络损失函数。Step 4: The specific implementation is as follows, setting the segmentation network loss function with constant scaling.
分割网络部分设置Loss函数为医学中常用的dice coefficient，具体公式为In the segmented network part, the Loss function is set to the dice coefficient commonly used in medicine, and the specific formula is
Figure PCTCN2021138825-appb-000012
Figure PCTCN2021138825-appb-000012
其中,|A∩B|表示集合A和集合B之间的共同元素，|A|表示A中的元素的个数，|B|表示B中的元素的个数，集合A中元素为真实的分割图像，输入数据集对多特征融合分割网络分割得到真实的分割图像，集合B中元素为原始图片中目标区域的真实分割图；Among them, |A∩B| represents the common elements between set A and set B, |A| represents the number of elements in A, |B| represents the number of elements in B, and the elements in set A are real To segment the image, the input data set is segmented by the multi-feature fusion segmentation network to obtain the real segmented image, and the elements in set B are the real segmented image of the target area in the original image;
为了计算预测的真实分割图的集合相似度度量函数，将|A|+|B|近似为实际分割得到的图像和真实分割图像之间的点乘，并将集合A和集合B中每个像素点的值相加；当损失函数最小时停止训练，得到训练好的分割网络模型为了计算预测的分割图的dice coefficient，将|A|+|B|近似为预测图和label之间的点乘，并将集合A和集合B中的元素相加。In order to calculate the set similarity measure function of the predicted real segmentation map, |A|+|B| is approximated as the point product between the actual segmented image and the real segmented image, and each pixel in set A and set B Add the values of the points; stop the training when the loss function is the smallest, and get the trained segmentation network model In order to calculate the dice coefficient of the predicted segmentation graph, |A|+|B| is approximated as the point product between the predicted graph and the label , and add the elements in set A and set B.
步骤5：训练分割网络；Step 5: train segmentation network;
为使步骤5中的损失函数最小，使用Adam优化器，先对每一阶段网络的权重参数进行初始化，使用平均值为0的高斯分布随机初始化权重参数；In order to minimize the loss function in step 5, use the Adam optimizer to initialize the weight parameters of the network at each stage, and use a Gaussian distribution with an average value of 0 to randomly initialize the weight parameters;
对每个样本图片x，先利用前向传播求出总误差，再利用反向传播求出各个权重参数的偏导数，最后根据梯度下降法对权重参数进行更新，重复此步骤直至损失函数达到最小，得到训练好的分割网络模型。For each sample image x, first use forward propagation to find the total error, then use back propagation to find the partial derivative of each weight parameter, and finally update the weight parameters according to the gradient descent method, repeat this step until the loss function reaches the minimum , to get the trained segmentation network model.
步骤6：将待分割的黑色素皮肤病图像输入本发明所述分割网络，得到分割好的黑色素皮肤病图像。Step 6: Input the melanotic skin disease image to be segmented into the segmentation network of the present invention to obtain the segmented melanotic skin disease image.
在对黑色素皮肤病数据集进行分割时，本发明通过改进原始U型网络结构，改善了浅层细节信息在下采样过程中丢失的问题，结合了恒定缩放的残差网络和循环神经网络进一步提升了深、浅层语义的融合，减少了语义差距，提升了医学图像前景和背景的分割准确性，同时对于不同场景下的医学影像分割，可以选择不同的权值组合进行应用，对于多场景，本发明提出的方法有很好的可用性。When segmenting the melanin skin disease data set, the present invention improves the original U-shaped network structure to improve the loss of shallow detail information during the downsampling process, and further improves The fusion of deep and shallow semantics reduces the semantic gap and improves the segmentation accuracy of medical image foreground and background. At the same time, for medical image segmentation in different scenarios, different weight combinations can be selected for application. For multiple scenarios, this The method proposed by the invention has good usability.
以上所述仅是本发明的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本发明技术原理的前提下，还可以做出若干改进和变形，这些改进和变形也应视为本发明的保护范围。The above is only a preferred embodiment of the present invention, it should be pointed out that for those of ordinary skill in the art, without departing from the technical principle of the present invention, some improvements and modifications can also be made. It should also be regarded as the protection scope of the present invention.

Claims (8)

  1. 一种基于U型网络的医学图像分割方法，所述方法由计算机实现，其特征是，包括以下步骤：A method for segmenting medical images based on a U-shaped network, said method being implemented by a computer, is characterized in that it comprises the following steps:
    步骤1：获取医学图像数据集；Step 1: Obtain a medical image dataset;
    步骤2：从所述医学图像数据集中获取成对的原始图片及对原始图片中目标区域的真实分割图，一起作为预先构建的恒定缩放分割网络的输入数据集，其中，所述输入数据集分为训练集、验证集和测试集；Step 2: Obtain a pair of original pictures and the real segmentation map of the target area in the original picture from the medical image data set, and use them together as the input data set of the pre-built constant scaling segmentation network, wherein the input data set is divided into are the training set, validation set and test set;
    步骤3：利用所述训练集对恒定缩放分割网络进行训练，得到训练好的分割网络模型，利用所述验证集对恒定缩放分割网络进行验证，其中，所述恒定缩放分割网络包括特征提取模块和分辨率增大模块，在解码过程中每个解码器层都与来自编码器的相同层次相应剪裁的特征图连接；Step 3: use the training set to train the constant scaling segmentation network to obtain a trained segmentation network model, and use the verification set to verify the constant scaling segmentation network, wherein the constant scaling segmentation network includes a feature extraction module and A resolution upscaling module, where each decoder layer is concatenated with the corresponding cropped feature maps from the same layer from the encoder during decoding;
    步骤4：将待分割的原始图片输入到所述分割网络模型中进行分割，得到真实分割图。Step 4: Input the original picture to be segmented into the segmentation network model for segmentation to obtain a real segmented image.
  2. 根据权利要求1所述的基于U型网络的医学图像分割方法，其特征是，所述步骤2中，所述输入数据集中训练集、验证集和测试集的比例为6：2：2。The medical image segmentation method based on U-shaped network according to claim 1, wherein in the step 2, the ratio of the training set, verification set and test set in the input data set is 6:2:2.
  3. 根据权利要求1所述的基于U型网络的医学图像分割方法，其特征是，所述步骤3中，特征提取模块包括5个恒定缩放模块，4个下采样模块，其中，所述恒定缩放模块之间通过下采样模块相连接；所述分辨率增大模块包括4个上采样模块和4个恒定缩放模块，其中，所述恒定缩放模块之间通过上采样模块相连接。The medical image segmentation method based on the U-shaped network according to claim 1, wherein in the step 3, the feature extraction module includes 5 constant scaling modules and 4 down-sampling modules, wherein the constant scaling module The downsampling modules are connected between them; the resolution increasing module includes 4 upsampling modules and 4 constant scaling modules, wherein the constant scaling modules are connected through an upsampling module.
  4. 根据权利要求3所述的基于U型网络的医学图像分割方法，其特征是，所述恒定缩放模块由恒定缩放的残差网络结构与循环神经网络结合而成，所 述恒定缩放的残差网络结构的输出由两部分相加组成：输入特征图乘以权值a；以及输入特征图经过两次权重层后乘以权值b；权值a与权值b应满足如下关系：The medical image segmentation method based on U-shaped network according to claim 3, wherein the constant scaling module is formed by combining a constant scaling residual network structure and a cyclic neural network, and the constant scaling residual network The output of the structure consists of two parts: the input feature map is multiplied by the weight a; and the input feature map is multiplied by the weight b after two weight layers; the weight a and the weight b should satisfy the following relationship:
    a+b＝1(1)。a+b=1(1).
  5. 根据权利要求4所述的基于U型网络的医学图像分割方法，其特征是，所述循环神经网络由输入特征图进入卷积层，后进行循环操作，使得每一次卷积操作都能获取上一次卷积操作得到的特征信息，最后经过ReLu激活函数得到输出。The medical image segmentation method based on U-shaped network according to claim 4, characterized in that, the cyclic neural network enters the convolution layer from the input feature map, and then performs a loop operation, so that each convolution operation can obtain the above The feature information obtained by a convolution operation is finally output through the ReLu activation function.
  6. 根据权利要求5所述的基于U型网络的医学图像分割方法，其特征是，使用所述循环神经网络替换恒定缩放模型的恒定缩放的残差网络结构中的权重层，从而形成恒定缩放模块，其输出由两部分相加组成：输入特征图乘以权值a；以及输入特征图经过两次包含卷积块以及ReLU激活函数的循环卷积块后乘以权值b；权值a与权值b应满足公式(1)。The medical image segmentation method based on U-shaped network according to claim 5, characterized in that, using the cyclic neural network to replace the weight layer in the constant scaling residual network structure of the constant scaling model, thereby forming a constant scaling module, Its output is composed of two parts: the input feature map is multiplied by the weight a; and the input feature map is multiplied by the weight b after two circular convolution blocks containing the convolution block and the ReLU activation function; the weight a and the weight The value b should satisfy formula (1).
  7. 根据权利要求1所述的基于U型网络的医学图像分割方法，其特征是，所述步骤3中，在所述恒定缩放分割网络中，设置损失函数为集合相似度度量函数，具体公式为：The medical image segmentation method based on U-shaped network according to claim 1, wherein in said step 3, in said constant scaling segmentation network, the loss function is set as a set similarity measurement function, and the specific formula is:
    Figure PCTCN2021138825-appb-100001
    Figure PCTCN2021138825-appb-100001
    其中，|A∩B|表示集合A和集合B之间的共同元素，|A|表示A中的元素的个数，|B|表示B中的元素的个数，集合A中元素为输入数据集对恒定缩放分割网络分割得到真实的分割图像，集合B中元素为原始图片中目标区域的真实分割图；Among them, |A∩B| represents the common elements between set A and set B, |A| represents the number of elements in A, |B| represents the number of elements in B, and the elements in set A are input data Set-to-constant scaling segmentation network segmentation to obtain the real segmented image, the elements in set B are the real segmented image of the target area in the original image;
    为了计算预测的真实分割图的集合相似度度量函数，将|A|+|B|近似为实际分割得到的图像和真实分割图像之间的点乘，并将集合A和集合B中每个像素 点的值相加；当损失函数最小时停止训练，得到训练好的分割网络模型。In order to calculate the set similarity measure function of the predicted real segmentation map, |A|+|B| is approximated as the point product between the actual segmented image and the real segmented image, and each pixel in set A and set B The values of the points are added; when the loss function is minimized, the training is stopped, and the trained segmentation network model is obtained.
  8. 根据权利要求7所述的基于U型网络的医学图像分割方法，其特征是，所述步骤3中，当损失函数最小时停止训练，得到训练好的分割网络模型，包括以下步骤：The medical image segmentation method based on the U-shaped network according to claim 7, wherein in the step 3, when the loss function is minimized, the training is stopped, and the trained segmentation network model is obtained, comprising the following steps:
    基于Adam优化器对每一阶段恒定缩放分割网络的权重参数进行初始化，使用平均值为0的高斯分布随机初始化权重参数；Based on the Adam optimizer, the weight parameters of the constant scaling segmentation network are initialized at each stage, and the weight parameters are randomly initialized using a Gaussian distribution with an average value of 0;
    对每个输入分割网络模型的训练集中的样本图片，样本图片包括合成图像和原始图片，先利用前向传播求出真实的分割图像与原始图片中目标区域的真实分割图间的总误差，再利用反向传播求出各个权重参数的偏导数，最后根据梯度下降法对权重参数进行更新；For each sample picture in the training set of the input segmentation network model, the sample picture includes the synthetic image and the original picture, first use the forward propagation to find the total error between the real segmentation image and the real segmentation map of the target area in the original picture, and then Use backpropagation to find the partial derivatives of each weight parameter, and finally update the weight parameters according to the gradient descent method;
    重复上述步骤直至损失函数达到最小，得到训练好的分割网络模型。Repeat the above steps until the loss function reaches the minimum, and the trained segmentation network model is obtained.
PCT/CN2021/138825 2021-06-10 2021-12-16 Medical image segmentation method based on u-shaped network WO2022257408A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/697,884 US20220398737A1 (en) 2021-06-10 2022-03-17 Medical image segmentation method based on u-network

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110654344.4 2021-06-10
CN202110654344.4A CN113240691B (en) 2021-06-10 2021-06-10 Medical image segmentation method based on U-shaped network

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/697,884 Continuation-In-Part US20220398737A1 (en) 2021-06-10 2022-03-17 Medical image segmentation method based on u-network

Publications (1)

Publication Number Publication Date
WO2022257408A1 true WO2022257408A1 (en) 2022-12-15

Family

ID=77139725

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/138825 WO2022257408A1 (en) 2021-06-10 2021-12-16 Medical image segmentation method based on u-shaped network

Country Status (2)

Country Link
CN (1) CN113240691B (en)
WO (1) WO2022257408A1 (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115984263A (en) * 2023-03-15 2023-04-18 三峡智控科技有限公司 Bolt looseness detection algorithm and detection system based on improved twin neural network
CN115984146A (en) * 2023-03-16 2023-04-18 中国海洋大学 Global consistency-based marine chlorophyll concentration image completion method and network
CN116188816A (en) * 2022-12-29 2023-05-30 广东省新黄埔中医药联合创新研究院 Acupoint positioning method based on cyclic consistency deformation image matching network
CN116363036A (en) * 2023-05-12 2023-06-30 齐鲁工业大学(山东省科学院) Infrared and visible light image fusion method based on visual enhancement
CN116363123A (en) * 2023-05-23 2023-06-30 杭州华得森生物技术有限公司 Fluorescence microscopic imaging system and method for detecting circulating tumor cells
CN116484905A (en) * 2023-06-20 2023-07-25 合肥高维数据技术有限公司 Deep neural network model training method for non-aligned samples
CN116612131A (en) * 2023-05-22 2023-08-18 山东省人工智能研究院 Cardiac MRI structure segmentation method based on ADC-UNet model
CN116934618A (en) * 2023-07-13 2023-10-24 江南大学 Image halftone method, system and medium based on improved residual error network
CN118135229A (en) * 2024-05-07 2024-06-04 南昌康德莱医疗科技有限公司 Medical image segmentation method, system and computer

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113240691B (en) * 2021-06-10 2023-08-01 南京邮电大学 Medical image segmentation method based on U-shaped network
CN114037714B (en) * 2021-11-02 2024-05-24 大连理工大学人工智能大连研究院 3D MR and TRUS image segmentation method for prostate system puncture
CN114387436B (en) * 2021-12-28 2022-10-25 北京安德医智科技有限公司 Wall coronary artery detection method and device, electronic device and storage medium
CN114170249B (en) * 2022-02-08 2022-04-19 湖南大学 Image semantic segmentation method based on CNUNet3+ network

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110570431A (en) * 2019-09-18 2019-12-13 东北大学 Medical image segmentation method based on improved convolutional neural network
WO2020093042A1 (en) * 2018-11-02 2020-05-07 Deep Lens, Inc. Neural networks for biomedical image analysis
CN111325750A (en) * 2020-02-25 2020-06-23 西安交通大学 Medical image segmentation method based on multi-scale fusion U-shaped chain neural network
CN111612790A (en) * 2020-04-29 2020-09-01 杭州电子科技大学 Medical image segmentation method based on T-shaped attention structure
CN112927240A (en) * 2021-03-08 2021-06-08 重庆邮电大学 CT image segmentation method based on improved AU-Net network
CN113240691A (en) * 2021-06-10 2021-08-10 南京邮电大学 Medical image segmentation method based on U-shaped network

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111681252B (en) * 2020-05-30 2022-05-03 重庆邮电大学 Medical image automatic segmentation method based on multipath attention fusion

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020093042A1 (en) * 2018-11-02 2020-05-07 Deep Lens, Inc. Neural networks for biomedical image analysis
CN110570431A (en) * 2019-09-18 2019-12-13 东北大学 Medical image segmentation method based on improved convolutional neural network
CN111325750A (en) * 2020-02-25 2020-06-23 西安交通大学 Medical image segmentation method based on multi-scale fusion U-shaped chain neural network
CN111612790A (en) * 2020-04-29 2020-09-01 杭州电子科技大学 Medical image segmentation method based on T-shaped attention structure
CN112927240A (en) * 2021-03-08 2021-06-08 重庆邮电大学 CT image segmentation method based on improved AU-Net network
CN113240691A (en) * 2021-06-10 2021-08-10 南京邮电大学 Medical image segmentation method based on U-shaped network

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116188816A (en) * 2022-12-29 2023-05-30 广东省新黄埔中医药联合创新研究院 Acupoint positioning method based on cyclic consistency deformation image matching network
CN116188816B (en) * 2022-12-29 2024-05-28 广东省新黄埔中医药联合创新研究院 Acupoint positioning method based on cyclic consistency deformation image matching network
CN115984263B (en) * 2023-03-15 2023-05-30 三峡智控科技有限公司 Bolt loosening detection method and detection system based on improved twin neural network
CN115984263A (en) * 2023-03-15 2023-04-18 三峡智控科技有限公司 Bolt looseness detection algorithm and detection system based on improved twin neural network
CN115984146A (en) * 2023-03-16 2023-04-18 中国海洋大学 Global consistency-based marine chlorophyll concentration image completion method and network
CN116363036B (en) * 2023-05-12 2023-10-10 齐鲁工业大学(山东省科学院) Infrared and visible light image fusion method based on visual enhancement
CN116363036A (en) * 2023-05-12 2023-06-30 齐鲁工业大学(山东省科学院) Infrared and visible light image fusion method based on visual enhancement
CN116612131B (en) * 2023-05-22 2024-02-13 山东省人工智能研究院 Cardiac MRI structure segmentation method based on ADC-UNet model
CN116612131A (en) * 2023-05-22 2023-08-18 山东省人工智能研究院 Cardiac MRI structure segmentation method based on ADC-UNet model
CN116363123B (en) * 2023-05-23 2023-12-22 杭州华得森生物技术有限公司 Fluorescence microscopic imaging system and method for detecting circulating tumor cells
CN116363123A (en) * 2023-05-23 2023-06-30 杭州华得森生物技术有限公司 Fluorescence microscopic imaging system and method for detecting circulating tumor cells
CN116484905B (en) * 2023-06-20 2023-08-29 合肥高维数据技术有限公司 Deep neural network model training method for non-aligned samples
CN116484905A (en) * 2023-06-20 2023-07-25 合肥高维数据技术有限公司 Deep neural network model training method for non-aligned samples
CN116934618A (en) * 2023-07-13 2023-10-24 江南大学 Image halftone method, system and medium based on improved residual error network
CN116934618B (en) * 2023-07-13 2024-06-11 江南大学 Image halftone method, system and medium based on improved residual error network
CN118135229A (en) * 2024-05-07 2024-06-04 南昌康德莱医疗科技有限公司 Medical image segmentation method, system and computer

Also Published As

Publication number Publication date
CN113240691B (en) 2023-08-01
CN113240691A (en) 2021-08-10

Similar Documents

Publication Publication Date Title
WO2022257408A1 (en) Medical image segmentation method based on u-shaped network
US11580646B2 (en) Medical image segmentation method based on U-Net
WO2022199143A1 (en) Medical image segmentation method based on u-shaped network
WO2020108362A1 (en) Body posture detection method, apparatus and device, and storage medium
WO2021227726A1 (en) Methods and apparatuses for training face detection and image detection neural networks, and device
CN114119638A (en) Medical image segmentation method integrating multi-scale features and attention mechanism
CN112580416A (en) Video tracking based on deep Siam network and Bayesian optimization
CN107610129B (en) CNN-based multi-modal nasopharyngeal tumor joint segmentation method
CN108830170B (en) End-to-end target tracking method based on layered feature representation
CN112150493A (en) Semantic guidance-based screen area detection method in natural scene
WO2023151237A1 (en) Face pose estimation method and apparatus, electronic device, and storage medium
CN107944437B (en) A kind of Face detection method based on neural network and integral image
US20220398737A1 (en) Medical image segmentation method based on u-network
CN110852330A (en) Behavior identification method based on single stage
CN116579982A (en) Pneumonia CT image segmentation method, device and equipment
CN113505634A (en) Double-flow decoding cross-task interaction network optical remote sensing image salient target detection method
Du High-precision portrait classification based on mtcnn and its application on similarity judgement
Hu et al. RGB-D image multi-target detection method based on 3D DSF R-CNN
Yang et al. VAMYOLOX: An accurate and efficient object detection algorithm based on visual attention mechanism for UAV optical sensors
Liu et al. Key algorithm for human motion recognition in virtual reality video sequences based on hidden markov model
CN117152507B (en) Tooth health state detection method, device, equipment and storage medium
CN111709269B (en) Human hand segmentation method and device based on two-dimensional joint information in depth image
Chen et al. Convolutional neural network using multi-scale information for stereo matching cost computation
WO2023109086A1 (en) Character recognition method, apparatus and device, and storage medium
CN113269734B (en) Tumor image detection method and device based on meta-learning feature fusion strategy

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21944904

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21944904

Country of ref document: EP

Kind code of ref document: A1