CN114612662A - Polyp image segmentation method based on boundary guidance - Google Patents

Polyp image segmentation method based on boundary guidance Download PDF

Info

Publication number
CN114612662A
CN114612662A CN202210228391.7A CN202210228391A CN114612662A CN 114612662 A CN114612662 A CN 114612662A CN 202210228391 A CN202210228391 A CN 202210228391A CN 114612662 A CN114612662 A CN 114612662A
Authority
CN
China
Prior art keywords
boundary
layer
polyp
image segmentation
convolution
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210228391.7A
Other languages
Chinese (zh)
Inventor
陈舒涵
陆露
徐秀奇
俞锦豪
陈泽宇
汤浩楠
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yangzhou University
Original Assignee
Yangzhou University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yangzhou University filed Critical Yangzhou University
Priority to CN202210228391.7A priority Critical patent/CN114612662A/en
Publication of CN114612662A publication Critical patent/CN114612662A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a polyp image segmentation method based on boundary guidance, which comprises the following steps: constructing a polyp image segmentation model; inputting the polyp image to be segmented into the polyp image segmentation model to obtain region prediction maps P output by each sideR4‑PR1And a boundary prediction map PB4‑PB1And upsampled to polyp to be segmentedObtaining a region prediction map with the same resolution
Figure DDA0003539687640000011
And boundary prediction map
Figure DDA0003539687640000012
Computing region prediction maps using cross entropy loss functions
Figure DDA0003539687640000013
Boundary prediction graph
Figure DDA0003539687640000014
Errors between the images and the real labeled graph are reversely transmitted to update the parameters of the polyp image segmentation model so as to train the polyp image segmentation model; inputting a polyp image to be segmented into a trained polyp image segmentation model, normalizing a region prediction graph output by a residual refinement decoder A to (0,1) through a Sigmoid layer, and outputting a final segmentation result after restoring the resolution of an original image; the invention greatly improves the accuracy of polyp image segmentation; more accurate region prediction is achieved.

Description

Polyp image segmentation method based on boundary guidance
Technical Field
The invention relates to the technical field of polyp image segmentation, in particular to a polyp image segmentation method based on boundary guidance.
Background
In recent years, the incidence of colon cancer is rapidly increased and endangers human health, colonoscopy is the most main way to detect colon cancer, and early treatment is an effective way to improve patient recovery. Colorectal polyps are accurately segmented from the resulting images of colonoscopy, contributing to the clinical treatment of colon cancer. The traditional manual segmentation method is complex in procedure, low in efficiency and high in technical requirements on medical staff, so that automatic segmentation of the image is a development trend of medical image application. However, the automatic segmentation of the polyp image has high requirement on accuracy, and due to the complex tissue structure, variable shape, fuzzy boundary and high similarity to the background, the segmentation of the polyp image is greatly challenged. Most of the traditional polyp segmentation methods are based on low-level features such as texture or iterative clustering super pixels, and the like, so that the improvement of the polyp image segmentation accuracy is promoted to a certain extent, but the following two problems still exist: firstly, the boundary segmentation of the polyp is still not accurate enough, and the factors of low contrast between the polyp and the background, fuzzy and tortuous boundary and the like are not considered in the boundary positioning; secondly, the adaptability to the multi-scale features of the polyp tissues is not high, the polyp tissues are different in size and shape, and the multi-scale feature extraction capability of the model should be improved, so that the current polyp image segmentation method still has certain limitations.
Disclosure of Invention
This section is for the purpose of summarizing some aspects of embodiments of the invention and to briefly introduce some preferred embodiments. In this section, as well as in the abstract and the title of the invention of this application, simplifications or omissions may be made to avoid obscuring the purpose of the section, the abstract and the title, and such simplifications or omissions are not intended to limit the scope of the invention.
The present invention has been made in view of the above-mentioned conventional problems.
In order to solve the above technical problems, the present invention provides the following technical solutions, including: constructing a polyp image segmentation model; inputting the polyp image to be segmented into the polyp image segmentation model to obtain region prediction maps P output by each sideR4-PR1And a boundary prediction map PB4-PB1And upsampling to the same resolution as the polyp image to be segmented to obtain a region prediction map
Figure BDA0003539687620000011
And boundary prediction map
Figure BDA0003539687620000012
Computing region prediction maps using cross entropy loss functions
Figure BDA0003539687620000013
Boundary prediction graph
Figure BDA0003539687620000014
Errors between the real labeling graphs and the polyp image segmentation model are reversely transmitted to update parameters of the polyp image segmentation model so as to train the polyp image segmentation model; and inputting the polyp image to be segmented into a trained polyp image segmentation model, normalizing the region prediction graph output by the residual refinement decoder A to (0,1) through a Sigmoid layer, and outputting a final segmentation result after restoring the resolution of the original image.
As a preferred embodiment of the polyp image segmentation method based on boundary guiding proposed by the present invention, wherein: the proposed construction of a polyp image segmentation model comprises: res2Net-50 is used as a backbone network, and the backbone networks are layerB to layerE and are used as side output characteristics Ft(t is 1,2,3 and 4) which are respectively sent to residual error refinement decoders A to C and an initial positioning decoder, wherein t represents the side output stage number of the backbone network, and 1 to 4 correspond to the side output stage numbers of the backbone networks layerB to E; detecting an initial coarse position of polyp tissue through an initial positioning decoder, wherein the resolution of an output result image of the initial positioning decoder is 1/32 of an original image; and repairing the lost target part in the initial coarse position of the polyp tissue through residual refinement decoders A-C, and refining the detail of the boundary.
As a preferred embodiment of the polyp image segmentation method based on boundary guiding proposed by the present invention, wherein: the operation of the initial positioning decoder comprises the following steps: input features F by initially positioning the decoder4Feeding into a 1 × 1 convolution layer, reducing the number of channels to 64 to obtain the input feature x4The parameter of the convolutional layer is 2048, 64,1 × 1, wherein in _ channel of { in _ channel, out _ channel, k × k } represents the number of channels of the input feature, out _ channel represents the number of channels of the output feature, k represents the size of the convolutional kernel, and a batch normalization layer Batchnorm and a nonlinear active layer ReLU are attached after the convolutional layer; inputting a feature x4Inputting into a layer division convolution HSC to obtain the boundary characteristics of the polyp tissue
Figure BDA0003539687620000021
Figure BDA0003539687620000022
Then reducing the channel number to 1 through a convolution layer, the convolution parameter is {64,1,3 x 3}, a batch normalization layer BatchNorm and a nonlinear active layer ReLU are attached to the convolution layer, and an initial boundary prediction graph of the initial positioning decoder is obtained
Figure BDA0003539687620000023
Figure BDA0003539687620000024
Wherein HSC (. cndot.) represents a hierarchical fission convolution operation, Conv3×3(. cndot.) represents a 3X 3 convolutional layer; characterizing the boundaries of polyp tissue
Figure BDA0003539687620000025
And input feature x4Splicing along the channel dimension, carrying out boundary-region guided learning, inputting learning region characteristics in 4 convolutional layers, wherein convolutional parameters are {128,64,3 × 3}, {64,64,3 × 3}, and {64,1,3 × 3}, a batch normalization layer BatchNorm and a nonlinear active layer ReLU are attached to the convolutional layers in sequence, and finally outputting a region prediction map of the initial positioning decoder
Figure BDA0003539687620000026
Figure BDA0003539687620000027
Where Cat (-) denotes the stitching operation along the channel dimension.
As a preferred embodiment of the polyp image segmentation method based on boundary guiding proposed by the present invention, wherein: the working process of the residual refinement decoders A-C comprises the following steps: input feature F1～F3Inputting a 1 × 1 convolution layer to reduce the number of channels to 64 to obtain the input characteristic xt,(t＝[1,2,3]) The convolution parameter is { in _ channel, 64,1 × 1},after the convolution layer, a batch normalization layer BatchNorm and a nonlinear active layer ReLU are added, and then the batch normalization layer BatchNorm and the nonlinear active layer ReLU are respectively sent to the boundary branch and the regional branch for residual correction.
As a preferred embodiment of the polyp image segmentation method based on boundary guiding proposed by the present invention, wherein: the step of sending the boundary branch to carry out residual error correction comprises the following steps: in boundary branching, deep region prediction
Figure BDA0003539687620000031
Upsampling to and input feature xtSame resolution, again with input features xtPhase splicing to obtain boundary features
Figure BDA0003539687620000032
Figure BDA0003539687620000033
Characterizing the boundary
Figure BDA0003539687620000034
Inputting a convolution layer to reduce the channel number to 1, the convolution parameter is {64,1,3 x 3}, the convolution layer is attached with a batch normalization layer BatchNorm and a nonlinear active layer ReLU to obtain the boundary prediction graph output by the decoder
Figure BDA0003539687620000035
Figure BDA0003539687620000036
Where Up (-) denotes the upsampling operation, Conv3×3(. cndot.) represents a 3X 3 convolutional layer.
As a preferred embodiment of the polyp image segmentation method based on boundary guiding proposed by the present invention, wherein: the sending into the regional branch for residual correction comprises the following steps: in zone branching, zone characteristics are obtained by a reverse attention mechanism module
Figure BDA0003539687620000037
Then, the boundary characteristics are related to the obtained data
Figure BDA0003539687620000038
Splicing along channel dimensions to obtain region features after fusion of boundary features and region features after fusion of boundary features
Figure BDA0003539687620000039
Figure BDA00035396876200000310
By four convolutional layer pairs
Figure BDA00035396876200000311
Residual error learning is carried out, convolution parameters are {128,64,3 × 3}, {64,64,3 × 3}, and {64,1,3 × 3}, a batch normalization layer BatchNorm and a nonlinear active layer ReLU are attached to the first three convolutional layers, the number of channels of the last convolutional layer is reduced to 1, and the result and the upsampled deep layer region prediction layer region are obtained
Figure BDA00035396876200000312
Adding to obtain the area prediction graph output by the residual error refinement decoders A-C
Figure BDA00035396876200000313
Figure BDA00035396876200000314
As a preferred embodiment of the polyp image segmentation method based on boundary guiding proposed by the present invention, wherein: the method comprises the following steps: the stratified fission convolutional HSCs are:
step 1: inputting a feature x4Divided into 6 split groups in channel dimension, each being [ x ]1,x2,x3,x4,x5,]x, wherein x1-x5Each having 11 channels, x6There are 9 channels, specifically denoted as:
[x1,x2,x3,x4,x5,x6]＝Split(x4)
where Split () means splitting along the channel dimension;
step 2: grouping the first split into x1Feeding expansion rate of R1The expansion convolution of (d) yields an output characteristic d1After the expansion convolution, a batch normalization layer BatchNorm and a nonlinear activation layer ReLU are attached, and the output characteristic d1The specific calculation formula is as follows:
Figure BDA0003539687620000041
wherein, the first and the second end of the pipe are connected with each other,
Figure BDA0003539687620000042
denotes an expansion ratio R1The expansion rates of the expansion convolutions used by the 1 st to 5 th split groups are Rk＝[1,2,4,6,8],(k＝1,2,3,4,5)；
And step 3: will output the characteristic d1Divided into two groups along the channel dimension, denoted d respectively1,1And d1,2There are 6 channels and 5 channels, respectively, specifically expressed as:
d1,1，d1,2＝Split(d1)
will d1,2With the next split group x2Splicing along channel dimension to obtain new split group
Figure BDA0003539687620000043
The concrete expression is as follows:
Figure BDA0003539687620000044
will be provided with
Figure BDA0003539687620000045
Inputting another expansion rate of R2The expansion convolution is added with a batch normalization layer BatchNorm and a nonlinear activation layer ReLU to obtain an output characteristic d2There are 16 channels, and the specific calculation formula is:
Figure BDA0003539687620000046
and 4, step 4: will output the characteristic d2Are divided into two groups, respectively denoted as d2,1And d2,2There are 8 channels and 8 channels, respectively, specifically expressed as:
d2,1，d2,2＝Split(d2)
will d2,2With the next split group x3Splicing along channel dimension to obtain new split group
Figure BDA0003539687620000047
The concrete expression is as follows:
Figure BDA0003539687620000048
will be provided with
Figure BDA0003539687620000049
Inputting another expansion rate of R3Is convolved to obtain an output characteristic d3There are 19 channels, and the specific calculation formula is:
Figure BDA00035396876200000410
and 5: repeating the steps 2 to 3 for a plurality of times until all the split groups are finished, finally splicing all the split groups to recover the channel number 64 to obtain the boundary characteristics of the polyp tissue
Figure BDA00035396876200000411
Figure BDA00035396876200000412
Wherein x is6The last split group is indicated.
As a preferred embodiment of the polyp image segmentation method based on boundary guiding proposed by the present invention, wherein: the reverse attention mechanism module comprises: predicting deep regions
Figure BDA00035396876200000413
Upsampling to and input feature xt(t＝[1,2,3]) The same resolution is input, then a Sigmoid layer is input to be normalized to (0,1), and then the normalization is subtracted from 1 to generate an inverse weight map
Figure BDA0003539687620000051
Figure BDA0003539687620000052
Inverse weight map
Figure BDA0003539687620000053
Multiplying by regional characteristics
Figure BDA0003539687620000054
Obtaining weighted regional characteristics for each channel of the plurality of channels
Figure BDA0003539687620000055
Figure BDA0003539687620000056
Wherein the content of the first and second substances,
Figure BDA0003539687620000057
representing regional characteristics
Figure BDA0003539687620000058
The c-th channel of (1).
The invention has the beneficial effects that: according to the invention, the initial positioning decoder and the residual refinement decoder are designed, so that the accuracy of polyp image segmentation is greatly improved; meanwhile, the boundary features are used for guiding the regional branches to carry out residual learning, and the regional branches combine the position information of the boundary features with the semantic information extracted by the regional branches, so that the challenges brought by irregular deformation of polyp tissues and similarity with the background are solved, and more accurate regional prediction is realized; in addition, a hierarchical fission convolution HSC is designed and embedded into a boundary branch, output characteristics of the HSC occupy different channels, polyp boundary characteristics of different scales can be captured corresponding to different receptive fields, and adaptability of the model to multi-scale polyp tissues is improved.
Drawings
Fig. 1 is a schematic diagram of the overall network architecture of a polyp image segmentation model of a polyp image segmentation method based on boundary guidance according to a first embodiment of the present invention;
fig. 2 is a block diagram of an initial positioning decoder and residual refinement decoders a-C of a polyp image segmentation method based on boundary guiding according to a first embodiment of the present invention;
FIG. 3 is a block diagram of a hierarchical split convolution HSC of the boundary-guided polyp image segmentation based method proposed in the first embodiment of the present invention;
fig. 4 is a diagram showing the results of the output of each decoder of the polyp image segmentation model in the method for segmenting a polyp image based on boundary guidance according to the second embodiment of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, specific embodiments accompanied with figures are described in detail below, and it is apparent that the described embodiments are a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making creative efforts based on the embodiments of the present invention, shall fall within the protection scope of the present invention.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways than those specifically described and will be readily apparent to those of ordinary skill in the art without departing from the spirit of the present invention, and therefore the present invention is not limited to the specific embodiments disclosed below.
Furthermore, reference herein to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one implementation of the invention. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments.
While the present invention has been described in detail with reference to the drawings, in describing the embodiments of the present invention in detail, the cross-sectional views illustrating the structure of the device are not enlarged partially according to the general scale for convenience of illustration, and the drawings are provided as examples only, which should not limit the scope of the present invention. In addition, the three-dimensional dimensions of length, width and depth should be included in the actual fabrication.
Also in the description of the present invention, it should be noted that the terms "upper, lower, inner and outer" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, which are only for convenience of description and simplification of description, but do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms first, second, or third are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
The terms "mounted, connected and connected" in the present invention are to be understood broadly, unless otherwise explicitly specified or limited, for example: can be fixedly connected, detachably connected or integrally connected; they may be mechanically, electrically, or directly connected, or indirectly connected through intervening media, or may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
Example 1
Referring to fig. 1 to 3, a first embodiment of the present invention provides a method for segmenting a polyp image based on boundary guiding, including:
s1: and constructing a polyp image segmentation model.
Referring to fig. 1, the polyp image segmentation model includes backbone networks layerA to layerE, residual refinement decoders a to C, and an initial positioning decoder.
(1) Res2Net-50 is used as a backbone network, and the backbone networks are layerB to layerE and are used as side output characteristics FtAnd (t is 1,2,3 and 4) respectively sent to the residual error refinement decoders A to C and the initial positioning decoder (as shown in fig. 2), wherein t represents the side output stage number of the backbone network, and t is 1 to 4 corresponding to the side output stage numbers of the backbone networks layerB to E.
(2) Detecting an initial coarse position of polyp tissue through an initial positioning decoder, wherein the resolution of an output result image of the initial positioning decoder is 1/32 of an original image;
the working process of the initial positioning decoder is specifically as follows:
firstly, input characteristic F is processed by initial positioning decoder4Feeding into a 1 × 1 convolution layer, reducing the number of channels to 64 to obtain the input feature x4The parameter of the convolutional layer is 2048, 64,1 × 1, wherein in _ channel of { in _ channel, out _ channel, k × k } represents the number of channels of the input feature, out _ channel represents the number of channels of the output feature, k represents the size of the convolutional kernel, and a batch normalization layer Batchnorm and a nonlinear active layer ReLU are attached after the convolutional layer;
will input the feature x4Input into a hierarchical fission convolution HSC (as shown in FIG. 3) to obtain the boundary characteristics of polyp tissue
Figure BDA0003539687620000071
Figure BDA0003539687620000072
Specifically, the working process of the layer splitting convolution HSC is as follows:
step 1: inputting a feature x4Divided into 6 split groups in channel dimension, each being [ x ]1,x2,x3,x4,x5,]x, wherein x1-x5Each having 11 channels, x6There are 9 channels, specifically denoted as:
[x1,x2,x3,x4,x5,x6]＝Split(x4)
where Split () means splitting along the channel dimension;
step 2: grouping the first split into x1Feeding an expansion rate of R1The expansion convolution of (d) yields an output characteristic d1After the expansion convolution, a batch normalization layer BatchNorm and a nonlinear activation layer ReLU are attached, and the output characteristic d1The specific calculation formula is as follows:
Figure BDA0003539687620000073
wherein the content of the first and second substances,
Figure BDA0003539687620000074
denotes an expansion ratio R1The expansion rates of the expansion convolutions used by the 1 st to 5 th split groups are Rk＝[1,2,4,6,8],(k＝1,2,3,4,5)；
And step 3: will output the characteristic d1Divided into two groups along the channel dimension, denoted d1,1And d1,2There are 6 channels and 5 channels, respectively, specifically expressed as:
d1,1，d1,2＝Split(d1)
will d1,2With the next split group x2Splicing along channel dimension to obtain new split group
Figure BDA0003539687620000075
Detailed description of the inventionComprises the following steps:
Figure BDA0003539687620000076
will be provided with
Figure BDA0003539687620000077
Inputting another expansion ratio of R2The expansion convolution is added with a batch normalization layer BatchNorm and a nonlinear activation layer ReLU to obtain an output characteristic d2There are 16 channels, and the specific calculation formula is:
Figure BDA0003539687620000081
and 4, step 4: will output the characteristic d2Are divided into two groups, respectively denoted as d2,1And d2,2There are 8 channels and 8 channels, respectively, specifically expressed as:
d2,1，d2,2＝Split(d2)
will d2,2With the next split group x3Splicing along channel dimension to obtain new split group
Figure BDA0003539687620000082
The concrete expression is as follows:
Figure BDA0003539687620000083
will be provided with
Figure BDA0003539687620000084
Inputting another expansion rate of R3Is convolved to obtain an output characteristic d3There are 19 channels, and the specific calculation formula is:
Figure BDA0003539687620000085
and 5: repeating the steps 2 to 3 for a plurality of times until all the split groups are finished, finally splicing all the split groups to recover the channel number 64 to obtain the boundary characteristics of the polyp tissue
Figure BDA0003539687620000086
Figure BDA0003539687620000087
Wherein x is6The last split group is indicated.
Thirdly, reducing the number of channels to 1 through a convolution layer, wherein the convolution parameter is {64,1,3 multiplied by 3}, and a batch normalization layer BatchNorm and a nonlinear active layer ReLU are attached to the convolution layer to obtain an initial boundary prediction graph of the initial positioning decoder
Figure BDA0003539687620000088
Figure BDA0003539687620000089
Wherein HSC (. cndot.) represents a hierarchical fission convolution operation, Conv3×3(. cndot.) represents a 3X 3 convolutional layer;
boundary characteristics of the meat texture
Figure BDA00035396876200000810
And input feature x4Splicing along the channel dimension, carrying out boundary-region guided learning, inputting learning region characteristics in 4 convolutional layers, wherein convolutional parameters are {128,64,3 × 3}, {64,64,3 × 3}, and {64,1,3 × 3}, a batch normalization layer BatchNorm and a nonlinear active layer ReLU are attached to the convolutional layers in sequence, and finally outputting a region prediction map of the initial positioning decoder
Figure BDA00035396876200000811
Figure BDA00035396876200000812
Where Cat (-) denotes the stitching operation along the channel dimension.
(3) And repairing the lost target part in the initial coarse position of the polyp tissue through residual refinement decoders A-C, and refining the boundary details.
The working process of the residual refinement decoders A-C is specifically as follows: input feature F1～F3Inputting a 1 × 1 convolution layer to reduce the number of channels to 64 to obtain the input characteristic xt,(t＝[1,2,3]) The convolution parameter is { in _ channel, 64,1 × 1}, a batch normalization layer BatchNorm and a nonlinear activation layer ReLU are attached to the convolution layer, and then the batch normalization layer BatchNorm and the nonlinear activation layer ReLU are sent to the boundary branch and the region branch respectively for residual correction.
Wherein, (1) the specific steps of sending into the boundary branch for residual correction are as follows:
first, in the boundary branch, the deep region is predicted
Figure BDA0003539687620000091
Upsampling to and input feature xtSame resolution, again with input features xtPhase splicing to obtain boundary characteristics
Figure BDA0003539687620000092
Figure BDA0003539687620000093
② characterizing the boundary
Figure BDA0003539687620000094
Inputting a convolution layer to reduce the channel number to 1, the convolution parameter is {64,1,3 x 3}, the convolution layer is attached with a batch normalization layer BatchNorm and a nonlinear active layer ReLU to obtain the boundary prediction graph output by the decoder
Figure BDA0003539687620000095
Figure BDA0003539687620000096
Wherein Up (-) denotes an upsampling operation, Conv3×3(. cndot.) represents a 3X 3 convolutional layer.
(2) The specific steps of sending the area branch to carry out residual error correction are as follows:
in the region branch, the region feature is obtained by the reverse attention mechanism module
Figure BDA0003539687620000097
Then, it is combined with the boundary feature
Figure BDA0003539687620000098
Splicing along channel dimension to obtain region features after boundary features are fused
Figure BDA0003539687620000099
Figure BDA00035396876200000910
Wherein, the working process of the reverse attention mechanism module is as follows:
predicting deep regions
Figure BDA00035396876200000911
Upsampling to and input feature xt(t＝[1,2,3]) The same resolution is input, then a Sigmoid layer is input to be normalized to (0,1), and then the result is subtracted from 1 to generate an inverse weight map
Figure BDA00035396876200000912
Figure BDA00035396876200000913
Inverse weight map
Figure BDA00035396876200000914
Multiplying by regional characteristics
Figure BDA00035396876200000915
Obtaining weighted regional characteristics for each channel of the plurality of channels
Figure BDA00035396876200000916
Figure BDA00035396876200000917
Wherein the content of the first and second substances,
Figure BDA00035396876200000918
representing regional characteristics
Figure BDA00035396876200000919
The c-th channel of (1).
② passing through four convolution layer pairs
Figure BDA00035396876200000920
Residual error learning is carried out, convolution parameters are {128,64,3 × 3}, {64,64,3 × 3}, and {64,1,3 × 3}, a batch normalization layer BatchNorm and a nonlinear active layer ReLU are attached to the first three convolutional layers, the number of channels of the last convolutional layer is reduced to 1, and the result and the upsampled deep layer region prediction layer region are obtained
Figure BDA00035396876200000921
Adding to obtain the area prediction graph output by the residual error refinement decoders A-C
Figure BDA00035396876200000922
Figure BDA00035396876200000923
S2: will be divided intoThe polyp image is input into a polyp image segmentation model, and a region prediction map P output from each side is obtainedR4-PR1And a boundary prediction map PB4-PB1And upsampling to the same resolution as the polyp image to be segmented to obtain a region prediction map
Figure BDA0003539687620000101
And boundary prediction map
Figure BDA0003539687620000102
S3: computing region prediction maps using cross entropy loss functions
Figure BDA0003539687620000103
Boundary prediction graph
Figure BDA0003539687620000104
And (4) errors with the real label graph are transmitted reversely to update the parameters of the polyp image segmentation model so as to train the polyp image segmentation model.
S4: and inputting the polyp image to be segmented into a trained polyp image segmentation model, normalizing the region prediction graph output by the residual refinement decoder A to (0,1) through a Sigmoid layer, and outputting a final segmentation result after restoring the resolution of the original image.
Example 2
In order to verify and explain the technical effect of the boundary-guided polyp image segmentation adopted in the method, different methods selected in the embodiment and the method adopted for carrying out contrast test are compared with the test result by means of scientific demonstration to verify the real effect of the method.
The boundary segmentation of polyps is still not accurate enough in the prior art, the factors of low contrast between the polyps and the background, fuzzy and zigzag boundary and the like are not considered in the boundary positioning, in addition, the adaptability of the prior art to the multi-scale features of polyp tissues is not high, and the prior art of polyp image segmentation has certain limitation.
Compared with the existing method, the method (our) is higher in the aspects of positioning and segmenting the polyp boundary and extracting the multi-scale features of the model. In this embodiment, the existing technical solution and the method are adopted to respectively perform experiments on four public polyp segmentation data sets, which are CVC-clicidb, CVC-ColonDB, ETIS-LaribPolypDB, and Lvasir. This experiment introduces four measurement methods that are widely used in the field of target detection: weighted set similarity metric FβPixel level precision measurement MAE, enhanced alignment measurement
Figure BDA0003539687620000105
Similarity measure S between prediction and truth mapα. The results of the experiments are shown in the following table.
Results of the experiment
Figure BDA0003539687620000106
(in the tables, the best performance indicators are indicated in grey.
The first column is the existing method model and the last row of the first column is the method model proposed by the present invention.
The first line represents the data set name, CVC-ClinicDB, CVC-ColonDB, ETIS-LaribPolypDB, Lvasir, respectively, with 62, 380, 196, 100 pictures, respectively.
The second row is the performance index, S α,
Figure BDA0003539687620000111
the larger F.beta.value is better, and the smaller M value is better.
The results were: the best performance in the four data sets is basically obtained by the network model provided by the invention
It should be recognized that embodiments of the present invention can be realized and implemented by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer readable memory. The methods may be implemented in a computer program using standard programming techniques, including a non-transitory computer-readable storage medium configured with the computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner, according to the methods and figures described in the detailed description. Each program may be implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Furthermore, the program can be run on a programmed application specific integrated circuit for this purpose.
Further, the operations of processes described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described herein (or variations and/or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) collectively executed on one or more processors, by hardware, or combinations thereof. The computer program includes a plurality of instructions executable by one or more processors.
Further, the method may be implemented in any type of computing platform operatively connected to a suitable interface, including but not limited to a personal computer, mini computer, mainframe, workstation, networked or distributed computing environment, separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, and the like. Aspects of the invention may be embodied in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optically read and/or write storage medium, RAM, ROM, or the like, such that it may be read by a programmable computer, which when read by the storage medium or device, is operative to configure and operate the computer to perform the procedures described herein. Further, the machine-readable code, or portions thereof, may be transmitted over a wired or wireless network. The invention described herein includes these and other different types of non-transitory computer-readable storage media when such media include instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor. The invention also includes the computer itself when programmed according to the methods and techniques described herein. A computer program can be applied to input data to perform the functions described herein to transform the input data to generate output data that is stored to non-volatile memory. The output information may also be applied to one or more output devices, such as a display. In a preferred embodiment of the invention, the transformed data represents physical and tangible objects, including particular visual depictions of physical and tangible objects produced on a display.
As used in this application, the terms "component," "module," "system," and the like are intended to refer to a computer-related entity, either hardware, firmware, a combination of hardware and software, or software in execution. For example, a component may be, but is not limited to being: a process running on a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of example, both an application running on a computing device and the computing device can be a component. One or more components can reside within a process and/or thread of execution and a component can be localized on one computer and/or distributed between two or more computers. In addition, these components can execute from various computer readable media having various data structures thereon. The components may communicate by way of local and/or remote processes such as in accordance with a signal having one or more data packets (e.g., data from one component interacting with another component in a local system, distributed system, and/or across a network such as the internet with other systems by way of the signal).
It should be noted that the above-mentioned embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, which should be covered by the claims of the present invention.

Claims (8)

1. A method for boundary-guided polyp image segmentation, comprising:
constructing a polyp image segmentation model;
inputting the polyp image to be segmented into the polyp image segmentation model to obtain region prediction maps P output by each sideR4-PR1And a boundary prediction map PB4-PB1And upsampling to the same resolution as the polyp image to be segmented to obtain a region prediction map
Figure FDA0003539687610000011
And boundary prediction map
Figure FDA0003539687610000012
Computing region prediction maps using cross entropy loss functions
Figure FDA0003539687610000013
Boundary prediction graph
Figure FDA0003539687610000014
Errors between the real labeling graphs and the polyp image segmentation model are reversely transmitted to update parameters of the polyp image segmentation model so as to train the polyp image segmentation model;
and inputting the polyp image to be segmented into a trained polyp image segmentation model, normalizing the region prediction graph output by the residual refinement decoder A to (0,1) through a Sigmoid layer, and outputting a final segmentation result after restoring the resolution of the original image.
2. The boundary-guided polyp image segmentation method as set forth in claim 1, wherein the constructing a polyp image segmentation model comprises:
res2Net-50 is used as a backbone network, and the backbone networks are layerB to layerE and are used as side output characteristics Ft(t is 1,2,3 and 4) which are respectively sent to residual error refinement decoders A to C and an initial positioning decoder, wherein t represents the side output stage number of the backbone network, and 1 to 4 correspond to the side output stage numbers of the backbone networks layerB to E;
detecting an initial coarse position of polyp tissue through an initial positioning decoder, wherein the resolution of an output result image of the initial positioning decoder is 1/32 of an original image;
and repairing the lost target part in the initial coarse position of the polyp tissue through residual refinement decoders A-C, and refining the detail of the boundary.
3. The boundary-guided polyp image segmentation method as set forth in claim 2, wherein the operation of the initial positioning decoder comprises:
input features F by initially positioning the decoder4Feeding into a 1 × 1 convolution layer, reducing the number of channels to 64 to obtain the input feature x4The parameter of the convolutional layer is 2048, 64,1 × 1, wherein in _ channel of { in _ channel, out _ channel, k × k } represents the number of channels of the input feature, out _ channel represents the number of channels of the output feature, k represents the size of the convolutional kernel, and a batch normalization layer Batchnorm and a nonlinear active layer ReLU are attached after the convolutional layer;
inputting a feature x4Inputting into a layer division convolution HSC to obtain the boundary characteristics of the polyp tissue
Figure FDA0003539687610000015
Figure FDA0003539687610000016
Then reducing the channel number to 1 through a convolution layer, the convolution parameter is {64,1,3 x 3}, a batch normalization layer BatchNorm and a nonlinear active layer ReLU are attached to the convolution layer, and an initial boundary prediction graph of the initial positioning decoder is obtained
Figure FDA0003539687610000021
Figure FDA0003539687610000022
Where HSC (. cndot.) represents a hierarchical splitting convolution operation, Conv3×3(. cndot.) represents a 3X 3 convolutional layer;
characterizing the boundaries of polyp tissue
Figure FDA0003539687610000023
And input feature x4Splicing along the channel dimension, carrying out boundary-region guided learning, inputting learning region characteristics in 4 convolutional layers, wherein convolutional parameters are {128,64,3 x 3}, {64,64,3 x 3}, and {64,1,3 x 3}, a batch normalization layer BatchNorm and a nonlinear active layer ReLU are attached to the convolutional layers in sequence, and finally outputting a region prediction map of the initial positioning decoder
Figure FDA0003539687610000024
Figure FDA0003539687610000025
Where Cat (-) indicates the stitching operation along the channel dimension.
4. The boundary-guided polyp image segmentation method as set forth in claim 2 or 3, wherein the residual refinement decoders A-C are operated by:
will input the feature F1～F3Inputting a 1 × 1 convolution layer to reduce the number of channels to 64 to obtain the input characteristic xt,(t＝[1,2,3]) The convolution parameters are { in _ channel, 64,1 × 1}, and a batch normalization layer Batchnorm and a nonlinear active layer ReLU are added after the convolution layer and then sent to the boundary branch and the region branch respectively for residual correction.
5. The boundary-guided polyp image segmentation method as set forth in claim 4, wherein the entering the boundary branch for residual correction comprises:
in boundary branching, deep region prediction
Figure FDA0003539687610000026
Upsampling to and input feature xtSame resolution, again with input features xtPhase splicing to obtain boundary features
Figure FDA0003539687610000027
Figure FDA0003539687610000028
Characterizing the boundary
Figure FDA0003539687610000029
Inputting a convolution layer to reduce the channel number to 1, the convolution parameter is {64,1,3 x 3}, the convolution layer is attached with a batch normalization layer BatchNorm and a nonlinear active layer ReLU to obtain the boundary prediction graph output by the decoder
Figure FDA00035396876100000210
Figure FDA00035396876100000211
Wherein Up (-) denotes an upsampling operation, Conv3×3(. cndot.) represents a 3X 3 convolutional layer.
6. The boundary-guided polyp image segmentation method as set forth in claim 5, wherein the entering region branch residual correction includes:
in zone branching, zone characteristics are obtained by a reverse attention mechanism module
Figure FDA00035396876100000212
Then, the boundary characteristics are related to the obtained data
Figure FDA00035396876100000314
Splicing along channel dimension to obtain region features after boundary features are fused
Figure FDA0003539687610000031
Figure FDA0003539687610000032
By four convolutional layer pairs
Figure FDA0003539687610000033
Residual error learning is carried out, convolution parameters are {128,64,3 × 3}, {64,64,3 × 3}, and {64,1,3 × 3}, a batch normalization layer BatchNorm and a nonlinear active layer ReLU are attached to the first three convolutional layers, the number of channels of the last convolutional layer is reduced to 1, and the result and the upsampled deep layer region prediction layer region are obtained
Figure FDA0003539687610000034
Adding to obtain the area prediction graph output by the residual error refinement decoders A-C
Figure FDA0003539687610000035
Figure FDA0003539687610000036
7. The boundary-guided polyp image segmentation method according to claim 5 or 6, comprising: the stratified fission convolutional HSCs are:
step 1: will input the feature x4Divided into 6 in channel dimensionSplit group of each of [ x1,x2,x3,x4,x5,x6]Wherein x is1-x5Each having 11 channels, x6There are 9 channels, specifically denoted as:
[x1,x2,x3,x4,x5,x6]＝Split(x4)
wherein Split () means splitting along the channel dimension;
step 2: grouping the first split into x1Feeding expansion rate of R1The expansion convolution of (d) yields an output characteristic d1After the expansion convolution, a batch normalization layer BatchNorm and a nonlinear activation layer ReLU are attached, and the output characteristic d1The specific calculation formula is as follows:
Figure FDA0003539687610000037
wherein, the first and the second end of the pipe are connected with each other,
Figure FDA0003539687610000038
denotes an expansion ratio R1The expansion rates of the expansion convolutions used by the 1 st to 5 th split groups are Rk＝[1,2,4,6,8],(k＝1,2,3,4,5)；
And 3, step 3: will output the characteristic d1Divided into two groups along the channel dimension, denoted d1,1And d1,2There are 6 channels and 5 channels, respectively, specifically expressed as:
d1,1，d1,2＝Split(d1)
will d1,2With the next split group x2Splicing along channel dimension to obtain new split group
Figure FDA0003539687610000039
The concrete expression is as follows:
Figure FDA00035396876100000310
wherein
Figure FDA00035396876100000311
As a new split group
Will be provided with
Figure FDA00035396876100000312
Inputting another expansion rate of R2The expansion convolution is added with a batch normalization layer BatchNorm and a nonlinear activation layer ReLU to obtain an output characteristic d2There are 16 channels, and the specific calculation formula is:
Figure FDA00035396876100000313
and 4, step 4: will output the characteristic d2Are divided into two groups, respectively denoted as d2,1And d2,2There are 8 channels and 8 channels, respectively, specifically expressed as:
d2,1，d2,2＝Split(d2)
will d2,2With the next split group x3Splicing along the channel dimension to obtain a new split group
Figure FDA0003539687610000041
The concrete expression is as follows:
Figure FDA0003539687610000042
will be provided with
Figure FDA0003539687610000043
Inputting another expansion rate of R3Is convolved to obtain an output characteristic d3There are 19 channels, and the specific calculation formula is:
Figure FDA0003539687610000044
and 5: repeating the steps 2 to 3 for a plurality of times until all the split groups are finished, finally splicing all the split groups to recover the channel number 64 to obtain the boundary characteristics of the polyp tissue
Figure FDA0003539687610000045
Figure FDA0003539687610000046
Wherein x is6The last split group is indicated.
8. The boundary-guided polyp image segmentation method as set forth in claim 7, wherein the reverse attention mechanism module comprises:
predicting deep regions
Figure FDA0003539687610000047
Upsampling to and inputting feature xt(t＝[1,2,3]) The same resolution is input, a Sigmoid layer is input to be normalized to (0,1), and then the normalization is subtracted from 1 to generate an inverse weight graph
Figure FDA0003539687610000048
Figure FDA0003539687610000049
Inverse weight map
Figure FDA00035396876100000410
Multiplying by regional characteristics
Figure FDA00035396876100000411
Each of which isTrack, obtaining weighted regional characteristics
Figure FDA00035396876100000412
Figure FDA00035396876100000413
Wherein the content of the first and second substances,
Figure FDA00035396876100000414
representing regional characteristics
Figure FDA00035396876100000415
The c-th channel of (1).
CN202210228391.7A 2022-03-10 2022-03-10 Polyp image segmentation method based on boundary guidance Pending CN114612662A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210228391.7A CN114612662A (en) 2022-03-10 2022-03-10 Polyp image segmentation method based on boundary guidance

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210228391.7A CN114612662A (en) 2022-03-10 2022-03-10 Polyp image segmentation method based on boundary guidance

Publications (1)

Publication Number Publication Date
CN114612662A true CN114612662A (en) 2022-06-10

Family

ID=81861910

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210228391.7A Pending CN114612662A (en) 2022-03-10 2022-03-10 Polyp image segmentation method based on boundary guidance

Country Status (1)

Country Link
CN (1) CN114612662A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117197166A (en) * 2023-11-06 2023-12-08 中南大学 Polyp image segmentation method and imaging method based on edge and neighborhood information
CN117830226A (en) * 2023-12-05 2024-04-05 广州恒沙云科技有限公司 Boundary constraint-based polyp segmentation method and system

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117197166A (en) * 2023-11-06 2023-12-08 中南大学 Polyp image segmentation method and imaging method based on edge and neighborhood information
CN117197166B (en) * 2023-11-06 2024-02-06 中南大学 Polyp image segmentation method and imaging method based on edge and neighborhood information
CN117830226A (en) * 2023-12-05 2024-04-05 广州恒沙云科技有限公司 Boundary constraint-based polyp segmentation method and system

Similar Documents

Publication Publication Date Title
CN113674253B (en) Automatic segmentation method for rectal cancer CT image based on U-transducer
WO2022001623A1 (en) Image processing method and apparatus based on artificial intelligence, and device and storage medium
CN111028242A (en) Automatic tumor segmentation system and method and electronic equipment
CN112102321A (en) Focal image segmentation method and system based on deep convolutional neural network
CN114612662A (en) Polyp image segmentation method based on boundary guidance
CN112288731A (en) Blood vessel stenosis analysis method and device
CN112884788B (en) Cup optic disk segmentation method and imaging method based on rich context network
CN113205524B (en) Blood vessel image segmentation method, device and equipment based on U-Net
CN113436173A (en) Abdomen multi-organ segmentation modeling and segmentation method and system based on edge perception
CN117132774B (en) Multi-scale polyp segmentation method and system based on PVT
CN112396605A (en) Network training method and device, image recognition method and electronic equipment
CN112561877B (en) Multi-scale double-channel convolution model training method, image processing method and device
Ta et al. Ble-net: boundary learning and enhancement network for polyp segmentation
CN112529930A (en) Context learning medical image segmentation method based on focus fusion
CN115512110A (en) Medical image tumor segmentation method related to cross-modal attention mechanism
CN112037212A (en) Pulmonary tuberculosis DR image identification method based on deep learning
CN116563533A (en) Medical image segmentation method and system based on target position priori information
CN113435469A (en) Kidney tumor enhanced CT image automatic identification system based on deep learning and training method thereof
CN116912829A (en) Small air channel segmentation method and device, electronic equipment and nonvolatile storage medium
CN113379770B (en) Construction method of nasopharyngeal carcinoma MR image segmentation network, image segmentation method and device
CN113989269B (en) Traditional Chinese medicine tongue image tooth trace automatic detection method based on convolutional neural network multi-scale feature fusion
CN115760868A (en) Colorectal and colorectal cancer segmentation method, system, device and medium based on topology perception
CN114998582A (en) Coronary artery blood vessel segmentation method, device and storage medium
CN114882282A (en) Neural network prediction method for colorectal cancer treatment effect based on MRI and CT images
CN113205454A (en) Segmentation model establishing and segmenting method and device based on multi-scale feature extraction

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination