CN115294086A - Medical image segmentation method, segmentation model training method, medium, and electronic device - Google Patents

Medical image segmentation method, segmentation model training method, medium, and electronic device Download PDF

Info

Publication number
CN115294086A
CN115294086A CN202210988644.0A CN202210988644A CN115294086A CN 115294086 A CN115294086 A CN 115294086A CN 202210988644 A CN202210988644 A CN 202210988644A CN 115294086 A CN115294086 A CN 115294086A
Authority
CN
China
Prior art keywords
image
mask
support
query
medical image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210988644.0A
Other languages
Chinese (zh)
Inventor
王润泽
周芹
郑国焱
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Jiaotong University
Original Assignee
Shanghai Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Jiaotong University filed Critical Shanghai Jiaotong University
Priority to CN202210988644.0A priority Critical patent/CN115294086A/en
Publication of CN115294086A publication Critical patent/CN115294086A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10088—Magnetic resonance imaging [MRI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • Databases & Information Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Data Mining & Analysis (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a medical image segmentation method, a segmentation model training method, a medium and an electronic device. The medical image segmentation method comprises the following steps: acquiring a target image to be segmented; acquiring at least one image from the target image as a first image, and acquiring at least two other images from the target image as second images; acquiring a marking mask of the first image; and processing the first image, the labeling mask thereof and the second image by using a medical image segmentation model to obtain a segmentation result of the second image, wherein the medical image segmentation model is a trained twin deep learning network. The medical image segmentation method does not need a pixel-level artificial labeling mask and has good generalization capability.

Description

Medical image segmentation method, segmentation model training method, medium, and electronic device
Technical Field
The present invention relates to a medical image processing method, and more particularly, to a medical image segmentation method, a segmentation model training method, a medium, and an electronic device.
Background
Abdominal magnetic resonance imaging is a common imaging modality for detecting abdominal diseases. The accurate segmentation of the abdominal organs is the basis of qualitative and quantitative analysis of abdominal related diseases, and can provide assistance for clinical medical treatment in multiple links such as disease screening, surgical planning and postoperative evaluation. In recent years, the medical image segmentation technology is mostly realized by adopting a deep convolutional neural network technology. However, the deep convolutional neural network technology depends on large-scale expert-level accurate artificial mask labeling, and such artificial labeling at a pixel level needs to consume a lot of time and energy, and is difficult to popularize in a large area in specific applications.
Disclosure of Invention
In view of the above-mentioned shortcomings of the prior art, an object of the present invention is to provide a medical image segmentation method, a segmentation model training method, a medium and an electronic device, which are used to solve the problem that the prior art relies on large-scale accurate manual mask labeling.
To achieve the above and other related objects, a first aspect of the present invention provides a medical image segmentation method, including: acquiring a target image to be segmented; acquiring at least one image from the target image as a first image, and acquiring at least two other images from the target image as second images; acquiring a marking mask of the first image; and processing the first image, the labeling mask thereof and the second image by using a medical image segmentation model to obtain a segmentation result of the second image, wherein the medical image segmentation model is a trained twin deep learning network.
In an embodiment of the first aspect, the number of the first images is less than the number of the second images.
In an embodiment of the first aspect, the processing the first image, the labeled mask thereof, and the second image includes: scaling the first image, the marking mask thereof and the second image so as to enable the sizes of the first image, the marking mask and the second image to be matched with the medical image segmentation model; processing the first image, the labeling mask thereof and the second image by using the medical image segmentation model to obtain a prediction mask of the second image; and carrying out scaling processing on the predicted mask of the second image so as to enable the size of the predicted mask of the second image to be matched with the original size of the second image.
In an embodiment of the first aspect, the method for training the medical image segmentation model includes: acquiring a training medical image; acquiring a support image, a support mask, a query image and a query mask according to the training medical image, wherein the support mask is a mask of a target object in the support image, and the query mask is a mask of the target object in the query image; processing the support image and the query image by using the medical image segmentation model to obtain a support feature and a query feature; training the medical image segmentation model according to the support features, the query features, the support mask and the query mask.
In an embodiment of the first aspect, acquiring a support image, a support mask, a query image and a query mask from the training medical image comprises: acquiring at least two images from the training medical image as the support images; acquiring a mask of the target object in the support image as the support mask; the same transformation is performed on the support image and the support mask to obtain the query image and the query mask.
In an embodiment of the first aspect, training the medical image segmentation model according to the support feature, the query feature, the support mask, and the query mask includes: training the medical image segmentation model by using a contrast learning strategy and a prototype learning strategy, wherein a loss function used in the training is determined by the support features, the query features, the support mask and the query mask.
In an embodiment of the first aspect, the object to be segmented in the target image is of the same type as or different from the target object.
A second aspect of the present invention provides a training method for a medical image segmentation model, the training method comprising: acquiring a training medical image; acquiring a support image, a support mask, a query image and a query mask according to the training medical image, wherein the support mask is a mask of a target object in the support image, and the query mask is a mask of the target object in the query image; processing the support image and the query image by using a medical image segmentation model to obtain a support feature and a query feature; training the medical image segmentation model according to the support features, the query features, the support mask and the query mask.
A third aspect of the present invention provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, implements the medical image segmentation method according to the first aspect of the present invention and/or the training method according to the second aspect of the present invention.
A fourth aspect of the present invention provides an electronic apparatus, comprising: a memory storing a computer program; a processor, communicatively coupled to the memory, for executing the medical image segmentation method according to the first aspect of the present invention and/or the training method according to the second aspect of the present invention when the computer program is invoked.
As described above, the medical image segmentation method, the segmentation model training method, the medium, and the electronic device according to the embodiments of the present invention have the following advantages:
the medical image segmentation method uses a first image, a marking mask of the first image and a second image to be segmented as input data of a twin deep learning network, and the input data are processed through the twin deep learning network to obtain a segmentation result of the second image. The method does not need manual mask marking at the pixel level, so that the method can be popularized in a large area in specific application. In addition, the method can realize accurate segmentation on new objects which do not appear in the training process, and has good generalization capability.
Drawings
Fig. 1A is a flowchart illustrating a medical image segmentation method according to an embodiment of the present invention.
Fig. 1B is a diagram illustrating a medical image segmentation method according to an embodiment of the invention.
FIG. 2 is a flowchart illustrating processing of a first image, a mark mask and a second image according to an embodiment of the present invention.
Fig. 3A is a flowchart illustrating a method for training a medical image segmentation model according to an embodiment of the present invention.
Fig. 3B is a diagram illustrating an example of a training process of a medical image segmentation model according to an embodiment of the present invention.
FIG. 4 is a flow chart illustrating the acquisition of a query image and a query mask in an embodiment of the present invention.
Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the invention.
Description of the element reference numerals
500. Electronic device
510. Memory device
520. Processor with a memory having a plurality of memory cells
530. Display device
S11 to S14
S21 to S23
S31 to S34 steps
S41 to S43 steps
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the drawings only show the components related to the present invention rather than being drawn according to the number, shape and size of the components in actual implementation, and the type, number and proportion of the components in actual implementation may be changed arbitrarily, and the layout of the components may be more complicated. Moreover, in this document, relational terms such as "first," "second," and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
In recent years, the medical image segmentation technology is mostly realized by adopting a deep convolutional neural network technology. However, the deep convolutional neural network technology relies on large-scale expert-level accurate artificial mask labeling, and the artificial labeling at the pixel level needs to consume a large amount of time and energy and is difficult to popularize in a large area in specific applications.
At least in view of the above problems, the present invention provides a medical image segmentation method. The medical image segmentation method uses a first image, a label mask of the first image and a second image to be segmented as input data of a twin deep learning network, and the input data are processed through the twin deep learning network to obtain a segmentation result of the second image. The method does not need a pixel-level manual marking mask, so that the method can be popularized in a large area in specific application. In addition, the method can realize accurate segmentation on new objects which do not appear in the training process, and has good generalization capability.
The medical image segmentation method provided by the invention will be described by means of specific embodiments in conjunction with the accompanying drawings.
Fig. 1A is a flowchart illustrating a medical image segmentation method according to an embodiment of the invention. As shown in fig. 1A, the medical image segmentation method includes the following steps S11 to S14.
S11, a target image to be segmented is acquired, which may be, for example, an abdominal magnetic resonance image of a patient.
And S12, acquiring at least one image from the target images as a first image, and acquiring at least two other images from the target images as a second image. Specifically, if the target image has N total images, N target images may be selected as the first image, and the remaining N-N target images may be selected as the second image, but the invention is not limited thereto. Wherein N is less than N and both are positive integers.
Preferably, the number of first images is smaller than the number of second images. It is further preferred that the number of first images is much smaller than the number of second images, for example, the number of first images may be several or tens of images, and the number of second images may be several thousands or tens of thousands or more.
S13, acquiring the marking mask of the first image. The marking mask of the first image can be obtained in a manual marking mode or in other modes. The mask is a template of an image filter, and is used for controlling the image processing area or the processing process. In some embodiments, the annotation mask of the first image may be a binary image consisting of 0 and 1.
And S14, processing the first image, the labeling mask and the second image by using the medical image segmentation model to obtain a segmentation result of the second image. Wherein, the medical image segmentation model is a well-trained twin deep learning network. Fig. 1B is a diagram illustrating an exemplary mask for acquiring a second image by using a medical image segmentation model according to the present embodiment. As shown in the figure, the first image and the second image are input into a medical image segmentation model, the image features of the first image and the second image are obtained by using the model, a prototype vector is obtained according to the image features of the first image and the labeled mask of the first image, and a predicted mask of the second image is obtained according to the prototype vector and the image features of the second image.
As can be seen from the above description, in the medical image segmentation method provided in this embodiment, the first image, the label mask thereof, and the second image to be segmented are used as input data of the twin deep learning network, and the input data is processed by the twin deep learning network to obtain a segmentation result of the second image. The method does not need a pixel-level manual marking mask, so that the method can be popularized in a large area in specific application. In addition, the method can realize accurate segmentation on new objects which do not appear in the training process, and has good generalization capability.
Referring to fig. 2, in an embodiment of the invention, processing the first image, the labeled mask thereof and the second image includes steps S21 to S23.
S21, the first image, the labeling mask and the second image are scaled so that the sizes of the first image, the labeling mask and the second image are matched with the medical image segmentation model. For example, the first image and the second image may be scaled to M1 × M2 pixel size using quadratic linear interpolation, and the labeling mask of the first image may be scaled to M1 × M2 pixel size using nearest neighbor interpolation. Where M1 and M2 are positive integers, and both values are 256, for example.
And S22, processing the first image, the labeling mask and the second image by using the medical image segmentation model to obtain a prediction mask of the second image.
And S23, performing scaling processing on the predicted mask of the second image so that the size of the predicted mask of the second image is matched with the original size of the second image. Specifically, the scaling processing in steps S23 and S21 are mutually reverse processes. For example, if the second video is reduced at the ratio of K:1 in step S21, the ratio of 1: the scale of K amplifies the prediction mask of the second image. Wherein K >1.
In this embodiment, the first image, the labeling mask thereof, and the second image are scaled, so that the sizes of the first image, the labeling mask thereof, and the second image are matched with the input size of the medical image segmentation model, thereby facilitating the medical image segmentation model to process data.
Fig. 3A is a flowchart illustrating a method for training a medical image segmentation model according to an embodiment of the invention. Fig. 3B is a schematic diagram illustrating a training process of the medical image segmentation model in this embodiment. As shown in fig. 3A, the training method of the medical image segmentation model in the present embodiment includes the following steps S31 to S34.
And S31, acquiring a training medical image. The training medical image and the target image may contain the same organ or different organs. The training medical image in this embodiment may be, for example, a two-dimensional slice image of an abdominal magnetic resonance image.
S32, acquiring a support image, a support mask, a query image and a query mask according to the training medical image, wherein the support mask is a mask supporting the target object in the image, and the query mask is a mask querying the target object in the image.
Alternatively, the type of the object to be segmented in the target image and the type of the target object in the training medical image may be the same or different. For example, in some embodiments, the object to be segmented in the target image and the target object in the training medical image may both be a liver. In other embodiments, the object to be segmented in the target image may be the spleen, and the target object in the training medical image may be the kidney.
And S33, processing the support image and the query image by using the medical image segmentation model to acquire the support characteristic and the query characteristic. Wherein the support features refer to image features derived from the support image, and the query features refer to image features derived from the query image.
And S34, training the medical image segmentation model according to the support features, the inquiry features, the support mask and the inquiry mask.
Fig. 4 is a flowchart illustrating the acquisition of a support image, a support mask, a query image, and a query mask according to the training medical image in the present embodiment. As shown in fig. 4, the above process includes the following steps S41 to S43.
S41, at least two images are obtained from the training medical images and used as support images. Optionally, after the support image is acquired, in this embodiment, a method such as two-dimensional linear interpolation may be further adopted to scale the size of the support image to match the input size of the medical image segmentation model.
And S42, acquiring a mask of the target object in the support image as a support mask. Optionally, for any support image a, the support image a may be processed by the super-pixel algorithm in step S42 to generate one or more pseudo masks, and any one of the pseudo masks may be used as the support mask of the support image a.
S43, carrying out the same transformation on the support image and the support mask to obtain a query image and a query mask. Specifically, for any support image B, in step S43, geometric transformation, such as rotation, inversion, translation, affine transformation, or the like, may be performed on the support image B to obtain the query image, and the same geometric transformation may be performed on the support mask of the support image B to obtain the query mask.
Optionally, in this embodiment, training the medical image segmentation model according to the support feature, the query feature, the support mask, and the query mask includes: the medical image segmentation model is trained by adopting a contrast learning strategy and a prototype learning strategy, and a loss function adopted in the training is determined by a support feature, a query feature, a support mask and a query mask.
If the support image is I s Query image is I q In step S33, the support image I can be processed s And query image I q Inputting a medical image segmentation model to obtain support features
Figure BDA0003803003600000061
And query feature
Figure BDA0003803003600000062
Wherein D, H and W represent the channel depth, length and width, respectively, of the feature map.
In contrast learning, the support feature F (I) s ) And a query feature F (I) q ) Respectively using support-based mask M s And query mask M q The mask averaging pooling extracts foreground-related and background-related features. Specifically, in the present embodiment, the foreground-related feature f extracted from the support feature s And foreground-related features f extracted on query features q Respectively as follows:
Figure BDA0003803003600000071
wherein H is less than or equal to H, W is less than or equal to W, F (I) s ) (h, w) represents a support feature at the feature point (h, w), M s (h, w) denotes a support mask at the feature point (h, w), F (I) q ) (h, w) represents a query feature at the feature point (h, w), M q (h, w) represents the query mask at feature point (h, w). In addition, a similar approach may be used in this embodiment to obtain context-dependent features on the support feature and the query feature.
If (t) u ,t v ) Is a pair of features obtained in the training of the medical image segmentation model when t is u And t v When belonging to the same category, e.g. both foreground-related or background-related features, regardless of t u And t v From the support feature or the query feature, both are considered positive feature pairs, and otherwise are considered negative feature pairs. Based on the above, the following loss function can be used to implement contrast learning in this embodiment:
Figure BDA0003803003600000072
wherein, for any two variables a and b,
Figure BDA0003803003600000073
r represents the total number of foreground-related features and background-related features, 1 (t) u ,t w ) τ is an empirical value, which can range, for example, to a positive number less than 0.1, for positive and negative feature pairs, respectively, taking values of 0 and 1. According to l (t) u ,t v ) The contrast loss function L can be obtained c ：
Figure BDA0003803003600000074
Wherein B (R, 2) represents the number of combinations.
In prototype learning, feature F (I) may be supported s ) Prototype vectors for the foreground and background classes are computed. Taking the foreground category as an example, the size (L) can be used in this embodiment H ,L W ) The average pooling window of (a) processes the supported features to obtain a size of
Figure BDA0003803003600000075
The size of the corresponding support mask is scaled to
Figure BDA0003803003600000076
Furthermore, on the processed support characteristics, each foreground pixel point is regarded as a prototype vector
Figure BDA0003803003600000077
Similarly, a prototype vector of the background category can be obtained
Figure BDA0003803003600000078
A prototype vector set can be obtained according to the prototype vectors of the foreground category and the prototype vectors of the background category
Figure BDA0003803003600000079
Wherein
Figure BDA00038030036000000710
And N c Number of prototype vectors, c, representing background class and foreground class, respectively 0 And c represents the background and foreground, respectively. According to the similarity measurement of the prototype vector set and the query features, the matching of the prototype vectors and the query features can be realized, and further, the segmentation of the query image is realized. The similarity calculation formula of each prototype vector and the query feature is as follows:
Figure BDA0003803003600000081
wherein the content of the first and second substances,
Figure BDA0003803003600000082
the similarity of the prototype vector representing the foreground class to the query feature,
Figure BDA0003803003600000083
the prototype vector representing the context category is similar to the query feature. The value of mu can be set according to practical experience and is used for assisting in realizing gradient back propagation in the training process. Based on the similarity
Figure BDA0003803003600000084
And
Figure BDA0003803003600000085
the method can respectively obtain the probability vectors of the softmax function at each characteristic point (h, w) by stacking the similarity measurement maps of the background and the foreground and calculating
Figure BDA0003803003600000086
And V q,c (h, w), the specific formula is:
Figure BDA0003803003600000087
according to the two probability vectors
Figure BDA0003803003600000088
And V q,c (h, w), and the weighted average of the similarity measures of all prototype vectors, the similarity of the background and foreground classes, respectively, can be obtained
Figure BDA0003803003600000089
And S q,c (h, w), the specific calculation formula is:
Figure BDA00038030036000000810
according to
Figure BDA00038030036000000811
And S q,c (h, w) the final prediction segmentation probability can be obtained, and the specific calculation formula is as follows:
Figure BDA00038030036000000812
wherein the content of the first and second substances,
Figure BDA00038030036000000813
optionally, in this embodiment, a cross-entropy loss function may be used to supervise the training process, where the calculation formula of the cross-entropy loss function is:
Figure BDA00038030036000000814
wherein when
Figure BDA00038030036000000815
Time of flight
Figure BDA00038030036000000816
A query mask that represents a background category,
Figure BDA00038030036000000817
time of flight
Figure BDA00038030036000000818
Query masks representing foreground categories.
Further, in this embodiment, a prototype vector and a support feature may be used to perform segmentation prediction based on similarity matching, and a support mask is used as a training supervision to further enhance the feature expression capability of the twin deep learning network. By adopting
Figure BDA00038030036000000819
And S q,c (h, w) in a similar manner, the similarity between the background and foreground categories can be calculated on the support features according to the embodiment
Figure BDA00038030036000000820
And S s,c (h, w) from which further segmentation probabilities on the support image can be derived
Figure BDA00038030036000000821
In particular, the amount of the solvent to be used,
Figure BDA00038030036000000822
the corresponding cross entropy loss function is:
Figure BDA0003803003600000091
according to the above-mentioned loss function L q 、L s And L c The total loss function L = L can be obtained q +λ 1 ×L s +λ 2 ×L c The total loss function L can be used for training a medical image segmentation model. Wherein λ is 1 And λ 2 The values of the two parameters, which correspond to the weighting parameters of the loss function, can be set according to actual requirements, and can be positive numbers less than or equal to 1, for example.
According to the above description, in the meta-learning training stage, the comparative learning optimization twin deep learning network is adopted in the embodiment, so that the same-class features in the generated support and query features are aggregated, and the heterogeneous features are far away. In addition, the embodiment also adopts a prototype learning optimization twin deep learning network, the prototype vectors generated by the support features and the support masks simultaneously calculate the similarity with the support features and the query features and generate corresponding predictions, and the loss function is calculated with the corresponding support masks and the query masks to train the network. In this way, good training performance can be achieved.
The present invention also provides a computer-readable storage medium having a computer program stored thereon. The computer program, when executed by a processor, implements a medical image segmentation method provided in an embodiment of the present invention, and/or implements a training method of a medical image segmentation model provided in an embodiment of the present invention.
Any combination of one or more storage media may be employed in the present invention. The storage medium may be a computer-readable signal medium or a computer-readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The invention also provides electronic equipment. Fig. 5 is a schematic structural diagram of an electronic device 500 according to an embodiment of the invention. As shown in fig. 5, the electronic device 500 in this embodiment includes a memory 510 and a processor 520.
The memory 510 is used to store computer programs; preferably, the memory 510 includes: various media that can store program codes, such as ROM, RAM, magnetic disk, U-disk, memory card, or optical disk.
In particular, memory 510 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) and/or cache memory. The electronic device 500 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. Memory 510 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
The processor 520 is connected to the memory 510 for executing the computer program stored in the memory 510 to make the electronic device 500 execute the medical image segmentation method provided in the embodiment of the present invention and/or execute the training method of the medical image segmentation model provided in the embodiment of the present invention.
Preferably, the Processor 520 may be a general-purpose Processor including a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, or discrete hardware components.
Preferably, the electronic device 500 in this embodiment may further include a display 530. A display 530 is communicatively coupled to the memory 510 and the processor 520 for displaying GUI interactive interfaces associated with the medical image segmentation method and/or the training method of the medical image segmentation model.
The scope of the method for segmenting medical images and/or the method for training the segmentation model of medical images according to the present invention is not limited to the order of executing steps listed in this embodiment, and all the solutions implemented by adding, subtracting, and replacing steps in the prior art according to the principles of the present invention are included in the scope of the present invention.
In summary, embodiments of the present invention provide a medical image segmentation method, a segmentation model training method, a medium, and an electronic device. According to the medical image segmentation method, the twin deep learning network has the capability of distinguishing similarities and differences through the meta-learning training strategy of small sample learning. During prediction, a small number of first images and corresponding labeling masks of the organ types to be segmented are selected, so that the network can segment the same type areas in the second images to obtain the corresponding prediction masks, and the purpose of segmenting the unseen types by using a small number of manual labeling is achieved. The method has high segmentation accuracy and good generalization capability. Therefore, the invention effectively overcomes various defects in the prior art and has high industrial utilization value.
The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.

Claims (10)

1. A medical image segmentation method, characterized in that the medical image segmentation method comprises:
acquiring a target image to be segmented;
acquiring at least one image from the target image as a first image, and acquiring at least two other images from the target image as second images;
acquiring a marking mask of the first image;
and processing the first image, the labeling mask thereof and the second image by using a medical image segmentation model to obtain a segmentation result of the second image, wherein the medical image segmentation model is a trained twin deep learning network.
2. The method of claim 1, wherein the number of the first images is less than the number of the second images.
3. The method of claim 1, wherein processing the first image and its labeling mask and the second image comprises:
scaling the first image, the marking mask thereof and the second image so as to enable the sizes of the first image, the marking mask and the second image to be matched with the medical image segmentation model;
processing the first image, the labeling mask thereof and the second image by using the medical image segmentation model to obtain a prediction mask of the second image;
and carrying out scaling processing on the predicted mask of the second image so as to enable the size of the predicted mask of the second image to be matched with the original size of the second image.
4. The medical image segmentation method according to claim 1, wherein the training method of the medical image segmentation model comprises:
acquiring a training medical image;
acquiring a support image, a support mask, a query image and a query mask according to the training medical image, wherein the support mask is a mask of a target object in the support image, and the query mask is a mask of the target object in the query image;
processing the support image and the query image by using the medical image segmentation model to obtain a support feature and a query feature;
training the medical image segmentation model according to the support features, the query features, the support mask and the query mask.
5. The medical image segmentation method of claim 4, wherein acquiring a support image, a support mask, a query image, and a query mask from the training medical image comprises:
acquiring at least two images from the training medical image as the support images;
acquiring a mask of the target object in the support image as the support mask;
the same transformation is performed on the support image and the support mask to obtain the query image and the query mask.
6. The medical image segmentation method of claim 4, wherein training the medical image segmentation model based on the support features, the query features, the support mask, and the query mask comprises:
training the medical image segmentation model by using a contrast learning strategy and a prototype learning strategy, wherein a loss function used in the training is determined by the support features, the query features, the support mask and the query mask.
7. The medical image segmentation method according to claim 4, wherein the type of the object to be segmented in the target image is the same as or different from that of the target object.
8. A training method of a medical image segmentation model is characterized by comprising the following steps:
acquiring a training medical image;
acquiring a support image, a support mask, a query image and a query mask according to the training medical image, wherein the support mask is a mask of a target object in the support image, and the query mask is a mask of the target object in the query image;
processing the support image and the query image by using a medical image segmentation model to obtain a support feature and a query feature;
training the medical image segmentation model according to the support features, the query features, the support mask and the query mask.
9. A computer-readable storage medium having stored thereon a computer program, characterized in that: the computer program, when executed by a processor, implements a medical image segmentation method according to any one of claims 1 to 7 and/or a training method according to claim 8.
10. An electronic device, characterized in that the electronic device comprises:
a memory storing a computer program;
a processor, communicatively connected to the memory, for executing the medical image segmentation method according to any one of claims 1 to 7 and/or the training method according to claim 8 when the computer program is invoked.
CN202210988644.0A 2022-08-17 2022-08-17 Medical image segmentation method, segmentation model training method, medium, and electronic device Pending CN115294086A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210988644.0A CN115294086A (en) 2022-08-17 2022-08-17 Medical image segmentation method, segmentation model training method, medium, and electronic device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210988644.0A CN115294086A (en) 2022-08-17 2022-08-17 Medical image segmentation method, segmentation model training method, medium, and electronic device

Publications (1)

Publication Number Publication Date
CN115294086A true CN115294086A (en) 2022-11-04

Family

ID=83829542

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210988644.0A Pending CN115294086A (en) 2022-08-17 2022-08-17 Medical image segmentation method, segmentation model training method, medium, and electronic device

Country Status (1)

Country Link
CN (1) CN115294086A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116486196A (en) * 2023-03-17 2023-07-25 哈尔滨工业大学（深圳） Focus segmentation model training method, focus segmentation method and apparatus
CN116664602A (en) * 2023-07-26 2023-08-29 中南大学 OCTA blood vessel segmentation method and imaging method based on few sample learning

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116486196A (en) * 2023-03-17 2023-07-25 哈尔滨工业大学（深圳） Focus segmentation model training method, focus segmentation method and apparatus
CN116486196B (en) * 2023-03-17 2024-01-23 哈尔滨工业大学（深圳） Focus segmentation model training method, focus segmentation method and apparatus
CN116664602A (en) * 2023-07-26 2023-08-29 中南大学 OCTA blood vessel segmentation method and imaging method based on few sample learning
CN116664602B (en) * 2023-07-26 2023-11-03 中南大学 OCTA blood vessel segmentation method and imaging method based on few sample learning

Similar Documents

Publication Publication Date Title
CN107506761B (en) Brain image segmentation method and system based on significance learning convolutional neural network
CN109949276B (en) Lymph node detection method for improving SegNet segmentation network
CN115294086A (en) Medical image segmentation method, segmentation model training method, medium, and electronic device
CN113129309B (en) Medical image semi-supervised segmentation system based on object context consistency constraint
CN111968138B (en) Medical image segmentation method based on 3D dynamic edge insensitivity loss function
CN111144486B (en) Heart nuclear magnetic resonance image key point detection method based on convolutional neural network
Wang et al. Blind single image super-resolution with a mixture of deep networks
CN113177592B (en) Image segmentation method and device, computer equipment and storage medium
Wang et al. A generalizable and robust deep learning algorithm for mitosis detection in multicenter breast histopathological images
CN113706562B (en) Image segmentation method, device and system and cell segmentation method
CN115359074A (en) Image segmentation and training method and device based on hyper-voxel clustering and prototype optimization
Wang et al. Medical matting: a new perspective on medical segmentation with uncertainty
CN117437423A (en) Weak supervision medical image segmentation method and device based on SAM collaborative learning and cross-layer feature aggregation enhancement
Wang et al. Cross-domain nuclei detection in histopathology images using graph-based nuclei feature alignment
Jin et al. Density-based one-shot active learning for image segmentation
Chenni et al. Patch clustering for representation of histopathology images
Gu et al. Endoscopic single image super-resolution based on transformer and convolutional neural network
Huang et al. Medical image super-resolution based on the generative adversarial network
CN114240844B (en) Unsupervised key point positioning and target detection method in medical image
CN113111879B (en) Cell detection method and system
Jia et al. A mix-supervised unified framework for salient object detection
Salsabili et al. Multiresolution semantic segmentation of biological structures in digital histopathology
CN113889233A (en) Cell positioning and counting method based on manifold regression network and application
Ma et al. A pathology image diagnosis network with visual interpretability and structured diagnostic report
CN111932557A (en) Image semantic segmentation method and device based on ensemble learning and probability map model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination