WO2021218215A1 - Image detection method and relevant model training method, relevant apparatuses, and device - Google Patents

Image detection method and relevant model training method, relevant apparatuses, and device Download PDF

Info

Publication number
WO2021218215A1
WO2021218215A1 PCT/CN2020/140325 CN2020140325W WO2021218215A1 WO 2021218215 A1 WO2021218215 A1 WO 2021218215A1 CN 2020140325 W CN2020140325 W CN 2020140325W WO 2021218215 A1 WO2021218215 A1 WO 2021218215A1
Authority
WO
WIPO (PCT)
Prior art keywords
detection model
image
organ
medical image
original
Prior art date
Application number
PCT/CN2020/140325
Other languages
French (fr)
Chinese (zh)
Inventor
黄锐
胡志强
张少霆
李鸿升
Original Assignee
上海商汤智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海商汤智能科技有限公司 filed Critical 上海商汤智能科技有限公司
Priority to KR1020217043241A priority Critical patent/KR20220016213A/en
Priority to JP2021576932A priority patent/JP2022538137A/en
Publication of WO2021218215A1 publication Critical patent/WO2021218215A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration using local operators
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10088—Magnetic resonance imaging [MRI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the present disclosure relates to the field of artificial intelligence technology, and in particular to an image detection method and related model training method and related devices and equipment.
  • Medical images such as CT (Computed Tomography) and MRI (Magnetic Resonance Imaging, MRI scan) have important clinical significance.
  • CT Computer Tomography
  • MRI Magnetic Resonance Imaging, MRI scan
  • multi-organ detection on medical images such as CT and MRI to determine the region corresponding to each organ on the medical image
  • training an image detection model suitable for multi-organ detection has high application value.
  • model training relies on a large number of labeled data sets.
  • obtaining a large number of high-quality multi-organ annotations is very time-consuming and labor-intensive, and usually only experienced radiologists have the ability to annotate data.
  • the existing image detection models often have the problem of low accuracy when performing multi-organ detection. In view of this, how to improve the accuracy of detection in multi-organ detection has become an urgent problem to be solved.
  • the present disclosure provides an image detection method, a training method of related models, and related devices and equipment.
  • an embodiment of the present disclosure provides a method for training an image detection model, including: obtaining a sample medical image, wherein the sample medical image pseudo-labels at least one actual region of an unlabeled organ; The image is detected to obtain a first detection result, where the first detection result includes the first predicted region of the unlabeled organ; and the sample medical image is detected using the image detection model to obtain the second detection result, where the second detection The results include the second prediction area of the unlabeled organ.
  • the network parameters of the image detection model are determined based on the network parameters of the original detection model; the differences between the first prediction area and the actual area and the second prediction area are used to adjust the original detection The network parameters of the model.
  • the sample medical image pseudo-labels the actual area of at least one unlabeled organ, there is no need to actually label multiple organs in the sample medical image
  • the original detection model is used to detect the sample medical image to obtain Contain the first detection result of the first preset area of the unlabeled organ, and use the image detection model to detect the sample medical image to obtain the second detection result of the second prediction area containing the unlabeled organ, and then use the first prediction area
  • the original detection model includes a first original detection model and a second original detection model
  • the image detection model includes a first image detection model corresponding to the first original detection model and a second image detection model corresponding to the second original detection model
  • Using the original detection model to detect the sample medical image to obtain the first detection result includes: using the first original detection model and the second original detection model to perform the step of detecting the sample medical image to obtain the first detection result; using the image The detection model detects the sample medical image to obtain the second detection result, including: using the first image detection model and the second image detection model to perform the step of detecting the sample medical image to obtain the second detection result; using the first prediction Adjust the network parameters of the original detection model based on the differences between the regions and the actual region and the second prediction region, including: using the first prediction region of the first original detection model to compare the second prediction of the actual region and the second image detection model Adjust the network parameters of the first original detection model; and adjust the difference between the first prediction area of the second original detection model and the actual area and the second prediction area of the
  • the original detection model is set to include the first original detection model and the second original detection model
  • the image detection model is set to include the first image detection model corresponding to the first original detection model and the image detection model corresponding to the second original detection model.
  • the second image detection model, and the first original detection model and the second original detection model are used to perform the step of detecting the sample medical image to obtain the first detection result
  • the first image detection model and the second detection model are used respectively to execute
  • the step of detecting the sample medical image to obtain the second detection result so as to use the difference between the first prediction area of the first original detection model and the actual area and the second prediction area of the second image detection model to adjust the first
  • the network parameters of the original detection model, and the difference between the first prediction area of the second original detection model and the actual area and the second prediction area of the first image detection model are used to adjust the network parameters of the second original detection model.
  • the first image detection model corresponding to the first original detection model can be used to supervise the training of the second original detection model
  • the second image detection model corresponding to the second original detection model can be used to supervise the training of the first original detection model. Constrain the cumulative error of the network parameters due to the pseudo-labeled real area during multiple training processes, and improve the accuracy of the image detection model.
  • using the difference between the first prediction area and the actual area and the second prediction area to adjust the network parameters of the original detection model includes: using the difference between the first prediction area and the actual area to determine the first prediction of the original detection model. Loss value; and, using the difference between the first prediction region and the second prediction region, determine the second loss value of the original detection model; use the first loss value and the second loss value to adjust the network parameters of the original detection model.
  • the first loss value of the original detection model is determined by the difference between the first prediction region and the actual region
  • the second loss value of the original detection model is determined by the difference between the first prediction region and the second prediction region
  • the two dimensions of the difference between the second prediction regions are used to measure the loss of the original detection model, which is helpful to improve the accuracy of loss calculation, which can help improve the accuracy of the network parameters of the original detection model, and thus can help improve the image detection model. Accuracy.
  • using the difference between the first prediction area and the actual area to determine the first loss value of the original detection model includes at least one of the following: using a focus loss function to process the first prediction area and the actual area to obtain the first focus loss Value; the first prediction area and the actual area are processed using the ensemble similarity loss function to obtain the first loss value of the ensemble similarity.
  • using the difference between the first prediction region and the second prediction region to determine the second loss value of the original detection model includes: using a consistency loss function to process the first prediction region and the second prediction region to obtain the second loss value.
  • using the first loss value and the second loss value to adjust the network parameters of the original detection model includes: weighting the first loss value and the second loss value to obtain the weighted loss value; using the weighted loss value to adjust the original detection model Network parameters.
  • the model can increase the focus on difficult samples, which can help improve the accuracy of the image detection model;
  • the ensemble similarity loss function processes the first prediction area and the actual area to obtain the first loss value of the ensemble similarity, which can make the model fit the pseudo-labeled actual area, which can help improve the accuracy of the image detection model;
  • the consistency loss function processes the first prediction area and the second prediction area to obtain the second loss value, which can improve the prediction consistency of the original model and the image detection model, and thus can help improve the accuracy of the image detection model; Perform weighting processing on the first loss value and the second loss value to obtain the weighted loss value, and use the weighted loss value to adjust the network parameters of the original detection model, which can balance the importance of each loss value in the training process, thereby improving the network
  • the accuracy of the parameters can help improve the accuracy of the image detection model.
  • the sample medical image also contains the actual area of the marked organ
  • the first detection result also includes the first prediction area of the marked organ
  • the second detection result also includes the second prediction area of the marked organ
  • using the first prediction area Determine the first loss value of the original detection model based on the difference between the actual area and the original detection model, including: using the difference between the first prediction area and the actual area of the unlabeled organ and the labeled organ to determine the first loss value of the original detection model
  • Using the difference between the first prediction area and the second prediction area to determine the second loss value of the original detection model, including: using the difference between the first prediction area of the unlabeled organ and the corresponding second prediction area to determine the original Check the second loss value of the model.
  • the second detection result also includes the second prediction region of the labeled organ
  • the difference between the first prediction area and the actual area is comprehensively considered.
  • the difference between the prediction area and the corresponding second prediction area can improve the robustness of the consistency constraints of the original detection model and the image detection model, and thus can improve the accuracy of the image detection model.
  • the original detection model after adjusting the network parameters of the original detection model by using the differences between the first prediction area and the actual area and the second prediction area, it also includes: using the network parameters adjusted during this training and several previous trainings to correct The network parameters of the image detection model are updated.
  • the network parameters can be further constrained due to pseudo-labeled real regions during multiple training sessions.
  • the resulting cumulative error improves the accuracy of the image detection model.
  • the network parameters adjusted during this training and several previous trainings to update the network parameters of the image detection model, including: statistics the average of the network parameters adjusted by the original detection model during this training and several previous trainings Value; update the network parameters of the image detection model to the average value of the network parameters of the corresponding original detection model.
  • acquiring the sample medical image includes: acquiring a medical image to be pseudo-labeled, wherein at least one unlabeled organ exists in the medical image to be pseudo-labeled; and detecting the pseudo-labeled medical image using a single-organ detection model corresponding to each unlabeled organ. , To obtain the organ prediction area of each unlabeled organ; pseudo-label the organ prediction area of the unlabeled organ as the actual area of the unlabeled organ, and use the pseudo-labeled medical image to be pseudo-labeled as the sample medical image.
  • the single-organ detection model can be used to avoid the manual labeling of multiple organs. Workload, which can help reduce the labor cost of training an image detection model for multi-organ detection, and improve the efficiency of training.
  • the medical image to be pseudo-labeled includes at least one labeled organ; before the single-organ detection model corresponding to each unlabeled organ is used to detect the pseudo-labeled medical image, the method further includes: using the medical image to be pseudo-labeled, Annotate the single-organ detection model corresponding to the annotated organ in the medical image for training.
  • the medical image to be pseudo-labeled including at least one labeled organ in the medical image to be pseudo-labeled, and using the medical image to be pseudo-labeled to train the single-organ detection model corresponding to the labeled organ in the pseudo-labeled medical image can improve the accuracy of the single-organ detection model. Therefore, it can help improve the accuracy of subsequent pseudo-labeling, and in turn, can help improve the accuracy of the subsequent training image detection model.
  • acquiring a medical image to be pseudo-labeled includes: acquiring a three-dimensional medical image and preprocessing the three-dimensional medical image; and performing cropping processing on the pre-processed three-dimensional medical image to obtain at least one two-dimensional medical image to be pseudo-labeled.
  • the pre-processed three-dimensional medical images are cropped to obtain at least one two-dimensional medical image to be pseudo-labeled, which can help to obtain a model training Medical images can help improve the accuracy of subsequent image detection model training.
  • the preprocessing of the three-dimensional medical image includes at least one of the following: adjusting the voxel resolution of the three-dimensional medical image to a preset resolution; using a preset window value to normalize the voxel value of the three-dimensional medical image to Within a preset range; Gaussian noise is added to at least part of the voxels of the three-dimensional medical image.
  • adjusting the voxel resolution of the 3D medical image to a preset resolution can facilitate subsequent model prediction processing; using the preset window value to normalize the voxel value of the 3D medical image to a preset range can be It is helpful for the model to extract accurate features; adding Gaussian noise to at least part of the voxels of the three-dimensional medical image can help achieve data augmentation, increase data diversity, and improve the accuracy of subsequent model training.
  • the embodiments of the present disclosure provide an image detection method, including: acquiring a medical image to be detected, wherein the medical image to be detected contains multiple organs; and using an image detection model to detect the medicine to be detected to obtain multiple organs The prediction area; wherein, the image detection model is obtained by using the training method of the image detection model in the first aspect.
  • the detection accuracy can be improved in the process of multiple organ detection.
  • an embodiment of the present disclosure provides a training device for an image detection model, including an image acquisition module, a first detection module, a second detection module, and a parameter adjustment module.
  • the image acquisition module is configured to acquire sample medical images, wherein , The sample medical image pseudo-labels the actual area of at least one unlabeled organ;
  • the first detection module is configured to use the original detection model to detect the sample medical image to obtain the first detection result, wherein the first detection result includes the unlabeled organ
  • the second detection module is configured to use the image detection model to detect the sample medical image to obtain the second detection result, and the network parameters of the image detection model are determined based on the network parameters of the original detection model, wherein ,
  • the second detection result includes a second prediction area of the unlabeled organ;
  • the parameter adjustment module is configured to adjust the network parameters of the original detection model by using the difference between the first prediction area and the actual area and the second prediction area, respectively.
  • an embodiment of the present disclosure provides an image detection device, including an image acquisition module and an image detection module, the image acquisition module is configured to acquire a medical image to be detected, wherein the medical image to be detected contains multiple organs; the image The detection module is configured to use the image detection model to detect the medicine to be detected to obtain the predicted regions of multiple organs; wherein the image detection model is obtained by training using the image detection model training device in the second aspect.
  • embodiments of the present disclosure provide an electronic device including a memory and a processor coupled to each other.
  • the processor is configured to execute program instructions stored in the memory to implement the image detection model in the first aspect. Training method, or implement the image detection method in the second aspect.
  • embodiments of the present disclosure provide a computer-readable storage medium on which program instructions are stored.
  • the program instructions are executed by a processor, the training method of the image detection model in the first aspect is realized, or the first aspect is realized.
  • the image detection method in the second aspect is realized.
  • the embodiments of the present disclosure also provide a computer program, including computer-readable code.
  • the processor in the electronic device executes the above-mentioned first aspect.
  • the sample medical image is acquired, and the sample medical image is pseudo-labeled with at least one actual region of an unlabeled organ, so there is no need to actually label multiple organs in the sample medical image, and the original detection model is used to detect the sample medical image.
  • the difference between the area and the actual area and the second predicted area, adjust the network parameters of the original detection model, and the network parameters of the image detection model are determined based on the network parameters of the original detection model, so the image detection model can supervise the original detection
  • the training of the model can constrain the cumulative error of the network parameters due to the pseudo-labeled real area during multiple training processes, and improve the accuracy of the image detection model, so that the image detection model can accurately supervise the training of the original detection model.
  • the original detection model can accurately adjust its network parameters during the training process. Therefore, the detection accuracy of the image detection model can be improved in the process of multi-organ detection.
  • FIG. 1 is a schematic flowchart of an embodiment of a training method for an image detection model provided by an embodiment of the present disclosure
  • FIG. 2 is a schematic flowchart of an embodiment of step S11 in FIG. 1;
  • FIG. 3 is a schematic flowchart of another embodiment of a training method for an image detection model provided by an embodiment of the present disclosure
  • FIG. 4 is a schematic diagram of an embodiment of the training process of an image detection model provided by an embodiment of the present disclosure
  • FIG. 5 is a schematic flowchart of an embodiment of an image detection method provided by an embodiment of the present disclosure
  • FIG. 6 is a schematic diagram of the framework of an embodiment of an image detection model training apparatus provided by an embodiment of the present disclosure
  • FIG. 7 is a schematic diagram of a framework of an embodiment of an image detection device provided by an embodiment of the present disclosure.
  • FIG. 8 is a schematic diagram of a framework of an embodiment of an electronic device provided by an embodiment of the present disclosure.
  • FIG. 9 is a schematic framework diagram of an embodiment of a computer-readable storage medium provided by an embodiment of the present disclosure.
  • system and "network” in this article are often used interchangeably in this article.
  • the term “and/or” in this article is only an association relationship describing the associated objects, which means that there can be three relationships, for example, A and/or B, which can mean: A alone exists, A and B exist at the same time, exist alone B these three situations.
  • the character "/” in this text generally indicates that the associated objects before and after are in an "or” relationship.
  • "many” in this document means two or more than two.
  • FIG. 1 is a schematic flowchart of an embodiment of a method for training an image detection model provided by an embodiment of the present disclosure. Among them, the following steps can be included:
  • Step S11 Obtain a sample medical image, where the sample medical image pseudo-labels at least one actual region of an unlabeled organ.
  • the sample medical images may include CT images and MR images, which are not limited here.
  • the sample medical image can be obtained by scanning the abdomen, chest, head, etc., and can be set according to actual application conditions, which is not limited here.
  • the organs in the sample medical image may include: kidney, spleen, liver, pancreas, etc.; or, when the chest is scanned, the organs in the sample medical image may include: heart, lung lobes, thyroid, etc.; or,
  • the head is scanned, and the organs in the sample medical image can include: brain stem, cerebellum, diencephalon, and telencephalon.
  • the actual area of the unlabeled organ may be detected by using a single-organ detection model corresponding to the unlabeled organ.
  • the unlabeled organ may include: At least one of the kidney, spleen, liver, and pancreas can use the single organ detection model corresponding to the kidney to detect the sample medical image to obtain the organ prediction area corresponding to the kidney, and the single organ detection model corresponding to the spleen can be used Detect the sample medical image to obtain the organ prediction area corresponding to the spleen, and use the single organ detection model corresponding to the liver to detect the sample medical image to obtain the organ prediction area corresponding to the liver, and use the single organ detection corresponding to the pancreas
  • the model detects the sample medical image and obtains the organ prediction region corresponding to the pancreas, so that the organ prediction regions corresponding to the kidney, spleen, liver, and pancreas are pseudo-labeled in the sample medical image,
  • pseudo-labeling refers to the process of taking the organ prediction regions of unlabeled organs detected by the single-organ detection model as the actual regions.
  • the organ is not marked as other organs, it can be deduced by analogy, and we will not give examples one by one here.
  • the single-organ detection model for unlabeled organs is trained using a single-organ data set labeled with the actual region of the unlabeled organ.
  • the single-organ detection model corresponding to the kidney uses the labeled kidney
  • the kidney data set of the actual area is trained, and the single-organ detection model corresponding to the spleen is trained using the spleen data set of the actual area marked with the spleen.
  • Step S12 Use the original detection model to detect the sample medical image to obtain a first detection result, where the first detection result includes a first prediction region of an unlabeled organ.
  • the original detection model can include any one of Mask R-CNN (Mask Region with Convolutional Neural Network), FCN (Fully Convolutional Network), PSP-net (Pyramid Scene Parsing Network, pyramid scene analysis network),
  • the original detection model can also be set-net, U-net, etc., which can be set according to the actual situation, which is not limited here.
  • the first detection result of the first prediction region containing the unlabeled organ can be obtained.
  • the sample medical image is an image obtained by scanning the abdomen.
  • the unlabeled organs include the kidney, spleen, and pancreas. Therefore, the original detection model is used to detect the sample medical image, and the first prediction area of the kidney and the first prediction area of the spleen can be obtained.
  • the first prediction area of the pancreas, and other scenarios can be deduced by analogy, so I won’t give an example one by one here.
  • Step S13 Use the image detection model to detect the sample medical image to obtain a second detection result, where the second detection result includes a second prediction region of an unlabeled organ.
  • the network structure of the original detection model and the network structure of the image detection model corresponding to the original detection model may be the same.
  • the corresponding image detection model can also be Mask R-CNN; or, when the original detection model is FCN, the corresponding image detection model can also be FCN; Or, when the original detection model is PSP-net, the corresponding image detection model can also be PSP-net; when the original detection model is another network, the analogy can be used, and no examples are given here.
  • the network parameters of the image detection model may be determined based on the network parameters of the original detection model.
  • the network parameters of the image detection model may be obtained based on the network parameters adjusted by the original detection model in multiple training processes.
  • the network parameters of the image detection model can be obtained by using the network parameters adjusted by the original detection model from the knth to the k-1th training process; or, in the k+th
  • the network parameters of the image detection model can be obtained by using the network parameters adjusted by the original detection model from the k+1-nth to the kth training process, and so on.
  • the number of times (ie, n) of the foregoing multiple trainings can be set according to actual conditions, for example, it can be set to 5, 10, 15, etc., which are not limited here.
  • the second detection result of the second prediction region containing the unlabeled organ can be obtained.
  • the unlabeled organs include the kidney, spleen, and pancreas. Therefore, the image detection model is used to detect the sample medical image, and the second prediction area of the kidney and the second prediction area of the spleen can be obtained.
  • the prediction area, the second prediction area of the pancreas, and other scenarios can be deduced by analogy, so we will not give examples one by one here.
  • the above steps S12 and S13 may be performed in a sequential order, for example, step S12 is performed first, and then step S13; or, step S13 is performed first, and then step S12 is performed.
  • the above step S12 and step S13 can also be performed at the same time, and can be set according to actual applications, which is not limited here.
  • Step S14 Use the differences between the first prediction area and the actual area and the second prediction area to adjust the network parameters of the original detection model.
  • the difference between the first prediction area and the actual area can be used to determine the first loss value of the original detection model.
  • the focal loss function can be used to process the first prediction area and the actual area to obtain the first focal loss value; or, in order to be able to make the model fit pseudo-labeled In the actual area, the first prediction area and the actual area can also be processed by using the dice loss function to obtain the first loss value of the dice loss.
  • the difference between the first prediction area and the second prediction area can also be used to determine the second loss value of the original detection model.
  • the consistency loss function can be used to process the first prediction area and the second prediction area to obtain the second loss value.
  • the performance loss function can be a cross-entropy loss function, which can be set according to actual application conditions, and is not limited here.
  • the above-mentioned first loss value and second loss value can also be used to adjust the network parameters of the original detection model.
  • the first loss value and the second loss value can be weighted to obtain a weighted loss value, so that the weighted loss value can be used to adjust the network parameters of the original detection model.
  • the weights corresponding to the first loss value and the second loss value can be set according to the actual situation, for example, both are set to 0.5; or, the weight corresponding to the first loss value is set to 0.6, and the weight corresponding to the second loss value is set Set to 0.4, which is not limited here.
  • the first loss value includes the first loss value of the focus and the first loss value of the set similarity
  • the first loss value of the focus, the first loss value of the set similarity, and the second loss value can be weighted to obtain The weighted loss value is used to adjust the network parameters of the original detection model.
  • Stochastic Gradient Descent (SGD), Batch Gradient Descent (BGD), Mini-Batch Gradient Descent (MBGD), etc. can be used, and weighted The loss value adjusts the network parameters of the original detection model.
  • batch gradient descent refers to the use of all samples for parameter updates during each iteration; stochastic gradient descent refers to the use of one during each iteration Samples are used to update parameters; mini-batch gradient descent refers to using a batch of samples to update parameters during each iteration, which will not be repeated here.
  • the sample medical image may also include the actual area of the marked organ
  • the first detection result may also include the first prediction area of the marked organ
  • the second detection result may also include the second area of the marked organ. Forecast area.
  • the unlabeled organs include the kidney, spleen, and pancreas
  • the labeled organs include the liver. Therefore, the original detection model is used to detect the sample medical image, and the corresponding kidneys of the unlabeled organs can be obtained.
  • the first prediction area corresponding to the unlabeled organ spleen, the first prediction area corresponding to the unlabeled organ pancreas, and the first prediction area corresponding to the labeled organ liver, and the image detection model corresponding to the original detection model is used Detecting the sample medical image can obtain the second prediction area corresponding to the unlabeled organ kidney, the second prediction area corresponding to the unlabeled organ spleen, the second prediction area corresponding to the unlabeled organ pancreas, and the second prediction area corresponding to the labeled organ liver. Forecast area.
  • the difference between the first prediction region and the actual region of the unlabeled organ and the labeled organ can be used to determine the first loss value of the original detection model, and the difference between the first prediction region of the unlabeled organ and the corresponding second prediction region can be used.
  • the difference between the two can determine the second loss value of the original detection model.
  • the unlabeled organs include the kidney, spleen, and pancreas
  • the labeled organs include the liver. You can use the first prediction area corresponding to the unlabeled organ kidney and the pseudo-labeled actual area.
  • the difference between the first prediction area corresponding to the unlabeled organ spleen and the pseudo-labeled actual area, the difference between the first prediction area corresponding to the unlabeled organ pancreas and the pseudo-labeled actual area, and the labeled organ liver Determine the first loss value of the original detection model according to the difference between the corresponding first prediction area and the actual area marked by the real label.
  • the first loss value may include at least one of the first loss value of focus and the first loss value of set similarity. Alternatively, please refer to the previous steps, which will not be repeated here.
  • the difference between the first prediction region and the second prediction region corresponding to the unlabeled organ kidney, the difference between the first prediction region and the second prediction region corresponding to the spleen of the unlabeled organ, and the pancreas corresponding to the unlabeled organ is determined to determine the second loss value of the original detection model.
  • the second loss value can be calculated by using the cross-entropy loss function. You can refer to the foregoing steps and will not be repeated here. Therefore, in the process of determining the first loss value of the original detection model, the difference between the first prediction area and the actual area is comprehensively considered, and in the process of determining the second loss value of the original detection model, only unlabeled organs are considered.
  • the difference between the first prediction region and the corresponding second prediction region can improve the robustness of the consistency constraint of the original detection model and the image detection model, and thus can improve the accuracy of the image detection model.
  • the network parameters of the image detection model may not be updated, but after a preset number of times (for example, 2 times, 3 times, etc.) training, reuse The network parameters adjusted during this training and several previous trainings will update the network parameters of the image detection model, which is not limited here. For example, during the kth training process, the network parameters of the image detection model may not be updated.
  • the original detection model can be used to train from the k+inth to the k+ith time.
  • i can be set to an integer not less than 1 according to the actual situation, for example, it can be set to 1, 2, 3, etc., which is not limited here.
  • the original detection model in the process of updating the network parameters of the image detection model, can be counted in this training and the average value of the network parameters adjusted by several previous trainings, and then the image detection model The network parameters of is updated to the average value of the network parameters of the corresponding original detection model.
  • the average value of network parameters refers to the average value corresponding to the same network parameter, which may be a certain weight (or bias) corresponding to the same neuron after being adjusted in multiple training processes.
  • the average value of the value, so the average value of each weight (or bias) of each neuron after adjustment in multiple training processes can be obtained by statistics, so as to use the average value to the corresponding weight of the corresponding neuron in the image detection model (Or offset) to update.
  • this training is the kth training, and the average value of the network parameters adjusted by the original detection model during this training and the previous n-1 training can be counted.
  • the value of n can be set according to the actual application, for example , Can be set to 5, 10, 15, etc., which is not limited here.
  • the network parameters of the image detection model are updated using the average value of the adjusted network parameters from the k-n+1 training process to the k training process, so as to be able to It is conducive to quickly constrain the accumulated errors generated in the process of multiple training, and improve the accuracy of the image detection model.
  • a preset training end condition can also be set. If the preset training end condition is not met, the above step S12 and subsequent steps can be re-executed to continue to perform the network parameters of the original detection model. Adjustment.
  • the preset training end conditions may include any of the following: the current number of training times reaches a preset number threshold (eg, 500 times, 1000 times, etc.), and the loss value of the original detection model is less than a preset loss threshold. For one, there is no limitation here.
  • the image detection model can be used to detect the medical image to be tested, so that the regions corresponding to multiple organs in the medical image to be tested can be directly obtained, thereby eliminating the need to use multiple units.
  • Organ detection performs separate detection operations on medical images to be detected, so the amount of detection calculations can be reduced.
  • the sample medical image is acquired, and the sample medical image is pseudo-labeled with at least one actual region of an unlabeled organ, so there is no need to actually label multiple organs in the sample medical image, and the original detection model is used to detect the sample medical image.
  • the difference between the area and the actual area and the second predicted area, adjust the network parameters of the original detection model, and the network parameters of the image detection model are determined by the network parameters of the original detection model, so the image detection model can supervise the original detection
  • the training of the model can constrain the cumulative error of the network parameters due to the pseudo-labeled real area during multiple training processes, and improve the accuracy of the image detection model, so that the image detection model can accurately supervise the training of the original detection model.
  • the original detection model can accurately adjust its network parameters during the training process. Therefore, the detection accuracy of the image detection model can be improved in the process of multi-organ detection.
  • FIG. 2 is a schematic flowchart of an embodiment of step S11 in FIG. 1.
  • FIG. 2 is a schematic diagram of an embodiment of obtaining a sample medical image, which includes the following steps:
  • Step S111 Obtain a medical image to be pseudo-labeled, where at least one unlabeled organ exists in the medical image to be pseudo-labeled.
  • the medical image to be pseudo-labeled can be obtained by scanning the abdomen, the unlabeled organs in the medical image to be pseudo-labeled can include: kidney, spleen, pancreas, etc., and the medical image to be pseudo-labeled can also be obtained by scanning other parts.
  • the chest, head, etc. can refer to the relevant steps in the foregoing embodiment, which is not limited here.
  • the acquired original medical image can be a three-dimensional medical image, for example, a three-dimensional CT image, a three-dimensional MR image, which is not limited here, so the three-dimensional medical image can be preprocessed and the preprocessed
  • the three-dimensional medical image is cropped to obtain at least one medical image to be pseudo-labeled.
  • the cropping process may be center cropping of the preprocessed three-dimensional medical image, which is not limited here.
  • cropping can be performed along a plane parallel to the three-dimensional medical image in the dimensions perpendicular to the plane to obtain a two-dimensional medical image to be pseudo-labeled.
  • the size of the medical image to be pseudo-labeled can be set according to the actual situation, for example, it can be 352*352, which is not limited here.
  • the preprocessing may include adjusting the voxel resolution of the three-dimensional medical image to a preset resolution.
  • the voxel of the 3D medical image is the smallest unit of 3D medical image segmentation in the 3D space.
  • the preset resolution can be 1*1*3mm, and the preset resolution can also be set to other resolutions according to the actual situation, for example, 1*1 *4mm, 2*2*3mm, etc., are not limited here. Adjusting the voxel resolution of the three-dimensional medical image to a preset resolution can facilitate subsequent model prediction processing.
  • the preprocessing may also include using a preset window value to normalize the voxel value of the three-dimensional medical image to a preset range.
  • the voxel value can be a different value depending on the three-dimensional medical image.
  • the voxel value can be a Hu (houns field unit) value.
  • the preset window value can be set according to the part corresponding to the 3D medical image.
  • the preset window value can be set from -125 to 275, and other parts can be set according to the actual situation. , I will not give an example one by one here.
  • the preset range can be set according to the actual application.
  • the preset range can be set from 0 to 1, still taking 3D CT images as an example.
  • the preset window value can be set from -125 to 275.
  • voxels with a voxel value less than or equal to -125 can be reset to a voxel value
  • voxels with a voxel value greater than or equal to 275 can be reset to a voxel uniformly.
  • Voxel value 1 you can reset voxels with voxel values between -125 to 275 to voxel values between 0 and 1, which can help enhance the contrast between different organs in the image, thereby improving the accuracy of the model extraction feature.
  • the preprocessing may also include adding Gaussian noise to at least part of the voxels of the three-dimensional medical image. At least part of the voxels can be set according to actual applications, for example, 1/3 voxels of 3D medical images, or 1/2 voxels of 3D medical images, or all voxels of 3D medical images, which are not limited here. .
  • Gaussian noise By adding Gaussian noise to at least part of the voxels of the three-dimensional medical image, the subsequent two-dimensional medical image to be pseudo-labeled can be cropped on the basis of the three-dimensional medical image and the three-dimensional medical image without Gaussian noise, so it can be beneficial to implementation Data augmentation, increase data diversity, and improve the accuracy of subsequent model training.
  • Step S112 Use the single-organ detection model corresponding to each unlabeled organ to detect the pseudo-labeled medical image to obtain the organ prediction area of each unlabeled organ.
  • the single-organ detection model corresponding to each unlabeled organ may be trained using a single-organ data set labeled with unlabeled organs.
  • the single-organ detection model corresponding to the kidney may be obtained by using a single-organ detection model labeled with kidneys.
  • the single-organ detection model corresponding to the spleen can be trained using the single-organ data set labeled with the spleen, and the other organs can be deduced by analogy, so we will not give an example one by one here.
  • the medical image to be pseudo-labeled may also include at least one labeled organ, and the medical image to be pseudo-labeled including the labeled organ may be used to treat a single organ corresponding to the labeled organ in the pseudo-labeled medical image.
  • the detection model is trained to obtain the corresponding single-organ detection model. For example, if the medical image to be pseudo-labeled includes the labeled liver, the medical image to be pseudo-labeled that includes the labeled liver can be used to train the single-organ detection model corresponding to the liver to obtain the single-organ detection model corresponding to the liver. It can be deduced by analogy, and I will not give examples one by one here.
  • single-organ detection models can include any of Mask R-CNN (Mask Region with Convolutional Neural Network), FCN (Fully Convolutional Network), and PSP-net (Pyramid Scene Parsing Network).
  • Mask R-CNN Mask Region with Convolutional Neural Network
  • FCN Full Convolutional Network
  • PSP-net Pyramid Scene Parsing Network
  • the single-organ detection model can also be set-net, U-net, etc., which can be set according to actual conditions, which is not limited here.
  • the organ prediction area of each unlabeled organ can be obtained.
  • the medical image to be pseudo-labeled is an image obtained by scanning the abdomen as an example
  • the unlabeled organs include the kidney, spleen, and pancreas.
  • the single-organ detection model corresponding to the kidney is used to detect the pseudo-labeled medical image, and the organ prediction area of the kidney can be obtained.
  • the single-organ detection model corresponding to the kidney can be used to detect pseudo-labeled medical images
  • the single-organ detection model corresponding to the spleen can be used to detect pseudo-labeled medical images
  • the use of The single-organ detection model corresponding to the pancreas requires the steps of detecting and detecting pseudo-labeled medical images, and finally uniformly pseudo-labeling the single-organ prediction regions of the kidney, spleen, and pancreas on the medical image to be pseudo-labeled; or, using each of the above
  • the single-organ detection model corresponding to the unlabeled organ can also perform the steps of detecting the pseudo-labeled medical image in sequence, so that it is no longer necessary to pseudo-label the organ prediction region of each unlabeled organ in the pseudo-labeled medical image.
  • the final medical image to be pseudo-labeled can include the single-organ prediction regions of the kidney, spleen, and pancreas. It can be set according to the actual situation and is not limited here.
  • Step S113 pseudo-label the organ prediction region of the unlabeled organ as the actual region of the unlabeled organ, and use the pseudo-labeled medical image to be pseudo-labeled as a sample medical image.
  • the organ prediction region of each unlabeled organ can be pseudo-labeled as the actual region of the unlabeled organ, and the pseudo-labeled medical image to be pseudo-labeled can be used as the sample medical image.
  • the organ prediction area of the non-labeled organ is pseudo-labeled as the actual area of the unlabeled organ, and the pseudo-labeled medical image after the pseudo-labeling is used as the sample medical image.
  • the single-organ detection model can be used to eliminate the need for manual pairing. The workload of organ labeling can help reduce the labor cost of training an image detection model for multi-organ detection and improve the efficiency of training.
  • FIG. 3 is a schematic flowchart of another embodiment of a training method for an image detection model provided by an embodiment of the present disclosure. Among them, the following steps can be included:
  • Step S31 Obtain a sample medical image, where the sample medical image pseudo-labels at least one actual region of an unlabeled organ.
  • step S31 can refer to related steps in the foregoing embodiment.
  • Step S32 using the first original detection model and the second original detection model to perform the step of detecting the sample medical image to obtain the first detection result.
  • the original detection model may include a first original detection model and a second original detection model.
  • the first original detection model can include any of Mask R-CNN (Mask Region with Convolutional Neural Network), FCN (Fully Convolutional Network), PSP-net (Pyramid Scene Parsing Network, pyramid scene analysis network)
  • the first original detection model can also be set-net, U-net, etc., which can be set according to the actual situation, which is not limited here.
  • the second original detection model can include any of Mask R-CNN (Mask Region with Convolutional Neural Network), FCN (Fully Convolutional Network), PSP-net (Pyramid Scene Parsing Network, pyramid scene analysis network)
  • the second original detection model can also be set-net, U-net, etc., which can be set according to the actual situation, which is not limited here.
  • the first detection result detected by the first original detection model may include the first prediction area of the unlabeled organ, or the first detection result detected by the first original detection model may also include the unlabeled organ. The first prediction area and the first prediction area of the labeled organ.
  • the first detection result detected by the second original detection model may include the first prediction region of the unlabeled organ, or the first detection result detected by the second original detection model may also include the unlabeled organ The first prediction area of and the first prediction area of the labeled organ.
  • FIG. 4 is a schematic diagram of an embodiment of the training process of the image detection model.
  • the first original detection model is denoted as net1
  • the second original detection model is denoted as net2.
  • the first original detection model net1 detects the sample medical image, and the first detection result corresponding to the first original detection model net1 is obtained.
  • the second original detection model net2 detects the sample medical image, and obtains the first detection result corresponding to the first original detection model net1. 2.
  • Step S33 using the first image detection model and the second image detection model to perform the step of detecting the sample medical image to obtain the second detection result.
  • the image detection model may include a first image detection model corresponding to the first original detection model and a second image detection model corresponding to the second original detection model, the network structure and network parameters of the first image detection model and the second image detection model You can refer to the relevant steps in the foregoing embodiment, which will not be repeated here.
  • the second detection result detected by the first image detection model may include the second prediction area of the unlabeled organ, or the second detection result detected by the first image detection model may also include the unlabeled organ.
  • the second detection result detected by the second image detection model may include the second prediction area of the unlabeled organ, or the second detection result detected by the second image detection model may also include the unlabeled organ The second prediction area of and the second prediction area of the labeled organ.
  • the first image detection model corresponding to the first original detection model net1 is denoted as EMA net1
  • the second image detection model corresponding to the second original detection model net2 is denoted as EMAnet2.
  • the first image detection model EMAnet1 detects the sample medical image
  • the second detection result corresponding to the first image detection model EMAnet1 is obtained
  • the second image detection model EMAnet2 detects the sample medical image.
  • steps S32 and S33 can be performed in a sequential order, for example, step S32 is performed first, and then step S33 is performed, or step S33 is performed first, and then step S32 is performed.
  • the above step S32 and step S33 can also be performed at the same time, and can be set according to actual applications, which is not limited here.
  • Step S34 Use the differences between the first prediction area of the first original detection model and the actual area and the second prediction area of the second image detection model to adjust the network parameters of the first original detection model.
  • the difference between the first prediction area of the first original detection model and the pseudo-labeled actual area can be used to determine the first loss value of the first original detection model, and the first prediction area of the first original detection model and the pseudo-labeled actual area can be used.
  • the difference between the second prediction regions of the second image detection model determines the second loss value of the first original detection model, so that the first loss value and the second loss value are used to adjust the network parameters of the first original detection model.
  • the calculation methods of the first loss value and the second loss value can refer to the relevant steps in the foregoing embodiment, and will not be repeated here.
  • the process of calculating the second loss value only the first prediction area and the second prediction area of the unlabeled organs can be calculated, so as to improve the consistency between the first original detection model and the second image detection model.
  • the robustness of sexual constraints can in turn improve the accuracy of the image detection model.
  • Step S35 Use the difference between the first prediction area of the second original detection model and the actual area and the second prediction area of the first image detection model to adjust the network parameters of the second original detection model.
  • the difference between the first prediction area of the second original detection model and the pseudo-labeled actual area can be used to determine the first loss value of the second original detection model, and the first prediction area and the pseudo-labeled actual area of the second original detection model can be used.
  • the difference between the second prediction regions of the first image detection model determines the second loss value of the second original detection model, so that the first loss value and the second loss value are used to adjust the network parameters of the second original detection model.
  • the calculation methods of the first loss value and the second loss value can refer to the relevant steps in the foregoing embodiment, and will not be repeated here.
  • only the first prediction area and the second prediction area of the unlabeled organ can be calculated, so as to improve the consistency between the second original detection model and the first image detection model.
  • the robustness of sexual constraints can in turn improve the accuracy of the image detection model.
  • steps S34 and S35 may be performed in a sequential order, for example, step S34 is performed first, and then step S35 is performed, or step S35 is performed first, and then step S34 is performed.
  • the above step S24 and step S35 can also be performed at the same time, and can be set according to actual applications, which is not limited here.
  • Step S36 Utilize the network parameters adjusted during the current training of the first original detection model and several previous trainings to update the network parameters of the first image detection model.
  • the average value of the network parameters adjusted by the first original detection model during this training and several previous trainings can be counted, and the network parameters of the first image detection model can be updated to the corresponding network parameters of the first original detection model. average value.
  • Step S37 The network parameters of the second image detection model are updated by using the network parameters adjusted during the current training of the second original detection model and several previous trainings.
  • the average value of the network parameters adjusted by the second original detection model during this training and several previous trainings can be counted, and the network parameters of the second image detection model can be updated to the corresponding network parameters of the second original detection model. average value.
  • steps S36 and S37 can be performed in a sequential order, for example, step S36 is performed first, and then step S37, or step S37 is performed first, and step S36 is performed later.
  • the above step S36 and step S37 can also be performed at the same time, and can be set according to actual applications, which is not limited here.
  • the above step S32 and subsequent steps can be re-executed to continue Adjust the network parameters of the first original detection model and the second original detection model, and adjust the network parameters of the first image detection model corresponding to the first original detection model and the second image detection model corresponding to the second original detection model
  • the network parameters are updated.
  • the preset training end conditions may include: the current number of training times reaches the preset number threshold (eg, 500 times, 1000 times, etc.), and the loss values of the first original detection model and the second original detection model are less than Any one of a preset loss threshold is not limited here.
  • any one of the first image detection model and the second image detection model can be used as the network model for subsequent image detection, so that the number of medical images to be detected can be directly obtained.
  • the area corresponding to each organ can eliminate the need to use multiple single organs to detect the medical image to be detected separately, so the amount of detection calculation can be reduced.
  • the original detection model is set to include the first original detection model and the second original detection model
  • the image detection model is set to include the first image detection model corresponding to the first original detection model and the second original detection model.
  • Detect the second image detection model corresponding to the model and use the first original detection model and the second original detection model to perform the step of detecting the sample medical image to obtain the first detection result, and use the first image detection model and the first image detection model respectively.
  • the second detection model executes the step of detecting the sample medical image to obtain the second detection result, thereby using the difference between the first prediction area of the first original detection model and the actual area and the second prediction area of the second image detection model.
  • Adjust the network parameters of the first original detection model and use the difference between the first prediction area of the second original detection model and the actual area and the second prediction area of the first image detection model to adjust the second original detection model Network parameters, so the first image detection model corresponding to the first original detection model can be used to supervise the training of the second original detection model, and the second image detection model corresponding to the second original detection model can be used to supervise the training of the first original detection model. Therefore, it is possible to further constrain the cumulative error of the network parameters due to the pseudo-labeled real region during multiple training processes, and improve the accuracy of the image detection model.
  • FIG. 5 is a schematic flowchart of an embodiment of an image detection method provided by an embodiment of the present disclosure. Among them, the following steps can be included:
  • Step S51 Obtain a medical image to be tested, where the medical image to be tested contains multiple organs.
  • the medical images to be detected may include CT images and MR images, which are not limited here.
  • the medical image to be detected can be obtained by scanning the abdomen, chest, head, etc., and can be set according to actual application conditions, which is not limited here.
  • the organs in the medical image to be tested may include: kidney, spleen, liver, pancreas, etc.; or scanning the chest, the organs in the medical image to be tested may include: heart, lung lobes, thyroid, etc.;
  • the head is scanned, and the organs in the medical image to be detected may include: brain stem, cerebellum, diencephalon, and telencephalon.
  • Step S52 Use the image detection model to detect the medicine to be detected to obtain predicted regions of multiple organs.
  • the image detection model is obtained by training using the steps in any of the above-mentioned image detection model training method embodiments. You can refer to the relevant steps in the foregoing embodiment, which will not be repeated here.
  • the image detection model to detect the medical image to be detected, the predicted regions of multiple organs can be directly obtained, and the operation of using multiple single organs to detect the medical image to be detected can be avoided, and the amount of detection calculation can be reduced.
  • the image detection model trained by using the steps in the embodiment of the training method of any of the above-mentioned image detection models detects and detects the medical image to be detected, and obtains the predicted regions of multiple organs, which can improve the detection in the process of multiple organ detection. accuracy.
  • FIG. 6 is a schematic diagram of an embodiment of an image detection model training apparatus provided by an embodiment of the present disclosure.
  • the training device 60 for the image detection model includes an image acquisition module 61, a first detection module 62, a second detection module 63, and a parameter adjustment module 64.
  • the image acquisition module 61 is configured to acquire sample medical images, wherein the sample medical images are pseudo-labeled At least one actual region of an unlabeled organ; the first detection module 62 is configured to use the original detection model to detect the sample medical image to obtain a first detection result, where the first detection result includes the first predicted region of the unlabeled organ; And, the second detection module 63 is configured to use the image detection model to detect the sample medical image to obtain a second detection result, wherein the second detection result includes a second predicted region of an unlabeled organ, and the network parameter of the image detection model is Determined by using the network parameters of the original detection model; the parameter adjustment module 64 is configured to adjust the network parameters of the original detection model by using the differences between the first prediction area and the actual area and the second prediction area, respectively.
  • the sample medical image is acquired, and the sample medical image is pseudo-labeled with at least one actual region of an unlabeled organ, so there is no need to actually label multiple organs in the sample medical image, and the original detection model is used to detect the sample medical image.
  • the difference between the area and the actual area and the second predicted area, adjust the network parameters of the original detection model, and the network parameters of the image detection model are determined by the network parameters of the original detection model, so the image detection model can supervise the original detection
  • the training of the model can constrain the cumulative error of the network parameters due to the pseudo-labeled real area during multiple training processes, and improve the accuracy of the image detection model, so that the image detection model can accurately supervise the training of the original detection model.
  • the original detection model can accurately adjust its network parameters during the training process. Therefore, the detection accuracy of the image detection model can be improved in the process of multi-organ detection.
  • the original detection model includes a first original detection model and a second original detection model
  • the image detection model includes a first image detection model corresponding to the first original detection model and a second image detection model corresponding to the second original detection model.
  • Image detection model the first detection module 62 is also configured to use the first original detection model and the second original detection model to perform the step of detecting the sample medical image to obtain the first detection result
  • the second detection model 63 is also configured
  • the parameter adjustment module 64 is further configured to use the first prediction area of the first original detection model respectively.
  • the difference between the actual area and the second prediction area of the second image detection model is adjusted to adjust the network parameters of the first original detection model.
  • the parameter adjustment module 64 is also configured to use the first prediction area of the second original detection model. The difference between the actual area and the second prediction area of the first image detection model is adjusted to adjust the network parameters of the second original detection model.
  • the original detection model is set to include the first original detection model and the second original detection model
  • the image detection model is set to include the first image detection model corresponding to the first original detection model and the second original detection model.
  • Detect the second image detection model corresponding to the model and use the first original detection model and the second original detection model to perform the step of detecting the sample medical image to obtain the first detection result, and use the first image detection model and the first image detection model respectively.
  • the second detection model performs the step of detecting the sample medical image to obtain the second detection result, thereby using the difference between the first prediction area of the first original detection model and the actual area and the second prediction area of the second image detection model.
  • Adjust the network parameters of the first original detection model and use the difference between the first prediction area of the second original detection model and the actual area and the second prediction area of the first image detection model to adjust the second original detection model Network parameters, so the first image detection model corresponding to the first original detection model can be used to supervise the training of the second original detection model, and the second image detection model corresponding to the second original detection model can be used to supervise the training of the first original detection model. Therefore, it is possible to further constrain the cumulative error of the network parameters due to the pseudo-labeled real area during multiple training processes, and improve the accuracy of the image detection model.
  • the parameter adjustment module 64 includes a first loss determination sub-module configured to use the difference between the first prediction area and the actual area to determine the first loss value of the original detection model, and the parameter adjustment module 64 includes a first loss value.
  • the second loss determination sub-module is configured to use the difference between the first prediction area and the second prediction area to determine the second loss value of the original detection model.
  • the parameter adjustment module 64 includes a parameter adjustment sub-module configured to use the first The loss value and the second loss value adjust the network parameters of the original detection model.
  • the first loss value of the original detection model is determined by the difference between the first prediction area and the actual area, and the difference between the first prediction area and the second prediction area is used to determine the value of the original detection model.
  • the second loss value and use the first loss value and the second loss value to adjust the network parameters of the original detection model, so that the first prediction area predicted from the original detection model can be detected with the pseudo-labeled actual area and the corresponding image respectively.
  • the two dimensions of the difference between the second prediction regions predicted by the model are used to measure the loss of the original detection model, which is conducive to improving the accuracy of the loss calculation, which can help improve the accuracy of the network parameters of the original detection model, which in turn can help Improve the accuracy of the image detection model.
  • the first loss determination submodule includes a focus loss determination unit configured to process the first prediction area and the actual area using a focus loss function to obtain the first focus loss value
  • the first loss determination submodule includes The collective similarity loss determination unit is configured to use the collective similarity loss function to process the first prediction region and the actual region to obtain the first loss value of the collective similarity
  • the second loss determination sub-module is also configured to use the consistency loss The function processes the first prediction area and the second prediction area to obtain the second loss value.
  • the parameter adjustment sub-module includes a weighting processing unit configured to perform weighting processing on the first loss value and the second loss value to obtain the weighted loss value ,
  • the parameter adjustment sub-module includes a parameter adjustment unit configured to adjust the network parameters of the original detection model by using the weighted loss value.
  • the model can increase the focus on difficult samples, which can help improve the accuracy of the image detection model.
  • the first loss value of the collective similarity is obtained, which can make the model fit the pseudo-labeled actual area, which can help improve the accuracy of the image detection model Performance
  • the consistency loss function to process the first prediction area and the second prediction area to obtain the second loss value, which can improve the prediction consistency of the original model and the image detection model, which can further improve the performance of the image detection model
  • Accuracy By weighting the first loss value and the second loss value, the weighted loss value is obtained, and the weighted loss value is used to adjust the network parameters of the original detection model, which can balance the importance of each loss value in the training process. Thereby, the accuracy of the network parameters can be improved, which in turn can help improve the accuracy of the image detection model.
  • the sample medical image further includes the actual region of the labeled organ
  • the first detection result further includes the first prediction region of the labeled organ
  • the second detection result further includes the second prediction region of the labeled organ.
  • the first loss determination submodule is further configured to determine the first loss value of the original detection model by using the difference between the first predicted region and the actual region of the unlabeled organ and the labeled organ
  • the second loss determination submodule is also configured In order to use the difference between the first prediction area of the unlabeled organ and the corresponding second prediction area, the second loss value of the original detection model is determined.
  • the second detection result also includes the second prediction of the marked organ
  • the difference between the first prediction area and the actual area is comprehensively considered, and in the process of determining the second loss value of the original detection model, only the unmarked value is considered
  • the difference between the first prediction region of the organ and the corresponding second prediction region can improve the robustness of the consistency constraint of the original detection model and the image detection model, and thus can improve the accuracy of the image detection model.
  • the training device 60 of the image detection model further includes a parameter update module configured to update the network parameters of the image detection model by using the network parameters adjusted during this training and several previous trainings.
  • the network parameters of the image detection model can be updated by using the network parameters adjusted by the original detection model in this training and several previous trainings, which can further restrict the network parameters in the process of multiple training due to false
  • the cumulative error generated by the marked real area improves the accuracy of the image detection model.
  • the parameter update module includes a statistics sub-module configured to count the average value of the network parameters adjusted by the original detection model during the current training and several previous trainings, and the parameter update module includes an update sub-module configured to Update the network parameters of the image detection model to the average value of the network parameters of the corresponding original detection model.
  • the average value of the network parameters adjusted by the original detection model during the current training and the previous training is counted, and the network parameters of the image detection model are updated to the average value of the network parameters of the corresponding original detection model.
  • the image acquisition module 61 includes an image acquisition sub-module configured to acquire a medical image to be pseudo-labeled, wherein at least one unlabeled organ exists in the medical image to be pseudo-labeled, and the image acquisition module 61 includes a single-organ detection sub-module , Is configured to detect pseudo-labeled medical images using a single-organ detection model corresponding to each unlabeled organ to obtain the organ prediction area of each unlabeled organ.
  • the image acquisition module 61 includes a pseudo-labeled sub-module and is configured In order to pseudo-label the organ prediction region of the unlabeled organ as the actual region of the unlabeled organ, and use the pseudo-labeled medical image to be pseudo-labeled as the sample medical image.
  • the organ prediction area of the non-labeled organ is pseudo-labeled as the actual area of the unlabeled organ, and the pseudo-labeled medical image after the pseudo-labeling is used as the sample medical image.
  • the single-organ detection model can be used to eliminate the need for manual pairing. The workload of organ labeling can help reduce the labor cost of training an image detection model for multi-organ detection and improve the efficiency of training.
  • the medical image to be pseudo-labeled includes at least one labeled organ
  • the image acquisition module 61 further includes a single organ training sub-module configured to use the medical image to be pseudo-labeled to use the labeled organ in the medical image to be pseudo-labeled.
  • the corresponding single-organ detection model is trained.
  • the medical image to be pseudo-labeled includes at least one labeled organ
  • the single-organ detection model corresponding to the labeled organ in the pseudo-labeled medical image is trained by using the medical image to be pseudo-labeled, which can improve the single organ
  • the accuracy of the detection model can thus help to improve the accuracy of subsequent pseudo-labeling, which in turn can help improve the accuracy of the subsequent training image detection model.
  • the image acquisition sub-module includes a three-dimensional image acquisition unit configured to acquire a three-dimensional medical image
  • the image acquisition sub-module includes a pre-processing unit configured to preprocess the three-dimensional medical image
  • the image acquisition sub-module includes an image
  • the cropping unit is configured to perform cropping processing on the preprocessed three-dimensional medical image to obtain at least one two-dimensional medical image to be pseudo-labeled.
  • the pre-processed three-dimensional medical images are cropped to obtain at least one two-dimensional medical image to be pseudo-labeled, which can be beneficial to obtain Medical images that meet model training can help improve the accuracy of subsequent image detection model training.
  • the preprocessing unit is further configured to perform at least one of the following: adjust the voxel resolution of the three-dimensional medical image to a preset resolution; use a preset window value to adjust the voxel value of the three-dimensional medical image Normalize to a preset range; add Gaussian noise to at least part of the voxels of the three-dimensional medical image.
  • adjusting the voxel resolution of the three-dimensional medical image to a preset resolution can facilitate subsequent model prediction processing; the preset window value is used to normalize the voxel value of the three-dimensional medical image to a preset Within the range, it can help the model to extract accurate features; adding Gaussian noise to at least part of the voxels of the three-dimensional medical image can help achieve data augmentation, increase data diversity, and improve the accuracy of subsequent model training.
  • FIG. 7 is a schematic diagram of a framework of an embodiment of an image detection device provided by an embodiment of the present disclosure.
  • the image detection device 70 includes an image acquisition module 71 and an image detection module 72.
  • the image acquisition module 71 is configured to acquire a medical image to be detected, wherein the medical image to be detected contains multiple organs;
  • the image detection module 72 is configured to use image detection
  • the model detects the medicine to be detected to obtain predicted regions of multiple organs; wherein, the image detection model is trained by the training device of the image detection model in any of the above-mentioned image detection model training device embodiments.
  • the image detection model trained by the training device of the image detection model in the embodiment of the training device for any of the above-mentioned image detection models is used for detection and detection of medical images to be detected, and the predicted regions of multiple organs are obtained. In the process, improve the detection accuracy.
  • FIG. 8 is a schematic diagram of a framework of an embodiment of an electronic device provided by an embodiment of the present disclosure.
  • the electronic device 80 includes a memory 81 and a processor 82 that are coupled to each other.
  • the processor 82 is configured to execute program instructions stored in the memory 81 to implement the steps of any of the foregoing image detection model training method embodiments, or implement any of the foregoing. Steps in an embodiment of an image detection method.
  • the electronic device 80 may include but is not limited to: a microcomputer and a server.
  • the electronic device 80 may also include mobile devices such as a notebook computer and a tablet computer, which are not limited herein.
  • the processor 82 is configured to control itself and the memory 81 to implement the steps of any of the foregoing image detection model training method embodiments, or implement the steps of any of the foregoing image detection method embodiments.
  • the processor 82 may also be referred to as a CPU (Central Processing Unit, central processing unit).
  • the processor 82 may be an integrated circuit chip with signal processing capability.
  • the processor 82 may also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (ASIC), a field programmable gate array (Field-Programmable Gate Array, FPGA) or other Programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the processor 82 may be jointly implemented by an integrated circuit chip.
  • the above solution can improve the accuracy of detection in the process of multi-organ detection.
  • FIG. 9 is a schematic framework diagram of an embodiment of a computer-readable storage medium provided by an embodiment of the present disclosure.
  • the computer-readable storage medium 90 stores program instructions 901 that can be executed by the processor.
  • the program instructions 901 are configured to implement the steps of any of the foregoing image detection model training method embodiments, or implement any of the foregoing image detection method embodiments. A step of.
  • the above solution can improve the accuracy of detection in the process of multi-organ detection.
  • the training method of the image detection model or the computer program product of the image detection method provided by the embodiments of the present disclosure includes a computer-readable storage medium storing program code, and the instructions included in the program code can be configured to execute the above method embodiments
  • the training method of the image detection model or the steps of the image detection method described in the above please refer to the above method embodiment, which will not be repeated here.
  • the embodiments of the present disclosure also provide a computer program, which, when executed by a processor, implements any one of the methods in the foregoing embodiments.
  • the computer program product can be implemented by hardware, software or a combination thereof.
  • the computer program product is embodied as a computer storage medium.
  • the computer program product is embodied as a software product, such as a software development kit (SDK) and so on.
  • SDK software development kit
  • the disclosed method and device may be implemented in other ways.
  • the device implementation described above is only illustrative, for example, the division of modules or units is only a logical function division, and there may be other divisions in the actual implementation process, for example, units or components can be combined or It can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of this embodiment.
  • the functional units in the various embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of the present disclosure essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium.
  • a computer device which may be a personal computer, a server, or a network device, etc.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes. .
  • a sample medical image is obtained, and the sample medical image pseudo-labels at least one actual region of an unlabeled organ; the original detection model is used to detect the sample medical image to obtain a first detection including the first predicted region of the unlabeled organ Results; use the image detection model to detect the sample medical image to obtain the second detection result including the second prediction area of the unlabeled organ, the network parameters of the image detection model are determined based on the network parameters of the original detection model; use the first prediction The difference between the area and the actual area and the second prediction area respectively, adjust the network parameters of the original detection model. In this way, the detection accuracy can be improved in the process of multi-organ detection.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

Provided are an image detection method and a relevant model training method, relevant apparatuses, and a device. The image detection model training method comprises: acquiring a sample medical image, an actual region of at least one unlabeled organ being pseudo-labeled on the sample medical image; using an original detection model to detect the sample medical image, so as to obtain a first detection result comprising a first prediction region of the unlabeled organ; using the image detection model to detect the sample medical image to obtain a second detection result comprising a second prediction region of the unlabeled organ, the network parameters of the image detection model being determined on the basis of the network parameters of the original detection model; and adjusting the network parameters of the original detection model by using a difference between the first prediction region and the actual region and a difference between the first prediction region and the second prediction region, respectively.

Description

图像检测方法及相关模型的训练方法和相关装置、设备Image detection method and related model training method and related device and equipment
相关申请的交叉引用Cross-references to related applications
本公开基于申请号为202010362766.X、申请日为2020年04月30日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本公开作为参考。This disclosure is based on a Chinese patent application with an application number of 202010362766.X and an application date of April 30, 2020, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is incorporated herein by reference.
技术领域Technical field
本公开涉及人工智能技术领域，特别是涉及一种图像检测方法及相关模型的训练方法和相关装置、设备。The present disclosure relates to the field of artificial intelligence technology, and in particular to an image detection method and related model training method and related devices and equipment.
背景技术Background technique
CT(Computed Tomography，计算机断层扫描)和MRI(Magnetic Resonance Imaging，核磁共振扫描)等医学图像在临床具有重要意义。其中，在CT、MRI等医学图像上进行多器官检测，以确定医学图像上各器官对应的区域，在临床实践中具有广泛的应用，例如，计算机辅助诊断、放疗计划制定等。故此，训练出适用于多器官检测的图像检测模型具有较高的应用价值。Medical images such as CT (Computed Tomography) and MRI (Magnetic Resonance Imaging, MRI scan) have important clinical significance. Among them, multi-organ detection on medical images such as CT and MRI to determine the region corresponding to each organ on the medical image has a wide range of applications in clinical practice, such as computer-aided diagnosis and radiotherapy planning. Therefore, training an image detection model suitable for multi-organ detection has high application value.
目前，模型训练依赖于大量的具有标注的数据集。然而，在医疗影像领域，获得大量的高质量的多器官标注是非常耗时耗力的，而且通常只有经验丰富的放射科医生才有能力对数据进行标注。受限于此，现有的图像检测模型在进行多器官检测时，往往存在准确性低的问题。有鉴于此，如何在多器官检测时，提高检测准确性成为亟待解决的问题。Currently, model training relies on a large number of labeled data sets. However, in the field of medical imaging, obtaining a large number of high-quality multi-organ annotations is very time-consuming and labor-intensive, and usually only experienced radiologists have the ability to annotate data. Limited by this, the existing image detection models often have the problem of low accuracy when performing multi-organ detection. In view of this, how to improve the accuracy of detection in multi-organ detection has become an urgent problem to be solved.
发明内容Summary of the invention
本公开提供一种图像检测方法及相关模型的训练方法和相关装置、设备。The present disclosure provides an image detection method, a training method of related models, and related devices and equipment.
第一方面，本公开实施例提供了一种图像检测模型的训练方法，包括：获取样本医学图像，其中，样本医学图像伪标注出至少一个未标注器官的实际区域；利用原始检测模型对样本医学图像进行检测以得到第一检测结果，其中，第一检测结果包括未标注器官的第一预测区域；以及，利用图像检测模型对样本医学图像进行检测以得到第二检测结果，其中，第二检测结果包括未标注器官的第二预测区域，图像检测模型的网络参数是基于原始检测模型的网络参数确定的；利用第一预测区域分别与实际区域、第二预测区域之间的差异，调整原始检测模型的网络参数。In a first aspect, an embodiment of the present disclosure provides a method for training an image detection model, including: obtaining a sample medical image, wherein the sample medical image pseudo-labels at least one actual region of an unlabeled organ; The image is detected to obtain a first detection result, where the first detection result includes the first predicted region of the unlabeled organ; and the sample medical image is detected using the image detection model to obtain the second detection result, where the second detection The results include the second prediction area of the unlabeled organ. The network parameters of the image detection model are determined based on the network parameters of the original detection model; the differences between the first prediction area and the actual area and the second prediction area are used to adjust the original detection The network parameters of the model.
因此，通过获取样本医学图像，且样本医学图像伪标注出至少一个未标注器官的实际区域，故样本医学图像中无需对多器官进行真实标注，从而利用原始检测模型对样本医学图像检测检测以得到包含未标注器官的第一预设区域的第一检测结果，并利用图像检测模型对样本医学图像进行检测以得到包含未标注器官的第二预测区域的第二检测结果，进而利用第一预测区域分别与实际区域、第二预测区域之间的差异，调整原始检测模型的网络参数，且图像检测模型的网络参数是基于原始检测模型的网络参数确定的，故能够使得图像检测模型监督原始检测模型的训练，故能够约束网络参数在多次训练过程中由于伪标注的真实区域所产生的累积误差，提高图像检测模型的准确性，从而使得图像检测模型得以准确地监督原始检测模型进行训练，进而使得原始检测模型在训练过程中能够准确地调整其网络参数，故此，能够在多器官检测的过程中，提升图像检测模型的检测准确性。Therefore, by acquiring the sample medical image, and the sample medical image pseudo-labels the actual area of at least one unlabeled organ, there is no need to actually label multiple organs in the sample medical image, and the original detection model is used to detect the sample medical image to obtain Contain the first detection result of the first preset area of the unlabeled organ, and use the image detection model to detect the sample medical image to obtain the second detection result of the second prediction area containing the unlabeled organ, and then use the first prediction area Adjust the network parameters of the original detection model based on the differences between the actual area and the second predicted area, and the network parameters of the image detection model are determined based on the network parameters of the original detection model, so the image detection model can supervise the original detection model Therefore, it is possible to constrain the cumulative error of the network parameters due to the pseudo-labeled real area during the multiple training process, improve the accuracy of the image detection model, so that the image detection model can accurately supervise the training of the original detection model, and then This allows the original detection model to accurately adjust its network parameters during the training process, so that the detection accuracy of the image detection model can be improved in the process of multi-organ detection.
其中，原始检测模型包括第一原始检测模型和第二原始检测模型，图像检测模型包括与第一原始检测模型对应的第一图像检测模型和与第二原始检测模型对应的第二图像检测模型；利用原始检测模型对样本医学图像进行检测以得到第一检测结果，包括：分别利用第一原始检测模型和第二原始检测模型执行对样本医学图像进行检测以得到第一检测结果的步骤；利用图像检测模型对样本医学图像进行检测以得到第二检测结果，包括：分别利用第一图像检测模型和第二图像检测模型执行对样本医学图像进行检测以得到第二检测结果的步骤；利用第一预测区域分别与实际区域、第二预测区域之间的差异，调整原始检测模型的网络参数，包括：利用第一原始检测模型的第一预测区域分别与实际区域、第二图像检测 模型的第二预测区域之间的差异，调整第一原始检测模型的网络参数；以及，利用第二原始检测模型的第一预测区域分别与实际区域、第一图像检测模型的第二预测区域之间的差异，调整第二原始检测模型的网络参数。Wherein, the original detection model includes a first original detection model and a second original detection model, and the image detection model includes a first image detection model corresponding to the first original detection model and a second image detection model corresponding to the second original detection model; Using the original detection model to detect the sample medical image to obtain the first detection result includes: using the first original detection model and the second original detection model to perform the step of detecting the sample medical image to obtain the first detection result; using the image The detection model detects the sample medical image to obtain the second detection result, including: using the first image detection model and the second image detection model to perform the step of detecting the sample medical image to obtain the second detection result; using the first prediction Adjust the network parameters of the original detection model based on the differences between the regions and the actual region and the second prediction region, including: using the first prediction region of the first original detection model to compare the second prediction of the actual region and the second image detection model Adjust the network parameters of the first original detection model; and adjust the difference between the first prediction area of the second original detection model and the actual area and the second prediction area of the first image detection model. The network parameters of the second original detection model.
因此，将原始检测模型设置为包括第一原始检测模型和第二原始检测模型，且图像检测模型设置为包括与第一原始检测模型对应的第一图像检测模型和与第二原始检测模型对应的第二图像检测模型，并分别利用第一原始检测模型和第二原始检测模型执行对样本医学图像进行检测以得到第一检测结果的步骤，并分别利用第一图像检测模型和第二检测模型执行对样本医学图像进行检测以得到第二检测结果的步骤，从而利用第一原始检测模型的第一预测区域分别与实际区域、第二图像检测模型的第二预测区域之间的差异，调整第一原始检测模型的网络参数，并利用第二原始检测模型的第一预测区域分别与实际区域、第一图像检测模型的第二预测区域之间的差异，调整第二原始检测模型的网络参数，故能够利用与第一原始检测模型对应的第一图像检测模型监督第二原始检测模型的训练，利用与第二原始检测模型对应的第二图像检测模型监督第一原始检测模型的训练，故能够进一步约束网络参数在多次训练过程中由于伪标注的真实区域所产生的累积误差，提高图像检测模型的准确性。Therefore, the original detection model is set to include the first original detection model and the second original detection model, and the image detection model is set to include the first image detection model corresponding to the first original detection model and the image detection model corresponding to the second original detection model. The second image detection model, and the first original detection model and the second original detection model are used to perform the step of detecting the sample medical image to obtain the first detection result, and the first image detection model and the second detection model are used respectively to execute The step of detecting the sample medical image to obtain the second detection result, so as to use the difference between the first prediction area of the first original detection model and the actual area and the second prediction area of the second image detection model to adjust the first The network parameters of the original detection model, and the difference between the first prediction area of the second original detection model and the actual area and the second prediction area of the first image detection model are used to adjust the network parameters of the second original detection model. The first image detection model corresponding to the first original detection model can be used to supervise the training of the second original detection model, and the second image detection model corresponding to the second original detection model can be used to supervise the training of the first original detection model. Constrain the cumulative error of the network parameters due to the pseudo-labeled real area during multiple training processes, and improve the accuracy of the image detection model.
其中，利用第一预测区域分别与实际区域、第二预测区域之间的差异，调整原始检测模型的网络参数包括：利用第一预测区域和实际区域之间的差异，确定原始检测模型的第一损失值；以及，利用第一预测区域和第二预测区域之间的差异，确定原始检测模型的第二损失值；利用第一损失值和第二损失值，调整原始检测模型的网络参数。Among them, using the difference between the first prediction area and the actual area and the second prediction area to adjust the network parameters of the original detection model includes: using the difference between the first prediction area and the actual area to determine the first prediction of the original detection model. Loss value; and, using the difference between the first prediction region and the second prediction region, determine the second loss value of the original detection model; use the first loss value and the second loss value to adjust the network parameters of the original detection model.
因此，通过第一预测区域和实际区域之间的差异，确定原始检测模型的第一损失值，并通过第一预测区域和第二预测区域之间的差异，确定原始检测模型的第二损失值，并利用第一损失值和第二损失值，调整原始检测模型的网络参数，从而能够从原始检测模型预测出的第一预测区域分别和伪标注的实际区域、对应的图像检测模型预测出的第二预测区域之间差异这两个维度来度量原始检测模型的损失，有利于提高损失计算的准确性，从而能够有利于提高原始检测模型网络参数的准确性，进而能够有利于提升图像检测模型的准确性。Therefore, the first loss value of the original detection model is determined by the difference between the first prediction region and the actual region, and the second loss value of the original detection model is determined by the difference between the first prediction region and the second prediction region , And use the first loss value and the second loss value to adjust the network parameters of the original detection model, so that the first prediction area predicted from the original detection model can be compared with the pseudo-labeled actual area and the corresponding image detection model. The two dimensions of the difference between the second prediction regions are used to measure the loss of the original detection model, which is helpful to improve the accuracy of loss calculation, which can help improve the accuracy of the network parameters of the original detection model, and thus can help improve the image detection model. Accuracy.
其中，利用第一预测区域和实际区域之间的差异，确定原始检测模型的第一损失值包括以下至少之一：利用焦点损失函数对第一预测区域和实际区域进行处理，得到焦点第一损失值；利用集合相似度损失函数对第一预测区域和实际区域进行处理，得到集合相似度第一损失值。Wherein, using the difference between the first prediction area and the actual area to determine the first loss value of the original detection model includes at least one of the following: using a focus loss function to process the first prediction area and the actual area to obtain the first focus loss Value; the first prediction area and the actual area are processed using the ensemble similarity loss function to obtain the first loss value of the ensemble similarity.
其中，利用第一预测区域和第二预测区域之间的差异，确定原始检测模型的第二损失值包括：利用一致性损失函数对第一预测区域和第二预测区域进行处理，得到第二损失值。Wherein, using the difference between the first prediction region and the second prediction region to determine the second loss value of the original detection model includes: using a consistency loss function to process the first prediction region and the second prediction region to obtain the second loss value.
其中，利用第一损失值和第二损失值，调整原始检测模型的网络参数包括：对第一损失值和第二损失值进行加权处理，得到加权损失值；利用加权损失值，调整原始检测模型的网络参数。Among them, using the first loss value and the second loss value to adjust the network parameters of the original detection model includes: weighting the first loss value and the second loss value to obtain the weighted loss value; using the weighted loss value to adjust the original detection model Network parameters.
因此，通过利用焦点损失函数对第一预测区域和实际区域进行处理，得到焦点第一损失值，能够使得模型提升对于难样本的关注度，从而能够有利于提高图像检测模型的准确性；通过利用集合相似度损失函数对第一预测区域和实际区域进行处理，得到集合相似度第一损失值，能够使得模型拟合伪标注的实际区域，从而能够有利于提高图像检测模型的准确性；通过利用一致性损失函数对第一预测区域和第二预测区域进行处理，得到第二损失值，从而能够提高原始模型和图像检测模型预测的一致性，进而能够有利于提高图像检测模型的准确性；通过对第一损失值和第二损失值进行加权处理，得到加权损失值，并利用加权损失值，调整原始检测模型的网络参数，能够平衡各损失值在训练过程中的重要程度，从而能够提高网络参数的准确性，进而能够有利于提高图像检测模型的准确性。Therefore, by using the focus loss function to process the first prediction area and the actual area to obtain the first loss value of the focus, the model can increase the focus on difficult samples, which can help improve the accuracy of the image detection model; The ensemble similarity loss function processes the first prediction area and the actual area to obtain the first loss value of the ensemble similarity, which can make the model fit the pseudo-labeled actual area, which can help improve the accuracy of the image detection model; The consistency loss function processes the first prediction area and the second prediction area to obtain the second loss value, which can improve the prediction consistency of the original model and the image detection model, and thus can help improve the accuracy of the image detection model; Perform weighting processing on the first loss value and the second loss value to obtain the weighted loss value, and use the weighted loss value to adjust the network parameters of the original detection model, which can balance the importance of each loss value in the training process, thereby improving the network The accuracy of the parameters, in turn, can help improve the accuracy of the image detection model.
其中，样本医学图像中还包含已标注器官的实际区域，第一检测结果还包括已标注器官的第一预测区域，第二检测结果还包括已标注器官的第二预测区域；利用第一预测区域和实际区域之间的差异，确 定原始检测模型的第一损失值，包括：利用未标注器官和已标注器官的第一预测区域和实际区域之间的差异，确定原始检测模型的第一损失值；利用第一预测区域和第二预测区域之间的差异，确定原始检测模型的第二损失值，包括：利用未标注器官的第一预测区域和对应第二预测区域之间的差异，确定原始检测模型的第二损失值。Wherein, the sample medical image also contains the actual area of the marked organ, the first detection result also includes the first prediction area of the marked organ, and the second detection result also includes the second prediction area of the marked organ; using the first prediction area Determine the first loss value of the original detection model based on the difference between the actual area and the original detection model, including: using the difference between the first prediction area and the actual area of the unlabeled organ and the labeled organ to determine the first loss value of the original detection model ; Using the difference between the first prediction area and the second prediction area to determine the second loss value of the original detection model, including: using the difference between the first prediction area of the unlabeled organ and the corresponding second prediction area to determine the original Check the second loss value of the model.
因此，通过在样本医学图像中设置已标注器官的实际区域，且第一检测结果中还包括已标注器官的第一预测区域，第二检测结果还包括已标注器官的第二预测区域，并在确定原始检测模型的第一损失值的过程中，综合考虑第一预测区域和实际区域之间的差异，而在确定原始检测模型的第二损失值的过程中，只考虑未标注器官的第一预测区域和对应的第二预测区域之间的差异，从而能够提升原始检测模型和图像检测模型一致性约束的鲁棒性，进而能够提高图像检测模型的准确性。Therefore, by setting the actual region of the labeled organ in the sample medical image, and the first detection result also includes the first prediction region of the labeled organ, the second detection result also includes the second prediction region of the labeled organ, and In the process of determining the first loss value of the original detection model, the difference between the first prediction area and the actual area is comprehensively considered. In the process of determining the second loss value of the original detection model, only the first loss value of the unlabeled organ is considered. The difference between the prediction area and the corresponding second prediction area can improve the robustness of the consistency constraints of the original detection model and the image detection model, and thus can improve the accuracy of the image detection model.
其中，利用第一预测区域分别与实际区域、第二预测区域之间的差异，调整原始检测模型的网络参数之后，还包括：利用本次训练以及之前若干次训练时调整后的网络参数，对图像检测模型的网络参数进行更新。Among them, after adjusting the network parameters of the original detection model by using the differences between the first prediction area and the actual area and the second prediction area, it also includes: using the network parameters adjusted during this training and several previous trainings to correct The network parameters of the image detection model are updated.
因此，通过利用原始检测模型在本次训练以及之前若干次训练时调整后的网络参数，对图像检测模型的网络参数进行更新，能够进一步约束网络参数在多次训练过程中由于伪标注的真实区域所产生的累积误差，提高图像检测模型的准确性。Therefore, by using the network parameters adjusted by the original detection model during this training and several previous trainings to update the network parameters of the image detection model, the network parameters can be further constrained due to pseudo-labeled real regions during multiple training sessions. The resulting cumulative error improves the accuracy of the image detection model.
其中，利用本次训练以及之前若干次训练时调整后的网络参数，对图像检测模型的网络参数进行更新，包括：统计原始检测模型在本次训练和之前若干次训练所调整的网络参数的平均值；将图像检测模型的网络参数更新为对应的原始检测模型的网络参数的平均值。Among them, using the network parameters adjusted during this training and several previous trainings to update the network parameters of the image detection model, including: statistics the average of the network parameters adjusted by the original detection model during this training and several previous trainings Value; update the network parameters of the image detection model to the average value of the network parameters of the corresponding original detection model.
因此，通过统计原始检测模型在本次训练和之前若干次训练所调整的网络参数的平均值，并将图像检测模型的网络参数更新为对应的原始检测模型的网络参数的平均值，能够有利于快速地约束多次训练过程中所产生的累积误差，提升图像检测模型的准确性。Therefore, by counting the average value of the network parameters adjusted by the original detection model in this training and several previous trainings, and updating the network parameters of the image detection model to the average value of the network parameters of the corresponding original detection model, it can be beneficial to Quickly constrain the accumulated errors generated during multiple training sessions and improve the accuracy of the image detection model.
其中，获取样本医学图像包括：获取待伪标注医学图像，其中，待伪标注医学图像存在至少一个未标注器官；分别利用与每一未标注器官对应的单器官检测模型对待伪标注医学图像进行检测，以得到每个未标注器官的器官预测区域；将未标注器官的器官预测区域伪标注为未标注器官的实际区域，并将伪标注后的待伪标注医学图像作为样本医学图像。Wherein, acquiring the sample medical image includes: acquiring a medical image to be pseudo-labeled, wherein at least one unlabeled organ exists in the medical image to be pseudo-labeled; and detecting the pseudo-labeled medical image using a single-organ detection model corresponding to each unlabeled organ. , To obtain the organ prediction area of each unlabeled organ; pseudo-label the organ prediction area of the unlabeled organ as the actual area of the unlabeled organ, and use the pseudo-labeled medical image to be pseudo-labeled as the sample medical image.
因此，通过获取存在至少一个未标注器官的待伪标注医学图像，并利用与每一未标注器官对应的单器官检测模型对待伪标注医学图像进行检测，以得到每个未标注器官的器官预测区域，并将未标注器官的器官预测区域伪标注为未标注器官的实际区域，将伪标注后的待伪标注医学图像作为样本医学图像，能够利用单器官检测模型免去人工对多器官进行标注的工作量，从而能够有利于降低训练用于多器官检测的图像检测模型的人工成本，并提升训练的效率。Therefore, by obtaining at least one unlabeled organ to be pseudo-labeled medical image, and using a single organ detection model corresponding to each unlabeled organ to detect the pseudo-labeled medical image, to obtain the organ prediction area of each unlabeled organ , And pseudo-label the organ prediction area of the unlabeled organ as the actual region of the unlabeled organ, and use the pseudo-labeled medical image to be pseudo-labeled as the sample medical image. The single-organ detection model can be used to avoid the manual labeling of multiple organs. Workload, which can help reduce the labor cost of training an image detection model for multi-organ detection, and improve the efficiency of training.
其中，待伪标注医学图像包括至少一个已标注器官；分别利用与每一未标注器官对应的单器官检测模型对待伪标注医学图像进行检测之前，方法还包括：利用待伪标注医学图像，对待伪标注医学图像中的已标注器官对应的单器官检测模型进行训练。Wherein, the medical image to be pseudo-labeled includes at least one labeled organ; before the single-organ detection model corresponding to each unlabeled organ is used to detect the pseudo-labeled medical image, the method further includes: using the medical image to be pseudo-labeled, Annotate the single-organ detection model corresponding to the annotated organ in the medical image for training.
因此，在待伪标注医学图像中包括至少一个已标注器官，并利用待伪标注医学图像对待伪标注医学图像中的已标注器官对应的单器官检测模型进行训练，能够提升单器官检测模型的准确性，从而能够有利于提升后续伪标注的准确性，进而能够有利于提升后续训练图像检测模型的准确性。Therefore, including at least one labeled organ in the medical image to be pseudo-labeled, and using the medical image to be pseudo-labeled to train the single-organ detection model corresponding to the labeled organ in the pseudo-labeled medical image can improve the accuracy of the single-organ detection model. Therefore, it can help improve the accuracy of subsequent pseudo-labeling, and in turn, can help improve the accuracy of the subsequent training image detection model.
其中，获取待伪标注医学图像，包括：获取三维医学图像，并对三维医学图像进行预处理；将预处理后的三维医学图像进行裁剪处理，得到至少一个二维的待伪标注医学图像。Wherein, acquiring a medical image to be pseudo-labeled includes: acquiring a three-dimensional medical image and preprocessing the three-dimensional medical image; and performing cropping processing on the pre-processed three-dimensional medical image to obtain at least one two-dimensional medical image to be pseudo-labeled.
因此，通过获取三维医学图像，并对三维医学图像进行预处理，从而对预处理后的三维医学图像进行裁剪处理，得到至少一个二维的待伪标注医学图像，能够有利于得到满足模型训练的医学图像，从而 能够有利于提升后续图像检测模型训练的准确性。Therefore, by acquiring three-dimensional medical images and preprocessing the three-dimensional medical images, the pre-processed three-dimensional medical images are cropped to obtain at least one two-dimensional medical image to be pseudo-labeled, which can help to obtain a model training Medical images can help improve the accuracy of subsequent image detection model training.
其中，对三维医学图像进行预处理包括以下至少一者：将三维医学图像的体素分辨率调整至一预设分辨率；利用一预设窗值将三维医学图像的体素值归一化至预设范围内；在三维医学图像的至少部分体素中加入高斯噪声。The preprocessing of the three-dimensional medical image includes at least one of the following: adjusting the voxel resolution of the three-dimensional medical image to a preset resolution; using a preset window value to normalize the voxel value of the three-dimensional medical image to Within a preset range; Gaussian noise is added to at least part of the voxels of the three-dimensional medical image.
因此，将三维医学图像的体素分辨率调整至一预设分辨率，能够有利于后续模型预测处理；利用预设窗值将三维医学图像的体素值归一化至预设范围内，能够有利于模型提取到准确的特征；在三维医学图像的至少部分体素中加入高斯噪声，能够有利于实现数据增广，提高数据多样性，提升后续模型训练的准确性。Therefore, adjusting the voxel resolution of the 3D medical image to a preset resolution can facilitate subsequent model prediction processing; using the preset window value to normalize the voxel value of the 3D medical image to a preset range can be It is helpful for the model to extract accurate features; adding Gaussian noise to at least part of the voxels of the three-dimensional medical image can help achieve data augmentation, increase data diversity, and improve the accuracy of subsequent model training.
第二方面，本公开实施例提供了一种图像检测方法，包括：获取待检测医学图像，其中，待检测医学图像中包含多个器官；利用图像检测模型对待检测医学进行检测，得到多个器官的预测区域；其中，图像检测模型是利用上述第一方面中的图像检测模型的训练方法训练得到的。In a second aspect, the embodiments of the present disclosure provide an image detection method, including: acquiring a medical image to be detected, wherein the medical image to be detected contains multiple organs; and using an image detection model to detect the medicine to be detected to obtain multiple organs The prediction area; wherein, the image detection model is obtained by using the training method of the image detection model in the first aspect.
因此，利用上述第一方面中训练得到的图像检测模型对待检测医学图像检测检测，得到多个器官的预测区域，能够在多器官检测的过程中，提高检测准确性。Therefore, by using the image detection model trained in the above first aspect to detect and detect the medical image to be detected, to obtain the predicted regions of multiple organs, the detection accuracy can be improved in the process of multiple organ detection.
第三方面，本公开实施例提供了一种图像检测模型的训练装置，包括图像获取模块、第一检测模块、第二检测模块、参数调整模块，图像获取模块被配置为获取样本医学图像，其中，样本医学图像伪标注出至少一个未标注器官的实际区域；第一检测模块被配置为利用原始检测模型对样本医学图像进行检测以得到第一检测结果，其中，第一检测结果包括未标注器官的第一预测区域；以及，第二检测模块被配置为利用图像检测模型对样本医学图像进行检测以得到第二检测结果，图像检测模型的网络参数是基于原始检测模型的网络参数确定的，其中，第二检测结果包括未标注器官的第二预测区域；参数调整模块被配置为利用第一预测区域分别与实际区域、第二预测区域之间的差异，调整原始检测模型的网络参数。In a third aspect, an embodiment of the present disclosure provides a training device for an image detection model, including an image acquisition module, a first detection module, a second detection module, and a parameter adjustment module. The image acquisition module is configured to acquire sample medical images, wherein , The sample medical image pseudo-labels the actual area of at least one unlabeled organ; the first detection module is configured to use the original detection model to detect the sample medical image to obtain the first detection result, wherein the first detection result includes the unlabeled organ And, the second detection module is configured to use the image detection model to detect the sample medical image to obtain the second detection result, and the network parameters of the image detection model are determined based on the network parameters of the original detection model, wherein , The second detection result includes a second prediction area of the unlabeled organ; the parameter adjustment module is configured to adjust the network parameters of the original detection model by using the difference between the first prediction area and the actual area and the second prediction area, respectively.
第四方面，本公开实施例提供了一种图像检测装置，包括图像获取模块和图像检测模块，图像获取模块被配置为获取待检测医学图像，其中，待检测医学图像中包含多个器官；图像检测模块被配置为利用图像检测模型对待检测医学进行检测，得到多个器官的预测区域；其中，图像检测模型是利用上述第二方面中的图像检测模型的训练装置训练得到的。In a fourth aspect, an embodiment of the present disclosure provides an image detection device, including an image acquisition module and an image detection module, the image acquisition module is configured to acquire a medical image to be detected, wherein the medical image to be detected contains multiple organs; the image The detection module is configured to use the image detection model to detect the medicine to be detected to obtain the predicted regions of multiple organs; wherein the image detection model is obtained by training using the image detection model training device in the second aspect.
第五方面，本公开实施例提供了一种电子设备，包括相互耦接的存储器和处理器，处理器被配置为执行存储器中存储的程序指令，以实现上述第一方面中的图像检测模型的训练方法，或实现上述第二方面中的图像检测方法。In a fifth aspect, embodiments of the present disclosure provide an electronic device including a memory and a processor coupled to each other. The processor is configured to execute program instructions stored in the memory to implement the image detection model in the first aspect. Training method, or implement the image detection method in the second aspect.
第六方面，本公开实施例提供了一种计算机可读存储介质，其上存储有程序指令，程序指令被处理器执行时实现上述第一方面中的图像检测模型的训练方法，或实现上述第二方面中的图像检测方法。In a sixth aspect, embodiments of the present disclosure provide a computer-readable storage medium on which program instructions are stored. When the program instructions are executed by a processor, the training method of the image detection model in the first aspect is realized, or the first aspect is realized. The image detection method in the second aspect.
第七方面，本公开实施例还提供了一种计算机程序，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行如上述第一方面中的图像检测模型的训练方法，或实现上述第二方面中的图像检测方法。In a seventh aspect, the embodiments of the present disclosure also provide a computer program, including computer-readable code. When the computer-readable code runs in an electronic device, the processor in the electronic device executes the above-mentioned first aspect. The training method of the image detection model in, or the image detection method in the second aspect mentioned above.
上述方案，通过获取样本医学图像，且样本医学图像伪标注出至少一个未标注器官的实际区域，故样本医学图像中无需对多器官进行真实标注，从而利用原始检测模型对样本医学图像检测检测以得到包含未标注器官的第一预设区域的第一检测结果，并利用图像检测模型对样本医学图像进行检测以得到包含未标注器官的第二预测区域的第二检测结果，进而利用第一预测区域分别与实际区域、第二预测区域之间的差异，调整原始检测模型的网络参数，且图像检测模型的网络参数是基于原始检测模型的网络参数确定的，故能够使得图像检测模型监督原始检测模型的训练，故能够约束网络参数在多次训练过程中由于伪标注的真实区域所产生的累积误差，提高图像检测模型的准确性，从而使得图像检测模型得以准确地监督原始检测模型进行训练，进而使得原始检测模型在训练过程中能够准确地调整其网络参数，故 此，能够在多器官检测的过程中，提升图像检测模型的检测准确性。In the above solution, the sample medical image is acquired, and the sample medical image is pseudo-labeled with at least one actual region of an unlabeled organ, so there is no need to actually label multiple organs in the sample medical image, and the original detection model is used to detect the sample medical image. Obtain the first detection result of the first preset region containing the unlabeled organ, and use the image detection model to detect the sample medical image to obtain the second detection result of the second prediction region containing the unlabeled organ, and then use the first prediction The difference between the area and the actual area and the second predicted area, adjust the network parameters of the original detection model, and the network parameters of the image detection model are determined based on the network parameters of the original detection model, so the image detection model can supervise the original detection The training of the model can constrain the cumulative error of the network parameters due to the pseudo-labeled real area during multiple training processes, and improve the accuracy of the image detection model, so that the image detection model can accurately supervise the training of the original detection model. In turn, the original detection model can accurately adjust its network parameters during the training process. Therefore, the detection accuracy of the image detection model can be improved in the process of multi-organ detection.
附图说明Description of the drawings
图1是本公开实施例提供的图像检测模型的训练方法一实施例的流程示意图；FIG. 1 is a schematic flowchart of an embodiment of a training method for an image detection model provided by an embodiment of the present disclosure;
图2是图1中步骤S11一实施例的流程示意图；FIG. 2 is a schematic flowchart of an embodiment of step S11 in FIG. 1;
图3是本公开实施例提供的图像检测模型的训练方法另一实施例的流程示意图；FIG. 3 is a schematic flowchart of another embodiment of a training method for an image detection model provided by an embodiment of the present disclosure;
图4是本公开实施例提供的图像检测模型的训练过程一实施例的示意图；4 is a schematic diagram of an embodiment of the training process of an image detection model provided by an embodiment of the present disclosure;
图5是本公开实施例提供的图像检测方法一实施例的流程示意图；FIG. 5 is a schematic flowchart of an embodiment of an image detection method provided by an embodiment of the present disclosure;
图6是本公开实施例提供的图像检测模型的训练装置一实施例的框架示意图；6 is a schematic diagram of the framework of an embodiment of an image detection model training apparatus provided by an embodiment of the present disclosure;
图7是本公开实施例提供的图像检测装置一实施例的框架示意图；FIG. 7 is a schematic diagram of a framework of an embodiment of an image detection device provided by an embodiment of the present disclosure;
图8是本公开实施例提供的电子设备一实施例的框架示意图；FIG. 8 is a schematic diagram of a framework of an embodiment of an electronic device provided by an embodiment of the present disclosure;
图9是本公开实施例提供的计算机可读存储介质一实施例的框架示意图。FIG. 9 is a schematic framework diagram of an embodiment of a computer-readable storage medium provided by an embodiment of the present disclosure.
具体实施方式Detailed ways
下面结合说明书附图，对本公开实施例的方案进行详细说明。The solutions of the embodiments of the present disclosure will be described in detail below in conjunction with the accompanying drawings of the specification.
以下描述中，为了说明而不是为了限定，提出了诸如特定***结构、接口、技术之类的细节，以便透彻理解本公开实施例。In the following description, for the purpose of illustration rather than limitation, details such as a specific system structure, interface, technology, etc. are proposed for a thorough understanding of the embodiments of the present disclosure.
本文中术语“***”和“网络”在本文中常被可互换使用。本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系。此外，本文中的“多”表示两个或者多于两个。The terms "system" and "network" in this article are often used interchangeably in this article. The term "and/or" in this article is only an association relationship describing the associated objects, which means that there can be three relationships, for example, A and/or B, which can mean: A alone exists, A and B exist at the same time, exist alone B these three situations. In addition, the character "/" in this text generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this document means two or more than two.
请参阅图1，图1是本公开实施例提供的图像检测模型的训练方法一实施例的流程示意图。其中，可以包括如下步骤：Please refer to FIG. 1. FIG. 1 is a schematic flowchart of an embodiment of a method for training an image detection model provided by an embodiment of the present disclosure. Among them, the following steps can be included:
步骤S11：获取样本医学图像，其中，样本医学图像伪标注出至少一个未标注器官的实际区域。Step S11: Obtain a sample medical image, where the sample medical image pseudo-labels at least one actual region of an unlabeled organ.
样本医学图像可以包括CT图像、MR图像，在此不做限定。在一个可能的实施场景中，样本医学图像可以是对腹部、胸部、头颅等部位进行扫描得到的，可以根据实际应用情况进行设置，在此不做限定。例如，对腹部进行扫描，样本医学图像中的器官可以包括：肾脏、脾脏、肝脏、胰腺等；或者，对胸部进行扫描，样本医学图像中的器官可以包括：心脏、肺叶、甲状腺等；或者，对头颅进行扫描，样本医学图像中的器官可以包括：脑干、小脑、间脑、端脑等。The sample medical images may include CT images and MR images, which are not limited here. In a possible implementation scenario, the sample medical image can be obtained by scanning the abdomen, chest, head, etc., and can be set according to actual application conditions, which is not limited here. For example, when the abdomen is scanned, the organs in the sample medical image may include: kidney, spleen, liver, pancreas, etc.; or, when the chest is scanned, the organs in the sample medical image may include: heart, lung lobes, thyroid, etc.; or, The head is scanned, and the organs in the sample medical image can include: brain stem, cerebellum, diencephalon, and telencephalon.
在一个可能的实施场景中，未标注器官的实际区域可以是利用与未标注器官对应的单器官检测模型检测得到的，例如，样本医学图像为腹部扫描得到的，则其中未标注器官可以包括：肾脏、脾脏、肝脏、胰腺中的至少一者，则可以利用与肾脏对应的单器官检测模型对样本医学图像进行检测，得到与肾脏对应的器官预测区域，可以利用与脾脏对应的单器官检测模型对样本医学图像进行检测，得到与脾脏对应的器官预测区域，可以利用与肝脏对应的单器官检测模型对样本医学图像进行检测，得到与肝脏对应的器官预测区域，利用与胰腺对应的单器官检测模型对样本医学图像进行检测，得到与胰腺对应的器官预测区域，从而在样本医学图像中将与肾脏、脾脏、肝脏、胰腺分别对应的器官预测区域进行伪标注，即得到未标注器官肾脏、脾脏、肝脏和胰腺伪标注出的实际区域，本公开实施例中，伪标注是指将单器官检测模型检测出的未标注器官的器官预测区域作为实际区域的过程。在未标注器官为其他器官的情况下，可以以此类推，在此不再一一举例。在一个可能的实施场景中，未标注器官的单器官检测模型是利用标注有未标注器官的实际区域的单器官数据集训练得到的，例如，与肾脏对应的单器官检测模型是利用标注有肾脏的实际区域的肾脏数据集训练得到的，与脾脏对应的单器官检测模型是利用标注有脾脏的实际 区域的脾脏数据集训练得到的，以此类推，在此不再一一举例。In a possible implementation scenario, the actual area of the unlabeled organ may be detected by using a single-organ detection model corresponding to the unlabeled organ. For example, if the sample medical image is obtained from an abdominal scan, the unlabeled organ may include: At least one of the kidney, spleen, liver, and pancreas can use the single organ detection model corresponding to the kidney to detect the sample medical image to obtain the organ prediction area corresponding to the kidney, and the single organ detection model corresponding to the spleen can be used Detect the sample medical image to obtain the organ prediction area corresponding to the spleen, and use the single organ detection model corresponding to the liver to detect the sample medical image to obtain the organ prediction area corresponding to the liver, and use the single organ detection corresponding to the pancreas The model detects the sample medical image and obtains the organ prediction region corresponding to the pancreas, so that the organ prediction regions corresponding to the kidney, spleen, liver, and pancreas are pseudo-labeled in the sample medical image, that is, the unlabeled organ kidney and spleen are obtained. The actual regions pseudo-labeled by the liver and pancreas. In the embodiments of the present disclosure, pseudo-labeling refers to the process of taking the organ prediction regions of unlabeled organs detected by the single-organ detection model as the actual regions. In the case where the organ is not marked as other organs, it can be deduced by analogy, and we will not give examples one by one here. In a possible implementation scenario, the single-organ detection model for unlabeled organs is trained using a single-organ data set labeled with the actual region of the unlabeled organ. For example, the single-organ detection model corresponding to the kidney uses the labeled kidney The kidney data set of the actual area is trained, and the single-organ detection model corresponding to the spleen is trained using the spleen data set of the actual area marked with the spleen.
步骤S12：利用原始检测模型对样本医学图像进行检测以得到第一检测结果，其中，第一检测结果包括未标注器官的第一预测区域。Step S12: Use the original detection model to detect the sample medical image to obtain a first detection result, where the first detection result includes a first prediction region of an unlabeled organ.
原始检测模型可以包括Mask R-CNN(Mask Region with Convolutional Neural Network)、FCN(Fully Convolutional Network，全卷积网络)、PSP-net(Pyramid Scene Parsing Network，金字塔场景分析网络)中的任一者，此外，原始检测模型还可以是set-net、U-net等，可以根据实际情况进行设置，在此不做限定。The original detection model can include any one of Mask R-CNN (Mask Region with Convolutional Neural Network), FCN (Fully Convolutional Network), PSP-net (Pyramid Scene Parsing Network, pyramid scene analysis network), In addition, the original detection model can also be set-net, U-net, etc., which can be set according to the actual situation, which is not limited here.
利用原始检测模型对样本医学图像进行检测，可以得到包含未标注器官的第一预测区域的第一检测结果。例如，样本医学图像是对腹部扫描得到的图像，未标注器官包括肾脏、脾脏、胰腺，故利用原始检测模型对样本医学图像进行检测，能够得到肾脏的第一预测区域、脾脏的第一预测区域、胰腺的第一预测区域，其他场景可以以此类推，在此不再一一举例。Using the original detection model to detect the sample medical image, the first detection result of the first prediction region containing the unlabeled organ can be obtained. For example, the sample medical image is an image obtained by scanning the abdomen. The unlabeled organs include the kidney, spleen, and pancreas. Therefore, the original detection model is used to detect the sample medical image, and the first prediction area of the kidney and the first prediction area of the spleen can be obtained. , The first prediction area of the pancreas, and other scenarios can be deduced by analogy, so I won’t give an example one by one here.
步骤S13：利用图像检测模型对样本医学图像进行检测以得到第二检测结果，其中，第二检测结果包括未标注器官的第二预测区域。Step S13: Use the image detection model to detect the sample medical image to obtain a second detection result, where the second detection result includes a second prediction region of an unlabeled organ.
原始检测模型的网络结构、与原始检测模型对应的图像检测模型的网络结构可以是相同的。例如，在原始检测模型为Mask R-CNN的情况下，对应的图像检测模型也可以是Mask R-CNN；或者，在原始检测模型为FCN的情况下，对应的图像检测模型也可以是FCN；或者，在原始检测模型为PSP-net的情况下，对应的图像检测模型也可以是PSP-net；在原始检测模型为其他网络的情况下，可以以此类推，在此不再一一举例。The network structure of the original detection model and the network structure of the image detection model corresponding to the original detection model may be the same. For example, when the original detection model is Mask R-CNN, the corresponding image detection model can also be Mask R-CNN; or, when the original detection model is FCN, the corresponding image detection model can also be FCN; Or, when the original detection model is PSP-net, the corresponding image detection model can also be PSP-net; when the original detection model is another network, the analogy can be used, and no examples are given here.
图像检测模型的网络参数可以是基于原始检测模型的网络参数确定的，例如，图像检测模型的网络参数可以是基于原始检测模型在多次训练过程中调整后的网络参数得到的。例如，在第k次训练的过程中，图像检测模型的网络参数可以是利用原始检测模型在第k-n次至第k-1次训练过程中调整后的网络参数得到的；或者，在第k+1次训练的过程中，图像检测模型的网络参数可以是利用原始检测模型在第k+1-n次至第k次训练过程中调整后的网络参数得到的，以此类推，在此不再一一举例。其中，上述多次训练的次数(即n)可以根据实际情况进行设置，如可以设置为5、10、15等等，在此不做限定。The network parameters of the image detection model may be determined based on the network parameters of the original detection model. For example, the network parameters of the image detection model may be obtained based on the network parameters adjusted by the original detection model in multiple training processes. For example, in the kth training process, the network parameters of the image detection model can be obtained by using the network parameters adjusted by the original detection model from the knth to the k-1th training process; or, in the k+th During the first training process, the network parameters of the image detection model can be obtained by using the network parameters adjusted by the original detection model from the k+1-nth to the kth training process, and so on. Give examples one by one. Among them, the number of times (ie, n) of the foregoing multiple trainings can be set according to actual conditions, for example, it can be set to 5, 10, 15, etc., which are not limited here.
利用图像检测模型对样本医学图像进行检测，可以得到包含未标注器官的第二预测区域的第二检测结果。仍以样本医学图像是对腹部扫描得到的图像为例，未标注器官包括肾脏、脾脏、胰腺，故利用图像检测模型对样本医学图像进行检测，能够得到肾脏的第二预测区域、脾脏的第二预测区域、胰腺的第二预测区域，其他场景可以以此类推，在此不再一一举例。Using the image detection model to detect the sample medical image, the second detection result of the second prediction region containing the unlabeled organ can be obtained. Still taking the sample medical image as an image obtained by scanning the abdomen, the unlabeled organs include the kidney, spleen, and pancreas. Therefore, the image detection model is used to detect the sample medical image, and the second prediction area of the kidney and the second prediction area of the spleen can be obtained. The prediction area, the second prediction area of the pancreas, and other scenarios can be deduced by analogy, so we will not give examples one by one here.
在一个可能的实施场景中，上述步骤S12和步骤S13可以按照先后顺序执行，例如，先执行步骤S12，后执行步骤S13；或者，先执行步骤S13，后执行步骤S12。在另一个可能的实施场景中，上述步骤S12和步骤S13还可以同时执行，可以根据实际应用进行设置，在此不做限定。In a possible implementation scenario, the above steps S12 and S13 may be performed in a sequential order, for example, step S12 is performed first, and then step S13; or, step S13 is performed first, and then step S12 is performed. In another possible implementation scenario, the above step S12 and step S13 can also be performed at the same time, and can be set according to actual applications, which is not limited here.
步骤S14：利用第一预测区域分别与实际区域、第二预测区域之间的差异，调整原始检测模型的网络参数。Step S14: Use the differences between the first prediction area and the actual area and the second prediction area to adjust the network parameters of the original detection model.
其中，可以利用第一预测区域和实际区域之间的差异，确定原始检测模型的第一损失值。例如，为了提升模型对于难样本的关注度，可以利用焦点损失(focal loss)函数对第一预测区域和实际区域进行处理，得到焦点第一损失值；或者，为了能够使模型拟合伪标注的实际区域，还可以利用集合相似度损失(dice loss)函数对第一预测区域和实际区域进行处理，得到集合相似度第一损失值。Wherein, the difference between the first prediction area and the actual area can be used to determine the first loss value of the original detection model. For example, in order to increase the model's attention to difficult samples, the focal loss function can be used to process the first prediction area and the actual area to obtain the first focal loss value; or, in order to be able to make the model fit pseudo-labeled In the actual area, the first prediction area and the actual area can also be processed by using the dice loss function to obtain the first loss value of the dice loss.
其中，还可以利用第一预测区域和第二预测区域之间的差异，确定原始检测模型的第二损失值。例如，为了提高原始检测模型和图像检测模型预测的一致性，可以利用一致性损失函数对第一预测区域和第二预测区域进行处理，得到第二损失值，在一个可能的实施场景中，一致性损失函数可以是交叉熵损 失函数，可以根据实际应用情况进行设置，在此不做限定。Wherein, the difference between the first prediction area and the second prediction area can also be used to determine the second loss value of the original detection model. For example, in order to improve the prediction consistency between the original detection model and the image detection model, the consistency loss function can be used to process the first prediction area and the second prediction area to obtain the second loss value. In a possible implementation scenario, the same The performance loss function can be a cross-entropy loss function, which can be set according to actual application conditions, and is not limited here.
其中，还可以利用上述第一损失值和第二损失值调整原始检测模型的网络参数。例如，为了能够平衡各损失值在训练过程中的重要程度，可以对第一损失值和第二损失值进行加权处理，得到加权损失值，从而利用加权损失值调整原始检测模型的网络参数。第一损失值和第二损失值对应的权值可以根据实际情况进行设置，例如，均设置为0.5；或者，将第一损失值对应的权值设置为0.6，第二损失值对应的权值设置为0.4，在此不做限定。此外，在第一损失值包括焦点第一损失值和集合相似度第一损失值的情况下，可以对焦点第一损失值、集合相似度第一损失值、第二损失值进行加权处理，得到加权损失值，并利用加权损失值调整原始检测模型的网络参数。在一个可能的实施场景中，可以采用随机梯度下降(Stochastic Gradient Descent，SGD)、批量梯度下降(Batch Gradient Descent，BGD)、小批量梯度下降(Mini-Batch Gradient Descent，MBGD)等方式，利用加权损失值对原始检测模型的网络参数进行调整，其中，批量梯度下降是指在每一次迭代的过程中，使用所有样本来进行参数更新；随机梯度下降是指在每一次迭代的过程中，使用一个样本来进行参数更新；小批量梯度下降是指在每一次迭代的过程中，使用一批样本来进行参数更新，在此不再赘述。Wherein, the above-mentioned first loss value and second loss value can also be used to adjust the network parameters of the original detection model. For example, in order to balance the importance of each loss value in the training process, the first loss value and the second loss value can be weighted to obtain a weighted loss value, so that the weighted loss value can be used to adjust the network parameters of the original detection model. The weights corresponding to the first loss value and the second loss value can be set according to the actual situation, for example, both are set to 0.5; or, the weight corresponding to the first loss value is set to 0.6, and the weight corresponding to the second loss value is set Set to 0.4, which is not limited here. In addition, when the first loss value includes the first loss value of the focus and the first loss value of the set similarity, the first loss value of the focus, the first loss value of the set similarity, and the second loss value can be weighted to obtain The weighted loss value is used to adjust the network parameters of the original detection model. In a possible implementation scenario, Stochastic Gradient Descent (SGD), Batch Gradient Descent (BGD), Mini-Batch Gradient Descent (MBGD), etc. can be used, and weighted The loss value adjusts the network parameters of the original detection model. Among them, batch gradient descent refers to the use of all samples for parameter updates during each iteration; stochastic gradient descent refers to the use of one during each iteration Samples are used to update parameters; mini-batch gradient descent refers to using a batch of samples to update parameters during each iteration, which will not be repeated here.
在一个实施场景中，样本医学图像中还可以包括已标注器官的实际区域，第一检测结果中还可以包括已标注器官的第一预测区域，第二检测结果还可以包括已标注器官的第二预测区域。仍以样本医学图像是对腹部扫描得到的图像为例，未标注器官包括肾脏、脾脏、胰腺，已标注器官包括肝脏，故利用原始检测模型对样本医学图像检测检测，能够得到未标注器官肾脏对应的第一预测区域、未标注器官脾脏对应的第一预测区域、未标注器官胰腺对应的第一预测区域和已标注器官肝脏对应的第一预测区域，而利用与原始检测模型对应的图像检测模型对样本医学图像进行检测，能够得到未标注器官肾脏对应的第二预测区域、未标注器官脾脏对应的第二预测区域、未标注器官胰腺对应的第二预测区域和已标注器官肝脏对应的第二预测区域。故此，可以利用未标注器官和已标注器官的第一预测区域和实际区域之间的差异，确定原始检测模型的第一损失值，利用未标注器官的第一预测区域和对应第二预测区域之间的差异，可以确定原始检测模型的第二损失值。仍以样本医学图像是对腹部扫描得到的图像为例，未标注器官包括肾脏、脾脏、胰腺，已标注器官包括肝脏，可以利用未标注器官肾脏对应的第一预测区域和伪标注的实际区域之间的差异、未标注器官脾脏对应的第一预测区域和伪标注的实际区域之间的差异、未标注器官胰腺对应的第一预测区域和伪标注的实际区域之间的差异和已标注器官肝脏对应的第一预测区域和真实标注的实际区域之间的差异，确定原始检测模型的第一损失值，第一损失值可以包括焦点第一损失值、集合相似度第一损失值中的至少一者，可以参阅前述步骤，在此不再赘述。此外，还可以利用未标注器官肾脏对应的第一预测区域和第二预测区域之间的差异、未标注器官脾脏对应的第一预测区域和第二预测区域之间的差异、未标注器官胰腺对应的第一预测区域和第二预测区域之间的差异，确定原始检测模型的第二损失值，第二损失值可以采用交叉熵损失函数进行计算，可以参阅前述步骤，在此不再赘述。故此，在确定原始检测模型的第一损失值的过程中，综合考虑第一预测区域和实际区域之间的差异，而在确定原始检测模型的第二损失值的过程中，只考虑未标注器官的第一预测区域和对应的第二预测区域之间的差异，从而能够提升原始检测模型和图像检测模型一致性约束的鲁棒性，进而能够提高图像检测模型的准确性。In an implementation scenario, the sample medical image may also include the actual area of the marked organ, the first detection result may also include the first prediction area of the marked organ, and the second detection result may also include the second area of the marked organ. Forecast area. Still taking the sample medical image as an image obtained by scanning the abdomen, the unlabeled organs include the kidney, spleen, and pancreas, and the labeled organs include the liver. Therefore, the original detection model is used to detect the sample medical image, and the corresponding kidneys of the unlabeled organs can be obtained. The first prediction area corresponding to the unlabeled organ spleen, the first prediction area corresponding to the unlabeled organ pancreas, and the first prediction area corresponding to the labeled organ liver, and the image detection model corresponding to the original detection model is used Detecting the sample medical image can obtain the second prediction area corresponding to the unlabeled organ kidney, the second prediction area corresponding to the unlabeled organ spleen, the second prediction area corresponding to the unlabeled organ pancreas, and the second prediction area corresponding to the labeled organ liver. Forecast area. Therefore, the difference between the first prediction region and the actual region of the unlabeled organ and the labeled organ can be used to determine the first loss value of the original detection model, and the difference between the first prediction region of the unlabeled organ and the corresponding second prediction region can be used. The difference between the two can determine the second loss value of the original detection model. Still taking the sample medical image as an image obtained by scanning the abdomen, the unlabeled organs include the kidney, spleen, and pancreas, and the labeled organs include the liver. You can use the first prediction area corresponding to the unlabeled organ kidney and the pseudo-labeled actual area. The difference between the first prediction area corresponding to the unlabeled organ spleen and the pseudo-labeled actual area, the difference between the first prediction area corresponding to the unlabeled organ pancreas and the pseudo-labeled actual area, and the labeled organ liver Determine the first loss value of the original detection model according to the difference between the corresponding first prediction area and the actual area marked by the real label. The first loss value may include at least one of the first loss value of focus and the first loss value of set similarity. Alternatively, please refer to the previous steps, which will not be repeated here. In addition, the difference between the first prediction region and the second prediction region corresponding to the unlabeled organ kidney, the difference between the first prediction region and the second prediction region corresponding to the spleen of the unlabeled organ, and the pancreas corresponding to the unlabeled organ The difference between the first prediction area and the second prediction area is determined to determine the second loss value of the original detection model. The second loss value can be calculated by using the cross-entropy loss function. You can refer to the foregoing steps and will not be repeated here. Therefore, in the process of determining the first loss value of the original detection model, the difference between the first prediction area and the actual area is comprehensively considered, and in the process of determining the second loss value of the original detection model, only unlabeled organs are considered. The difference between the first prediction region and the corresponding second prediction region can improve the robustness of the consistency constraint of the original detection model and the image detection model, and thus can improve the accuracy of the image detection model.
在另一个实施场景中，在对原始检测模型的网络参数进行调整之后，还可以利用本次训练以及之前若干次训练时调整后的网络参数，对图像检测模型的网络参数进行更新，以进一步约束网络参数在多次训练过程中由于伪标注的真实区域所产生的累积误差，提高图像检测模型的准确性。此外，在对原始检测模型的网络参数进行调整之后，根据需要，也可以不对图像检测模型的网络参数进行更新，而在预设数量次(例如，2次、3次等)训练之后，再利用本次训练以及之前若干次训练时调整后的网络参数， 对图像检测模型的网络参数进行更新，在此不做限定。例如，在第k次训练的过程中，可以不对图像检测模型的网络参数进行更新，在第k+i次训练的过程中，可以利用原始检测模型在第k+i-n至第k+i次训练，其中，i可以根据实际情况设置为不小于1的整数，如可以设置为1、2、3等等，在此不做限定。In another implementation scenario, after adjusting the network parameters of the original detection model, you can also use the network parameters adjusted during this training and several previous trainings to update the network parameters of the image detection model to further restrict The cumulative error of the network parameters due to the pseudo-labeled real regions during multiple training processes improves the accuracy of the image detection model. In addition, after adjusting the network parameters of the original detection model, if necessary, the network parameters of the image detection model may not be updated, but after a preset number of times (for example, 2 times, 3 times, etc.) training, reuse The network parameters adjusted during this training and several previous trainings will update the network parameters of the image detection model, which is not limited here. For example, during the kth training process, the network parameters of the image detection model may not be updated. During the k+ith training process, the original detection model can be used to train from the k+inth to the k+ith time. , Where i can be set to an integer not less than 1 according to the actual situation, for example, it can be set to 1, 2, 3, etc., which is not limited here.
在一个可能的实施场景中，在对图像检测模型的网络参数进行更新的过程中，可以统计原始检测模型在本次训练和之前若干次训练所调整的网络参数的平均值，再将图像检测模型的网络参数更新为对应的原始检测模型的网络参数的平均值。本公开实施例中，网络参数的平均值均是指对应于同一网络参数的平均值，其中，可以是对应于同一神经元的某一权重(或偏置)在多次训练过程中调整后的数值的平均值，故可以统计得到各神经元的各个权重(或偏置)在多次训练过程中调整后的数值的平均值，从而利用该平均值对图像检测模型中对应神经元的对应权重(或偏置)进行更新。例如，本次训练为第k次训练，可以统计原始检测模型在本次训练和之前n-1次训练所调整的网络参数的平均值，其中，n的数值可以根据实际应用情况进行设置，例如，可以设置为5、10、15等，在此不做限定。故此，在第k+1次训练的过程中，图像检测模型的网络参数是利用第k-n+1次至第k次训练过程中所调整后的网络参数的平均值更新得到的，从而能够有利于快速地约束多次训练过程中所产生的累积误差，提升图像检测模型的准确性。In a possible implementation scenario, in the process of updating the network parameters of the image detection model, the original detection model can be counted in this training and the average value of the network parameters adjusted by several previous trainings, and then the image detection model The network parameters of is updated to the average value of the network parameters of the corresponding original detection model. In the embodiments of the present disclosure, the average value of network parameters refers to the average value corresponding to the same network parameter, which may be a certain weight (or bias) corresponding to the same neuron after being adjusted in multiple training processes. The average value of the value, so the average value of each weight (or bias) of each neuron after adjustment in multiple training processes can be obtained by statistics, so as to use the average value to the corresponding weight of the corresponding neuron in the image detection model (Or offset) to update. For example, this training is the kth training, and the average value of the network parameters adjusted by the original detection model during this training and the previous n-1 training can be counted. The value of n can be set according to the actual application, for example , Can be set to 5, 10, 15, etc., which is not limited here. Therefore, in the k+1 training process, the network parameters of the image detection model are updated using the average value of the adjusted network parameters from the k-n+1 training process to the k training process, so as to be able to It is conducive to quickly constrain the accumulated errors generated in the process of multiple training, and improve the accuracy of the image detection model.
在又一个实施场景中，还可以设置一预设训练训练结束条件，在不满足预设训练结束条件的情况下，可以重新执行上述步骤S12以及后续步骤，以继续对原始检测模型的网络参数进行调整。在一个可能的实施场景中，预设训练结束条件可以包括：当前训练次数达到预设次数阈值(如，500次、1000次等)、原始检测模型的损失值小于一预设损失阈值中的任一者，在此不做限定。在另一个可能的实施场景中，在训练结束后，可以利用图像检测模型对待检测医学图像进行检测，从而能够直接得到待检测医学图像中多个器官对应的区域，进而能够免去利用多个单器官检测对待检测医学图像进行分别检测的操作，故能够降低检测计算量。In another implementation scenario, a preset training end condition can also be set. If the preset training end condition is not met, the above step S12 and subsequent steps can be re-executed to continue to perform the network parameters of the original detection model. Adjustment. In a possible implementation scenario, the preset training end conditions may include any of the following: the current number of training times reaches a preset number threshold (eg, 500 times, 1000 times, etc.), and the loss value of the original detection model is less than a preset loss threshold. For one, there is no limitation here. In another possible implementation scenario, after the training is completed, the image detection model can be used to detect the medical image to be tested, so that the regions corresponding to multiple organs in the medical image to be tested can be directly obtained, thereby eliminating the need to use multiple units. Organ detection performs separate detection operations on medical images to be detected, so the amount of detection calculations can be reduced.
上述方案，通过获取样本医学图像，且样本医学图像伪标注出至少一个未标注器官的实际区域，故样本医学图像中无需对多器官进行真实标注，从而利用原始检测模型对样本医学图像检测检测以得到包含未标注器官的第一预设区域的第一检测结果，并利用图像检测模型对样本医学图像进行检测以得到包含未标注器官的第二预测区域的第二检测结果，进而利用第一预测区域分别与实际区域、第二预测区域之间的差异，调整原始检测模型的网络参数，且图像检测模型的网络参数是利用原始检测模型的网络参数确定的，故能够使得图像检测模型监督原始检测模型的训练，故能够约束网络参数在多次训练过程中由于伪标注的真实区域所产生的累积误差，提高图像检测模型的准确性，从而使得图像检测模型得以准确地监督原始检测模型进行训练，进而使得原始检测模型在训练过程中能够准确地调整其网络参数，故此，能够在多器官检测的过程中，提升图像检测模型的检测准确性。In the above solution, the sample medical image is acquired, and the sample medical image is pseudo-labeled with at least one actual region of an unlabeled organ, so there is no need to actually label multiple organs in the sample medical image, and the original detection model is used to detect the sample medical image. Obtain the first detection result of the first preset region containing the unlabeled organ, and use the image detection model to detect the sample medical image to obtain the second detection result of the second prediction region containing the unlabeled organ, and then use the first prediction The difference between the area and the actual area and the second predicted area, adjust the network parameters of the original detection model, and the network parameters of the image detection model are determined by the network parameters of the original detection model, so the image detection model can supervise the original detection The training of the model can constrain the cumulative error of the network parameters due to the pseudo-labeled real area during multiple training processes, and improve the accuracy of the image detection model, so that the image detection model can accurately supervise the training of the original detection model. In turn, the original detection model can accurately adjust its network parameters during the training process. Therefore, the detection accuracy of the image detection model can be improved in the process of multi-organ detection.
请参阅图2，图2是图1中步骤S11一实施例的流程示意图。其中，图2是获取样本医学图像一实施例的流程示意图，包括如下步骤：Please refer to FIG. 2, which is a schematic flowchart of an embodiment of step S11 in FIG. 1. Wherein, FIG. 2 is a schematic diagram of an embodiment of obtaining a sample medical image, which includes the following steps:
步骤S111：获取待伪标注医学图像，其中，待伪标注医学图像存在至少一个未标注器官。Step S111: Obtain a medical image to be pseudo-labeled, where at least one unlabeled organ exists in the medical image to be pseudo-labeled.
待伪标注医学图像可以是对腹部进行扫描得到的，待伪标注医学图像中的未标注器官可以包括：肾脏、脾脏、胰腺等，待伪标注医学图像也可以是对其他部位进行扫描得到的，例如，胸部、头颅等，可以参阅前述实施例中的相关步骤，在此不做限定。The medical image to be pseudo-labeled can be obtained by scanning the abdomen, the unlabeled organs in the medical image to be pseudo-labeled can include: kidney, spleen, pancreas, etc., and the medical image to be pseudo-labeled can also be obtained by scanning other parts. For example, the chest, head, etc., can refer to the relevant steps in the foregoing embodiment, which is not limited here.
在一个实施场景中，采集得到的原始医学图像可以是三维医学图像，例如，三维CT图像、三维MR图像，在此不做限定，故可以对三维医学图像进行预处理，并将预处理后的三维医学图像进行裁剪处理，得到至少一个待伪标注医学图像。裁剪处理可以是对预处理后的三维医学图像进行中心裁剪，在此不做限定。例如，可以沿平行于三维医学图像的某一平面在垂直与该平面的维度上进行裁剪，得到二 维的待伪标注医学图像。待伪标注医学图像的尺寸可以根据实际情况进行设置，例如，可以为352*352，在此不做限定。In an implementation scenario, the acquired original medical image can be a three-dimensional medical image, for example, a three-dimensional CT image, a three-dimensional MR image, which is not limited here, so the three-dimensional medical image can be preprocessed and the preprocessed The three-dimensional medical image is cropped to obtain at least one medical image to be pseudo-labeled. The cropping process may be center cropping of the preprocessed three-dimensional medical image, which is not limited here. For example, cropping can be performed along a plane parallel to the three-dimensional medical image in the dimensions perpendicular to the plane to obtain a two-dimensional medical image to be pseudo-labeled. The size of the medical image to be pseudo-labeled can be set according to the actual situation, for example, it can be 352*352, which is not limited here.
在一个可能的实施场景中，预处理可以包括将三维医学图像的体素分辨率调整至一预设分辨率。三维医学图像的体素是三维医学图像在三维空间分割的最小单位，预设分辨率可以为1*1*3mm，预设分辨率还可以根据实际情况设置为其他分辨率，例如，1*1*4mm、2*2*3mm等等，在此不做限定。通过将三维医学图像的体素分辨率调整至一预设分辨率，能够有利于后续模型预测处理。In a possible implementation scenario, the preprocessing may include adjusting the voxel resolution of the three-dimensional medical image to a preset resolution. The voxel of the 3D medical image is the smallest unit of 3D medical image segmentation in the 3D space. The preset resolution can be 1*1*3mm, and the preset resolution can also be set to other resolutions according to the actual situation, for example, 1*1 *4mm, 2*2*3mm, etc., are not limited here. Adjusting the voxel resolution of the three-dimensional medical image to a preset resolution can facilitate subsequent model prediction processing.
在另一个可能的实施场景中，预处理还可以包括利用一预设窗值将三维医学图像的体素值归一化至预设范围内。体素值根据三维医学图像的不同，可以为不同的数值，例如，对于三维CT图像而言，体素值可以为Hu(houns field unit，即亨氏单位)值。预设窗值可以根据三维医学图像所对应的部位进行设置，仍以三维CT图像为例，对于腹部CT而言，预设窗值可以设置为-125至275，其他部位可以根据实际情况进行设置，在此不再一一举例。预设范围可以根据实际应用进行设置，例如，预设范围可以设置为0至1，仍以三维CT图像为例，对于腹部CT而言，预设窗值可以设置为-125至275，则在预设范围为0至1的情况下，可以统一将体素值小于或等于-125的体素重置为体素值0，可以统一将体素值大于或等于275的体素重置为体素值1，可以将体素值位于-125至275的体素重置为体素值0至1之间，从而能够有利于加强图像内不同器官间的对比度，进而能够提高模型提取到准确的特征。In another possible implementation scenario, the preprocessing may also include using a preset window value to normalize the voxel value of the three-dimensional medical image to a preset range. The voxel value can be a different value depending on the three-dimensional medical image. For example, for a three-dimensional CT image, the voxel value can be a Hu (houns field unit) value. The preset window value can be set according to the part corresponding to the 3D medical image. Still taking the 3D CT image as an example, for abdominal CT, the preset window value can be set from -125 to 275, and other parts can be set according to the actual situation. , I will not give an example one by one here. The preset range can be set according to the actual application. For example, the preset range can be set from 0 to 1, still taking 3D CT images as an example. For abdominal CT, the preset window value can be set from -125 to 275. When the preset range is 0 to 1, voxels with a voxel value less than or equal to -125 can be reset to a voxel value 0, and voxels with a voxel value greater than or equal to 275 can be reset to a voxel uniformly. Voxel value 1, you can reset voxels with voxel values between -125 to 275 to voxel values between 0 and 1, which can help enhance the contrast between different organs in the image, thereby improving the accuracy of the model extraction feature.
在又一个可能的实施场景中，预处理还可以包括在三维医学图像的至少部分体素中加入高斯噪声。至少部分体素可以根据实际应用进行设置，例如，三维医学图像的1/3体素，或者，三维医学图像的1/2体素，或者，三维医学图像的全部体素，在此不做限定。通过在三维医学图像的至少部分体素中加入高斯噪声，能够使得后续在三维医学图像和未加入高斯噪声的三维医学图像基础上裁剪得到的二维的待伪标注医学图像，故能够有利于实现数据增广，提高数据多样性，提升后续模型训练的准确性。In another possible implementation scenario, the preprocessing may also include adding Gaussian noise to at least part of the voxels of the three-dimensional medical image. At least part of the voxels can be set according to actual applications, for example, 1/3 voxels of 3D medical images, or 1/2 voxels of 3D medical images, or all voxels of 3D medical images, which are not limited here. . By adding Gaussian noise to at least part of the voxels of the three-dimensional medical image, the subsequent two-dimensional medical image to be pseudo-labeled can be cropped on the basis of the three-dimensional medical image and the three-dimensional medical image without Gaussian noise, so it can be beneficial to implementation Data augmentation, increase data diversity, and improve the accuracy of subsequent model training.
步骤S112：分别利用与每一未标注器官对应的单器官检测模型对待伪标注医学图像进行检测，以得到每个未标注器官的器官预测区域。Step S112: Use the single-organ detection model corresponding to each unlabeled organ to detect the pseudo-labeled medical image to obtain the organ prediction area of each unlabeled organ.
在一个实施场景中，每一未标注器官对应的单器官检测模型可以是利用标注有未标注器官的单器官数据集训练得到的，例如，与肾脏对应的单器官检测模型可以是利用标注有肾脏的单器官数据集训练得到的，与脾脏对应的单器官检测模型可以是利用标注有脾脏的单器官数据集训练得到的，其他器官可以以此类推，在此不再一一举例。In an implementation scenario, the single-organ detection model corresponding to each unlabeled organ may be trained using a single-organ data set labeled with unlabeled organs. For example, the single-organ detection model corresponding to the kidney may be obtained by using a single-organ detection model labeled with kidneys. The single-organ detection model corresponding to the spleen can be trained using the single-organ data set labeled with the spleen, and the other organs can be deduced by analogy, so we will not give an example one by one here.
在另一个实施场景中，待伪标注医学图像中还可以包括至少一个已标注器官，则可以利用包括已标注器官的待伪标注医学图像，对待伪标注医学图像中的已标注器官对应的单器官检测模型进行训练，从而得到对应的单器官检测模型。例如，待伪标注医学图像中包括已标注肝脏，则可以利用包括已标注肝脏的待伪标注医学图像，对肝脏对应的单器官检测模型进行训练，从而得到肝脏对应的单器官检测模型，其他器官可以以此类推，在此不再一一举例。In another implementation scenario, the medical image to be pseudo-labeled may also include at least one labeled organ, and the medical image to be pseudo-labeled including the labeled organ may be used to treat a single organ corresponding to the labeled organ in the pseudo-labeled medical image. The detection model is trained to obtain the corresponding single-organ detection model. For example, if the medical image to be pseudo-labeled includes the labeled liver, the medical image to be pseudo-labeled that includes the labeled liver can be used to train the single-organ detection model corresponding to the liver to obtain the single-organ detection model corresponding to the liver. It can be deduced by analogy, and I will not give examples one by one here.
此外，单器官检测模型可以包括Mask R-CNN(Mask Region with Convolutional Neural Network)、FCN(Fully Convolutional Network，全卷积网络)、PSP-net(Pyramid Scene Parsing Network，金字塔场景分析网络)中的任一者，或者，单器官检测模型还可以是set-net、U-net等，可以根据实际情况进行设置，在此不做限定。In addition, single-organ detection models can include any of Mask R-CNN (Mask Region with Convolutional Neural Network), FCN (Fully Convolutional Network), and PSP-net (Pyramid Scene Parsing Network). One, alternatively, the single-organ detection model can also be set-net, U-net, etc., which can be set according to actual conditions, which is not limited here.
通过利用与每一未标注器官对应的单器官检测模型对待伪标注医学图像进行检测，能够得到每个未标注器官的器官预测区域。以待伪标注医学图像是对腹部扫描得到的图像为例，未标注器官包括肾脏、脾脏、胰腺，利用与肾脏对应的单器官检测模型对待伪标注医学图像进行检测，能够得到肾脏的器官预测区域，利用与脾脏对应的单器官检测模型对待伪标注医学图像进行检测，能够得到脾脏的器官预测区域，利用与胰腺对应的单器官检测模型对待伪标注医学图像检测检测，能够得到胰腺的器官预测区域， 上述利用每一未标注器官对应的单器官检测模型对待伪标注医学图像进行检测的步骤可以同时执行，最终将每个未标注器官的器官预测区域在待伪标注医学图像进行伪标注即可，从而能够提升伪标注的效率，例如，可以同时执行上述利用与肾脏对应的单器官检测模型对待伪标注医学图像进行检测、利用与脾脏对应的单器官检测模型对待伪标注医学图像进行检测，以及利用与胰腺对应的单器官检测模型对待伪标注医学图像检测检测的步骤，最终统一在待伪标注医学图像上对肾脏、脾脏和胰腺的单器官预测区域进行伪标注即可；或者，上述利用每一未标注器官对应的单器官检测模型对待伪标注医学图像进行检测的步骤还可以依次执行，从而无需再将每个未标注器官的器官预测区域在待伪标注医学图像进行伪标注，例如，可以依次执行上述利用与肾脏对应的单器官检测模型对待伪标注医学图像进行检测、利用与脾脏对应的单器官检测模型对待伪标注医学图像进行检测，以及利用与胰腺对应的单器官检测模型对待伪标注医学图像检测检测的步骤，最终得到的待伪标注医学图像中即可包含肾脏、脾脏和胰腺的单器官预测区域。可以根据实际情况进行设置，在此不做限定。By using the single-organ detection model corresponding to each unlabeled organ to detect the pseudo-labeled medical image, the organ prediction area of each unlabeled organ can be obtained. Taking the medical image to be pseudo-labeled is an image obtained by scanning the abdomen as an example, the unlabeled organs include the kidney, spleen, and pancreas. The single-organ detection model corresponding to the kidney is used to detect the pseudo-labeled medical image, and the organ prediction area of the kidney can be obtained. , Use the single-organ detection model corresponding to the spleen to detect the pseudo-labeled medical image to obtain the organ prediction area of the spleen, and use the single-organ detection model corresponding to the pancreas to detect the pseudo-labeled medical image to obtain the organ prediction area of the pancreas The above steps of using the single-organ detection model corresponding to each unlabeled organ to detect the pseudo-labeled medical image can be performed at the same time, and finally the organ prediction area of each unlabeled organ is pseudo-labeled in the pseudo-labeled medical image. Therefore, the efficiency of pseudo-labeling can be improved. For example, the single-organ detection model corresponding to the kidney can be used to detect pseudo-labeled medical images, the single-organ detection model corresponding to the spleen can be used to detect pseudo-labeled medical images, and the use of The single-organ detection model corresponding to the pancreas requires the steps of detecting and detecting pseudo-labeled medical images, and finally uniformly pseudo-labeling the single-organ prediction regions of the kidney, spleen, and pancreas on the medical image to be pseudo-labeled; or, using each of the above The single-organ detection model corresponding to the unlabeled organ can also perform the steps of detecting the pseudo-labeled medical image in sequence, so that it is no longer necessary to pseudo-label the organ prediction region of each unlabeled organ in the pseudo-labeled medical image. For example, you can sequentially Perform the above-mentioned use of the single-organ detection model corresponding to the kidney to detect pseudo-labeled medical images, use the single-organ detection model corresponding to the spleen to detect pseudo-labeled medical images, and use the single-organ detection model corresponding to the pancreas to detect pseudo-labeled medical images In the image detection and detection steps, the final medical image to be pseudo-labeled can include the single-organ prediction regions of the kidney, spleen, and pancreas. It can be set according to the actual situation and is not limited here.
步骤S113：将未标注器官的器官预测区域伪标注为未标注器官的实际区域，并将伪标注后的待伪标注医学图像作为样本医学图像。Step S113: pseudo-label the organ prediction region of the unlabeled organ as the actual region of the unlabeled organ, and use the pseudo-labeled medical image to be pseudo-labeled as a sample medical image.
在得到每个未标注器官的器官预测区域之后，即可将未标注器官的器官预测区域伪标注为未标注器官的实际区域，并将伪标注后的待伪标注医学图像作为样本医学图像。After the organ prediction region of each unlabeled organ is obtained, the organ prediction region of the unlabeled organ can be pseudo-labeled as the actual region of the unlabeled organ, and the pseudo-labeled medical image to be pseudo-labeled can be used as the sample medical image.
区别于前述实施例，通过获取存在至少一个未标注器官的待伪标注医学图像，并利用与每一未标注器官对应的单器官检测模型对待伪标注医学图像进行检测，以得到每个未标注器官的器官预测区域，并将未标注器官的器官预测区域伪标注为未标注器官的实际区域，将伪标注后的待伪标注医学图像作为样本医学图像，能够利用单器官检测模型免去人工对多器官进行标注的工作量，从而能够有利于降低训练用于多器官检测的图像检测模型的人工成本，并提升训练的效率。Different from the foregoing embodiment, by acquiring at least one unlabeled organ to be pseudo-labeled medical image, and using a single organ detection model corresponding to each unlabeled organ to detect the pseudo-labeled medical image, to obtain each unlabeled organ The organ prediction area of the non-labeled organ is pseudo-labeled as the actual area of the unlabeled organ, and the pseudo-labeled medical image after the pseudo-labeling is used as the sample medical image. The single-organ detection model can be used to eliminate the need for manual pairing. The workload of organ labeling can help reduce the labor cost of training an image detection model for multi-organ detection and improve the efficiency of training.
请参阅图3，图3是本公开实施例提供的图像检测模型的训练方法另一实施例的流程示意图。其中，可以包括如下步骤：Please refer to FIG. 3, which is a schematic flowchart of another embodiment of a training method for an image detection model provided by an embodiment of the present disclosure. Among them, the following steps can be included:
步骤S31：获取样本医学图像，其中，样本医学图像伪标注出至少一个未标注器官的实际区域。Step S31: Obtain a sample medical image, where the sample medical image pseudo-labels at least one actual region of an unlabeled organ.
其中，步骤S31可以参阅前述实施例中的相关步骤。Wherein, step S31 can refer to related steps in the foregoing embodiment.
步骤S32：分别利用第一原始检测模型和第二原始检测模型执行对样本医学图像进行检测以得到第一检测结果的步骤。Step S32: using the first original detection model and the second original detection model to perform the step of detecting the sample medical image to obtain the first detection result.
原始检测模型可以包括第一原始检测模型和第二原始检测模型。第一原始检测模型可以包括Mask R-CNN(Mask Region with Convolutional Neural Network)、FCN(Fully Convolutional Network，全卷积网络)、PSP-net(Pyramid Scene Parsing Network，金字塔场景分析网络)中的任一者，此外，第一原始检测模型还可以是set-net、U-net等，可以根据实际情况进行设置，在此不做限定。第二原始检测模型可以包括Mask R-CNN(Mask Region with Convolutional Neural Network)、FCN(Fully Convolutional Network，全卷积网络)、PSP-net(Pyramid Scene Parsing Network，金字塔场景分析网络)中的任一者，此外，第二原始检测模型还可以是set-net、U-net等，可以根据实际情况进行设置，在此不做限定。The original detection model may include a first original detection model and a second original detection model. The first original detection model can include any of Mask R-CNN (Mask Region with Convolutional Neural Network), FCN (Fully Convolutional Network), PSP-net (Pyramid Scene Parsing Network, pyramid scene analysis network) In addition, the first original detection model can also be set-net, U-net, etc., which can be set according to the actual situation, which is not limited here. The second original detection model can include any of Mask R-CNN (Mask Region with Convolutional Neural Network), FCN (Fully Convolutional Network), PSP-net (Pyramid Scene Parsing Network, pyramid scene analysis network) In addition, the second original detection model can also be set-net, U-net, etc., which can be set according to the actual situation, which is not limited here.
利用第一原始检测模型和第二原始检测模型执行对样本医学图像进行检测以得到第一检测结果的步骤，可以参阅前述实施例中的相关步骤，在此不再赘述。在一个实施场景中，第一原始检测模型检测得到的第一检测结果可以包括未标注器官的第一预测区域，或者，第一原始检测模型检测得到的第一检测结果还可以包括未标注器官的第一预测区域和已标注器官的第一预测区域。在另一个实施场景中，第二原始检测模型检测得到的第一检测结果可以包括未标注器官的第一预测区域，或者，第二原始检测模型检测得到的第一检测结果还可以包括未标注器官的第一预测区域和已标注器官的第一预测区域。Using the first original detection model and the second original detection model to perform the step of detecting the sample medical image to obtain the first detection result, please refer to the relevant steps in the foregoing embodiment, which will not be repeated here. In an implementation scenario, the first detection result detected by the first original detection model may include the first prediction area of the unlabeled organ, or the first detection result detected by the first original detection model may also include the unlabeled organ. The first prediction area and the first prediction area of the labeled organ. In another implementation scenario, the first detection result detected by the second original detection model may include the first prediction region of the unlabeled organ, or the first detection result detected by the second original detection model may also include the unlabeled organ The first prediction area of and the first prediction area of the labeled organ.
请结合参阅图4，图4是图像检测模型的训练过程一实施例的示意图。如图4所示，为了便于描述， 第一原始检测模型表示为net1，第二原始检测模型表示为net2。如图4所示，第一原始检测模型net1对样本医学图像进行检测，得到与第一原始检测模型net1对应的第一检测结果，第二原始检测模型net2对样本医学图像进行检测，得到与第二原始检测模型net2对应的第一检测结果。Please refer to FIG. 4 in combination. FIG. 4 is a schematic diagram of an embodiment of the training process of the image detection model. As shown in FIG. 4, for ease of description, the first original detection model is denoted as net1, and the second original detection model is denoted as net2. As shown in Figure 4, the first original detection model net1 detects the sample medical image, and the first detection result corresponding to the first original detection model net1 is obtained. The second original detection model net2 detects the sample medical image, and obtains the first detection result corresponding to the first original detection model net1. 2. The first detection result corresponding to the original detection model net2.
步骤S33：分别利用第一图像检测模型和第二图像检测模型执行对样本医学图像进行检测以得到第二检测结果的步骤。Step S33: using the first image detection model and the second image detection model to perform the step of detecting the sample medical image to obtain the second detection result.
图像检测模型可以包括与第一原始检测模型对应的第一图像检测模型和与第二原始检测模型对应的第二图像检测模型，第一图像检测模型和第二图像检测模型的网络结构、网络参数可以参阅前述实施例中的相关步骤，在此不再赘述。The image detection model may include a first image detection model corresponding to the first original detection model and a second image detection model corresponding to the second original detection model, the network structure and network parameters of the first image detection model and the second image detection model You can refer to the relevant steps in the foregoing embodiment, which will not be repeated here.
利用第一图像检测模型和第二图像检测模型执行对样本医学图像进行检测以得到第二检测结果的步骤，可以参阅前述实施例中的相关步骤，在此不再赘述。在一个实施场景中，第一图像检测模型检测得到的第二检测结果可以包括未标注器官的第二预测区域，或者，第一图像检测模型检测得到的第二检测结果还可以包括未标注器官的第二预测区域和已标注器官的第二预测区域。在另一个实施场景中，第二图像检测模型检测得到的第二检测结果可以包括未标注器官的第二预测区域，或者，第二图像检测模型检测得到的第二检测结果还可以包括未标注器官的第二预测区域和已标注器官的第二预测区域。Using the first image detection model and the second image detection model to perform the step of detecting the sample medical image to obtain the second detection result, please refer to the relevant steps in the foregoing embodiment, which will not be repeated here. In an implementation scenario, the second detection result detected by the first image detection model may include the second prediction area of the unlabeled organ, or the second detection result detected by the first image detection model may also include the unlabeled organ. The second prediction area and the second prediction area of the marked organ. In another implementation scenario, the second detection result detected by the second image detection model may include the second prediction area of the unlabeled organ, or the second detection result detected by the second image detection model may also include the unlabeled organ The second prediction area of and the second prediction area of the labeled organ.
请结合参阅图4，为了便于描述，与第一原始检测模型net1对应的第一图像检测模型表示为EMA net1，与第二原始检测模型net2对应的第二图像检测模型表示为EMA net2。如图4所示，第一图像检测模型EMA net1对样本医学图像进行检测，得到与第一图像检测模型EMA net1对应的第二检测结果，第二图像检测模型EMA net2对样本医学图像进行检测，得到与第二图像检测模型EMA net2对应的第二检测结果。Please refer to FIG. 4 in combination. For ease of description, the first image detection model corresponding to the first original detection model net1 is denoted as EMA net1, and the second image detection model corresponding to the second original detection model net2 is denoted as EMAnet2. As shown in Figure 4, the first image detection model EMAnet1 detects the sample medical image, and the second detection result corresponding to the first image detection model EMAnet1 is obtained, and the second image detection model EMAnet2 detects the sample medical image. Obtain the second detection result corresponding to the second image detection model EMAnet2.
在一个实施场景中，上述步骤S32和步骤S33可以按照先后顺序执行，例如，先执行步骤S32，后执行步骤S33，或者，先执行步骤S33，后执行步骤S32。在另一个实施场景中，上述步骤S32和步骤S33还可以同时执行，可以根据实际应用进行设置，在此不做限定。In an implementation scenario, the above steps S32 and S33 can be performed in a sequential order, for example, step S32 is performed first, and then step S33 is performed, or step S33 is performed first, and then step S32 is performed. In another implementation scenario, the above step S32 and step S33 can also be performed at the same time, and can be set according to actual applications, which is not limited here.
步骤S34：利用第一原始检测模型的第一预测区域分别与实际区域、第二图像检测模型的第二预测区域之间的差异，调整第一原始检测模型的网络参数。Step S34: Use the differences between the first prediction area of the first original detection model and the actual area and the second prediction area of the second image detection model to adjust the network parameters of the first original detection model.
其中，可以利用第一原始检测模型的第一预测区域与伪标注的实际区域之间的差异，确定第一原始检测模型的第一损失值，并利用第一原始检测模型的第一预测区域与第二图像检测模型的第二预测区域之间的差异，确定第一原始检测模型的第二损失值，从而利用第一损失值和第二损失值，调整第一原始检测模型的网络参数。第一损失值和第二损失值的计算方式可以参阅前述实施例中的相关步骤，在此不再赘述。在一个可能的实施场景中，在计算第二损失值的过程中，可以仅对未标注器官的第一预测区域和第二预测区域，从而能够提升第一原始检测模型和第二图像检测模型一致性约束的鲁棒性，进而能够提高图像检测模型的准确性。Wherein, the difference between the first prediction area of the first original detection model and the pseudo-labeled actual area can be used to determine the first loss value of the first original detection model, and the first prediction area of the first original detection model and the pseudo-labeled actual area can be used. The difference between the second prediction regions of the second image detection model determines the second loss value of the first original detection model, so that the first loss value and the second loss value are used to adjust the network parameters of the first original detection model. The calculation methods of the first loss value and the second loss value can refer to the relevant steps in the foregoing embodiment, and will not be repeated here. In a possible implementation scenario, in the process of calculating the second loss value, only the first prediction area and the second prediction area of the unlabeled organs can be calculated, so as to improve the consistency between the first original detection model and the second image detection model. The robustness of sexual constraints can in turn improve the accuracy of the image detection model.
步骤S35：利用第二原始检测模型的第一预测区域分别与实际区域、第一图像检测模型的第二预测区域之间的差异，调整第二原始检测模型的网络参数。Step S35: Use the difference between the first prediction area of the second original detection model and the actual area and the second prediction area of the first image detection model to adjust the network parameters of the second original detection model.
其中，可以利用第二原始检测模型的第一预测区域与伪标注的实际区域之间的差异，确定第二原始检测模型的第一损失值，并利用第二原始检测模型的第一预测区域和第一图像检测模型的第二预测区域之间的差异，确定第二原始检测模型的第二损失值，从而利用第一损失值和第二损失值，调整第二原始检测模型的网络参数。第一损失值和第二损失值的计算方式可以参阅前述实施例中的相关步骤，在此不再赘述。在一个可能的实施场景中，在计算第二损失值的过程中，可以仅对未标注器官的第一预测区域和第二预测区域，从而能够提升第二原始检测模型和第一图像检测模型一致性约束的鲁棒性，进而能够提高图像检测模型的准确性。Wherein, the difference between the first prediction area of the second original detection model and the pseudo-labeled actual area can be used to determine the first loss value of the second original detection model, and the first prediction area and the pseudo-labeled actual area of the second original detection model can be used. The difference between the second prediction regions of the first image detection model determines the second loss value of the second original detection model, so that the first loss value and the second loss value are used to adjust the network parameters of the second original detection model. The calculation methods of the first loss value and the second loss value can refer to the relevant steps in the foregoing embodiment, and will not be repeated here. In a possible implementation scenario, in the process of calculating the second loss value, only the first prediction area and the second prediction area of the unlabeled organ can be calculated, so as to improve the consistency between the second original detection model and the first image detection model. The robustness of sexual constraints can in turn improve the accuracy of the image detection model.
在一个实施场景中，上述步骤S34和步骤S35可以按照先后顺序执行，例如，先执行步骤S34，后执行步骤S35，或者，先执行步骤S35，后执行步骤S34。在另一个实施场景中，上述步骤S24和步骤S35还可以同时执行，可以根据实际应用进行设置，在此不做限定。In an implementation scenario, the above steps S34 and S35 may be performed in a sequential order, for example, step S34 is performed first, and then step S35 is performed, or step S35 is performed first, and then step S34 is performed. In another implementation scenario, the above step S24 and step S35 can also be performed at the same time, and can be set according to actual applications, which is not limited here.
步骤S36：利用第一原始检测模型本次训练以及之前若干次训练时调整后的网络参数，对第一图像检测模型的网络参数进行更新。Step S36: Utilize the network parameters adjusted during the current training of the first original detection model and several previous trainings to update the network parameters of the first image detection model.
其中，可以统计第一原始检测模型在本次训练和之前若干次训练所调整的网络参数的平均值，并将第一图像检测模型的网络参数更新为对应的第一原始检测模型的网络参数的平均值。可以参阅前述实施例中的相关步骤，在此不再赘述。Among them, the average value of the network parameters adjusted by the first original detection model during this training and several previous trainings can be counted, and the network parameters of the first image detection model can be updated to the corresponding network parameters of the first original detection model. average value. You can refer to the relevant steps in the foregoing embodiment, which will not be repeated here.
请结合参阅图4，可以统计第一原始检测模型net1在本次训练和之前若干次训练所调整的网络参数的平均值，并将第一图像检测模型EMA net1的网络参数更新为第一原始检测模型net1网络参数的平均值。Please refer to Figure 4, you can count the average of the network parameters adjusted by the first original detection model net1 in this training and several previous trainings, and update the network parameters of the first image detection model EMA net1 to the first original detection The average value of the network parameters of the model net1.
步骤S37：利用第二原始检测模型本次训练以及之前若干次训练时调整后的网络参数，对第二图像检测模型的网络参数进行更新。Step S37: The network parameters of the second image detection model are updated by using the network parameters adjusted during the current training of the second original detection model and several previous trainings.
其中，可以统计第二原始检测模型在本次训练和之前若干次训练所调整的网络参数的平均值，并将第二图像检测模型的网络参数更新为对应的第二原始检测模型的网络参数的平均值。可以参阅前述实施例中的相关步骤，在此不再赘述。Among them, the average value of the network parameters adjusted by the second original detection model during this training and several previous trainings can be counted, and the network parameters of the second image detection model can be updated to the corresponding network parameters of the second original detection model. average value. You can refer to the relevant steps in the foregoing embodiment, which will not be repeated here.
请结合参阅图4，可以统计第二原始检测模型net2在本次训练和之前若干次训练所调整的网络参数的平均值，并将第二图像检测模型EMA net2的网络参数更新为第二原始检测模型net2网络参数的平均值。Please refer to Figure 4, you can count the average value of the network parameters adjusted by the second original detection model net2 in this training and several previous trainings, and update the network parameters of the second image detection model EMA net2 to the second original detection The average value of the network parameters of the model net2.
在一个实施场景中，上述步骤S36和步骤S37可以按照先后顺序执行，例如，先执行步骤S36，后执行步骤S37，或者，先执行步骤S37，后执行步骤S36。在另一个实施场景中，上述步骤S36和步骤S37还可以同时执行，可以根据实际应用进行设置，在此不做限定。In an implementation scenario, the above steps S36 and S37 can be performed in a sequential order, for example, step S36 is performed first, and then step S37, or step S37 is performed first, and step S36 is performed later. In another implementation scenario, the above step S36 and step S37 can also be performed at the same time, and can be set according to actual applications, which is not limited here.
在一个实施场景中，在对第一图像检测模型和第二图像检测模型的网络参数进行更新之后，在不满足预设训练结束条件的情况下，可以重新执行上述步骤S32以及后续步骤，以继续对第一原始检测模型和第二原始检测模型的网络参数进行调整，并对与第一原始检测模型对应的第一图像检测模型的网络参数和与第二原始检测模型对应的第二图像检测模型的网络参数进行更新。在一个可能的实施场景中，预设训练结束条件可以包括：当前训练次数达到预设次数阈值(如，500次、1000次等)、第一原始检测模型和第二原始检测模型的损失值小于一预设损失阈值中的任一者，在此不做限定。在另一个可能的实施场景中，在训练结束后，可以将第一图像检测模型、第二图像检测模型中的任一者作为后续图像检测的网络模型，从而能够直接得到待检测医学图像中多个器官对应的区域，进而能够免去利用多个单器官检测对待检测医学图像进行分别检测的操作，故能够降低检测计算量。In an implementation scenario, after the network parameters of the first image detection model and the second image detection model are updated, if the preset training end condition is not met, the above step S32 and subsequent steps can be re-executed to continue Adjust the network parameters of the first original detection model and the second original detection model, and adjust the network parameters of the first image detection model corresponding to the first original detection model and the second image detection model corresponding to the second original detection model The network parameters are updated. In a possible implementation scenario, the preset training end conditions may include: the current number of training times reaches the preset number threshold (eg, 500 times, 1000 times, etc.), and the loss values of the first original detection model and the second original detection model are less than Any one of a preset loss threshold is not limited here. In another possible implementation scenario, after the training is completed, any one of the first image detection model and the second image detection model can be used as the network model for subsequent image detection, so that the number of medical images to be detected can be directly obtained. The area corresponding to each organ can eliminate the need to use multiple single organs to detect the medical image to be detected separately, so the amount of detection calculation can be reduced.
区别于前述实施例，将原始检测模型设置为包括第一原始检测模型和第二原始检测模型，且图像检测模型设置为包括与第一原始检测模型对应的第一图像检测模型和与第二原始检测模型对应的第二图像检测模型，并分别利用第一原始检测模型和第二原始检测模型执行对样本医学图像进行检测以得到第一检测结果的步骤，并分别利用第一图像检测模型和第二检测模型执行对样本医学图像进行检测以得到第二检测结果的步骤，从而利用第一原始检测模型的第一预测区域分别与实际区域、第二图像检测模型的第二预测区域之间的差异，调整第一原始检测模型的网络参数，并利用第二原始检测模型的第一预测区域分别与实际区域、第一图像检测模型的第二预测区域之间的差异，调整第二原始检测模型的网络参数，故能够利用与第一原始检测模型对应的第一图像检测模型监督第二原始检测模型的训练，利用与第二原始检测模型对应的第二图像检测模型监督第一原始检测模型的训练，故能够进一步约束网络参数在 多次训练过程中由于伪标注的真实区域所产生的累积误差，提高图像检测模型的准确性。Different from the foregoing embodiment, the original detection model is set to include the first original detection model and the second original detection model, and the image detection model is set to include the first image detection model corresponding to the first original detection model and the second original detection model. Detect the second image detection model corresponding to the model, and use the first original detection model and the second original detection model to perform the step of detecting the sample medical image to obtain the first detection result, and use the first image detection model and the first image detection model respectively. The second detection model executes the step of detecting the sample medical image to obtain the second detection result, thereby using the difference between the first prediction area of the first original detection model and the actual area and the second prediction area of the second image detection model. , Adjust the network parameters of the first original detection model, and use the difference between the first prediction area of the second original detection model and the actual area and the second prediction area of the first image detection model to adjust the second original detection model Network parameters, so the first image detection model corresponding to the first original detection model can be used to supervise the training of the second original detection model, and the second image detection model corresponding to the second original detection model can be used to supervise the training of the first original detection model. Therefore, it is possible to further constrain the cumulative error of the network parameters due to the pseudo-labeled real region during multiple training processes, and improve the accuracy of the image detection model.
请参阅图5，图5是本公开实施例提供的图像检测方法一实施例的流程示意图。其中，可以包括如下步骤：Please refer to FIG. 5, which is a schematic flowchart of an embodiment of an image detection method provided by an embodiment of the present disclosure. Among them, the following steps can be included:
步骤S51：获取待检测医学图像，其中，待检测医学图像中包含多个器官。Step S51: Obtain a medical image to be tested, where the medical image to be tested contains multiple organs.
待检测医学图像可以包括CT图像、MR图像，在此不做限定。在一个可能的实施场景中，待检测医学图像可以是对腹部、胸部、头颅等部位进行扫描得到的，可以根据实际应用情况进行设置，在此不做限定。例如，对腹部进行扫描，待检测医学图像中的器官可以包括：肾脏、脾脏、肝脏、胰腺等；或者，对胸部进行扫描，待检测医学图像中的器官可以包括：心脏、肺叶、甲状腺等；或者，对头颅进行扫描，待检测医学图像中的器官可以包括：脑干、小脑、间脑、端脑等。The medical images to be detected may include CT images and MR images, which are not limited here. In a possible implementation scenario, the medical image to be detected can be obtained by scanning the abdomen, chest, head, etc., and can be set according to actual application conditions, which is not limited here. For example, when scanning the abdomen, the organs in the medical image to be tested may include: kidney, spleen, liver, pancreas, etc.; or scanning the chest, the organs in the medical image to be tested may include: heart, lung lobes, thyroid, etc.; Alternatively, the head is scanned, and the organs in the medical image to be detected may include: brain stem, cerebellum, diencephalon, and telencephalon.
步骤S52：利用图像检测模型对待检测医学进行检测，得到多个器官的预测区域。Step S52: Use the image detection model to detect the medicine to be detected to obtain predicted regions of multiple organs.
图像检测模型是利用上述任一图像检测模型的训练方法实施例中的步骤训练得到的，可以参阅前述实施例中的相关步骤，在此不再赘述。通过利用图像检测模型对待检测医学图像进行检测，能够直接得到多个器官的预测区域，进而能够免去利用多个单器官检测对待检测医学图像进行分别检测的操作，故能够降低检测计算量。The image detection model is obtained by training using the steps in any of the above-mentioned image detection model training method embodiments. You can refer to the relevant steps in the foregoing embodiment, which will not be repeated here. By using the image detection model to detect the medical image to be detected, the predicted regions of multiple organs can be directly obtained, and the operation of using multiple single organs to detect the medical image to be detected can be avoided, and the amount of detection calculation can be reduced.
上述方案，利用上述任一图像检测模型的训练方法实施例中的步骤训练得到的图像检测模型对待检测医学图像检测检测，得到多个器官的预测区域，能够在多器官检测的过程中，提高检测准确性。In the above solution, the image detection model trained by using the steps in the embodiment of the training method of any of the above-mentioned image detection models detects and detects the medical image to be detected, and obtains the predicted regions of multiple organs, which can improve the detection in the process of multiple organ detection. accuracy.
请参阅图6，图6是本公开实施例提供的图像检测模型的训练装置一实施例的框架示意图。图像检测模型的训练装置60包括图像获取模块61、第一检测模块62、第二检测模块63、参数调整模块64，图像获取模块61被配置为获取样本医学图像，其中，样本医学图像伪标注出至少一个未标注器官的实际区域；第一检测模块62被配置为利用原始检测模型对样本医学图像进行检测以得到第一检测结果，其中，第一检测结果包括未标注器官的第一预测区域；以及，第二检测模块63被配置为利用图像检测模型对样本医学图像进行检测以得到第二检测结果，其中，第二检测结果包括未标注器官的第二预测区域，图像检测模型的网络参数是利用原始检测模型的网络参数确定的；参数调整模块64被配置为利用第一预测区域分别与实际区域、第二预测区域之间的差异，调整原始检测模型的网络参数。Please refer to FIG. 6. FIG. 6 is a schematic diagram of an embodiment of an image detection model training apparatus provided by an embodiment of the present disclosure. The training device 60 for the image detection model includes an image acquisition module 61, a first detection module 62, a second detection module 63, and a parameter adjustment module 64. The image acquisition module 61 is configured to acquire sample medical images, wherein the sample medical images are pseudo-labeled At least one actual region of an unlabeled organ; the first detection module 62 is configured to use the original detection model to detect the sample medical image to obtain a first detection result, where the first detection result includes the first predicted region of the unlabeled organ; And, the second detection module 63 is configured to use the image detection model to detect the sample medical image to obtain a second detection result, wherein the second detection result includes a second predicted region of an unlabeled organ, and the network parameter of the image detection model is Determined by using the network parameters of the original detection model; the parameter adjustment module 64 is configured to adjust the network parameters of the original detection model by using the differences between the first prediction area and the actual area and the second prediction area, respectively.
上述方案，通过获取样本医学图像，且样本医学图像伪标注出至少一个未标注器官的实际区域，故样本医学图像中无需对多器官进行真实标注，从而利用原始检测模型对样本医学图像检测检测以得到包含未标注器官的第一预设区域的第一检测结果，并利用图像检测模型对样本医学图像进行检测以得到包含未标注器官的第二预测区域的第二检测结果，进而利用第一预测区域分别与实际区域、第二预测区域之间的差异，调整原始检测模型的网络参数，且图像检测模型的网络参数是利用原始检测模型的网络参数确定的，故能够使得图像检测模型监督原始检测模型的训练，故能够约束网络参数在多次训练过程中由于伪标注的真实区域所产生的累积误差，提高图像检测模型的准确性，从而使得图像检测模型得以准确地监督原始检测模型进行训练，进而使得原始检测模型在训练过程中能够准确地调整其网络参数，故此，能够在多器官检测的过程中，提升图像检测模型的检测准确性。In the above solution, the sample medical image is acquired, and the sample medical image is pseudo-labeled with at least one actual region of an unlabeled organ, so there is no need to actually label multiple organs in the sample medical image, and the original detection model is used to detect the sample medical image. Obtain the first detection result of the first preset region containing the unlabeled organ, and use the image detection model to detect the sample medical image to obtain the second detection result of the second prediction region containing the unlabeled organ, and then use the first prediction The difference between the area and the actual area and the second predicted area, adjust the network parameters of the original detection model, and the network parameters of the image detection model are determined by the network parameters of the original detection model, so the image detection model can supervise the original detection The training of the model can constrain the cumulative error of the network parameters due to the pseudo-labeled real area during multiple training processes, and improve the accuracy of the image detection model, so that the image detection model can accurately supervise the training of the original detection model. In turn, the original detection model can accurately adjust its network parameters during the training process. Therefore, the detection accuracy of the image detection model can be improved in the process of multi-organ detection.
在一些实施例中，原始检测模型包括第一原始检测模型和第二原始检测模型，图像检测模型包括与第一原始检测模型对应的第一图像检测模型和与第二原始检测模型对应的第二图像检测模型，第一检测模块62还被配置为分别利用第一原始检测模型和第二原始检测模型执行对样本医学图像进行检测以得到第一检测结果的步骤，第二检测模型63还被配置为分别利用第一图像检测模型和第二图像检测模型执行对样本医学图像进行检测以得到第二检测结果的步骤，参数调整模块64还被配置为利用第一原始检测模型的第一预测区域分别与实际区域、第二图像检测模型的第二预测区域之间的差异，调整第一原始检测模型的网络参数，参数调整模块64还还被配置为利用第二原始检测模型的第一预测区域分别与 实际区域、第一图像检测模型的第二预测区域之间的差异，调整第二原始检测模型的网络参数。In some embodiments, the original detection model includes a first original detection model and a second original detection model, and the image detection model includes a first image detection model corresponding to the first original detection model and a second image detection model corresponding to the second original detection model. Image detection model, the first detection module 62 is also configured to use the first original detection model and the second original detection model to perform the step of detecting the sample medical image to obtain the first detection result, and the second detection model 63 is also configured In order to use the first image detection model and the second image detection model to perform the step of detecting the sample medical image to obtain the second detection result, the parameter adjustment module 64 is further configured to use the first prediction area of the first original detection model respectively. The difference between the actual area and the second prediction area of the second image detection model is adjusted to adjust the network parameters of the first original detection model. The parameter adjustment module 64 is also configured to use the first prediction area of the second original detection model. The difference between the actual area and the second prediction area of the first image detection model is adjusted to adjust the network parameters of the second original detection model.
区别于前述实施例，将原始检测模型设置为包括第一原始检测模型和第二原始检测模型，且图像检测模型设置为包括与第一原始检测模型对应的第一图像检测模型和与第二原始检测模型对应的第二图像检测模型，并分别利用第一原始检测模型和第二原始检测模型执行对样本医学图像进行检测以得到第一检测结果的步骤，并分别利用第一图像检测模型和第二检测模型执行对样本医学图像进行检测以得到第二检测结果的步骤，从而利用第一原始检测模型的第一预测区域分别与实际区域、第二图像检测模型的第二预测区域之间的差异，调整第一原始检测模型的网络参数，并利用第二原始检测模型的第一预测区域分别与实际区域、第一图像检测模型的第二预测区域之间的差异，调整第二原始检测模型的网络参数，故能够利用与第一原始检测模型对应的第一图像检测模型监督第二原始检测模型的训练，利用与第二原始检测模型对应的第二图像检测模型监督第一原始检测模型的训练，故能够进一步约束网络参数在多次训练过程中由于伪标注的真实区域所产生的累积误差，提高图像检测模型的准确性。Different from the foregoing embodiment, the original detection model is set to include the first original detection model and the second original detection model, and the image detection model is set to include the first image detection model corresponding to the first original detection model and the second original detection model. Detect the second image detection model corresponding to the model, and use the first original detection model and the second original detection model to perform the step of detecting the sample medical image to obtain the first detection result, and use the first image detection model and the first image detection model respectively. The second detection model performs the step of detecting the sample medical image to obtain the second detection result, thereby using the difference between the first prediction area of the first original detection model and the actual area and the second prediction area of the second image detection model. , Adjust the network parameters of the first original detection model, and use the difference between the first prediction area of the second original detection model and the actual area and the second prediction area of the first image detection model to adjust the second original detection model Network parameters, so the first image detection model corresponding to the first original detection model can be used to supervise the training of the second original detection model, and the second image detection model corresponding to the second original detection model can be used to supervise the training of the first original detection model. Therefore, it is possible to further constrain the cumulative error of the network parameters due to the pseudo-labeled real area during multiple training processes, and improve the accuracy of the image detection model.
在一些实施例中，参数调整模块64包括第一损失确定子模块，被配置为利用第一预测区域和实际区域之间的差异，确定原始检测模型的第一损失值，参数调整模块64包括第二损失确定子模块，被配置为利用第一预测区域和第二预测区域之间的差异，确定原始检测模型的第二损失值，参数调整模块64包括参数调整子模块，被配置为利用第一损失值和第二损失值，调整原始检测模型的网络参数。In some embodiments, the parameter adjustment module 64 includes a first loss determination sub-module configured to use the difference between the first prediction area and the actual area to determine the first loss value of the original detection model, and the parameter adjustment module 64 includes a first loss value. The second loss determination sub-module is configured to use the difference between the first prediction area and the second prediction area to determine the second loss value of the original detection model. The parameter adjustment module 64 includes a parameter adjustment sub-module configured to use the first The loss value and the second loss value adjust the network parameters of the original detection model.
区别于前述实施例，通过第一预测区域和实际区域之间的差异，确定原始检测模型的第一损失值，并通过第一预测区域和第二预测区域之间的差异，确定原始检测模型的第二损失值，并利用第一损失值和第二损失值，调整原始检测模型的网络参数，从而能够从原始检测模型预测出的第一预测区域分别和伪标注的实际区域、对应的图像检测模型预测出的第二预测区域之间差异这两个维度来度量原始检测模型的损失，有利于提高损失计算的准确性，从而能够有利于提高原始检测模型网络参数的准确性，进而能够有利于提升图像检测模型的准确性。Different from the foregoing embodiment, the first loss value of the original detection model is determined by the difference between the first prediction area and the actual area, and the difference between the first prediction area and the second prediction area is used to determine the value of the original detection model. The second loss value, and use the first loss value and the second loss value to adjust the network parameters of the original detection model, so that the first prediction area predicted from the original detection model can be detected with the pseudo-labeled actual area and the corresponding image respectively. The two dimensions of the difference between the second prediction regions predicted by the model are used to measure the loss of the original detection model, which is conducive to improving the accuracy of the loss calculation, which can help improve the accuracy of the network parameters of the original detection model, which in turn can help Improve the accuracy of the image detection model.
在一些实施例中，第一损失确定子模块包括焦点损失确定单元，被配置为利用焦点损失函数对第一预测区域和实际区域进行处理，得到焦点第一损失值，第一损失确定子模块包括集合相似度损失确定单元，被配置为利用集合相似度损失函数对第一预测区域和实际区域进行处理，得到集合相似度第一损失值，第二损失确定子模块还被配置为利用一致性损失函数对第一预测区域和第二预测区域进行处理，得到第二损失值，参数调整子模块包括加权处理单元，被配置为对第一损失值和第二损失值进行加权处理，得到加权损失值，参数调整子模块包括参数调整单元，被配置为利用加权损失值，调整原始检测模型的网络参数。In some embodiments, the first loss determination submodule includes a focus loss determination unit configured to process the first prediction area and the actual area using a focus loss function to obtain the first focus loss value, and the first loss determination submodule includes The collective similarity loss determination unit is configured to use the collective similarity loss function to process the first prediction region and the actual region to obtain the first loss value of the collective similarity, and the second loss determination sub-module is also configured to use the consistency loss The function processes the first prediction area and the second prediction area to obtain the second loss value. The parameter adjustment sub-module includes a weighting processing unit configured to perform weighting processing on the first loss value and the second loss value to obtain the weighted loss value , The parameter adjustment sub-module includes a parameter adjustment unit configured to adjust the network parameters of the original detection model by using the weighted loss value.
区别于前述实施例，通过利用焦点损失函数对第一预测区域和实际区域进行处理，得到焦点第一损失值，能够使得模型提升对于难样本的关注度，从而能够有利于提高图像检测模型的准确性；通过利用集合相似度损失函数对第一预测区域和实际区域进行处理，得到集合相似度第一损失值，能够使得模型拟合伪标注的实际区域，从而能够有利于提高图像检测模型的准确性；通过利用一致性损失函数对第一预测区域和第二预测区域进行处理，得到第二损失值，从而能够提高原始模型和图像检测模型预测的一致性，进而能够有利于提高图像检测模型的准确性；通过对第一损失值和第二损失值进行加权处理，得到加权损失值，并利用加权损失值，调整原始检测模型的网络参数，能够平衡各损失值在训练过程中的重要程度，从而能够提高网络参数的准确性，进而能够有利于提高图像检测模型的准确性。Different from the foregoing embodiment, by using the focus loss function to process the first prediction area and the actual area to obtain the first focus loss value, the model can increase the focus on difficult samples, which can help improve the accuracy of the image detection model.性; By using the collective similarity loss function to process the first prediction area and the actual area, the first loss value of the collective similarity is obtained, which can make the model fit the pseudo-labeled actual area, which can help improve the accuracy of the image detection model Performance; by using the consistency loss function to process the first prediction area and the second prediction area to obtain the second loss value, which can improve the prediction consistency of the original model and the image detection model, which can further improve the performance of the image detection model Accuracy: By weighting the first loss value and the second loss value, the weighted loss value is obtained, and the weighted loss value is used to adjust the network parameters of the original detection model, which can balance the importance of each loss value in the training process. Thereby, the accuracy of the network parameters can be improved, which in turn can help improve the accuracy of the image detection model.
在一些实施例中，样本医学图像中还包含已标注器官的实际区域，第一检测结果还包括已标注器官的第一预测区域，第二检测结果还包括已标注器官的第二预测区域。第一损失确定子模块还被配置为利用未标注器官和已标注器官的第一预测区域和实际区域之间的差异，确定原始检测模型的第一损失值，第二损失确定子模块还被配置为利用未标注器官的第一预测区域和对应第二预测区域之间的差异，确定 原始检测模型的第二损失值。In some embodiments, the sample medical image further includes the actual region of the labeled organ, the first detection result further includes the first prediction region of the labeled organ, and the second detection result further includes the second prediction region of the labeled organ. The first loss determination submodule is further configured to determine the first loss value of the original detection model by using the difference between the first predicted region and the actual region of the unlabeled organ and the labeled organ, and the second loss determination submodule is also configured In order to use the difference between the first prediction area of the unlabeled organ and the corresponding second prediction area, the second loss value of the original detection model is determined.
区别于前述实施例，通过在样本医学图像中设置已标注器官的实际区域，且第一检测结果中还包括已标注器官的第一预测区域，第二检测结果还包括已标注器官的第二预测区域，并在确定原始检测模型的第一损失值的过程中，综合考虑第一预测区域和实际区域之间的差异，而在确定原始检测模型的第二损失值的过程中，只考虑未标注器官的第一预测区域和对应的第二预测区域之间的差异，从而能够提升原始检测模型和图像检测模型一致性约束的鲁棒性，进而能够提高图像检测模型的准确性。Different from the foregoing embodiment, by setting the actual area of the marked organ in the sample medical image, and the first detection result also includes the first prediction area of the marked organ, the second detection result also includes the second prediction of the marked organ In the process of determining the first loss value of the original detection model, the difference between the first prediction area and the actual area is comprehensively considered, and in the process of determining the second loss value of the original detection model, only the unmarked value is considered The difference between the first prediction region of the organ and the corresponding second prediction region can improve the robustness of the consistency constraint of the original detection model and the image detection model, and thus can improve the accuracy of the image detection model.
在一些实施例中，图像检测模型的训练装置60还包括参数更新模块，被配置为利用本次训练以及之前若干次训练时调整后的网络参数，对图像检测模型的网络参数进行更新。In some embodiments, the training device 60 of the image detection model further includes a parameter update module configured to update the network parameters of the image detection model by using the network parameters adjusted during this training and several previous trainings.
区别于前述实施例，通过利用原始检测模型在本次训练以及之前若干次训练时调整后的网络参数，对图像检测模型的网络参数进行更新，能够进一步约束网络参数在多次训练过程中由于伪标注的真实区域所产生的累积误差，提高图像检测模型的准确性。Different from the foregoing embodiment, the network parameters of the image detection model can be updated by using the network parameters adjusted by the original detection model in this training and several previous trainings, which can further restrict the network parameters in the process of multiple training due to false The cumulative error generated by the marked real area improves the accuracy of the image detection model.
在一些实施例中，参数更新模块包括统计子模块，被配置为统计原始检测模型在本次训练和之前若干次训练所调整的网络参数的平均值，参数更新模块包括更新子模块，被配置为将图像检测模型的网络参数更新为对应的原始检测模型的网络参数的平均值。In some embodiments, the parameter update module includes a statistics sub-module configured to count the average value of the network parameters adjusted by the original detection model during the current training and several previous trainings, and the parameter update module includes an update sub-module configured to Update the network parameters of the image detection model to the average value of the network parameters of the corresponding original detection model.
区别于前述实施例，通过统计原始检测模型在本次训练和之前若干次训练所调整的网络参数的平均值，并将图像检测模型的网络参数更新为对应的原始检测模型的网络参数的平均值，能够有利于快速地约束多次训练过程中所产生的累积误差，提升图像检测模型的准确性。Different from the foregoing embodiment, the average value of the network parameters adjusted by the original detection model during the current training and the previous training is counted, and the network parameters of the image detection model are updated to the average value of the network parameters of the corresponding original detection model. , Which can help to quickly constrain the accumulated errors generated during multiple training sessions and improve the accuracy of the image detection model.
在一些实施例中，图像获取模块61包括图像获取子模块，被配置为获取待伪标注医学图像，其中，待伪标注医学图像存在至少一个未标注器官，图像获取模块61包括单器官检测子模块，被配置为分别利用与每一未标注器官对应的单器官检测模型对待伪标注医学图像进行检测，以得到每个未标注器官的器官预测区域，图像获取模块61包括伪标注子模块，被配置为将未标注器官的器官预测区域伪标注为未标注器官的实际区域，并将伪标注后的待伪标注医学图像作为样本医学图像。In some embodiments, the image acquisition module 61 includes an image acquisition sub-module configured to acquire a medical image to be pseudo-labeled, wherein at least one unlabeled organ exists in the medical image to be pseudo-labeled, and the image acquisition module 61 includes a single-organ detection sub-module , Is configured to detect pseudo-labeled medical images using a single-organ detection model corresponding to each unlabeled organ to obtain the organ prediction area of each unlabeled organ. The image acquisition module 61 includes a pseudo-labeled sub-module and is configured In order to pseudo-label the organ prediction region of the unlabeled organ as the actual region of the unlabeled organ, and use the pseudo-labeled medical image to be pseudo-labeled as the sample medical image.
区别于前述实施例，通过获取存在至少一个未标注器官的待伪标注医学图像，并利用与每一未标注器官对应的单器官检测模型对待伪标注医学图像进行检测，以得到每个未标注器官的器官预测区域，并将未标注器官的器官预测区域伪标注为未标注器官的实际区域，将伪标注后的待伪标注医学图像作为样本医学图像，能够利用单器官检测模型免去人工对多器官进行标注的工作量，从而能够有利于降低训练用于多器官检测的图像检测模型的人工成本，并提升训练的效率。Different from the foregoing embodiment, by acquiring at least one unlabeled organ to be pseudo-labeled medical image, and using a single organ detection model corresponding to each unlabeled organ to detect the pseudo-labeled medical image, to obtain each unlabeled organ The organ prediction area of the non-labeled organ is pseudo-labeled as the actual area of the unlabeled organ, and the pseudo-labeled medical image after the pseudo-labeling is used as the sample medical image. The single-organ detection model can be used to eliminate the need for manual pairing. The workload of organ labeling can help reduce the labor cost of training an image detection model for multi-organ detection and improve the efficiency of training.
在一些实施例中，待伪标注医学图像包括至少一个已标注器官，图像获取模块61还包括单器官训练子模块，被配置为利用待伪标注医学图像，对待伪标注医学图像中的已标注器官对应的单器官检测模型进行训练。In some embodiments, the medical image to be pseudo-labeled includes at least one labeled organ, and the image acquisition module 61 further includes a single organ training sub-module configured to use the medical image to be pseudo-labeled to use the labeled organ in the medical image to be pseudo-labeled. The corresponding single-organ detection model is trained.
区别于前述实施例，在待伪标注医学图像中包括至少一个已标注器官，并利用待伪标注医学图像对待伪标注医学图像中的已标注器官对应的单器官检测模型进行训练，能够提升单器官检测模型的准确性，从而能够有利于提升后续伪标注的准确性，进而能够有利于提升后续训练图像检测模型的准确性。Different from the foregoing embodiment, the medical image to be pseudo-labeled includes at least one labeled organ, and the single-organ detection model corresponding to the labeled organ in the pseudo-labeled medical image is trained by using the medical image to be pseudo-labeled, which can improve the single organ The accuracy of the detection model can thus help to improve the accuracy of subsequent pseudo-labeling, which in turn can help improve the accuracy of the subsequent training image detection model.
在一些实施例中，图像获取子模块包括三维图像获取单元，被配置为获取三维医学图像，图像获取子模块包括预处理单元，被配置为对三维医学图像进行预处理，图像获取子模块包括图像裁剪单元，被配置为将预处理后的三维医学图像进行裁剪处理，得到至少一个二维的待伪标注医学图像。In some embodiments, the image acquisition sub-module includes a three-dimensional image acquisition unit configured to acquire a three-dimensional medical image, the image acquisition sub-module includes a pre-processing unit configured to preprocess the three-dimensional medical image, and the image acquisition sub-module includes an image The cropping unit is configured to perform cropping processing on the preprocessed three-dimensional medical image to obtain at least one two-dimensional medical image to be pseudo-labeled.
区别于前述实施例，通过获取三维医学图像，并对三维医学图像进行预处理，从而对预处理后的三维医学图像进行裁剪处理，得到至少一个二维的待伪标注医学图像，能够有利于得到满足模型训练的医学图像，从而能够有利于提升后续图像检测模型训练的准确性。Different from the foregoing embodiment, by acquiring three-dimensional medical images and preprocessing the three-dimensional medical images, the pre-processed three-dimensional medical images are cropped to obtain at least one two-dimensional medical image to be pseudo-labeled, which can be beneficial to obtain Medical images that meet model training can help improve the accuracy of subsequent image detection model training.
在一些实施例中，预处理单元还被配置为执行以下至少一者：将三维医学图像的体素分辨率调整至 一预设分辨率；利用一预设窗值将三维医学图像的体素值归一化至预设范围内；在三维医学图像的至少部分体素中加入高斯噪声。In some embodiments, the preprocessing unit is further configured to perform at least one of the following: adjust the voxel resolution of the three-dimensional medical image to a preset resolution; use a preset window value to adjust the voxel value of the three-dimensional medical image Normalize to a preset range; add Gaussian noise to at least part of the voxels of the three-dimensional medical image.
区别于前述实施例，将三维医学图像的体素分辨率调整至一预设分辨率，能够有利于后续模型预测处理；利用预设窗值将三维医学图像的体素值归一化至预设范围内，能够有利于模型提取到准确的特征；在三维医学图像的至少部分体素中加入高斯噪声，能够有利于实现数据增广，提高数据多样性，提升后续模型训练的准确性。Different from the foregoing embodiment, adjusting the voxel resolution of the three-dimensional medical image to a preset resolution can facilitate subsequent model prediction processing; the preset window value is used to normalize the voxel value of the three-dimensional medical image to a preset Within the range, it can help the model to extract accurate features; adding Gaussian noise to at least part of the voxels of the three-dimensional medical image can help achieve data augmentation, increase data diversity, and improve the accuracy of subsequent model training.
请参阅图7，图7是本公开实施例提供的图像检测装置一实施例的框架示意图。图像检测装置70包括图像获取模块71和图像检测模块72，图像获取模块71被配置为获取待检测医学图像，其中，待检测医学图像中包含多个器官；图像检测模块72被配置为利用图像检测模型对待检测医学进行检测，得到多个器官的预测区域；其中，图像检测模型是利用上述任一图像检测模型的训练装置实施例中的图像检测模型的训练装置训练得到的。Please refer to FIG. 7. FIG. 7 is a schematic diagram of a framework of an embodiment of an image detection device provided by an embodiment of the present disclosure. The image detection device 70 includes an image acquisition module 71 and an image detection module 72. The image acquisition module 71 is configured to acquire a medical image to be detected, wherein the medical image to be detected contains multiple organs; the image detection module 72 is configured to use image detection The model detects the medicine to be detected to obtain predicted regions of multiple organs; wherein, the image detection model is trained by the training device of the image detection model in any of the above-mentioned image detection model training device embodiments.
上述方案，利用上述任一图像检测模型的训练装置实施例中的图像检测模型的训练装置训练得到的图像检测模型对待检测医学图像检测检测，得到多个器官的预测区域，能够在多器官检测的过程中，提高检测准确性。In the above solution, the image detection model trained by the training device of the image detection model in the embodiment of the training device for any of the above-mentioned image detection models is used for detection and detection of medical images to be detected, and the predicted regions of multiple organs are obtained. In the process, improve the detection accuracy.
请参阅图8，图8是本公开实施例提供的电子设备一实施例的框架示意图。电子设备80包括相互耦接的存储器81和处理器82，处理器82被配置为执行存储器81中存储的程序指令，以实现上述任一图像检测模型的训练方法实施例的步骤，或实现上述任一图像检测方法实施例中的步骤。在一个可能的实施场景中，电子设备80可以包括但不限于：微型计算机、服务器，此外，电子设备80还可以包括笔记本电脑、平板电脑等移动设备，在此不做限定。Please refer to FIG. 8, which is a schematic diagram of a framework of an embodiment of an electronic device provided by an embodiment of the present disclosure. The electronic device 80 includes a memory 81 and a processor 82 that are coupled to each other. The processor 82 is configured to execute program instructions stored in the memory 81 to implement the steps of any of the foregoing image detection model training method embodiments, or implement any of the foregoing. Steps in an embodiment of an image detection method. In a possible implementation scenario, the electronic device 80 may include but is not limited to: a microcomputer and a server. In addition, the electronic device 80 may also include mobile devices such as a notebook computer and a tablet computer, which are not limited herein.
其中，处理器82被配置为控制其自身以及存储器81以实现上述任一图像检测模型的训练方法实施例的步骤，或实现上述任一图像检测方法实施例中的步骤。处理器82还可以称为CPU(Central Processing Unit，中央处理单元)。处理器82可能是一种集成电路芯片，具有信号的处理能力。处理器82还可以是通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。另外，处理器82可以由集成电路芯片共同实现。Wherein, the processor 82 is configured to control itself and the memory 81 to implement the steps of any of the foregoing image detection model training method embodiments, or implement the steps of any of the foregoing image detection method embodiments. The processor 82 may also be referred to as a CPU (Central Processing Unit, central processing unit). The processor 82 may be an integrated circuit chip with signal processing capability. The processor 82 may also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (ASIC), a field programmable gate array (Field-Programmable Gate Array, FPGA) or other Programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like. In addition, the processor 82 may be jointly implemented by an integrated circuit chip.
上述方案，能够在多器官检测的过程中，提高检测准确性。The above solution can improve the accuracy of detection in the process of multi-organ detection.
请参阅图9，图9为本公开实施例提供的计算机可读存储介质一实施例的框架示意图。计算机可读存储介质90存储有能够被处理器运行的程序指令901，程序指令901被配置为实现上述任一图像检测模型的训练方法实施例的步骤，或实现上述任一图像检测方法实施例中的步骤。Please refer to FIG. 9, which is a schematic framework diagram of an embodiment of a computer-readable storage medium provided by an embodiment of the present disclosure. The computer-readable storage medium 90 stores program instructions 901 that can be executed by the processor. The program instructions 901 are configured to implement the steps of any of the foregoing image detection model training method embodiments, or implement any of the foregoing image detection method embodiments. A step of.
上述方案，能够在多器官检测的过程中，提高检测准确性。The above solution can improve the accuracy of detection in the process of multi-organ detection.
本公开实施例所提供的图像检测模型的训练方法或图像检测方法的计算机程序产品，包括存储了程序代码的计算机可读存储介质，所述程序代码包括的指令可被配置为执行上述方法实施例中所述的图像检测模型的训练方法或图像检测方法的步骤，可参见上述方法实施例，在此不再赘述。The training method of the image detection model or the computer program product of the image detection method provided by the embodiments of the present disclosure includes a computer-readable storage medium storing program code, and the instructions included in the program code can be configured to execute the above method embodiments For the training method of the image detection model or the steps of the image detection method described in the above, please refer to the above method embodiment, which will not be repeated here.
本公开实施例还提供一种计算机程序，该计算机程序被处理器执行时实现前述实施例的任意一种方法。该计算机程序产品可以通过硬件、软件或其结合的方式实现。在一个可选实施例中，所述计算机程序产品体现为计算机存储介质，在另一个可选实施例中，计算机程序产品体现为软件产品，例如软件开发包(Software Development Kit，SDK)等等。The embodiments of the present disclosure also provide a computer program, which, when executed by a processor, implements any one of the methods in the foregoing embodiments. The computer program product can be implemented by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and so on.
在本公开所提供的几个实施例中，应该理解到，所揭露的方法和装置，可以通过其它的方式实现。例如，以上所描述的装置实施方式仅仅是示意性的，例如，模块或单元的划分，仅仅为一种逻辑功能划 分，实际实现的过程中可以有另外的划分方式，例如单元或组件可以结合或者可以集成到另一个***，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性、机械或其它的形式。In the several embodiments provided in the present disclosure, it should be understood that the disclosed method and device may be implemented in other ways. For example, the device implementation described above is only illustrative, for example, the division of modules or units is only a logical function division, and there may be other divisions in the actual implementation process, for example, units or components can be combined or It can be integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施方式方案的目的。The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of this embodiment.
另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。In addition, the functional units in the various embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或处理器(processor)执行本公开各个实施方式方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of the present disclosure essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium. , Including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor execute all or part of the steps of the methods of the various embodiments of the present disclosure. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes. .
工业实用性Industrial applicability
本公开实施例通过获取样本医学图像，样本医学图像伪标注出至少一个未标注器官的实际区域；利用原始检测模型对样本医学图像进行检测以得到包括未标注器官的第一预测区域的第一检测结果；利用图像检测模型对样本医学图像进行检测以得到包括未标注器官的第二预测区域的第二检测结果，图像检测模型的网络参数是基于原始检测模型的网络参数确定的；利用第一预测区域分别与实际区域、第二预测区域之间的差异，调整原始检测模型的网络参数。这样，能够在多器官检测的过程中，提高检测准确性。In the embodiments of the present disclosure, a sample medical image is obtained, and the sample medical image pseudo-labels at least one actual region of an unlabeled organ; the original detection model is used to detect the sample medical image to obtain a first detection including the first predicted region of the unlabeled organ Results; use the image detection model to detect the sample medical image to obtain the second detection result including the second prediction area of the unlabeled organ, the network parameters of the image detection model are determined based on the network parameters of the original detection model; use the first prediction The difference between the area and the actual area and the second prediction area respectively, adjust the network parameters of the original detection model. In this way, the detection accuracy can be improved in the process of multi-organ detection.

Claims (19)

  1. 一种图像检测模型的训练方法，包括：An image detection model training method, including:
    获取样本医学图像，其中，所述样本医学图像伪标注出至少一个未标注器官的实际区域；Acquiring a sample medical image, wherein the sample medical image pseudo-labels at least one actual region of an unlabeled organ;
    利用原始检测模型对所述样本医学图像进行检测以得到第一检测结果，其中，所述第一检测结果包括所述未标注器官的第一预测区域；以及，Detecting the sample medical image using an original detection model to obtain a first detection result, wherein the first detection result includes the first prediction region of the unlabeled organ; and,
    利用图像检测模型对所述样本医学图像进行检测以得到第二检测结果，其中，所述第二检测结果包括所述未标注器官的第二预测区域；所述图像检测模型的网络参数是基于所述原始检测模型的网络参数确定的；Use an image detection model to detect the sample medical image to obtain a second detection result, wherein the second detection result includes the second prediction area of the unlabeled organ; the network parameters of the image detection model are based on the The network parameters of the original detection model are determined;
    利用所述第一预测区域分别与所述实际区域、所述第二预测区域之间的差异，调整所述原始检测模型的网络参数。Using differences between the first prediction area and the actual area and the second prediction area to adjust the network parameters of the original detection model.
  2. 根据权利要求1所述的训练方法，其中，所述原始检测模型包括第一原始检测模型和第二原始检测模型，所述图像检测模型包括与所述第一原始检测模型对应的第一图像检测模型和与所述第二原始检测模型对应的第二图像检测模型；The training method according to claim 1, wherein the original detection model includes a first original detection model and a second original detection model, and the image detection model includes a first image detection corresponding to the first original detection model. A model and a second image detection model corresponding to the second original detection model;
    所述利用原始检测模型对所述样本医学图像进行检测以得到第一检测结果，包括：The using the original detection model to detect the sample medical image to obtain the first detection result includes:
    分别利用所述第一原始检测模型和所述第二原始检测模型执行所述对所述样本医学图像进行检测以得到第一检测结果的步骤；Using the first original detection model and the second original detection model to perform the step of detecting the sample medical image to obtain a first detection result;
    所述利用图像检测模型对所述样本医学图像进行检测以得到第二检测结果，包括：The using the image detection model to detect the sample medical image to obtain a second detection result includes:
    分别利用所述第一图像检测模型和第二图像检测模型执行所述对所述样本医学图像进行检测以得到第二检测结果的步骤；Using the first image detection model and the second image detection model to perform the step of detecting the sample medical image to obtain a second detection result;
    所述利用所述第一预测区域分别与所述实际区域、所述第二预测区域之间的差异，调整所述原始检测模型的网络参数，包括：The adjusting the network parameters of the original detection model by using the differences between the first prediction area and the actual area and the second prediction area respectively includes:
    利用所述第一原始检测模型的第一预测区域分别与所述实际区域、所述第二图像检测模型的第二预测区域之间的差异，调整所述第一原始检测模型的网络参数；以及，Using the differences between the first prediction area of the first original detection model and the actual area and the second prediction area of the second image detection model to adjust the network parameters of the first original detection model; and ,
    利用所述第二原始检测模型的第一预测区域分别与所述实际区域、所述第一图像检测模型的第二预测区域之间的差异，调整所述第二原始检测模型的网络参数。The difference between the first prediction area of the second original detection model and the actual area and the second prediction area of the first image detection model is used to adjust the network parameters of the second original detection model.
  3. 根据权利要求1或2所述的训练方法，其中，所述利用所述第一预测区域分别与所述实际区域、所述第二预测区域之间的差异，调整所述原始检测模型的网络参数包括：The training method according to claim 1 or 2, wherein the difference between the first prediction area and the actual area and the second prediction area is used to adjust the network parameters of the original detection model include:
    利用所述第一预测区域和所述实际区域之间的差异，确定所述原始检测模型的第一损失值；以及，Using the difference between the first prediction area and the actual area to determine the first loss value of the original detection model; and,
    利用所述第一预测区域和所述第二预测区域之间的差异，确定所述原始检测模型的第二损失值；Using the difference between the first prediction area and the second prediction area to determine the second loss value of the original detection model;
    利用所述第一损失值和所述第二损失值，调整所述原始检测模型的网络参数。Using the first loss value and the second loss value to adjust the network parameters of the original detection model.
  4. 根据权利要求3所述的训练方法，其中，所述利用所述第一预测区域和所述实际区域之间的差异，确定所述原始检测模型的第一损失值包括以下至少之一：The training method according to claim 3, wherein the determining the first loss value of the original detection model by using the difference between the first prediction area and the actual area comprises at least one of the following:
    利用焦点损失函数对所述第一预测区域和所述实际区域进行处理，得到焦点第一损失值；Processing the first prediction area and the actual area by using a focus loss function to obtain a first focus loss value;
    利用集合相似度损失函数对所述第一预测区域和所述实际区域进行处理，得到集合相似度第一损失值。The first prediction area and the actual area are processed by using the collective similarity loss function to obtain the first loss value of the collective similarity.
  5. 根据权利要求3所述的训练方法，其中，所述利用所述第一预测区域和所述第二预测区域之间的差异，确定所述原始检测模型的第二损失值包括：The training method according to claim 3, wherein the determining the second loss value of the original detection model by using the difference between the first prediction area and the second prediction area comprises:
    利用一致性损失函数对所述第一预测区域和所述第二预测区域进行处理，得到所述第二损失值。The first prediction area and the second prediction area are processed by using a consistency loss function to obtain the second loss value.
  6. 根据权利要求3所述的训练方法，其中，所述利用所述第一损失值和所述第二损失值，调整所述原始检测模型的网络参数包括：The training method according to claim 3, wherein the adjusting the network parameters of the original detection model by using the first loss value and the second loss value comprises:
    对所述第一损失值和所述第二损失值进行加权处理，得到加权损失值；Weighting the first loss value and the second loss value to obtain a weighted loss value;
    利用所述加权损失值，调整所述原始检测模型的网络参数。Using the weighted loss value, adjust the network parameters of the original detection model.
  7. 根据权利要求3至6任一项所述的训练方法，其中，所述样本医学图像中还包含已标注器官的实际区域，所述第一检测结果还包括所述已标注器官的第一预测区域，所述第二检测结果还包括所述已标注器官的第二预测区域；The training method according to any one of claims 3 to 6, wherein the sample medical image further includes the actual region of the labeled organ, and the first detection result further includes the first prediction region of the labeled organ , The second detection result further includes a second prediction area of the marked organ;
    所述利用所述第一预测区域和所述实际区域之间的差异，确定所述原始检测模型的第一损失值，包括：The using the difference between the first prediction area and the actual area to determine the first loss value of the original detection model includes:
    利用所述未标注器官和所述已标注器官的第一预测区域和所述实际区域之间的差异，确定所述原始检测模型的第一损失值；Determine the first loss value of the original detection model by using the difference between the first prediction area and the actual area of the unlabeled organ and the labeled organ;
    所述利用所述第一预测区域和所述第二预测区域之间的差异，确定所述原始检测模型的第二损失值，包括：The using the difference between the first prediction area and the second prediction area to determine the second loss value of the original detection model includes:
    利用所述未标注器官的第一预测区域和对应所述第二预测区域之间的差异，确定所述原始检测模型的第二损失值。Using the difference between the first prediction region of the unlabeled organ and the corresponding second prediction region, the second loss value of the original detection model is determined.
  8. 根据权利要求1至7任一项所述的训练方法，其中，所述利用所述第一预测区域分别与所述实际区域、所述第二预测区域之间的差异，调整所述原始检测模型的网络参数之后，所述方法还包括：The training method according to any one of claims 1 to 7, wherein the difference between the first prediction area and the actual area and the second prediction area is used to adjust the original detection model After the network parameters of, the method further includes:
    利用本次训练以及之前若干次训练时调整后的网络参数，对所述图像检测模型的网络参数进行更新。The network parameters of the image detection model are updated by using the network parameters adjusted during this training and several previous trainings.
  9. 根据权利要求8所述的训练方法，其中，所述利用本次训练以及之前若干次训练时调整后的网络参数，对所述图像检测模型的网络参数进行更新，包括：The training method according to claim 8, wherein said using the network parameters adjusted during this training and several previous trainings to update the network parameters of the image detection model comprises:
    统计所述原始检测模型在本次训练和之前若干次训练所调整的网络参数的平均值；Count the average values of the network parameters adjusted by the original detection model during this training and several previous trainings;
    将所述图像检测模型的网络参数更新为对应的所述原始检测模型的所述网络参数的平均值。The network parameter of the image detection model is updated to the average value of the network parameter of the corresponding original detection model.
  10. 根据权利要求1至9任一项所述的训练方法，其中，所述获取样本医学图像包括：The training method according to any one of claims 1 to 9, wherein said obtaining a sample medical image comprises:
    获取待伪标注医学图像，其中，所述待伪标注医学图像存在至少一个所述未标注器官；Acquiring a medical image to be pseudo-labeled, where at least one unlabeled organ exists in the medical image to be pseudo-labeled;
    分别利用与每一所述未标注器官对应的单器官检测模型对所述待伪标注医学图像进行检测，以得到每个所述未标注器官的器官预测区域；Detecting the medical image to be pseudo-labeled by using a single-organ detection model corresponding to each of the unlabeled organs to obtain the organ prediction region of each of the unlabeled organs;
    将所述未标注器官的器官预测区域伪标注为所述未标注器官的实际区域，并将所述伪标注后的待伪标注医学图像作为所述样本医学图像。The organ prediction region of the unlabeled organ is pseudo-labeled as the actual region of the unlabeled organ, and the pseudo-labeled medical image after the pseudo-labeling is used as the sample medical image.
  11. 根据权利要求10所述的训练方法，其中，所述待伪标注医学图像包括至少一个已标注器官；所述分别利用与每一所述未标注器官对应的单器官检测模型对所述待伪标注医学图像进行检测之前，所述方法还包括：The training method according to claim 10, wherein the medical image to be pseudo-labeled includes at least one labeled organ; and the single-organ detection model corresponding to each of the unlabeled organs is used for each of the unlabeled organs. Before the medical image is detected, the method further includes:
    利用所述待伪标注医学图像，对所述待伪标注医学图像中的已标注器官对应的单器官检测模型进行训练。Using the medical image to be pseudo-labeled, the single-organ detection model corresponding to the labeled organ in the medical image to be pseudo-labeled is trained.
  12. 根据权利要求10所述的训练方法，其中，所述获取待伪标注医学图像，包括：The training method according to claim 10, wherein said acquiring the medical image to be pseudo-labeled comprises:
    获取三维医学图像，并对所述三维医学图像进行预处理；Acquiring a three-dimensional medical image, and preprocessing the three-dimensional medical image;
    将预处理后的所述三维医学图像进行裁剪处理，得到至少一个二维的待伪标注医学图像。The preprocessed three-dimensional medical image is cropped to obtain at least one two-dimensional medical image to be pseudo-labeled.
  13. 根据权利要求12所述的训练方法，其中，所述对所述三维医学图像进行预处理包括以下至少之一：The training method according to claim 12, wherein said preprocessing said three-dimensional medical image comprises at least one of the following:
    将所述三维医学图像的体素分辨率调整至一预设分辨率；Adjusting the voxel resolution of the three-dimensional medical image to a preset resolution;
    利用一预设窗值将所述三维医学图像的体素值归一化至预设范围内；Using a preset window value to normalize the voxel value of the three-dimensional medical image to a preset range;
    在所述三维医学图像的至少部分体素中加入高斯噪声。Gaussian noise is added to at least part of the voxels of the three-dimensional medical image.
  14. 一种图像检测方法，包括：An image detection method, including:
    获取待检测医学图像，其中，所述待检测医学图像中包含多个器官；Acquiring a medical image to be tested, wherein the medical image to be tested includes a plurality of organs;
    利用图像检测模型对所述待检测医学进行检测，得到所述多个器官的预测区域；Use an image detection model to detect the medicine to be detected to obtain the predicted regions of the multiple organs;
    其中，所述图像检测模型是利用权利要求1至13任一项所述的图像检测模型的训练方法训练得到的。Wherein, the image detection model is obtained by training using the training method of the image detection model according to any one of claims 1 to 13.
  15. 一种图像检测模型的训练装置，包括：A training device for an image detection model includes:
    图像获取模块，被配置为获取样本医学图像，其中，所述样本医学图像伪标注出至少一个未标注器官的实际区域；An image acquisition module configured to acquire a sample medical image, wherein the sample medical image pseudo-marks the actual area of at least one unmarked organ;
    第一检测模块，被配置为利用原始检测模型对所述样本医学图像进行检测以得到第一检测结果，其中，所述第一检测结果包括所述未标注器官的第一预测区域；以及，A first detection module configured to detect the sample medical image using an original detection model to obtain a first detection result, wherein the first detection result includes the first prediction region of the unlabeled organ; and,
    第二检测模块，被配置为利用图像检测模型对所述样本医学图像进行检测以得到第二检测结果，其中，所述第二检测结果包括所述未标注器官的第二预测区域，所述图像检测模型的网络参数是基于所述原始检测模型的网络参数确定的；The second detection module is configured to use an image detection model to detect the sample medical image to obtain a second detection result, wherein the second detection result includes a second prediction area of the unlabeled organ, and the image The network parameters of the detection model are determined based on the network parameters of the original detection model;
    参数调整模块，被配置为利用所述第一预测区域分别与所述实际区域、所述第二预测区域之间的差异，调整所述原始检测模型的网络参数。The parameter adjustment module is configured to adjust the network parameters of the original detection model by using the differences between the first prediction area and the actual area and the second prediction area, respectively.
  16. 一种图像检测装置，包括：An image detection device includes:
    图像获取模块，被配置为获取待检测医学图像，其中，所述待检测医学图像中包含多个器官；An image acquisition module configured to acquire a medical image to be detected, wherein the medical image to be detected contains a plurality of organs;
    图像检测模块，被配置为利用图像检测模型对所述待检测医学进行检测，得到所述多个器官的预测区域；An image detection module configured to detect the medicine to be detected by using an image detection model to obtain the predicted regions of the multiple organs;
    其中，所述图像检测模型是利用权利要求15所述的图像检测模型的训练装置训练得到的。Wherein, the image detection model is obtained by training using the image detection model training device of claim 15.
  17. 一种电子设备，包括相互耦接的存储器和处理器，所述处理器被配置为执行所述存储器中存储的程序指令，以实现权利要求1至13任一项所述的图像检测模型的训练方法，或实现权利要求14所述的图像检测方法。An electronic device comprising a memory and a processor coupled to each other, the processor being configured to execute program instructions stored in the memory to implement the training of the image detection model according to any one of claims 1 to 13 Method, or implement the image detection method of claim 14.
  18. 一种计算机可读存储介质，其上存储有程序指令，所述程序指令被处理器执行时实现权利要求1至13任一项所述的图像检测模型的训练方法，或实现权利要求14所述的图像检测方法。A computer-readable storage medium, on which program instructions are stored, when the program instructions are executed by a processor, implement the method for training an image detection model according to any one of claims 1 to 13, or implement the method described in claim 14 Image detection method.
  19. 一种计算机程序，包括计算机可读代码，在所述计算机可读代码在电子设备中运行的过程中，所述电子设备中的处理器执行用于实现权利要求1至13任一项所述的图像检测模型的训练方法，或实现权利要求14所述的图像检测方法。A computer program, comprising computer readable code, in the process of running the computer readable code in an electronic device, the processor in the electronic device executes for realizing any one of claims 1 to 13 The training method of the image detection model, or the realization of the image detection method of claim 14.
PCT/CN2020/140325 2020-04-30 2020-12-28 Image detection method and relevant model training method, relevant apparatuses, and device WO2021218215A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
KR1020217043241A KR20220016213A (en) 2020-04-30 2020-12-28 Image detection method and related model training method and related apparatus and apparatus
JP2021576932A JP2022538137A (en) 2020-04-30 2020-12-28 Image detection method, related model training method, and related devices and equipment

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010362766.XA CN111539947B (en) 2020-04-30 2020-04-30 Image detection method, related model training method, related device and equipment
CN202010362766.X 2020-04-30

Publications (1)

Publication Number Publication Date
WO2021218215A1 true WO2021218215A1 (en) 2021-11-04

Family

ID=71967825

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/140325 WO2021218215A1 (en) 2020-04-30 2020-12-28 Image detection method and relevant model training method, relevant apparatuses, and device

Country Status (5)

Country Link
JP (1) JP2022538137A (en)
KR (1) KR20220016213A (en)
CN (1) CN111539947B (en)
TW (1) TW202145249A (en)
WO (1) WO2021218215A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114391828A (en) * 2022-03-01 2022-04-26 郑州大学 Active psychological nursing intervention system for stroke patient
CN117041531A (en) * 2023-09-04 2023-11-10 无锡维凯科技有限公司 Mobile phone camera focusing detection method and system based on image quality evaluation

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111539947B (en) * 2020-04-30 2024-03-29 上海商汤智能科技有限公司 Image detection method, related model training method, related device and equipment
CN112132206A (en) * 2020-09-18 2020-12-25 青岛商汤科技有限公司 Image recognition method, training method of related model, related device and equipment
CN113850179A (en) * 2020-10-27 2021-12-28 深圳市商汤科技有限公司 Image detection method, and training method, device, equipment and medium of related model
CN112200802B (en) * 2020-10-30 2022-04-26 上海商汤智能科技有限公司 Training method of image detection model, related device, equipment and storage medium
CN112669293A (en) * 2020-12-31 2021-04-16 上海商汤智能科技有限公司 Image detection method, training method of detection model, related device and equipment
CN112785573A (en) * 2021-01-22 2021-05-11 上海商汤智能科技有限公司 Image processing method and related device and equipment
CN112749801A (en) * 2021-01-22 2021-05-04 上海商汤智能科技有限公司 Neural network training and image processing method and device
CN114049344A (en) * 2021-11-23 2022-02-15 上海商汤智能科技有限公司 Image segmentation method, training method of model thereof, related device and electronic equipment
CN114429459A (en) * 2022-01-24 2022-05-03 上海商汤智能科技有限公司 Training method of target detection model and corresponding detection method
CN114155365B (en) * 2022-02-07 2022-06-14 北京航空航天大学杭州创新研究院 Model training method, image processing method and related device

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090116737A1 (en) * 2007-10-30 2009-05-07 Siemens Corporate Research, Inc. Machine Learning For Tissue Labeling Segmentation
CN109166107A (en) * 2018-04-28 2019-01-08 北京市商汤科技开发有限公司 A kind of medical image cutting method and device, electronic equipment and storage medium
CN109658419A (en) * 2018-11-15 2019-04-19 浙江大学 The dividing method of organella in a kind of medical image
CN110097557A (en) * 2019-01-31 2019-08-06 卫宁健康科技集团股份有限公司 Automatic medical image segmentation method and system based on 3D-UNet
CN110188829A (en) * 2019-05-31 2019-08-30 北京市商汤科技开发有限公司 The training method of neural network, the method for target identification and Related product
CN111539947A (en) * 2020-04-30 2020-08-14 上海商汤智能科技有限公司 Image detection method, training method of related model, related device and equipment

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018033154A1 (en) * 2016-08-19 2018-02-22 北京市商汤科技开发有限公司 Gesture control method, device, and electronic apparatus
CN108229267B (en) * 2016-12-29 2020-10-16 北京市商汤科技开发有限公司 Object attribute detection, neural network training and region detection method and device
JP6931579B2 (en) * 2017-09-20 2021-09-08 株式会社Ｓｃｒｅｅｎホールディングス Live cell detection methods, programs and recording media
EP3474192A1 (en) * 2017-10-19 2019-04-24 Koninklijke Philips N.V. Classifying data
JP7325414B2 (en) * 2017-11-20 2023-08-14 コーニンクレッカ フィリップス エヌ ヴェ Training a First Neural Network Model and a Second Neural Network Model
JP7066385B2 (en) * 2017-11-28 2022-05-13 キヤノン株式会社 Information processing methods, information processing equipment, information processing systems and programs
CN109086656B (en) * 2018-06-06 2023-04-18 平安科技（深圳）有限公司 Airport foreign matter detection method, device, computer equipment and storage medium
CN109523526B (en) * 2018-11-08 2021-10-22 腾讯科技（深圳）有限公司 Tissue nodule detection and model training method, device, equipment and system thereof
CN110148142B (en) * 2019-05-27 2023-04-18 腾讯科技（深圳）有限公司 Training method, device and equipment of image segmentation model and storage medium
JP2021039748A (en) * 2019-08-30 2021-03-11 キヤノン株式会社 Information processor, information processing method, information processing system, and program
CN111028206A (en) * 2019-11-21 2020-04-17 万达信息股份有限公司 Prostate cancer automatic detection and classification system based on deep learning
CN111062390A (en) * 2019-12-18 2020-04-24 北京推想科技有限公司 Region-of-interest labeling method, device, equipment and storage medium
CN110969245B (en) * 2020-02-28 2020-07-24 北京深睿博联科技有限责任公司 Target detection model training method and device for medical image

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090116737A1 (en) * 2007-10-30 2009-05-07 Siemens Corporate Research, Inc. Machine Learning For Tissue Labeling Segmentation
CN109166107A (en) * 2018-04-28 2019-01-08 北京市商汤科技开发有限公司 A kind of medical image cutting method and device, electronic equipment and storage medium
CN109658419A (en) * 2018-11-15 2019-04-19 浙江大学 The dividing method of organella in a kind of medical image
CN110097557A (en) * 2019-01-31 2019-08-06 卫宁健康科技集团股份有限公司 Automatic medical image segmentation method and system based on 3D-UNet
CN110188829A (en) * 2019-05-31 2019-08-30 北京市商汤科技开发有限公司 The training method of neural network, the method for target identification and Related product
CN111539947A (en) * 2020-04-30 2020-08-14 上海商汤智能科技有限公司 Image detection method, training method of related model, related device and equipment

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114391828A (en) * 2022-03-01 2022-04-26 郑州大学 Active psychological nursing intervention system for stroke patient
CN117041531A (en) * 2023-09-04 2023-11-10 无锡维凯科技有限公司 Mobile phone camera focusing detection method and system based on image quality evaluation
CN117041531B (en) * 2023-09-04 2024-03-15 无锡维凯科技有限公司 Mobile phone camera focusing detection method and system based on image quality evaluation

Also Published As

Publication number Publication date
CN111539947A (en) 2020-08-14
TW202145249A (en) 2021-12-01
KR20220016213A (en) 2022-02-08
JP2022538137A (en) 2022-08-31
CN111539947B (en) 2024-03-29

Similar Documents

Publication Publication Date Title
WO2021218215A1 (en) Image detection method and relevant model training method, relevant apparatuses, and device
CN109584254B (en) Heart left ventricle segmentation method based on deep full convolution neural network
Bi et al. Automatic liver lesion detection using cascaded deep residual networks
US11941807B2 (en) Artificial intelligence-based medical image processing method and medical device, and storage medium
WO2021128825A1 (en) Three-dimensional target detection method, method and device for training three-dimensional target detection model, apparatus, and storage medium
CN110363760B (en) Computer system for recognizing medical images
Wang et al. CheXLocNet: Automatic localization of pneumothorax in chest radiographs using deep convolutional neural networks
US9142030B2 (en) Systems, methods and computer readable storage media storing instructions for automatically segmenting images of a region of interest
CN109215014B (en) Training method, device and equipment of CT image prediction model and storage medium
EP3961561A1 (en) Method for designing a module for image segmentation
CN109949280B (en) Image processing method, image processing apparatus, device storage medium, and growth evaluation system
Yang et al. A deep learning segmentation approach in free‐breathing real‐time cardiac magnetic resonance imaging
US20220335600A1 (en) Method, device, and storage medium for lesion segmentation and recist diameter prediction via click-driven attention and dual-path connection
KR102328198B1 (en) Method and apparatus for measuring volume of organ using artificial neural network
CN112767504A (en) System and method for image reconstruction
EP3973508A1 (en) Sampling latent variables to generate multiple segmentations of an image
CN111724371A (en) Data processing method and device and electronic equipment
CN116130090A (en) Ejection fraction measuring method and device, electronic device, and storage medium
US20210110520A1 (en) Method and system for simulating and constructing original medical images from one modality to other modality
CN113284145A (en) Image processing method and device, computer readable storage medium and electronic device
CN115862119B (en) Attention mechanism-based face age estimation method and device
CN114787816A (en) Data enhancement for machine learning methods
CN115496703A (en) Pneumonia area detection method and system
US20240177839A1 (en) Image annotation systems and methods
TWI778670B (en) Method and system for pneumonia area detection

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20933616

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021576932

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 20217043241

Country of ref document: KR

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20933616

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 20933616

Country of ref document: EP

Kind code of ref document: A1