CN115170795B - Image small target segmentation method, device, terminal and storage medium - Google Patents
Image small target segmentation method, device, terminal and storage medium Download PDFInfo
- Publication number
- CN115170795B CN115170795B CN202210520684.2A CN202210520684A CN115170795B CN 115170795 B CN115170795 B CN 115170795B CN 202210520684 A CN202210520684 A CN 202210520684A CN 115170795 B CN115170795 B CN 115170795B
- Authority
- CN
- China
- Prior art keywords
- target
- sample image
- result corresponding
- loss
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method, a device, a terminal and a storage medium for segmenting small targets of images. The method and the device can improve the accuracy of segmenting the targets with different sizes in the image.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method, an apparatus, a terminal, and a storage medium for segmenting small objects of an image.
Background
Medical institutions generate a large amount of medical images each year, which account for about 90% of the entire medical data and often contain a large amount of potential information. Currently, these medical image data require a professional radiologist to spend a lot of time and effort in reading and analyzing, and the accuracy of the reading result is easily restricted by the clinical experience of the radiologist. In recent years, deep learning has been highly successful in the field of computer vision, and provides a new idea and a new solution for automated processing of medical images. Image segmentation and detection are very important in imaging diagnostics.
There are often some small objects in medical images that need to be identified. This is determined by the characteristics of the medical image. Unlike natural images, tiny focal zones or small organs are ubiquitous in medical imaging. The existing two-classification segmentation algorithm is influenced by other similar backgrounds around the small organ region when the small organ region is segmented and neglected, namely, the small target comprises targets with different sizes in the same class, and the small target is difficult to segment due to the influence of the large target and the background, so the existing segmentation algorithm cannot have good performance on the small target.
Thus, there is a need for improvements and enhancements in the art.
Disclosure of Invention
In view of the above defects in the prior art, the present invention provides a method, an apparatus, a terminal and a storage medium for segmenting small objects in an image, and aims to solve the problem in the prior art that small objects in an image cannot be segmented well.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows:
in a first aspect of the present invention, a method for segmenting a small image target is provided, the method comprising:
determining target training data in each training data, wherein the target training data comprise a sample image and a target labeling result corresponding to the sample image, inputting the sample image into a target segmentation model, and acquiring a target prediction result corresponding to the sample image output by the target segmentation model;
preprocessing a target labeling result corresponding to the sample image, and dividing the target labeling result corresponding to the sample image into a first target and a second target, wherein the number of pixel points of the first target is greater than that of the second target;
obtaining a first training loss corresponding to the target training data according to the target prediction result corresponding to the sample image and the first target, and obtaining a second training loss corresponding to the target training data according to the target prediction result corresponding to the sample image and the second target;
obtaining a total training loss corresponding to the target training data according to the first training loss and the second training loss;
updating a learnable parameter of the target segmentation model according to the total training loss;
and re-executing the step of determining the target training data in each training data until the learnable parameters of the target segmentation model are converged, wherein the target segmentation model with the converged learnable parameters is used for performing target segmentation on the image to be processed.
The image small target segmentation method comprises the following steps of:
and taking the target with the number of the pixel points larger than a preset threshold value as the first target, and taking the target with the number of the pixel points smaller than the preset threshold value as the second target.
The image small target segmentation method, wherein obtaining a first training loss corresponding to the target training data according to the target prediction result corresponding to the sample image and the first target includes:
taking the intersection of the target prediction result corresponding to the sample image and the first target;
and calculating the first training loss according to the intersection of the target prediction result corresponding to the sample image and the first target.
The image small target segmentation method, wherein the calculating the first training loss according to the intersection of the target prediction result corresponding to the sample image and the first target, includes:
calculating the first training loss using a first formula;
the first formula is:
LOFP＝Pre-LOTP
LOFN＝LGT-LOTP
wherein LOTP is an intersection of a target prediction result corresponding to the sample image and the first target and represents a true positive of the first target, LOFP represents a false positive of the first target, LOFN represents a false negative of the first target, pre is a target prediction result corresponding to the sample image, LGT is the first target, smooth is a constant, and loss is a constant large Is the first training loss.
The image small target segmentation method, wherein obtaining a second training loss corresponding to the target training data according to the target prediction result corresponding to the sample image and the second target includes:
taking the intersection of the target prediction result corresponding to the sample image and the second target;
and calculating the loss of the target prediction result corresponding to the sample image relative to the second target according to the intersection of the target prediction result corresponding to the sample image and the second target.
The image small target segmentation method, wherein the calculating the second training loss according to the intersection of the target prediction result corresponding to the sample image and the second target, includes:
calculating the second training loss using a second formula;
the second formula is:
SOFP＝Pre-SOTP
SOFN＝SGT-SOTP
wherein SOTP is an intersection of the target prediction result corresponding to the sample image and the second target, and represents a true positive of the first target, SOFP represents a false positive of the second target, SOFN represents a false negative of the second target, pre is a target prediction result corresponding to the sample image, SGT is the second target, smooth is a constant, exp = (SGT/(SGT + smooth)), and loss is a constant small Is the second training loss.
The image small target segmentation method, wherein the obtaining of the total training loss corresponding to the target training data according to the first training loss and the second training loss includes:
and performing geometric average on the first training loss and the second training loss to obtain a total training loss corresponding to the target training data.
In a second aspect of the present invention, there is provided an image small object segmentation apparatus, comprising:
the target prediction unit is used for determining target training data in each piece of training data, inputting the sample images into a target segmentation model, and acquiring target prediction results corresponding to the sample images output by the target segmentation model, wherein the target training data comprise sample images and target annotation results corresponding to the sample images;
the data preprocessing module is used for preprocessing a target labeling result corresponding to the sample image and dividing the target labeling result corresponding to the sample image into a first target and a second target, wherein the number of pixel points of the first target is larger than that of the second target;
a loss obtaining module, configured to obtain a first training loss corresponding to the target training data according to a target prediction result corresponding to the sample image and the first target, and obtain a second training loss corresponding to the target training data according to a target prediction result corresponding to the sample image and the second target;
an updating module, configured to obtain a total training loss corresponding to the target training data according to the first training loss and the second training loss;
updating learnable parameters of the target segmentation model according to the total training loss;
and re-executing the step of determining the target training data in each training data until the learnable parameters of the target segmentation model converge.
In a third aspect of the present invention, there is provided a terminal, including a processor, and a computer-readable storage medium communicatively connected to the processor, the computer-readable storage medium being adapted to store a plurality of instructions, and the processor being adapted to call the instructions in the computer-readable storage medium to execute the steps of implementing the image small-object segmentation method according to any one of the above.
In a fourth aspect of the present invention, there is provided a computer readable storage medium storing one or more programs, which are executable by one or more processors, to implement the steps of the image small object segmentation method described in any one of the above.
Compared with the prior art, the image small target segmentation method determines target training data in each training data, the target training data comprise a sample image and a target labeling result corresponding to the sample image, the target labeling result corresponding to the sample image is divided into a first target and a second target, a target prediction result corresponding to the sample image output by a target segmentation model is obtained according to the sample image in the target training data, a first training loss and a second training loss are obtained through calculation according to the target prediction result corresponding to the sample image and the first target and the second target respectively, a total training loss is obtained according to the first training loss and the second training loss, a learnable parameter of the target segmentation model is updated according to the total training loss, and the steps are repeatedly executed until the learnable parameter of the target segmentation model is converged. According to the method, the first target and the second target are respectively calculated and then are fused into the final training loss, so that the influence of the second target in calculating the loss function is improved, and the accuracy in segmenting the targets with different sizes in the image is improved.
Drawings
FIG. 1 is a flowchart of an embodiment of a method for segmenting a small object in an image according to the present invention;
FIG. 2 is a schematic diagram of the position of a liver organ in a CT image;
FIG. 3 is a schematic diagram of a first object and a second object separated according to an embodiment of the image small object segmentation method provided by the present invention;
FIG. 4 is a schematic diagram of a visualization result of an embodiment of a method for segmenting a small image target according to the present invention;
FIG. 5 is a schematic structural diagram of an embodiment of an image small object segmentation apparatus provided in the present invention;
fig. 6 is a schematic diagram illustrating the principle of an embodiment of the terminal according to the present invention.
Detailed Description
In order to make the objects, technical solutions and effects of the present invention clearer and clearer, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. As used herein, the term "and/or" includes all or any element and all combinations of one or more of the associated listed items.
It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The image small target segmentation method provided by the invention can be applied to a terminal with computing power, and the terminal can execute the image small target segmentation method provided by the invention to segment the target in the image to be processed.
Example one
The present embodiment is a method designed for small objects that are difficult to segment in an image. In practical applications, there may be a requirement that a small target needs to be segmented from an image, for example, in a medical CT image, an organ that needs to be segmented from the image sometimes occupies a small portion, as shown in fig. 2, a target liver only occupies a small portion compared to a background region; however, due to the limitation of the imaging technology, the medical image has the disadvantages of non-uniform gray scale, large contrast variation, noise, etc., as shown in fig. 2, all organs have similar gray scales and have a large difference from the background, the target liver has similar gray scales and similar boundary characteristics to the surrounding organs, and especially similar targets with different sizes may exist in the CT images of slices at different positions, such as small liver blocks in a box, and the segmentation effect for small targets in the prior art is not good. In the method provided by the embodiment, in the training process of the segmentation model, the loss is calculated by separating the large target and the small target in one image, so that the influence of the small target in calculating the loss function is improved. The method provided by the embodiment can be used for correctly segmenting the objects with different sizes on the image classification data set with low resolution, non-uniform gray scale and large amount of noise.
As shown in fig. 1, in an embodiment of the image small object segmentation method, the method includes the steps of:
s100, determining target training data in each training data, wherein the target training data comprise a sample image and a target labeling result corresponding to the sample image, inputting the sample image into a target segmentation model, and acquiring a target prediction result corresponding to the sample image output by the target segmentation model.
Each year, medical institutions generate a large amount of medical images, the image data contains a large amount of potential information, when the medical image data needs to be subjected to interpretation analysis, target areas (such as organs, focuses and the like) are firstly segmented in the medical images, and doctors can be well assisted in quantitatively evaluating the effects before and after treatment.
In the method provided by this embodiment, an image to be processed is segmented by a trained segmentation model, the segmentation model is trained based on a plurality of sets of training data, each set of training data includes a sample image of the same type as the image to be processed and a target labeling result corresponding to the sample image, and the target labeling result corresponding to the sample image is information of a target object that is manually labeled in advance on the sample image, including a position and an occupied area of the target object on the sample image. The image to be processed may be a CT image, for example, the CT image including a liver organ in fig. 2, in which case, the sample image is a CT image including a liver organ of different patients, and the target labeling result corresponding to the sample image is an image in which a liver part in the CT image including a liver organ has been manually segmented.
Inputting the sample image into a target segmentation model, and obtaining a target prediction result corresponding to the sample image output by the target segmentation model, wherein the target prediction result comprises prediction information of the size and the occupied area of a target object in the sample image.
S200, preprocessing a target labeling result corresponding to the sample image, and dividing the target labeling result corresponding to the sample image into a first target and a second target, wherein the number of pixel points of the first target is larger than that of the second target.
Referring to fig. 3, a schematic diagram of distinguishing a first target from a second target in an embodiment of the image small target segmentation method provided by the present invention is shown.
Dividing the target labeling result corresponding to the sample image into a first target and a second target, including:
s210, taking the target with the number of the pixel points larger than a preset threshold value as the first target, and taking the target with the number of the pixel points smaller than the preset threshold value as the second target.
Since most of the loss functions existing are insensitive to small objects in the foreground when categorizing because these smaller objects have much fewer pixels inside than the large object. That is, in pixel-level segmentation, there is intra-class imbalance between the foreground large target pixels and the foreground small target pixels, resulting in inaccurate segmentation of the small target region.
Therefore, in this embodiment, before calculating the loss, the target labeling result corresponding to the sample image is preprocessed, the target labeling result corresponding to the sample image is divided into a large target and a small target, and the loss is calculated based on the large target and the small target, respectively. And calculating losses of the large target and the small target respectively, so that the small target can occupy larger weight when calculating the losses and obtain more attention of the model compared with a method for calculating the losses without distinguishing the large target and the small target.
Specifically, in the binary segmentation algorithm, the Large target and the Small target belong to the same class and cannot be directly separated to calculate the loss, so in this embodiment, the Large target and the Small target in the image tag are separated by a preset threshold, a target having a pixel point larger than the preset threshold in the target labeling result corresponding to the sample image is taken as the first target (Large group route, LGT), a target having a pixel point smaller than the preset threshold in the target labeling result corresponding to the sample image is taken as the second target (Small group route, SGT), and then the target and the predicted value (Prediction, pre) are respectively used to calculate the loss.
In this embodiment, the preset threshold is set to 4000 pixels.
Due to different sampling positions, a situation that a larger target in the target labeling results corresponding to the sample image is smaller than a preset threshold value may exist, and at this time, only a unique target exists in the target labeling results corresponding to the sample image.
Specifically, if only a unique target exists in the target labeling result corresponding to the sample image and the number of pixel points of the unique target is smaller than the preset threshold, it is determined that no second target exists, and the unique target is taken as the first target.
S300, obtaining a first training loss corresponding to the target training data according to the target prediction result corresponding to the sample image and the first target, and obtaining a second training loss corresponding to the target training data according to the target prediction result corresponding to the sample image and the second target.
In this embodiment, the obtaining a first training loss corresponding to the target training data according to the target prediction result corresponding to the sample image and the first target includes:
s310, taking the intersection of the target prediction result corresponding to the sample image and the first target;
s320, calculating the first training loss according to the intersection of the target prediction result corresponding to the sample image and the first target.
In a possible implementation manner, the calculating the first training loss according to the intersection of the target prediction result corresponding to the sample image and the first target includes calculating the first training loss by using a first formula, where the specific formula is as follows:
LOTP＝Pre∩LGT
LOFP＝Pre-LOTP
LOFN＝LGT-LOTP
wherein LOTP is an intersection of a target prediction result corresponding to the sample image and the first target, pre is a target prediction result corresponding to the sample image, LGT is the first target, smooth is a constant, and loss is large Is the first training loss.
Specifically, LOTP, LOFP, LOFN represent true positive of the first target, false positive of the first target, and false negative of the first target, respectively, smooth is a constant having a small value for preventing the formula from being meaningless, and smooth may take a value less than 0.1, for example, may take 0.00001.
In this embodiment, the obtaining a second training loss corresponding to the target training data according to the target prediction result corresponding to the sample image and the second target includes:
s330, taking the intersection of the target prediction result corresponding to the sample image and the second target;
s340, calculating the loss relative to the second target in the target prediction result corresponding to the sample image according to the intersection of the target prediction result corresponding to the sample image and the second target.
In a possible implementation manner, the calculating the second training loss according to the intersection of the target prediction result corresponding to the sample image and the second target includes calculating the second training loss by using a second formula, where the specific formula is as follows:
SOTP＝Pre∩SGT
SOFP＝Pre-SOTP
SOFN＝SGT-SOTP
wherein SOTP is an intersection of the target prediction result corresponding to the sample image and the second target, pre is the target prediction result corresponding to the sample image, SGT is the second target, smooth is a constant, exp = (SGT/(SGT + smooth)), and loss small Is the second training loss.
Specifically, SOTP, SOFP, SOFN represent the true positive of the second target, the false positive of the second target and the false negative of the second target, respectively; smooth is a constant with a small value to prevent the formula from being meaningless, and smooth may take a value less than 0.1, for example, may take 0.00001; smooth is a smoothing parameter for preventing denominator from being 0, setting smooth to a constant having a value of 0.1 or less can prevent denominator from being 0, and setting exp = (SGT/(SGT + smooth)) can prevent samples without the second target from having loss at the time of calculation loss small The value of (2) is constant 0, so that the loss value of the whole sample is constant 0, and normal training is influenced. It should be noted that the above is only a calculation formula for calculating the first training loss and the second training loss, and in other implementations, the calculation formula can be integrated into any other distribution and area-based loss function to calculate the first training loss and the second training loss. After the target labeling results corresponding to the sample images are processed separatelyAnd carrying on other loss functions, obtaining the first training loss based on the first target and the target prediction result by adopting a calculation formula of other loss functions, and obtaining the second training loss based on the second target and the target prediction result.
And after the first training loss and the second training loss are obtained, fusing the first training loss and the second training loss to obtain the loss of learnable parameters for updating the segmentation model.
Referring to fig. 1 again, it can be seen that the image small object segmentation method provided by the present invention further includes:
s400, obtaining a total training loss corresponding to the target training data according to the first training loss and the second training loss;
the obtaining of the total training loss corresponding to the target training data according to the first training loss and the second training loss includes:
s410, geometric averaging is conducted on the first training loss and the second training loss, and a total training loss corresponding to the target training data is obtained.
In this embodiment, calculating the total training loss corresponding to the target training data includes fusing the first training loss and the second training loss in a geometric mean manner, where the formula is:
and S500, updating the learnable parameters of the target segmentation model according to the total training loss.
S600, the step of determining the target training data in the training data is executed again until learnable parameters of the target segmentation model are converged, and the target segmentation model after the learnable parameters are converged is used for performing target segmentation on the image to be processed.
And updating the learnable parameters of the target segmentation model according to the total training loss, and then re-executing the step of determining the target training data in each training data until the learnable parameters of the target segmentation model are converged to obtain the target segmentation model with the converged learnable parameters, and using the target segmentation model with the converged learnable parameters for target segmentation of the image to be processed.
The following is a performance evaluation of an image small target segmentation model segmented by using the present embodiment, and four kinds of classical and novel networks are adopted as the segmentation model in the method provided by the present embodiment to verify the method provided by the present embodiment, including UNet, res50-UNet, attention-UNet and CA-Net.
To evaluate the performance of the image small object segmentation model, verification was performed using indices including Dice Similarity Coefficient (DSC), hausdorff distance (95% hd), average Surface Distance (ASD), accuracy (Precision), recall (Recall). The dataset used for validation is the CHAOs-CT dataset in the CHAOs challenge, which segments the liver from a Computed Tomography (CT) dataset. The data set consists of 2874 pictures and corresponding labels for 20 patients, wherein the data set is divided by patient according to a total number 8:2, and only the sample with the second object is retained. As shown in table 1, is the behavior of different loss functions under the classical UNet network:
| Method | DSC | 95％HD | ASD | Precision | Recall |
| BCE loss | 0.9572 | 2.6772 | 0.4482 | 0.9683 | 0.9499 |
| Dice loss | 0.9570 | 2.3615 | 0.5034 | 0.9651 | 0.9526 |
| the method provided by the embodiment | 0.9588 | 2.1858 | 0.3437 | 0.9614 | 0.9588 |
TABLE 1
As can be seen from Table 1, the method provided in this example gave the highest scores on DSC, 95% HD and ASD on classical UNet. This shows that the image small object segmentation method provided by the embodiment is superior to the comparative Dice-loss and BCE-loss in both the whole and the boundary. Meanwhile, because a plurality of small targets exist in the data set, the method provided by the embodiment can better divide the region than other methods, and lower FN is obtained, so that the method provided by the embodiment also obtains a higher score on Recall.
Tables 2, 3 and 4 show the performance of different loss functions in Res50-UNet, attention-UNet and CA-Net networks, respectively:
| Method | DSC | 95％HD | ASD | Precision | Sensitivity |
| BCE loss | 0.9507 | 3.8213 | 1.0631 | 0.9549 | 0.9518 |
| Dice loss | 0.9548 | 3.8154 | 0.7098 | 0.9557 | 0.9584 |
| the method provided by the embodiment | 0.9567 | 2.9782 | 0.9250 | 0.9562 | 0.9605 |
TABLE 2
| Method | DSC | 95％HD | ASD | Precision | Sensitivity |
| BCE loss | 0.9535 | 2.8255 | 0.5651 | 0.9628 | 0.9521 |
| Dice loss | 0.9548 | 2.8156 | 0.5358 | 0.9634 | 0.9516 |
| The method provided by the embodiment | 0.9594 | 2.4861 | 0.5095 | 0.9571 | 0.9645 |
TABLE 3
| Method | DSC | 95％HD | ASD | Precision | Sensitivity |
| BCE loss | 0.9512 | 5.5595 | 1.0777 | 0.9551 | 0.9513 |
| Dice loss | 0.9555 | 5.9419 | 0.6695 | 0.9616 | 0.9527 |
| The method provided by the embodiment | 0.9574 | 3.1452 | 0.4317 | 0.9600 | 0.9592 |
TABLE 4
As can be seen from tables 2, 3 and 4, the segmentation results using the method provided in this example are superior to the most commonly used Dice loss and BCE loss, both in the classical Res50-UNet and Attention-UNet, and in the novel CA-Net.
The visualization result can be shown in fig. 4, and it is obvious that the method provided by the embodiment has better performance on small targets.
In summary, this embodiment provides an image small target segmentation method, where target training data is determined in each training data, where the target training data includes a sample image and a target labeling result corresponding to the sample image, the target labeling result corresponding to the sample image is divided into a first target and a second target, a target prediction result corresponding to the sample image output by a target segmentation model is obtained according to the sample image in the target training data, a first training loss and a second training loss are obtained according to the target prediction result corresponding to the sample image and the first target and the second target, a total training loss is obtained according to the first training loss and the second training loss, a learnable parameter of the target segmentation model is updated according to the total training loss, and the above steps are repeated until the learnable parameter of the target segmentation model converges. By calculating the first training loss and the second training loss respectively, the influence of the second target in calculating the loss function is improved, and the accuracy in segmenting the targets with different sizes in the image is improved.
It should be understood that, although the steps in the flowcharts shown in the figures of the present specification are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, databases, or other media used in embodiments provided herein may include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), rambus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
Example two
Based on the foregoing embodiment, the present invention further provides an image small target segmentation apparatus, as shown in fig. 5, the image small target segmentation apparatus includes:
the target prediction unit is used for determining target training data in each training data, the target training data comprise sample images and target labeling results corresponding to the sample images, the sample images are input into a target segmentation model, and target prediction results corresponding to the sample images output by the target segmentation model are obtained;
the data preprocessing module is used for preprocessing a target labeling result corresponding to the sample image and dividing the target labeling result corresponding to the sample image into a first target and a second target, wherein the number of pixel points of the first target is larger than that of the second target;
a loss obtaining module, configured to obtain a first training loss corresponding to the target training data according to a target prediction result corresponding to the sample image and the first target, and obtain a second training loss corresponding to the target training data according to a target prediction result corresponding to the sample image and the second target;
an updating module, configured to obtain a total training loss corresponding to the target training data according to the first training loss and the second training loss;
updating a learnable parameter of the target segmentation model according to the total training loss;
and re-executing the step of determining the target training data in each training data until the learnable parameters of the target segmentation model converge.
EXAMPLE III
Based on the above embodiments, the present invention further provides a terminal, as shown in fig. 6, where the terminal includes a processor 10 and a memory 20. Fig. 6 shows only some of the components of the terminal, but it is to be understood that not all of the shown components are required to be implemented, and that more or fewer components may be implemented instead.
The memory 20 may in some embodiments be an internal storage unit of the terminal, such as a hard disk or a memory of the terminal. The memory 20 may also be an external storage device of the terminal in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the terminal. Further, the memory 20 may also include both an internal storage unit and an external storage device of the terminal. The memory 20 is used for storing application software installed in the terminal and various data. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, the memory 20 stores an image small object segmentation program 30, and the image small object segmentation program 30 can be executed by the processor 10 to implement the image small object segmentation method of the present application.
The processor 10 may be a Central Processing Unit (CPU), a microprocessor or other chip in some embodiments, and is used for running program codes stored in the memory 20 or Processing data, such as executing the image small target segmentation method.
In one embodiment, when the processor 10 executes the image small object segmentation program 30 in the memory 20, the following steps are implemented:
determining target training data in each piece of training data, wherein the target training data comprise sample images and target labeling results corresponding to the sample images, inputting the sample images into a target segmentation model, and acquiring target prediction results corresponding to the sample images output by the target segmentation model;
preprocessing a target labeling result corresponding to the sample image, and dividing the target labeling result corresponding to the sample image into a first target and a second target, wherein the number of pixel points of the first target is greater than that of the second target;
obtaining a first training loss corresponding to the target training data according to the target prediction result corresponding to the sample image and the first target, and obtaining a second training loss corresponding to the target training data according to the target prediction result corresponding to the sample image and the second target;
obtaining a total training loss corresponding to the target training data according to the first training loss and the second training loss;
updating a learnable parameter of the target segmentation model according to the total training loss;
and re-executing the step of determining the target training data in each training data until the learnable parameters of the target segmentation model are converged, wherein the target segmentation model with the converged learnable parameters is used for performing target segmentation on the image to be processed.
Dividing the target labeling result corresponding to the sample image into a first target and a second target comprises:
and taking the target with the number of the pixel points larger than a preset threshold value as the first target, and taking the target with the number of the pixel points smaller than the preset threshold value as the second target.
Obtaining a first training loss corresponding to the target training data according to the target prediction result corresponding to the sample image and the first target includes:
taking the intersection of the target prediction result corresponding to the sample image and the first target;
and calculating the first training loss according to the intersection of the target prediction result corresponding to the sample image and the first target.
Wherein the calculating the first training loss according to the intersection of the target prediction result corresponding to the sample image and the first target comprises:
calculating the first training loss using a first formula;
the first formula is:
LOFP＝Pre-LOTP
LOFN＝LGT-LOTP
wherein LOTP is an intersection of a target prediction result corresponding to the sample image and the first target and represents a true positive of the first target, LOFP represents a false positive of the first target, LOFN represents a false negative of the first target, pre is a target prediction result corresponding to the sample image, LGT is the first target, smooth is a constant, and loss is large Is the first training loss.
Obtaining a second training loss corresponding to the target training data according to the target prediction result corresponding to the sample image and the second target comprises:
taking the intersection of the target prediction result corresponding to the sample image and the second target;
and calculating the loss relative to the second target in the target prediction result corresponding to the sample image according to the intersection of the target prediction result corresponding to the sample image and the second target.
Wherein the calculating the second training loss according to the intersection of the target prediction result corresponding to the sample image and the second target includes:
calculating the second training loss using a second formula;
the second formula is:
SOFP＝Pre-SOTP
SOFN＝SGT-SOTP
wherein SOTP represents the intersection of the target prediction result corresponding to the sample image and the second target, and represents the true positive of the first target, SOFP represents the false positive of the second target, SOFN represents the false negative of the second target, pre represents the target prediction result corresponding to the sample image, SGT represents the second target, smooth represents a constant, exp = (SGT/(SGT + smooth)), and loss represents the true positive of the second target, and SGT represents the false negative of the second target, and small is the second training loss.
Wherein the obtaining of the total training loss corresponding to the target training data according to the first training loss and the second training loss comprises:
and performing geometric average on the first training loss and the second training loss to obtain a total training loss corresponding to the target training data.
Example four
The present invention also provides a computer readable storage medium, in which one or more programs are stored, the one or more programs being executable by one or more processors for implementing the steps of the image small object segmentation method as described above.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, and not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (8)
1. An image small object segmentation method, characterized in that the method comprises:
determining target training data in each training data, wherein the target training data comprise a sample image and a target labeling result corresponding to the sample image, inputting the sample image into a target segmentation model, and acquiring a target prediction result corresponding to the sample image output by the target segmentation model;
preprocessing a target labeling result corresponding to the sample image, and dividing the target labeling result corresponding to the sample image into a first target and a second target, wherein the number of pixel points of the first target is greater than that of the second target;
obtaining a first training loss corresponding to the target training data according to the target prediction result corresponding to the sample image and the first target, and obtaining a second training loss corresponding to the target training data according to the target prediction result corresponding to the sample image and the second target;
the obtaining a first training loss corresponding to the target training data according to the target prediction result corresponding to the sample image and the first target includes:
acquiring an intersection of a target prediction result corresponding to the sample image and the first target;
calculating the first training loss according to the intersection of the target prediction result corresponding to the sample image and the first target;
the calculating the first training loss according to the intersection of the target prediction result corresponding to the sample image and the first target includes:
calculating the first training loss using a first formula;
the first formula is:
LOFP＝Pre-LOTP；
LOFN＝LGT-LOTP；
wherein LOTP is an intersection of a target prediction result corresponding to the sample image and the first target and represents a true positive of the first target, LOFP represents a false positive of the first target, LOFN represents a false negative of the first target, pre is a target prediction result corresponding to the sample image, LGT is the first target, smooth is a constant, and loss is large Is the first training loss;
obtaining a total training loss corresponding to the target training data according to the first training loss and the second training loss;
updating a learnable parameter of the target segmentation model according to the total training loss;
and re-executing the step of determining the target training data in each training data until the learnable parameters of the target segmentation model are converged, wherein the target segmentation model with the converged learnable parameters is used for performing target segmentation on the image to be processed.
2. The image small-target segmentation method according to claim 1, wherein the dividing the target labeling result corresponding to the sample image into a first target and a second target comprises:
and taking the target with the number of the pixel points larger than a preset threshold value as the first target, and taking the target with the number of the pixel points smaller than the preset threshold value as the second target.
3. The image small target segmentation method according to claim 1, wherein the obtaining a second training loss corresponding to the target training data according to the target prediction result corresponding to the sample image and the second target includes:
acquiring an intersection of a target prediction result corresponding to the sample image and the second target;
and calculating the loss relative to the second target in the target prediction result corresponding to the sample image according to the intersection of the target prediction result corresponding to the sample image and the second target.
4. The image small target segmentation method according to claim 3, wherein the calculating the second training loss according to the intersection of the target prediction result corresponding to the sample image and the second target includes:
calculating the second training loss using a second formula;
the second formula is:
SOFP＝Pre-SOTP
SOFN＝SGT-SOTP
wherein SOTP is an intersection of the target prediction result corresponding to the sample image and the second target, and represents a true positive of the second target, SOFP represents a false positive of the second target, SOFN represents a false negative of the second target, pre is a target prediction result corresponding to the sample image, SGT is the second target, smooth is a constant, exp = (SGT/(SGT + smooth)), and loss is a constant small Is the second training loss.
5. The image small target segmentation method according to claim 1, wherein the obtaining of the total training loss corresponding to the target training data according to the first training loss and the second training loss comprises:
and performing geometric average on the first training loss and the second training loss to obtain a total training loss corresponding to the target training data.
6. An image small object segmentation apparatus, comprising:
the target prediction unit is used for determining target training data in each training data, the target training data comprise sample images and target labeling results corresponding to the sample images, the sample images are input into a target segmentation model, and target prediction results corresponding to the sample images output by the target segmentation model are obtained;
the data preprocessing module is used for preprocessing a target labeling result corresponding to the sample image and dividing the target labeling result corresponding to the sample image into a first target and a second target, wherein the number of pixel points of the first target is larger than that of the second target;
a loss obtaining module, configured to obtain a first training loss corresponding to the target training data according to a target prediction result corresponding to the sample image and the first target, and obtain a second training loss corresponding to the target training data according to a target prediction result corresponding to the sample image and the second target;
the obtaining a first training loss corresponding to the target training data according to the target prediction result corresponding to the sample image and the first target includes:
acquiring an intersection of a target prediction result corresponding to the sample image and the first target;
calculating the first training loss according to the intersection of the target prediction result corresponding to the sample image and the first target;
the calculating the first training loss according to the intersection of the target prediction result corresponding to the sample image and the first target includes:
calculating the first training loss using a first formula;
the first formula is:
LOFP＝Pre-LOTP；
LOFN＝LGT-LOTP；
wherein LOTP is an intersection of a target prediction result corresponding to the sample image and the first target and represents a true positive of the first target, LOFP represents a false positive of the first target, LOFN represents a false negative of the first target, pre is a target prediction result corresponding to the sample image, LGT is the first target, smooth is a constant, and loss is large Is the first training loss;
an updating module, configured to obtain a total training loss corresponding to the target training data according to the first training loss and the second training loss;
updating a learnable parameter of the target segmentation model according to the total training loss;
and re-executing the step of determining the target training data in each training data until the learnable parameters of the target segmentation model converge.
7. A terminal, characterized in that the terminal comprises: a processor, a computer readable storage medium communicatively connected to the processor, the computer readable storage medium adapted to store a plurality of instructions, the processor adapted to invoke the instructions in the computer readable storage medium to perform the steps of implementing the image small object segmentation method according to any one of the preceding claims 1 to 5.
8. A computer readable storage medium, storing one or more programs, which are executable by one or more processors, for performing the steps of the image small object segmentation method as claimed in any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210520684.2A CN115170795B (en) | 2022-05-13 | 2022-05-13 | Image small target segmentation method, device, terminal and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210520684.2A CN115170795B (en) | 2022-05-13 | 2022-05-13 | Image small target segmentation method, device, terminal and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115170795A CN115170795A (en) | 2022-10-11 |
| CN115170795B true CN115170795B (en) | 2023-03-21 |
Family
ID=83483598
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210520684.2A Active CN115170795B (en) | 2022-05-13 | 2022-05-13 | Image small target segmentation method, device, terminal and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115170795B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115375955B (en) * | 2022-10-25 | 2023-04-18 | 北京鹰瞳科技发展股份有限公司 | Training method of target detection model, target detection method and related product |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108257134A (en) * | 2017-12-21 | 2018-07-06 | 深圳大学 | Nasopharyngeal Carcinoma Lesions automatic division method and system based on deep learning |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110163234B (en) * | 2018-10-10 | 2023-04-18 | 腾讯科技（深圳）有限公司 | Model training method and device and storage medium |
| CN109598290A (en) * | 2018-11-22 | 2019-04-09 | 上海交通大学 | A kind of image small target detecting method combined based on hierarchical detection |
| CN113807472B (en) * | 2021-11-19 | 2022-02-22 | 智道网联科技(北京)有限公司 | Hierarchical target detection method and device |
| CN114419083A (en) * | 2021-12-16 | 2022-04-29 | 山东众阳健康科技集团有限公司 | ResUnet medical image segmentation system based on edge operator improvement |
-
2022
- 2022-05-13 CN CN202210520684.2A patent/CN115170795B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108257134A (en) * | 2017-12-21 | 2018-07-06 | 深圳大学 | Nasopharyngeal Carcinoma Lesions automatic division method and system based on deep learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115170795A (en) | 2022-10-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Lu et al. | Fully automatic liver segmentation combining multi-dimensional graph cut with shape information in 3D CT images | |
| US10853409B2 (en) | Systems and methods for image search | |
| Sedai et al. | Deep multiscale convolutional feature learning for weakly supervised localization of chest pathologies in x-ray images | |
| Pourreza et al. | Brain tumor segmentation in MRI scans using deeply-supervised neural networks | |
| Dandıl | A Computer‐Aided Pipeline for Automatic Lung Cancer Classification on Computed Tomography Scans | |
| US10878564B2 (en) | Systems and methods for processing 3D anatomical volumes based on localization of 2D slices thereof | |
| Hsiao et al. | A deep learning-based precision and automatic kidney segmentation system using efficient feature pyramid networks in computed tomography images | |
| Poh et al. | Automatic segmentation of ventricular cerebrospinal fluid from ischemic stroke CT images | |
| CN115170795B (en) | Image small target segmentation method, device, terminal and storage medium | |
| CN114092450A (en) | Real-time image segmentation method, system and device based on gastroscopy video | |
| US20220301177A1 (en) | Updating boundary segmentations | |
| CN110992439B (en) | Fiber bundle tracking method, computer device, and storage medium | |
| Kloenne et al. | Domain-specific cues improve robustness of deep learning-based segmentation of CT volumes | |
| US9538920B2 (en) | Standalone annotations of axial-view spine images | |
| CN113920137B (en) | Lymph node metastasis prediction method, device, equipment and storage medium | |
| Chandra et al. | Segmented lung boundary correction in chest radiograph using context-aware adaptive scan algorithm | |
| Zhu et al. | Improved maximally stable extremal regions based method for the segmentation of ultrasonic liver images | |
| CN113822846A (en) | Method, apparatus, device and medium for determining region of interest in medical image | |
| CN112529900A (en) | Method, device, terminal and storage medium for matching ROI in mammary gland image | |
| Shin et al. | A novel cascade classifier for automatic microcalcification detection | |
| CN113379770B (en) | Construction method of nasopharyngeal carcinoma MR image segmentation network, image segmentation method and device | |
| CN113269137B (en) | Non-matching face recognition method combining PCANet and shielding positioning | |
| KN et al. | Comparison of 3-segmentation techniques for intraventricular and intracerebral hemorrhages in unenhanced computed tomography scans | |
| Li et al. | SIFT-GVF-based lung edge correction method for correcting the lung region in CT images | |
| Liu et al. | Lung CT Image Segmentation via Dilated U-Net Model and Multi-scale Gray Correlation-Based Approach |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |