CN115018857B - Image segmentation method, image segmentation device, computer-readable storage medium and computer equipment - Google Patents
Image segmentation method, image segmentation device, computer-readable storage medium and computer equipment Download PDFInfo
- Publication number
- CN115018857B CN115018857B CN202210953897.4A CN202210953897A CN115018857B CN 115018857 B CN115018857 B CN 115018857B CN 202210953897 A CN202210953897 A CN 202210953897A CN 115018857 B CN115018857 B CN 115018857B
- Authority
- CN
- China
- Prior art keywords
- image segmentation
- image
- loss
- pixel
- training sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
An image segmentation method, an image segmentation device, a computer-readable storage medium and a computer device are provided, wherein the method comprises the following steps: carrying out pixel normalization processing on the training sample image; inputting the training sample image subjected to the normalization processing into an image segmentation model for image segmentation processing; inputting the prediction probability output by the image segmentation model and the segmentation true value of the training sample image into a target loss function for loss value calculation, wherein the target loss function is used for performing cross loss calculation, mean absolute error loss calculation, loss calculation in pixel classes and loss calculation between pixel classes; when the calculated loss value is larger than a preset value, optimizing parameters of the image segmentation model according to the loss value; and when the calculated loss value is less than or equal to a preset value, inputting the image to be detected into the image segmentation model to obtain an image segmentation result of the image to be detected. The image segmentation precision and the image segmentation efficiency of the invention are superior to those of the prior art.
Description
Technical Field
The present invention relates to the field of image processing, and in particular, to an image segmentation method, an image segmentation apparatus, a computer-readable storage medium, and a computer device.
Background
In industrial visual defect detection, deep learning has been widely used. Wherein the deep learning based semantic segmentation model exhibits more advantages over the conventional threshold segmentation in defective pixel segmentation. However, the types of defects in industrial scenes are various, and some common semantic segmentation loss functions cannot be well applied to some special scenes for industrial detection.
In image segmentation scenarios, the general task is to classify pixels, so the penalty for pixel classification usually employs a cross-entropy penalty function. The cross entropy loss function is defined as:
The cross entropy loss function is better in performance under ideal conditions of balanced category number, noiseless data and the like, but in a defect detection scene, scenes of uneven defect type distribution, small defect area, low-contrast scratch and the like often exist, and image labeling of industrial defect detection is often calculated on defect pixel labeling noise, for example, when labeling is not uniform due to unclear defect outline, model training is performed by adopting the cross entropy loss function, and the obtained model is not ideal in image segmentation effect.
Disclosure of Invention
In view of the above situation, it is desirable to provide an image segmentation method, an image segmentation apparatus, a computer-readable storage medium, and a computer device, which are directed to solve the problem in the prior art that the image segmentation effect is not ideal.
The invention provides an image segmentation method, which comprises the following steps:
carrying out pixel normalization processing on the training sample image;
inputting the training sample image subjected to the normalization processing into an image segmentation model for image segmentation processing;
inputting the prediction probability output by the image segmentation model and the segmentation true value of the training sample image into a target loss function for loss value calculation, wherein the target loss function is used for performing cross loss calculation, mean absolute error loss calculation, loss calculation in pixel classes and loss calculation between pixel classes;
when the calculated loss value is larger than a preset value, optimizing parameters of the image segmentation model according to the loss value, and returning to the step of inputting the training sample image subjected to the normalization processing into the image segmentation model for image segmentation processing;
and when the calculated loss value is less than or equal to the preset value, inputting the image to be detected into the image segmentation model to obtain an image segmentation result of the image to be detected.
Further, in the image segmentation method, the step of inputting the prediction probability output by the image segmentation model and the segmentation true value of the training sample image into the target loss function to calculate the loss value includes:
counting the number of defective pixels and the number of non-defective pixels in the training sample image, and screening out the non-defective pixels with the probability lower than a threshold value by a preset number according to the prediction probability output by the image segmentation model;
and taking the defective pixels in the training sample image and the screened non-defective images as input pixels, and inputting the prediction probability of the input pixels and the segmentation true value of the training sample image into a target loss function for loss value calculation.
Further, in the image segmentation method, the preset number ish n Is composed ofAnd the training sample image is a non-defective pixel image, wherein p is the number of defective pixels in the training sample image, and n is the number of non-defective pixels in the training sample image.
Further, the image segmentation method described above, wherein the step of inputting the prediction probability output by the image segmentation model and the true segmentation value of the training sample image into the objective loss function to calculate the loss value further includes:
constructing an initial loss function according to the cross entropy function and the average absolute error loss function;
determining a loss function corresponding to each pixel category according to the pixels of each pixel category and the initial loss function;
and introducing a mean value to the loss function corresponding to each pixel type to calculate to obtain a target loss function.
Further, in the image segmentation method, the objective loss function is:
wherein, P t Predicting the probability, S, for the model of pixel t n Is a set of pixels of the nth pixel class,the number of pixels in the pixel set of the nth pixel class is N, the total number of the pixel classes is N, and lambda is a coefficient for balancing cross entropy and average absolute error loss.
Further, in the image segmentation method, λ is 2.
Further, in the image segmentation method, the image segmentation model adopts a UNet image segmentation network or an FCN image segmentation network.
The present invention also provides an image segmentation apparatus, comprising:
the normalization processing module is used for carrying out pixel normalization processing on the training sample image;
the first image segmentation module is used for inputting the training sample image subjected to the normalization processing into an image segmentation model for image segmentation processing;
the calculation module is used for inputting the prediction probability output by the image segmentation model and the segmentation true value of the training sample image into a target loss function for loss value calculation, wherein the target loss function is used for performing cross loss calculation, mean absolute error loss calculation, loss calculation in pixel classes and loss calculation between pixel classes;
the model optimization module is used for optimizing parameters of the image segmentation model according to the loss value when the calculated loss value is larger than a preset value, and returning to execute the step of inputting the training sample image subjected to the normalization processing into the image segmentation model for image segmentation processing;
and the second image segmentation module is used for inputting the image to be detected into the image segmentation model when the calculated loss value is less than or equal to the preset value, so as to obtain the image segmentation result of the image to be detected.
Further, the image segmentation apparatus further includes a target loss function construction module, where the target loss function construction module is configured to:
constructing an initial loss function according to the cross entropy function and the average absolute error loss function;
determining a loss function corresponding to each pixel category according to the pixels of each pixel category and the initial loss function;
and introducing a mean value to the loss function corresponding to each pixel type to calculate to obtain a target loss function.
The invention also provides a computer device comprising a memory and a processor, the memory storing a program which, when executed by the processor, implements any of the methods described above.
The invention also provides a computer readable storage medium having a program stored thereon, which when executed by a processor implements any of the methods described above.
In the invention, an image is input into an image segmentation model for image segmentation processing, a prediction probability is output, the prediction probability and a segmentation true value of the image are input into a target loss function for loss value calculation, parameters of the image segmentation model are optimized according to the calculated loss value, and the optimization of the image segmentation model is completed when the loss value is lower than a preset value. The target loss function definition defined in the invention can unify cross entropy, average absolute error, loss calculation in pixel categories, loss among pixel categories and the like, and improves the image segmentation precision.
Drawings
FIG. 1 is a flowchart of an image segmentation method according to a first embodiment of the present invention;
FIG. 2 is a flowchart of an image segmentation method according to a second embodiment of the present invention;
FIG. 3 is a block diagram of an image segmentation apparatus according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of a computer device in an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
These and other aspects of embodiments of the invention will be apparent with reference to the following description and attached drawings. In the description and drawings, particular embodiments of the invention have been disclosed in detail as being indicative of some of the ways in which the principles of the embodiments of the invention may be practiced, but it is understood that the scope of the embodiments of the invention is not limited correspondingly. On the contrary, the embodiments of the invention include all changes, modifications and equivalents coming within the spirit and terms of the claims appended hereto.
Referring to fig. 1, an image segmentation method according to a first embodiment of the present invention includes steps S11 to S15.
And step S11, carrying out pixel normalization processing on the training sample image.
And S12, inputting the training sample image subjected to the normalization processing into an image segmentation model for image segmentation processing.
In this embodiment, the image segmentation model is trained by using a training sample data set and a loss function. The training sample data set comprises a plurality of training sample images, and each training sample image needs to be preprocessed. The pre-processing process mainly includes normalization, i.e., dividing the pixel values of the training sample image by 255 to normalize them to be between 0~1.
The image segmentation model can adopt a deep learning model, such as UNet, FCN and other segmentation networks. The image segmentation model is used for predicting the pixel category of each pixel, namely calculating the probability that the pixel is consistent with the labeling result, and determining the pixel category of the pixel according to the probability.
And S13, inputting the prediction probability output by the image segmentation model and the segmentation true value of the training sample image into a target loss function for loss value calculation, wherein the target loss function is used for performing cross entropy loss calculation, mean absolute error loss calculation, loss calculation in pixel categories and loss calculation between pixel categories.
And inputting the output of the image segmentation model and a segmentation true value (Ground Truth) of the training sample image into a loss function calculation module to obtain a loss value. The output of the image segmentation model is the class prediction probability of the pixel, i.e. the probability of being consistent with the labeling result. The true segmentation value is the pixel type labeled by each pixel.
The pixel type is a defect type of the image, such as crack, slip line, scratch, bump, and the like.
When loss calculation is performed by the loss function according to input information, the following three calculation parts are mainly involved:
calculating cross entropy loss;
calculating the average absolute error loss;
loss calculations within pixel classes and loss calculations between pixel classes.
The cross entropy function used for cross entropy loss calculation may be:
The cross entropy function is defined as standard cross entropy, and the standard cross entropy function is a standard loss function in image segmentation and is suitable for most scenes with larger sizes.
In order to improve the class data labeling noise scene, average Absolute Error loss calculation and an average Absolute Error loss function (MAE) are introduced, so that the noise scene can be effectively improved.
The mean absolute error loss function is more robust in noisy data sets than the cross entropy loss function, which is defined as follows:
the gradient of cross entropy is more negative for negative samples, while the mean absolute error is unity. Therefore, if there are many noisy samples in the sample, then using a cross-entropy model will try to fit the noisy data, where it is preferable to use MAE as the loss function.
In an industrial defect detection scene, the number of different types of defects is often unbalanced, and in order to improve the problem of class imbalance, in this embodiment, a method of averaging all pixels is not adopted, but loss averaging is performed inside each pixel class first, and then averaging between the pixel classes is performed. This way the class imbalance situation is greatly alleviated.
And S14, when the calculated loss value is larger than a preset value, optimizing parameters of the image segmentation model according to the loss value, and returning to the step of inputting the training sample image subjected to the normalization processing into the image segmentation model for image segmentation processing.
And S15, when the calculated loss value is less than or equal to the preset value, inputting the image to be detected into the image segmentation model to obtain an image segmentation result of the image to be detected.
The smaller the calculated loss value is, the more accurate the image segmentation of the image segmentation model is, and the better the segmentation effect is. When the calculated loss value is less than or equal to the preset value, it can be said that the image segmentation model is trained. The trained model can be used for image segmentation of the image to be detected.
In this embodiment, an image is input into an image segmentation model to perform image segmentation processing, a prediction probability is output, the prediction probability and a true segmentation value of the image are input into a target loss function to perform loss value calculation, and parameters of the image segmentation model are optimized according to the calculated loss value until the loss value is lower than a preset value, so that the optimization of the image segmentation model is completed. The target loss function defined in this embodiment may unify cross entropy, average absolute error, loss calculation in pixel categories, loss between pixel categories, and the like, and improve image segmentation accuracy, and the target loss function in this embodiment does not require complex coefficient search, and simple coefficients have already reached the improvement of data set accuracy.
Referring to FIG. 2, a method for segmenting an image according to a second embodiment of the present invention includes steps S21 to S28.
And S21, carrying out pixel normalization processing on the training sample image.
The normalization process mainly adjusts the size of the characteristic value to a similar range, so that the model training is stable in convergence. In one embodiment, the pixel values of the training sample image may be divided by 255 to normalize them to 0~1.
And S22, inputting the training sample image subjected to the normalization processing into an image segmentation model for image segmentation processing.
The image segmentation model can adopt a deep learning model, such as UNet, FCN and other segmentation networks. The image segmentation model is used for predicting the pixel type of each pixel, namely calculating the probability that the pixel is consistent with the labeling result, and determining the pixel type of the pixel according to the probability.
Step S23, an objective loss function is constructed.
Specifically, the step of constructing the target loss function includes:
s231, constructing an initial loss function according to the cross entropy function and the average absolute error loss function;
s232, determining a loss function corresponding to each pixel type according to the pixels of each pixel type and the initial loss function;
and S233, calculating the average value of the loss functions corresponding to the pixel types to obtain a target loss function.
The target loss function is:
wherein, P t Predicting the probability, S, for the model of pixel t n Is a set of pixels of the nth pixel class,the number of pixels in the pixel set of the nth pixel class is N, the total number of the pixel classes is N, and lambda is a coefficient for balancing cross entropy and average absolute error loss.
The objective loss function in this embodiment mainly includes the following three components.
1. Cross entropy part
The cross entropy formula adopted is,P t Is the prediction probability. This section is defined as the standard cross entropy, which is the standard loss function in image segmentation that applies to larger and more scenes.
2. Mean absolute error part
In order to improve class data labeling noise scenes, an average absolute error part is introduced, and the average absolute error is verified, so that the noise scenes can be effectively improved. In this embodiment, the loss function of the average absolute error part may be:
this portion is superimposed with the cross entropy, where λ is the coefficient that balances the cross entropy and the mean absolute error loss, reducing the effect of noisy data. In this embodiment, λ =2, and a good segmentation effect can be achieved for most data sets by using the coefficient.
3. Loss per class and mean between classes part:
in an industrial defect detection scenario, different defect types tend to be uneven in numberIn order to improve the problem of category imbalance, in this embodiment, a method of averaging all pixels is not adopted, but loss averaging is performed inside each category first, and then inter-category averaging is performed. This way the class imbalance situation is greatly alleviated. In thatIn this section, the penalty for each pixel class is obtained. After the loss of each pixel class is obtained, the overall average is obtained, which is equivalent to equalizing the weights of the classes. The method can effectively relieve the conditions of small defect area and unbalanced defect types in industrial defect detection.
And S24, counting the number of defective pixels and the number of non-defective pixels in the training sample image, and screening out the non-defective pixels with the probability lower than a threshold value according to the prediction probability output by the image segmentation model.
And S25, taking the defective pixel in the training sample image and the screened non-defective image as input pixels, and inputting the prediction probability of the input pixels and the segmentation true value of the training sample image into a target loss function for loss value calculation.
For the current training sample image, firstly, screening the defective pixels and the pixels with low confidence level, and taking the pixels of the screened training sample image as the pixel sample for calculating loss. Counting the number p of the existing marked defective pixels and the number n of the non-defective pixels, and screening the number with lower probability asP defective pixels, and the selected oneOne non-defective pixel, which is a pixel in a single image for which loss is calculated, is denoted as a pixel set S.
And inputting the prediction probability of each pixel in the pixel set S and the corresponding segmentation true value into the target loss function for loss value calculation.
In step S26, it is determined whether the calculated loss value is less than or equal to the predetermined value, if not, step S27 is performed, and if so, step S28 is performed.
And S27, optimizing the parameters of the image segmentation model according to the loss values, and returning to the step S22.
And S28, inputting the image to be detected into the image segmentation model to obtain an image segmentation result of the image to be detected.
The smaller the calculated loss value is, the more accurate the image segmentation of the image segmentation model is, and the better the segmentation effect is. And when the calculated loss value is larger than the preset value, adjusting the model parameters, and returning to the step of carrying out image segmentation and loss value calculation again. When the calculated loss value is less than or equal to the preset value, the image segmentation model can be proved to be trained, and the trained model can be used for image segmentation of the image to be detected.
To verify the effect of the image segmentation method in this embodiment, the following experiment was performed:
the defect detection data sets of a plurality of products are adopted, 4 data sets are provided, each data set comprises 100 sample images, the number of pixel categories is from 2 to 5, and the number of the sample images in the pixel categories is unbalanced. The 4 data sets are respectively subjected to training set division and test set division, wherein the proportion of the training set is 80%, the proportion of the verification set is 10%, the proportion of the test set is 10%, the batch size is unified into 16, the size of the image training set is 448 x 448, the model backbone network can comprise UNET, FCN and other common backbone networks, and DDRNet-slim is adopted in the calculation efficiency considered in the scheme. The DDRNet is a high-speed real-time image segmentation backbone network, has high calculation efficiency, and adopts a mode of testing set defects mIOU in an accuracy statistical mode, namely, an intersection/union set of the positions of the pixels with the defects and the marked pixels is detected. Based on the training set, the loss function and the conventional standard cross entropy loss function L in the embodiment are respectively utilized CE Loss function L 1 And a loss function L 2 Training and training the image segmentation modelThe trained image segmentation models were tested with a test set, and the test results are shown in table 1. Wherein substrate-1, substrate-2, epi-1, and epi-2 represent four data sets. L represents a loss function in the present embodiment;,L CE representing a standard cross entropy loss function; l1 is,L 1 Representing the introduction of an average absolute error loss function in a standard cross entropy loss function; l2 isEquivalent to a weighted cross entropy loss function.
TABLE 1
The experimental result shows that after the average absolute error loss, the intra-class and inter-class average losses are introduced in the loss value calculation process, the image segmentation model has the highest accuracy in the test set and the best segmentation effect.
Referring to fig. 3, an image segmentation apparatus according to a third embodiment of the present invention includes:
a normalization processing module 31, configured to perform pixel normalization processing on the training sample image;
the first image segmentation module 32 is configured to input the training sample image after the normalization processing into an image segmentation model for image segmentation processing;
a calculating module 33, configured to input the prediction probability output by the image segmentation model and the segmentation true value of the training sample image into a target loss function for performing loss value calculation, where the target loss function is configured to perform cross loss calculation, mean absolute error loss calculation, and intra-pixel loss calculation and inter-pixel loss calculation;
the model optimization module 34 is configured to, when the calculated loss value is greater than a preset value, optimize parameters of the image segmentation model according to the loss value, and return to the step of inputting the training sample image after the normalization processing into the image segmentation model for image segmentation processing;
and the second image segmentation module 35 is configured to, when the calculated loss value is less than or equal to the preset value, input the image to be detected into the image segmentation model, so as to obtain an image segmentation result of the image to be detected.
Further, the image segmentation apparatus further includes a target loss function construction module, where the target loss function construction module is configured to:
constructing an initial loss function according to the cross entropy function and the average absolute error loss function;
determining a loss function corresponding to each pixel category according to the pixels of each pixel category and the initial loss function;
and introducing a mean value to the loss function corresponding to each pixel category to calculate to obtain a target loss function.
The image segmentation apparatus provided in the embodiment of the present invention has the same implementation principle and technical effect as those of the foregoing method embodiments, and for brief description, reference may be made to corresponding contents in the foregoing method embodiments for the sake of brevity.
Referring to fig. 4, a computer device according to an embodiment of the present invention is shown, which includes a processor 10, a memory 20, and a computer program 30 stored in the memory and executable on the processor, and the processor 10 implements the image segmentation method as described above when executing the computer program 30.
The computer device may be, but is not limited to, a personal computer, a server, and the like. The processor 10 may be, in some embodiments, a Central Processing Unit (CPU), controller, microcontroller, microprocessor or other data Processing chip for executing program codes stored in the memory 20 or Processing data.
The memory 20 includes at least one type of readable storage medium, which includes a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, and the like. The memory 20 may in some embodiments be an internal storage unit of the computer device, for example a hard disk of the computer device. The memory 20 may also be an external storage device of the computer device in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the computer device. Further, the memory 20 may also include both an internal storage unit and an external storage device of the computer apparatus. The memory 20 may be used not only to store application software installed in the computer device, various types of data, and the like, but also to temporarily store data that has been output or is to be output.
Optionally, the computer device may further comprise a user interface, which may comprise a Display (Display), an input unit such as a Keyboard (Keyboard), a network interface, a communication bus, etc., and the optional user interface may also comprise a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the computer device and for displaying a visualized user interface. The network interface may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), typically used to establish a communication link between the device and other electronic devices. The communication bus is used to enable connection communication between these components.
It should be noted that the configuration shown in fig. 4 does not constitute a limitation of the computer device, and in other embodiments, the computer device may include fewer or more components than those shown, or some components may be combined, or a different arrangement of components may be used.
The invention also proposes a computer-readable storage medium on which a computer program is stored which, when being executed by a processor, carries out the image segmentation method as described above.
Those of skill in the art will understand that the logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be viewed as implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus (e.g., a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or execute the instructions). For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Further, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (9)
1. An image segmentation method, comprising:
carrying out pixel normalization processing on the training sample image;
inputting the training sample image subjected to the normalization processing into an image segmentation model for image segmentation processing;
inputting the prediction probability output by the image segmentation model and the segmentation true value of the training sample image into a target loss function for loss value calculation, wherein the target loss function is used for performing cross loss calculation, mean absolute error loss calculation, intra-pixel loss calculation and inter-pixel loss calculation on the prediction probability output by the image segmentation model, and the target loss function is as follows:
wherein L is an objective loss function, P t Predicting the probability, S, for the model of the pixel t n Is a set of pixels of the nth pixel class,the number of pixels in a pixel set of the nth pixel category is N, the number of the total pixel categories is N, and lambda is a coefficient for balancing cross entropy and average absolute error loss;
when the calculated loss value is larger than a preset value, optimizing parameters of the image segmentation model according to the loss value, and returning to the step of inputting the training sample image subjected to the normalization processing into the image segmentation model for image segmentation processing;
and when the calculated loss value is less than or equal to the preset value, inputting the image to be detected into the image segmentation model to obtain an image segmentation result of the image to be detected.
2. The image segmentation method as claimed in claim 1, wherein the step of inputting the prediction probability output by the image segmentation model and the segmentation true value of the training sample image into an objective loss function for loss value calculation comprises:
counting the number of defective pixels and the number of non-defective pixels in the training sample image, and screening out the non-defective pixels with the probability lower than a threshold value by a preset number according to the prediction probability output by the image segmentation model;
and taking the defective pixels in the training sample image and the screened non-defective images as input pixels, and inputting the prediction probability of the input pixels and the segmentation true value of the training sample image into a target loss function for loss value calculation.
3. The image segmentation method according to claim 2, wherein the preset numberh n Is composed ofAnd the training sample image is a non-defective pixel image, wherein p is the number of defective pixels in the training sample image, and n is the number of non-defective pixels in the training sample image.
4. The image segmentation method according to claim 1, wherein the step of inputting the prediction probability output by the image segmentation model and the segmentation true value of the training sample image into the target loss function for loss value calculation is preceded by the step of:
constructing an initial loss function according to the cross entropy function and the average absolute error loss function;
determining a loss function corresponding to each pixel category according to the pixels of each pixel category and the initial loss function;
and introducing a mean value to the loss function corresponding to each pixel type to calculate to obtain a target loss function.
5. The image segmentation method as set forth in claim 1, wherein λ is 2.
6. The image segmentation method according to claim 1, wherein the image segmentation model employs a UNet image segmentation network or an FCN image segmentation network.
7. An image segmentation apparatus, comprising:
the normalization processing module is used for carrying out pixel normalization processing on the training sample image;
the first image segmentation module is used for inputting the training sample image subjected to the normalization processing into an image segmentation model for image segmentation processing;
a calculating module, configured to input the prediction probability output by the image segmentation model and the segmentation true value of the training sample image into an objective loss function for performing loss value calculation, where the objective loss function is configured to perform cross loss calculation, mean absolute error loss calculation, and intra-pixel loss calculation and inter-pixel loss calculation on the prediction probability output by the image segmentation model, and the objective loss function is:
wherein L is an objective loss function, P t Predicting the probability, S, for the model of pixel t n Is a set of pixels of the nth pixel class,the number of pixels in a pixel set of the nth pixel category is N, the number of the total pixel categories is N, and lambda is a coefficient for balancing cross entropy and average absolute error loss;
the model optimization module is used for optimizing parameters of the image segmentation model according to the loss value when the calculated loss value is larger than a preset value, and returning to execute the step of inputting the training sample image subjected to the normalization processing into the image segmentation model for image segmentation processing;
and the second image segmentation module is used for inputting the image to be detected into the image segmentation model when the calculated loss value is less than or equal to the preset value, so as to obtain the image segmentation result of the image to be detected.
8. A computer device comprising a memory and a processor, the memory storing a program that, when executed by the processor, implements the method of any of claims 1-6.
9. A computer-readable storage medium, on which a program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1 to 6.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210953897.4A CN115018857B (en) | 2022-08-10 | 2022-08-10 | Image segmentation method, image segmentation device, computer-readable storage medium and computer equipment |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210953897.4A CN115018857B (en) | 2022-08-10 | 2022-08-10 | Image segmentation method, image segmentation device, computer-readable storage medium and computer equipment |
Publications (2)
Publication Number | Publication Date |
---|---|
CN115018857A CN115018857A (en) | 2022-09-06 |
CN115018857B true CN115018857B (en) | 2022-11-11 |
Family
ID=83065502
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210953897.4A Active CN115018857B (en) | 2022-08-10 | 2022-08-10 | Image segmentation method, image segmentation device, computer-readable storage medium and computer equipment |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN115018857B (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115239719A (en) * | 2022-09-22 | 2022-10-25 | 南昌昂坤半导体设备有限公司 | Defect detection method, system, electronic device and storage medium |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107403430A (en) * | 2017-06-15 | 2017-11-28 | 中山大学 | A kind of RGBD image, semantics dividing method |
CN108921839A (en) * | 2018-07-02 | 2018-11-30 | 北京百度网讯科技有限公司 | Continuous casting billet quality detection method, device, electronic equipment and storage medium |
CN108961238A (en) * | 2018-07-02 | 2018-12-07 | 北京百度网讯科技有限公司 | Display screen quality determining method, device, electronic equipment and storage medium |
CN111179272A (en) * | 2019-12-10 | 2020-05-19 | 中国科学院深圳先进技术研究院 | Rapid semantic segmentation method for road scene |
CN111667027A (en) * | 2020-07-03 | 2020-09-15 | 腾讯科技(深圳)有限公司 | Multi-modal image segmentation model training method, image processing method and device |
CN112419248A (en) * | 2020-11-13 | 2021-02-26 | 复旦大学 | Ear sclerosis focus detection and diagnosis system based on small target detection neural network |
CN113468977A (en) * | 2021-06-10 | 2021-10-01 | 汉王科技股份有限公司 | Text line language identification method and device and electronic equipment |
CN113538480A (en) * | 2020-12-15 | 2021-10-22 | 腾讯科技(深圳)有限公司 | Image segmentation processing method and device, computer equipment and storage medium |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20200126236A1 (en) * | 2018-10-22 | 2020-04-23 | The Board Of Trustees Of The Leland Stanford Junior University | Systems and Methods for Image Segmentation using IOU Loss Functions |
CN111260665B (en) * | 2020-01-17 | 2022-01-21 | 北京达佳互联信息技术有限公司 | Image segmentation model training method and device |
CN112613515A (en) * | 2020-11-23 | 2021-04-06 | 上海眼控科技股份有限公司 | Semantic segmentation method and device, computer equipment and storage medium |
CN113763387B (en) * | 2021-07-22 | 2023-09-01 | 深圳大学 | Placenta ultramicro blood vessel segmentation method, storage medium and terminal equipment |
-
2022
- 2022-08-10 CN CN202210953897.4A patent/CN115018857B/en active Active
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107403430A (en) * | 2017-06-15 | 2017-11-28 | 中山大学 | A kind of RGBD image, semantics dividing method |
CN108921839A (en) * | 2018-07-02 | 2018-11-30 | 北京百度网讯科技有限公司 | Continuous casting billet quality detection method, device, electronic equipment and storage medium |
CN108961238A (en) * | 2018-07-02 | 2018-12-07 | 北京百度网讯科技有限公司 | Display screen quality determining method, device, electronic equipment and storage medium |
CN111179272A (en) * | 2019-12-10 | 2020-05-19 | 中国科学院深圳先进技术研究院 | Rapid semantic segmentation method for road scene |
CN111667027A (en) * | 2020-07-03 | 2020-09-15 | 腾讯科技(深圳)有限公司 | Multi-modal image segmentation model training method, image processing method and device |
CN112419248A (en) * | 2020-11-13 | 2021-02-26 | 复旦大学 | Ear sclerosis focus detection and diagnosis system based on small target detection neural network |
CN113538480A (en) * | 2020-12-15 | 2021-10-22 | 腾讯科技(深圳)有限公司 | Image segmentation processing method and device, computer equipment and storage medium |
CN113468977A (en) * | 2021-06-10 | 2021-10-01 | 汉王科技股份有限公司 | Text line language identification method and device and electronic equipment |
Non-Patent Citations (1)
Title |
---|
《3D SE-ResNet:一种从CT图像中自动分割COVID-19肺部感染模型》;朱勇等;《中南民族大学学报(自然科学版)》;20220331;第41卷(第2期);第3.1节以及摘要 * |
Also Published As
Publication number | Publication date |
---|---|
CN115018857A (en) | 2022-09-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN108921206B (en) | Image classification method and device, electronic equipment and storage medium | |
CN108171203B (en) | Method and device for identifying vehicle | |
CN111275660B (en) | Flat panel display defect detection method and device | |
CN107729848B (en) | Method for checking object and device | |
CN110874604A (en) | Model training method and terminal equipment | |
US20140286527A1 (en) | Systems and methods for accelerated face detection | |
CN115240011A (en) | Image classification method and device, computer readable storage medium and computer equipment | |
CN115018857B (en) | Image segmentation method, image segmentation device, computer-readable storage medium and computer equipment | |
CN109740553B (en) | Image semantic segmentation data screening method and system based on recognition | |
CN116894985B (en) | Semi-supervised image classification method and semi-supervised image classification system | |
CN111723815A (en) | Model training method, image processing method, device, computer system, and medium | |
CN117147561B (en) | Surface quality detection method and system for metal zipper | |
CN111901594B (en) | Visual analysis task-oriented image coding method, electronic device and medium | |
CN110135428B (en) | Image segmentation processing method and device | |
US20150058579A1 (en) | Systems and methods for memory utilization for object detection | |
CN110751170A (en) | Panel quality detection method, system, terminal device and computer readable medium | |
CN106407991B (en) | Image attribute recognition method and system and related network training method and system | |
CN111803956B (en) | Method and device for determining game plug-in behavior, electronic equipment and storage medium | |
CN110796129A (en) | Text line region detection method and device | |
CN111784053A (en) | Transaction risk detection method, device and readable storage medium | |
CN116977271A (en) | Defect detection method, model training method, device and electronic equipment | |
CN115880533A (en) | Bridge apparent crack identification method based on adaptive subset search and deep learning | |
CN115512203A (en) | Information detection method, device, equipment and storage medium | |
CN116188940A (en) | Method and device for training model, storage medium and electronic equipment | |
CN113762120A (en) | Insulator image segmentation method and device, electronic equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |