CN117333443A - Defect detection method and device, electronic equipment and storage medium - Google Patents
Defect detection method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117333443A CN117333443A CN202311265857.1A CN202311265857A CN117333443A CN 117333443 A CN117333443 A CN 117333443A CN 202311265857 A CN202311265857 A CN 202311265857A CN 117333443 A CN117333443 A CN 117333443A
- Authority
- CN
- China
- Prior art keywords
- determining
- model
- defect detection
- defect
- mask
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 296
- 238000001514 detection method Methods 0.000 title claims abstract description 132
- 238000003860 storage Methods 0.000 title claims abstract description 18
- 230000011218 segmentation Effects 0.000 claims abstract description 134
- 238000012360 testing method Methods 0.000 claims abstract description 48
- 238000005457 optimization Methods 0.000 claims abstract description 22
- 230000006870 function Effects 0.000 claims description 62
- 230000000007 visual effect Effects 0.000 claims description 47
- 238000012549 training Methods 0.000 claims description 41
- 238000000034 method Methods 0.000 claims description 24
- 238000013140 knowledge distillation Methods 0.000 claims description 22
- 238000004590 computer program Methods 0.000 claims description 16
- 238000013145 classification model Methods 0.000 claims description 12
- 238000010276 construction Methods 0.000 claims 1
- 238000004891 communication Methods 0.000 description 8
- 238000012545 processing Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 238000005299 abrasion Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000032683 aging Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000011031 large-scale manufacturing process Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/096—Transfer learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a defect detection method, a defect detection device, electronic equipment and a storage medium. The invention relates to the technical field of equipment surface defect detection, which comprises the following steps: obtaining a defect test picture; inputting a defect test picture into a pre-trained target defect detection model, and determining a defect test result; the target defect detection model is constructed based on the segmentation model and the two-class segmentation model, and is trained and determined by taking an optimization rule as an index. According to the technical scheme, a target defect detection model is established based on a segmentation model and a binary segmentation model, and is trained and determined by taking an optimization rule as an index, and then a defect test picture is obtained; the defect test picture is input into a pre-trained target defect detection model, a defect test result is determined, the aim is to retrain and obtain the defect edge characteristics of industrial equipment, the accuracy of the detection result is improved, potential hazards of the equipment are found in time, and the working safety of the equipment is improved.
Description
Technical Field
The present invention relates to the field of device surface defect detection technologies, and in particular, to a defect detection method, device, electronic device, and storage medium.
Background
In the industrial production process, the safety risk coefficient of large-scale production processing equipment is higher and higher along with the time due to abrasion, aging, corrosion and the like. The service life set by the device instruction manual cannot accurately reflect the actual condition of the device, and periodic quality inspection supervision only by manpower is time-consuming and labor-consuming, and some non-standard problems with lighter degree are easily ignored.
Under the prior art condition, the surface defect detection task of large-scale industrial equipment based on machine vision and artificial intelligence technology has the following problems: the samples are rare, the similar devices are few, and a sufficient quantity of defect picture training models are difficult to collect; the foreground and background of the defect picture are not obviously distinguished, and the defect characteristic extraction is difficult due to the diversity of the background; the common 'frame selection' target detection model can not extract the defect edge characteristics, so that the accuracy of the detection result is low.
Disclosure of Invention
The invention provides a defect detection method, a device, electronic equipment and a storage medium, and aims to retrain and acquire defect edge characteristics of industrial equipment, improve the accuracy of detection results, discover potential hidden danger of the equipment in time and improve the working safety of the equipment.
According to an aspect of the present invention, there is provided a defect detection method including:
Obtaining a defect test picture;
inputting a defect test picture into a pre-trained target defect detection model, and determining a defect test result; the target defect detection model is constructed based on the segmentation model and the two-class segmentation model, and is trained and determined by taking an optimization rule as an index.
Optionally, the method for training the target defect detection model includes: acquiring a surface defect sample image and a visual segmentation data image of industrial equipment; carrying out knowledge distillation on the segmentation model and the two classification segmentation models according to the surface defect sample image to obtain a surface defect detection model; and training the surface defect detection model by taking the optimization rule as an index according to the surface defect sample image and the visual segmentation data image, and determining the target defect detection model.
Optionally, performing knowledge distillation on the segmentation model and the binary segmentation model according to the surface defect sample image and the visual segmentation data image to obtain a surface defect detection model, including: inputting the surface defect sample picture and the visual segmentation data image into a segmentation model for knowledge distillation, determining a first sample picture mask and a first data image mask, inputting the surface defect sample picture and the visual segmentation data image into a two-class segmentation model for training, and determining a second sample picture mask and a second data image mask; determining a first target according to the first data image mask and the second data image mask, and determining a second target according to the first sample picture mask and the second sample picture mask; determining a loss function value from the first objective and the second objective; determining whether the loss function value is less than or equal to a preset function value; if the loss function value is larger than the preset function value, returning to execute the steps of inputting the surface defect sample picture and the visual segmentation data image into the segmentation model for knowledge distillation, determining a first sample picture mask and a first data image mask, inputting the surface defect sample picture and the visual segmentation data image into the two-classification segmentation model for training, and determining a second sample picture mask and a second data image mask; and if the loss function value is smaller than or equal to the preset function value, determining the current two-class segmentation model as a surface defect detection model.
Optionally, determining the first target according to the first data image mask and the second data image mask includes: determining the pixel number of the surface defect sample picture, and determining the first pixel coordinate of a first data image mask corresponding to the pixel number and the second pixel coordinate of a second data image mask corresponding to the pixel number; and determining a first target according to the pixel number, the first pixel coordinate and the second pixel coordinate.
Optionally, determining the second target according to the first sample picture mask and the second sample picture mask includes: determining the pixel number of the surface defect sample picture, and determining the third pixel coordinate of the first sample picture mask corresponding to the pixel number and the fourth pixel coordinate of the second sample picture mask corresponding to the pixel number; and determining a second target according to the pixel number, the third pixel coordinate and the fourth pixel coordinate.
Optionally, determining the loss function value according to the first objective and the second objective includes: determining a loss weight; the loss function value is determined based on the loss weight, the first objective and the second objective.
Optionally, training the surface defect detection model with the optimization rule as an index according to the surface defect sample image, and determining the target defect detection model includes: acquiring a main framework model of the surface defect detection model; determining a classification model according to the main architecture model, the classification output layer and the network model; inputting the surface defect sample picture into a classification model to determine a defect type label; and (3) maintaining the parameters of the bottom layer of the surface defect detection model unchanged, inputting the surface defect sample picture and the defect type label into the surface defect detection model for retraining, and determining the target defect detection model.
According to another aspect of the present invention, there is provided a defect detecting apparatus including:
the image acquisition module is used for acquiring a defect test image;
the result determining module is used for inputting the defect test picture into a pre-trained target defect detection model and determining a defect test result; the target defect detection model is constructed based on the segmentation model and the two-class segmentation model, and is trained and determined by taking an optimization rule as an index.
According to another aspect of the present invention, there is provided an electronic device including:
at least one processor; and a memory communicatively coupled to the at least one processor;
wherein the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the defect detection method of any one of the embodiments of the present invention.
According to another aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to execute a defect detection method of any embodiment of the present invention.
According to the technical scheme, the defect test picture is obtained; inputting a defect test picture into a pre-trained target defect detection model, and determining a defect test result; the target defect detection model is constructed based on the segmentation model and the two-class segmentation model, and is trained and determined by taking an optimization rule as an index. According to the technical scheme, a target defect detection model is established based on a segmentation model and a binary segmentation model, and is trained and determined by taking an optimization rule as an index, and then a defect test picture is obtained; the defect test picture is input into a pre-trained target defect detection model, a defect test result is determined, the aim is to retrain and obtain the defect edge characteristics of industrial equipment, the accuracy of the detection result is improved, potential hazards of the equipment are found in time, and the working safety of the equipment is improved.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a defect detection method according to the first embodiment;
FIG. 2 is a schematic diagram of a defect detecting device according to a second embodiment;
fig. 3 is a schematic structural diagram of an electronic device according to the third embodiment.
Detailed Description
In order that the manner in which the invention may be better understood, a more particular description of the invention will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
Fig. 1 is a schematic flow chart of a defect detection method provided in the first embodiment, which is applicable to the surface defect detection of an industrial device, and the method may be performed by a defect detection device, which may be implemented in the form of hardware and/or software, and the defect detection device may be configured in an electronic device. As shown in fig. 1, the method includes:
s101, obtaining a defect test picture.
The defect test picture is a sample picture of the surface defect of the industrial equipment and is used for testing the surface defect of the industrial equipment.
Specifically, defect test pictures of surface defects of industrial equipment are collected and acquired.
S102, inputting the defect test picture into a pre-trained target defect detection model, and determining a defect test result.
The target defect detection model is constructed based on the segmentation model and the two-class segmentation model, and is trained and determined by taking an optimization rule as an index. The defect test result includes a defect picture and a defect feature, the defect picture is used for reflecting a defect type of a portion of the industrial equipment in the defect test picture, the defect type may be, for example, abrasion, aging, etc., which is not limited in this embodiment, the defect feature is used for indicating a defect degree corresponding to the defect type of the industrial equipment in the defect picture, for example, the defect degree of defect abrasion is 30 percent, etc., which is not limited in this embodiment.
The segmentation model is typically used to segment the defect test picture, thereby obtaining a picture mask for the defect test picture. The segmentation model can be (SAM, segment Anything Model), for example, the SAM segmentation model is a general model, can segment any seen or not seen object in an image or video, and outputs a mask conforming to the edge characteristics of the foreground object, and has the effect similar to automatic matting. Such as object instances, materials, parts, contours, and text segmentation. Experimental results in various tasks show great ability. The SAM segmentation model has the advantages of strong universality, realization of semantic segmentation of images with any size and format, unified conversion of masks into image formats for subsequent processing, flexibility, universality and one-step in place; the capability is stronger: experimental results in various tasks show greater capability and higher performance. Compared with other technical schemes, the method has obvious advantages in the aspects of accuracy, speed, efficiency and the like; the method has strong generalization capability, is rare in daily life such as industrial equipment defects, has no images which can participate in training, has good adaptability, and can accurately outline the defect position.
The picture mask refers to various bit operation operations between pictures, and is used for partially or completely hiding the image operation of the object or the part of the element, and the effect of applying the eye mask to the object of the picture is that the image object is coated on the background through a mask, so that the various parts of the image object are completely or partially covered, and the image in the cover is unchanged. The two-class segmentation model is a preset basic model for carrying out network training.
Specifically, after a defect test picture is obtained, the defect test picture is input into a pre-trained target defect detection model, and a defect test result is determined.
In one embodiment, a method of training a target defect detection model includes: acquiring a surface defect sample image and a visual segmentation data image of industrial equipment; carrying out knowledge distillation on the segmentation model and the two classification segmentation models according to the surface defect sample image to obtain a surface defect detection model; and training the surface defect detection model by taking the optimization rule as an index according to the surface defect sample image and the visual segmentation data image, and determining the target defect detection model.
The surface defect sample image is a surface defect picture of the industrial equipment which is collected in advance and used for carrying out model training; the visual segmentation data image is a visual segmentation data set.
Specifically, a surface defect sample image and a visual segmentation data image of industrial equipment are obtained, and knowledge distillation is carried out on a segmentation model and a binary segmentation model according to the surface defect sample image to obtain a surface defect detection model; and training the surface defect detection model by taking the optimization rule as an index according to the surface defect sample image and the visual segmentation data image, and determining the target defect detection model.
In one embodiment, performing knowledge distillation on the segmentation model and the binary segmentation model according to the surface defect sample image and the visual segmentation data image to obtain a surface defect detection model, wherein the method comprises the following steps of: inputting the surface defect sample picture and the visual segmentation data image into a segmentation model for knowledge distillation, determining a first sample picture mask and a first data image mask, inputting the surface defect sample picture and the visual segmentation data image into a two-class segmentation model for training, and determining a second sample picture mask and a second data image mask; determining a first target according to the first data image mask and the second data image mask, and determining a second target according to the first sample picture mask and the second sample picture mask; determining a loss function value from the first objective and the second objective; determining whether the loss function value is less than or equal to a preset function value; if the loss function value is larger than the preset function value, returning to execute the steps of inputting the surface defect sample picture and the visual segmentation data image into the segmentation model for knowledge distillation, determining a first sample picture mask and a first data image mask, inputting the surface defect sample picture and the visual segmentation data image into the two-classification segmentation model for training, and determining a second sample picture mask and a second data image mask; and if the loss function value is smaller than or equal to the preset function value, determining the current two-class segmentation model as a surface defect detection model.
The first sample picture mask is a picture mask after the segmentation model masks the surface defect sample image; the first data image mask is a picture mask after the segmentation model masks the visual segmentation data image; the second sample picture mask is a picture mask obtained by masking the surface defect sample image by the two-class segmentation model; the second data image mask is a picture mask after the visual segmentation data image is masked by the binary segmentation model.
The first object is also called a soft object and is determined through Euclidean distance calculation of the first data image mask and the second data image mask; the second object, also called hard object, is determined by euclidean distance calculations of the first sample picture mask and the second sample picture mask.
Wherein loss function value loss=loss soft +(1-a)loss hard A is a preset weight. The preset function value is a preset function value for judging whether the loss function value meets the requirement.
Specifically, after the surface defect sample image and the visual segmentation data image are obtainedAnd then, firstly inputting the surface defect sample image and the visual segmentation data image into a segmentation model for knowledge distillation, determining a first sample picture mask and a first data image mask, and simultaneously inputting the surface defect sample image and the visual segmentation data image into a two-class segmentation model for training, and determining a second sample picture mask and a second data image mask. Then determining a first target based on the first data image mask and the second data image mask, and determining a second target based on the first sample picture mask and the second sample picture mask, further, the first target Wherein N is the number of pixels of the currently input surface defect sample picture, (x) 11i -x 21i ) Mask pixel coordinates for the first data image, (y) 11i -y 21i ) For the pixel coordinates of the second data image mask, i is the number of pixel points, and is the second targetWherein N is the number of pixels of the currently input surface defect sample picture, (x) 12i -x 22i ) Pixel coordinates of the first sample picture mask, (y) 12i -y 22i ) And the pixel coordinates of the second sample picture mask are the number of pixel points. Then, a loss function value is determined from the first target and the second target, the loss function value loss=aloss soft +(1-a)loss hard Determining whether the loss function value is greater than or equal to a preset function value; if the loss function value is larger than the preset function value, the steps of inputting the surface defect sample picture and the visual segmentation data image into the segmentation model for knowledge distillation, determining a first sample picture mask and a first data image mask, inputting the surface defect sample picture and the visual segmentation data image into the two-class segmentation model for training, determining a second sample picture mask and a second data image mask, continuously obtaining the masks, and continuously determining the loss function value are performed; if the loss function value is smaller than or equal to the preset function value, determining that the current iterative training binary segmentation model is a surface And (5) a defect detection model.
In one embodiment, determining the first target from the first data image mask and the second data image mask includes: determining the pixel number of the surface defect sample picture, and determining the first pixel coordinate of a first data image mask corresponding to the pixel number and the second pixel coordinate of a second data image mask corresponding to the pixel number; and determining a first target according to the pixel number, the first pixel coordinate and the second pixel coordinate.
The first pixel coordinates are coordinates corresponding to all pixel points in the first data image mask, and the second pixel coordinates are coordinates corresponding to all pixel points in the second data image mask.
Specifically, determining the pixel number of the surface defect sample picture, and determining a first pixel coordinate of a first data image mask corresponding to the pixel number and a second pixel coordinate of a second data image mask corresponding to the pixel number; determining a first target according to the pixel number, the first pixel coordinate and the second pixel coordinate, wherein the first target is a targetWherein N is the number of pixels of the currently input surface defect sample picture, (x) 11i -x 21i ) Mask pixel coordinates for the first data image, (y) 11i -y 21i ) For the pixel coordinates of the second data image mask, i is the number of pixel points.
The advantage of this is that the accuracy of model training is improved by determining the pixel coordinates of the first data image mask and the pixel coordinates of the second data image mask, thereby determining the first target based on the euclidean distance.
In one embodiment, determining the second target from the first sample picture mask and the second sample picture mask includes: determining the pixel number of the surface defect sample picture, and determining the third pixel coordinate of the first sample picture mask corresponding to the pixel number and the fourth pixel coordinate of the second sample picture mask corresponding to the pixel number; and determining a second target according to the pixel number, the third pixel coordinate and the fourth pixel coordinate.
The third pixel coordinates are coordinates corresponding to each pixel point in the first sample picture mask, and the fourth pixel coordinates are coordinates corresponding to each pixel point in the second sample picture mask.
Specifically, determining the pixel number of the surface defect sample picture, and determining the third pixel coordinate of the first sample picture mask corresponding to the pixel number and the fourth pixel coordinate of the second sample picture mask corresponding to the pixel number; determining a second target according to the pixel number, the third pixel coordinate and the fourth pixel coordinate, wherein the second target Wherein N is the number of pixels of the currently input surface defect sample picture, (x) 12i -x 22i ) Pixel coordinates of the first sample picture mask, (y) 12i -y 22i ) And the pixel coordinates of the second sample picture mask are the number of pixel points.
In one embodiment, determining the loss function value from the first objective and the second objective includes: determining a loss weight; the loss function value is determined based on the loss weight, the first objective and the second objective.
The loss weight is a preset weight for determining a loss function value according to the first target and the second target. The loss function value is the weighted summation of the first target and the second target, wherein the larger the weighting coefficient of the first target is, the more the migration induction depends on the contribution of the segmentation model and the visual segmentation data image, which is necessary for the initial stage of training, and is helpful for the two-class segmentation model to identify simple samples more easily, but the later stage of training needs to properly reduce the specific gravity of the first target, so that the true labeling helps to identify true defect samples.
Specifically, determining a loss weight; determining a loss function value from the loss weight, the first objective and the second objective, the loss function value loss=all soft +(1-a)loss hard 。
In a specific embodiment, training the surface defect detection model with the optimization rule as an index according to the surface defect sample image to determine a target defect detection model, including: acquiring a main framework model of the surface defect detection model; determining a classification model according to the main architecture model, the classification output layer and the network model; inputting the surface defect sample picture into a classification model to determine a defect type label; and (3) maintaining the parameters of the bottom layer of the surface defect detection model unchanged, inputting the surface defect sample picture and the defect type label into the surface defect detection model for retraining, and determining the target defect detection model.
Specifically, a main framework model of the surface defect detection model is obtained, the main framework model of the surface defect detection model is used as a basic framework of a network model, the main framework model of the surface defect detection model is modified at the tail end of a neural network, a classification output layer is added to determine a classification model, and the classification output layer can be added to be a deep convolution network structure; inputting the surface defect sample picture into a classification model to determine a defect type label; maintaining the parameters of the bottom layer of the surface defect detection model unchanged, carrying out original context understanding and semantic segmentation capability of the model, inputting a surface defect sample picture and a defect type label into the surface defect detection model for retraining, and determining a target defect detection model.
The method has the advantages that the efficiency and the accuracy of model training are improved, the defect edge characteristics are focused, and the use experience of a user is improved.
Specifically, a surface defect sample image and a visual segmentation data image of industrial equipment are obtained; carrying out knowledge distillation on the segmentation model and the two classification segmentation models according to the surface defect sample image to obtain a surface defect detection model; inputting the surface defect sample picture and the visual segmentation data image into a segmentation model for knowledge distillation, determining a first sample picture mask and a first data image mask, inputting the surface defect sample picture and the visual segmentation data image into a two-class segmentation model for training, and determining a second sample picture mask and a second data image mask; determining a first target according to the first data image mask and the second data image mask, and determining a second target according to the first sample picture mask and the second sample picture mask; determining a loss function value from the first objective and the second objective; determining whether the loss function value is less than or equal to a preset function value; if the loss function value is larger than the preset function value, returning to execute the steps of inputting the surface defect sample picture and the visual segmentation data image into the segmentation model for knowledge distillation, determining a first sample picture mask and a first data image mask, inputting the surface defect sample picture and the visual segmentation data image into the two-classification segmentation model for training, and determining a second sample picture mask and a second data image mask; and if the loss function value is smaller than or equal to the preset function value, determining the current two-class segmentation model as a surface defect detection model. Training the surface defect detection model by taking an optimization rule as an index according to the surface defect sample image and the visual segmentation data image, and determining a target defect detection model, namely acquiring a main framework model of the surface defect detection model; determining a classification model according to the main architecture model, the classification output layer and the network model; inputting the surface defect sample picture into a classification model to determine a defect type label; and (3) maintaining the parameters of the bottom layer of the surface defect detection model unchanged, inputting a surface defect sample picture and a defect type label into the surface defect detection model for retraining, determining a target defect detection model, and finally, inputting a defect test picture into the pre-trained target defect detection model to determine a defect test result. The method aims at retraining and acquiring the defect edge characteristics of the industrial equipment, improving the accuracy of detection results, finding potential hidden danger of the equipment in time, improving the working safety of the equipment, scanning the surface defects of the industrial equipment, finding potential hidden danger in time, facilitating local repair and part replacement, and prolonging the service life of the whole equipment.
According to the technical scheme, the defect test picture is obtained; inputting a defect test picture into a pre-trained target defect detection model, and determining a defect test result; the target defect detection model is constructed based on the segmentation model and the two-class segmentation model, and is trained and determined by taking an optimization rule as an index. According to the technical scheme, a target defect detection model is established based on a segmentation model and a binary segmentation model, and is trained and determined by taking an optimization rule as an index, and then a defect test picture is obtained; the defect test picture is input into a pre-trained target defect detection model, a defect test result is determined, the aim is to retrain and obtain the defect edge characteristics of industrial equipment, the effect of small sample detection is improved, the accuracy of the detection result is improved, potential hazards of the equipment are found in time, and the safety of the equipment operation is improved.
Example two
Fig. 2 is a schematic structural diagram of a defect detecting device according to the second embodiment. As shown in fig. 2, the apparatus includes: a picture acquisition module 201 and a result determination module 202; wherein,
the picture obtaining module 201 is configured to obtain a defect test picture.
The result determining module 202 is configured to input a defect test picture to a pre-trained target defect detection model, and determine a defect test result; the target defect detection model is constructed based on the segmentation model and the two-class segmentation model, and is trained and determined by taking an optimization rule as an index.
Optionally, the device further comprises a model training module for acquiring a surface defect sample image and a visual segmentation data image of the industrial equipment; carrying out knowledge distillation on the segmentation model and the two classification segmentation models according to the surface defect sample image to obtain a surface defect detection model; and training the surface defect detection model by taking the optimization rule as an index according to the surface defect sample image and the visual segmentation data image, and determining the target defect detection model.
Optionally, the model training module performs knowledge distillation on the segmentation model and the binary segmentation model according to the surface defect sample image and the visual segmentation data image to obtain a surface defect detection model, which is specifically used for: inputting the surface defect sample picture and the visual segmentation data image into a segmentation model for knowledge distillation, determining a first sample picture mask and a first data image mask, inputting the surface defect sample picture and the visual segmentation data image into a two-class segmentation model for training, and determining a second sample picture mask and a second data image mask; determining a first target according to the first data image mask and the second data image mask, and determining a second target according to the first sample picture mask and the second sample picture mask; determining a loss function value from the first objective and the second objective; determining whether the loss function value is less than or equal to a preset function value; if the loss function value is larger than the preset function value, returning to execute the steps of inputting the surface defect sample picture and the visual segmentation data image into the segmentation model for knowledge distillation, determining a first sample picture mask and a first data image mask, inputting the surface defect sample picture and the visual segmentation data image into the two-classification segmentation model for training, and determining a second sample picture mask and a second data image mask; and if the loss function value is smaller than or equal to the preset function value, determining the current two-class segmentation model as a surface defect detection model.
Optionally, the model training module determines a first target according to the first data image mask and the second data image mask, and is specifically configured to: determining the pixel number of the surface defect sample picture, and determining the first pixel coordinate of a first data image mask corresponding to the pixel number and the second pixel coordinate of a second data image mask corresponding to the pixel number; and determining a first target according to the pixel number, the first pixel coordinate and the second pixel coordinate.
Optionally, the model training module determines a second target according to the first sample picture mask and the second sample picture mask, and is specifically configured to: determining the pixel number of the surface defect sample picture, and determining the third pixel coordinate of the first sample picture mask corresponding to the pixel number and the fourth pixel coordinate of the second sample picture mask corresponding to the pixel number; and determining a second target according to the pixel number, the third pixel coordinate and the fourth pixel coordinate.
Optionally, the model training module is configured to determine a loss function value according to the first objective and the second objective, and specifically configured to: determining a loss weight; the loss function value is determined based on the loss weight, the first objective and the second objective.
Optionally, the model training module trains the surface defect detection model with the optimization rule as an index according to the surface defect sample image, and determines a target defect detection model, which is specifically used for: acquiring a main framework model of the surface defect detection model; determining a classification model according to the main architecture model, the classification output layer and the network model; inputting the surface defect sample picture into a classification model to determine a defect type label; and (3) maintaining the parameters of the bottom layer of the surface defect detection model unchanged, inputting the surface defect sample picture and the defect type label into the surface defect detection model for retraining, and determining the target defect detection model.
The defect detection device provided by the invention can execute the defect detection method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example III
Fig. 3 is a schematic structural diagram of an electronic device according to the third embodiment. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 3, the electronic device 10 includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, in which the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 may also be stored. The processor 11, the ROM 12 and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
Various components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as defect detection methods.
In some embodiments, the defect detection method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the defect detection method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the defect detection method by any other suitable means (e.g. by means of firmware).
Various implementations of the systems and techniques described here above can be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server) or that includes a middleware component (e.g., an application server) or that includes a front-end component through which a user can interact with an implementation of the systems and techniques described here, or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (10)
1. A method of defect detection, the method comprising:
obtaining a defect test picture;
inputting the defect test picture into a pre-trained target defect detection model, and determining a defect test result; the target defect detection model is constructed based on a segmentation model and a two-class segmentation model, and is a model which is determined by training with an optimization rule as an index.
2. The defect detection method of claim 1, wherein the method of training the target defect detection model comprises:
Acquiring a surface defect sample image and a visual segmentation data image of industrial equipment;
carrying out knowledge distillation on the segmentation model and the two-classification segmentation model according to the surface defect sample image to obtain a surface defect detection model;
and training the surface defect detection model by taking the optimization rule as an index according to the surface defect sample image and the visual segmentation data image, and determining the target defect detection model.
3. The defect detection method of claim 2, wherein performing knowledge distillation on the segmentation model and the bi-classification segmentation model based on the surface defect sample image and the visual segmentation data image to obtain a surface defect detection model comprises:
inputting the surface defect sample picture and the visual segmentation data image into the segmentation model for knowledge distillation, determining a first sample picture mask and a first data image mask, inputting the surface defect sample picture and the visual segmentation data image into a two-class segmentation model for training, and determining a second sample picture mask and a second data image mask;
determining a first target according to the first data image mask and the second data image mask, and determining a second target according to the first sample picture mask and the second sample picture mask;
Determining a loss function value from the first objective and the second objective;
determining whether the loss function value is less than or equal to a preset function value;
if the loss function value is larger than the preset function value, returning to execute the steps of inputting the surface defect sample picture and the visual segmentation data image into the segmentation model for knowledge distillation, determining a first sample picture mask and a first data image mask, inputting the surface defect sample picture and the visual segmentation data image into a two-class segmentation model for training, and determining a second sample picture mask and a second data image mask;
and if the loss function value is smaller than or equal to the preset function value, determining the current two-classification segmentation model as the surface defect detection model.
4. A defect detection method according to claim 3, wherein said determining a first target from said first data image mask and said second data image mask comprises:
determining the pixel number of the surface defect sample picture, and determining a first pixel coordinate of the first data image mask corresponding to the pixel number and a second pixel coordinate of the second data image mask corresponding to the pixel number;
And determining the first target according to the pixel number, the first pixel coordinate and the second pixel coordinate.
5. A defect detection method according to claim 3, wherein said determining a second target from said first sample picture mask and said second sample picture mask comprises:
determining the pixel number of the surface defect sample picture, and determining the third pixel coordinate of the first sample picture mask corresponding to the pixel number and the fourth pixel coordinate of the second sample picture mask corresponding to the pixel number;
and determining the second target according to the pixel number, the third pixel coordinate and the fourth pixel coordinate.
6. A defect detection method according to claim 3, wherein said determining a loss function value from said first target and said second target comprises:
determining a loss weight;
and determining the loss function value according to the loss weight, the first target and the second target.
7. The defect detection method according to claim 2, wherein the training the surface defect detection model with the optimization rule as an index according to the surface defect sample image, and determining the target defect detection model, comprises:
Acquiring a main framework model of the surface defect detection model;
determining a classification model according to the main construction model, the classification output layer and the network model;
inputting the surface defect sample picture into the classification model to determine a defect type label;
and maintaining the parameters of the bottom layer of the surface defect detection model unchanged, inputting the surface defect sample picture and the defect type label into the surface defect detection model for retraining, and determining the target defect detection model.
8. A defect detection apparatus, the apparatus comprising:
the image acquisition module is used for acquiring a defect test image;
the result determining module is used for inputting the defect test picture into a pre-trained target defect detection model and determining a defect test result; the target defect detection model is constructed based on a segmentation model and a two-class segmentation model, and is a model which is determined by training with an optimization rule as an index.
9. An electronic device, the electronic device comprising:
at least one processor; and a memory communicatively coupled to the at least one processor;
wherein the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the defect detection method of any one of claims 1 to 7.
10. A computer readable storage medium storing computer instructions for causing a processor to perform the defect detection method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311265857.1A CN117333443A (en) | 2023-09-27 | 2023-09-27 | Defect detection method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311265857.1A CN117333443A (en) | 2023-09-27 | 2023-09-27 | Defect detection method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117333443A true CN117333443A (en) | 2024-01-02 |
Family
ID=89294539
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311265857.1A Pending CN117333443A (en) | 2023-09-27 | 2023-09-27 | Defect detection method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117333443A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117710366A (en) * | 2024-02-02 | 2024-03-15 | 杭州百子尖科技股份有限公司 | Quality inspection method and device for thermos cup and storage medium |
-
2023
- 2023-09-27 CN CN202311265857.1A patent/CN117333443A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117710366A (en) * | 2024-02-02 | 2024-03-15 | 杭州百子尖科技股份有限公司 | Quality inspection method and device for thermos cup and storage medium |
| CN117710366B (en) * | 2024-02-02 | 2024-05-14 | 杭州百子尖科技股份有限公司 | Quality inspection method and device for thermos cup and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112949767B (en) | Sample image increment, image detection model training and image detection method | |
| CN112633276B (en) | Training method, recognition method, device, equipment and medium | |
| CN113436100B (en) | Method, apparatus, device, medium, and article for repairing video | |
| CN116952958B (en) | Defect detection method, device, electronic equipment and storage medium | |
| CN113378832B (en) | Text detection model training method, text prediction box method and device | |
| CN117333443A (en) | Defect detection method and device, electronic equipment and storage medium | |
| CN115358392A (en) | Deep learning network training method, text detection method and text detection device | |
| CN114429633A (en) | Text recognition method, model training method, device, electronic equipment and medium | |
| CN116740355A (en) | Automatic driving image segmentation method, device, equipment and storage medium | |
| CN115471476A (en) | Method, device, equipment and medium for detecting component defects | |
| CN117521768A (en) | Training method, device, equipment and storage medium of image search model | |
| CN116309963A (en) | Batch labeling method and device for images, electronic equipment and storage medium | |
| CN114724144B (en) | Text recognition method, training device, training equipment and training medium for model | |
| CN117197517A (en) | Insulator pollution classification method and device, electronic equipment and medium | |
| CN116363444A (en) | Fuzzy classification model training method, fuzzy image recognition method and device | |
| CN115359322A (en) | Target detection model training method, device, equipment and storage medium | |
| CN114612971A (en) | Face detection method, model training method, electronic device, and program product | |
| CN116503407B (en) | Method and device for detecting foreign object region in image and electronic equipment | |
| CN113361524B (en) | Image processing method and device | |
| CN117746069B (en) | Graph searching model training method and graph searching method | |
| CN117333487B (en) | Acne classification method, device, equipment and storage medium | |
| CN117649683B (en) | Acne grading method, device, equipment and storage medium | |
| CN116823924B (en) | Determination method and device for defect area, electronic equipment and storage medium | |
| CN116580050A (en) | Medical image segmentation model determination method, device, equipment and medium | |
| CN117893510A (en) | Defect detection method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |