CN112700432A - Texture surface defect detection method and system based on abnormal synthesis and decomposition - Google Patents
Texture surface defect detection method and system based on abnormal synthesis and decomposition Download PDFInfo
- Publication number
- CN112700432A CN112700432A CN202110033380.9A CN202110033380A CN112700432A CN 112700432 A CN112700432 A CN 112700432A CN 202110033380 A CN202110033380 A CN 202110033380A CN 112700432 A CN112700432 A CN 112700432A
- Authority
- CN
- China
- Prior art keywords
- abnormal
- defect
- image
- texture
- decomposition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000007547 defect Effects 0.000 title claims abstract description 141
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 120
- 238000000354 decomposition reaction Methods 0.000 title claims abstract description 45
- 238000001514 detection method Methods 0.000 title claims abstract description 41
- 230000015572 biosynthetic process Effects 0.000 title claims abstract description 27
- 238000003786 synthesis reaction Methods 0.000 title claims abstract description 27
- 238000000034 method Methods 0.000 claims abstract description 33
- 238000012549 training Methods 0.000 claims abstract description 32
- 238000005070 sampling Methods 0.000 claims abstract description 30
- 230000011218 segmentation Effects 0.000 claims abstract description 22
- 238000012545 processing Methods 0.000 claims abstract description 4
- 230000008569 process Effects 0.000 claims description 14
- 230000002547 anomalous effect Effects 0.000 claims description 9
- 238000009826 distribution Methods 0.000 claims description 7
- 230000002950 deficient Effects 0.000 claims description 5
- 230000002194 synthesizing effect Effects 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 2
- 238000001308 synthesis method Methods 0.000 abstract description 6
- 230000006870 function Effects 0.000 description 16
- 239000013598 vector Substances 0.000 description 14
- 238000004519 manufacturing process Methods 0.000 description 8
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000005192 partition Methods 0.000 description 3
- 230000003042 antagnostic effect Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000001788 irregular Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 239000002023 wood Substances 0.000 description 2
- 229910000831 Steel Inorganic materials 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 239000000919 ceramic Substances 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 239000002994 raw material Substances 0.000 description 1
- 239000010959 steel Substances 0.000 description 1
- 239000004753 textile Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F18/2132—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on discrimination criteria, e.g. discriminant analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/088—Non-supervised learning, e.g. competitive learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a texture surface defect detection method and system based on abnormal synthesis and decomposition, and belongs to the field of image processing. The invention constructs a defect generation network guided by segmentation, can generate a large number of defect samples similar to real defects by using a small number of real defect training samples, and simultaneously provides an abnormal synthesis method based on Gaussian sampling, which can randomly synthesize abnormal negative samples by using defect-free positive samples, thereby solving the problem of small number of defect samples in industry and further improving the defect detection precision; according to the method, the abnormal negative sample is decomposed into the texture background image and the abnormal mask image by adopting the abnormal decomposition network, so that the defect can be effectively inhibited from being reconstructed into the texture background, the texture background reconstruction precision is improved, the defect area can be accurately segmented, the residual image and the abnormal segmented image are fused, the defect detection rate is improved, and the defect overdetection rate is reduced.
Description
Technical Field
The invention belongs to the field of image processing, and particularly relates to a texture surface defect detection method and system based on abnormal synthesis and decomposition.
Background
In the field of industrial manufacturing, various industrial products have different raw material quality types and complex production and manufacturing process. It is difficult to avoid various surface defects on the surface of products, such as textiles, novel display devices, ceramics, steel, and the like. Surface defects refer to local areas that differ from the surrounding texture and pattern, or local areas with irregular brightness variations. Because the surfaces of various products often present different texture characteristics, the texture surface defects can directly reduce the product quality and influence the user experience. To improve production quality, all types of surface defects should be tightly controlled during the manufacturing process. The texture surface defect detection technology based on machine vision is the most important technology for detecting the texture surface defects, and intelligent manufacturing is realized by utilizing an artificial intelligence technology. Artificial intelligence based detection of textured surface defects is fundamental and critical to the overall manufacturing industry.
In the manufacturing process of various industrial products, the texture surface defects have the following characteristics: different types, large-scale change, low contrast, irregular brightness change and variable shapes, and meanwhile, the number of defective products relative to good products is extremely small, so that the number of available defect samples is small, and great difficulty is brought to visual detection. Therefore, texture surface defect detection algorithms remain a challenging task in industrial product quality control.
At present, a large number of algorithms are proposed to solve the problem of detecting the texture surface defects. The existing method can only be suitable for one or more types of textures (such as only a display device can be detected, and the method cannot be suitable for the surface of wood), can only detect fixed type texture defects (such as only high contrast defects can be detected, and low contrast defects cannot be detected), and is difficult to solve all conditions. Therefore, a more robust and more adaptive deep learning texture surface defect detection algorithm needs to be provided, which is suitable for various textures and has better robustness for different types of texture surface defects.
Disclosure of Invention
In view of the above defects or improvement needs in the prior art, the present invention provides a texture surface defect detection method and system based on abnormal synthesis and decomposition, which aims to improve the accuracy of surface defect detection.
In order to achieve the above object, the present invention provides a texture surface defect detection method based on abnormal synthesis and decomposition, comprising:
s1, constructing a defect generation network for segmentation guidance; the defect generation network comprises a generator and a discriminator of segmentation guide; the generator is used for synthesizing the first type negative sample image IcnConversion into a first type abnormal negative sample image I more similar to a real defect imagen(ii) a The first type of synthetic negative sample image IcnFrom defect-free textured background IpAnd a defect image IdDefective label image I oflBy regional superposition synthesis; the partition-guided discriminator is used for respectively discriminating abnormal negative sample images InWith the real defect image IdCarrying out defect segmentation;
s2, aiming at defect segmentation, training a defect generation network by using a counterstudy method;
s3, converting the second type of synthetic negative sample image I 'by using a trained generator'cnObtaining a second abnormal negative sample image I'n(ii) a The second type of synthetic negative sample image is composed of a defect-free texture background IpAbnormal mask image I generated by random samplingaBy regional superposition synthesis;
s4, constructing an abnormal decomposition network and using a defect-free texture background IpAnd a second-type abnormal negative sample image I'nTraining an abnormal decomposition network as a training set; the abnormal decomposition network is used for decomposing the defect image into a texture background image and an abnormal image;
and S5, inputting the texture surface image to be detected into the trained abnormal decomposition network to obtain a corresponding texture background image and an abnormal image, and fusing a residual image between the defect image and the texture background image with the abnormal image to obtain a defect detection result and obtain a defect detection result.
Further, a first class of synthetic negative sample images IcnFrom defect-free textured background IpAnd defective label image IlPerforming regional superposition synthesis by using the following expression;
Icn(x,y)=(1-Il(x,y))·Ip(x,y)+λ·Il(x,y)
where λ represents a random variable, x 1., W, y 1., H, W, H represent the image width and height, respectively.
Further, the defect generation network is trained using the following loss function:
wherein L isdis(θsd) Representing the loss function of the training arbiter, Lgen(θg) A loss function representing a training generator;respectively representing expectation, matrix dot product, L1、L2Norm, λa、λbRepresenting the weight corresponding to each loss.
Further, the random sampling comprises Gaussian sampling, student t distribution sampling and uniform sampling.
Further, the training process of the abnormal decomposition network comprises the following steps:
defect-free textured background I using an encoderpCoded as corresponding implicit spatial features Zp(ii) a Using a feature embedding encoder to embed a spatial feature ZpEncoding into corresponding texture featuresDecoding texture features using a decoderSign forObtaining a defect-free textured background IpIs reconstructed image of
Utilizing an encoder to obtain a second type abnormal negative sample image I'nCoded as corresponding implicit spatial features ZnAnd abnormal feature Za(ii) a Using a feature embedding encoder to embed a spatial feature ZnEncoding into corresponding texture featuresMake itAndsharing a feature distribution space; decoding texture features with a decoderCorrespondingly obtaining a second abnormal negative sample image I'nReconstructed textured background imageDecoding of exception features Z with exception decoderaObtaining an abnormal foreground image
Respectively processing the second type abnormal negative sample image I 'by a discriminator of segmentation guidance'nWith reconstructed textured background imageCarrying out defect segmentation;
an encoder, a feature embedding encoder, a decoder and a discriminator are trained using a counterlearning method.
Further, the anomaly decomposition network is trained using the following loss function:
Lrrepresenting reconstruction losses for training the encoder, the feature embedding encoder and the decoder; l iszRepresenting implicit spatial feature constraint loss; for training the encoder and the feature embedding encoder; l isaRepresenting an anomalous decoding loss for training the encoder and the anomalous decoder; l isadRepresents the discrimination loss of the discriminator, LagDiscriminator generation penalty for training an encoder, a feature embedding encoder and decoder, lambda1,λ2,λ3,λ4The weights of the four losses are respectively.
In general, the above technical solutions contemplated by the present invention can achieve the following advantageous effects compared to the prior art.
(1) The invention constructs a defect generation network guided by segmentation, can generate a large number of defect samples similar to real defects by using a small number of real defect training samples, and can randomly synthesize abnormal negative samples by using defect-free positive samples by the abnormal synthesis method based on Gaussian sampling, thereby solving the problem of small quantity of defect samples in industry and further improving the defect detection precision.
(2) The existing defect detection method comprises an unsupervised learning method and a supervised learning method, wherein the unsupervised learning method only utilizes non-defective good images for training and utilizes background reconstruction to detect defects, but because defects are not used in the training process, the defect feature expression cannot be learned, the method can only be suitable for one or more types of textures (such as only a display device can be detected and the surface of wood cannot be suitable), only can detect fixed types of texture defects (such as only high-contrast defects can be detected and low-contrast defects cannot be detected), and all conditions are difficult to solve; the supervised learning method needs a large amount of labeled defect samples for training, and the industrial field is difficult to collect a large amount of defect samples, so the method cannot be applied to the industrial field; according to the method, the abnormal negative sample is decomposed into the texture background image and the abnormal mask image by adopting the abnormal decomposition network, so that the defect can be effectively inhibited from being reconstructed into the texture background, the texture background reconstruction precision is improved, the defect area can be accurately segmented, the residual image and the abnormal segmented image are fused, the defect detection rate is improved, and the defect overdetection rate is reduced.
Drawings
FIG. 1 is a schematic flow chart of a texture surface defect detection method based on abnormal composition and decomposition according to the present invention;
FIG. 2 is a schematic diagram of a feature embedding encoder implemented in accordance with the present invention;
FIG. 3 is a comparison of defect detection effects.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
Abnormal negative sample image InCan be decomposed into defect-free texture background IpAnd abnormal image IaThe assumptions of (a) are as follows:
In(x,y)=Ib(x,y)·Ip(x,y)+Ia(x,y)·If(x,y) (0-1)
wherein, In,Ip,Ib,Ia,If∈RW×H×1W and H respectively indicate an image width and a height, and in this study, 256 pixels are used, x is 1. I isaRepresenting an abnormal mask image, IfRepresenting the corresponding abnormal region image, IbRepresenting a textured background mask image, i.e. Ib(x,y)=1-Ia(x,y)。
Based on the above assumptions, as shown in fig. 1, the present invention provides a texture surface defect detection method based on abnormal synthesis and decomposition, and the methodBased on the assumption of abnormal synthesis in equation (0-1), the synthesis of the abnormality in the decomposition process is simulated. Specifically, the method comprises a training process and a testing process, wherein the training process comprises three steps: the method comprises the steps of dividing guided defect generation network, abnormal synthesis based on Gaussian sampling and abnormal decomposition network. The defect generation network comprises a generator and a discriminator, and the generator can utilize a defect-free texture background I through a counterstudy processpGenerating an anomalous negative sample image In(ii) a An abnormal synthesis step based on Gaussian sampling generates a large number of abnormal negative sample images I 'through a random Gaussian sampling and generator'nThe abnormal decomposition network converts the abnormal negative sample image I'nDecomposition into textured background imagesAnd abnormal foreground imageRealizing an abnormal decomposition process; the testing process utilizes the trained abnormal decomposition network to carry out texture defect image IdDecomposition into textured background imagesAnd abnormal segmentation of imagesAnd by fusing texture defect images IdWith the texture background imageAnd detecting the texture surface defects by the residual image and the abnormal segmentation image. The proposed algorithm is explained in detail below.
(1) Split-guided defect generation network
The segmentation-guided defect generation network is used to generate a large number of abnormal negative examples similar to real defects.
Firstly, a defect-free texture background IpAnd a defect image IdDefective label image I oflSynthesizing a synthesized negative sample image I by regional superpositioncn:
Icn(x,y)=(1-Il(x,y))·Ip(x,y)+λ·Il(x,y) (0-2)
Wherein IlCorresponds to I in the formula (0-1) with a random variable λ (0 < λ < 1)aAnd If。
Then, the generator of the segmentation-guided defect generation network will synthesize a negative sample image IcnTransition to an anomalous negative sample image InMaking it more similar to the real defect image:
In=fg(Icn;θg) (0-3)
wherein In∈RW×H×1,fg(. o) and θgRepresenting the function and the parameters of the generator, respectively. Abnormal negative sample image InShould be in IlThe corresponding masked areas generate defects close to real defects and keep other texture areas unchanged. Thus, the segmentation-guided counterlearning approach presented herein provides different optimization objectives for mask regions versus texture regions. Abnormal negative sample image InWith the real defect image IdThe discriminator division defects that are simultaneously input to the division guide:
wherein Ins,Ids∈RW×H×1Representing the output pixel-by-pixel prediction result image, fsd(. o) and θsdRespectively representing functions and parameters of the discriminators of the segmentation guide. Purpose of the generator is IlThe corresponding mask region generates a defect close to a real defect, and the discriminator for segmentation guidance discriminates the defect as the real defect.
The pixel-by-pixel cross-entropy loss function is used to perform a segmentation-guided countermeasure learning process:
wherein L isdis(θsd) Representing the loss function of the training arbiter, Lgen(θg) A loss function representing a training generator;respectively representing expectation, matrix dot product, L1、L2Norm, λa、λbRepresenting the weight corresponding to each loss. To further ensure the generation of abnormal negative sample images InTexture region and defect-free positive sample image IpConsistently, the reconstruction loss is also used as an auxiliary loss training generator in equations (0-6).
(2) Anomaly synthesis based on Gaussian sampling
As shown in fig. 1, the abnormal synthesis method based on gaussian sampling may randomly synthesize an abnormal negative sample using a defect-free positive sample, and the defect-free positive sample is fused with an abnormal mask image of the random gaussian sampling to obtain a synthesized abnormal negative sample; and then, converting the synthesized abnormal negative sample into an abnormal negative sample which is closer to the real defect sample by using a generator in the trained segmentation-guided defect generation network.
The first step of the anomaly synthesis method based on gaussian sampling is gaussian sampling. In this step, an abnormal mask image IaGenerated by random Gaussian sampling and matched with a defect-free texture background IpFusing and synthesizing the image Icn. Based on the assumption that each defect region obeys Gaussian distribution, two-dimensional Gaussian function is used for randomly sampling an abnormal weight map
wherein,is a parameter of each two-dimensional gaussian function, i 1a,NaIndicating the number of defective areas. In this study, the random sampling range of each parameter is: a is more than or equal to 0.1i≤2,-90°≤θi≤90°,1≤NaLess than or equal to 3. By contrast against defect AiCenter position of defectShape of defectDefect angle thetaiThe random sampling of (2) can be carried out on a large number of defect samples.
where max (-) denotes maximum operation, TaThe threshold value is set to 0.7, and in practical application, the threshold value is set according to the requirement of defect contrast. Then, the ith abnormal region imageCan be obtained by fusionAnd defect-free positive samples IpObtaining:
where α ═ {1, -1}, is used to obtain bright defects or dark defects. The anomaly image and the anomaly mask image can be constructed by fusing all sampled regions:
finally, a synthetic abnormal image I can be obtainedcn. Therefore, based on Gaussian sampling, by fusing defect-free positive sample images IpAnd random sampling abnormal mask image Ia。
In the semi-supervised mode of the method, the anomaly synthesis method based on Gaussian sampling further performs anomaly transformation. A generator for generating a trained segmentation-guided defect generation network, synthesizing the abnormal image I by the formula (0-3)cnTransition to an anomalous negative sample image InSo as to make it more approximate to the real defect and synthesize a large amount of abnormal negative sample images. Gaussian sampling is easier to operate, and in practical application, other random sampling methods can be used for generating abnormal negative samples.
(3) Exception decomposition network
As shown in fig. 1, the abnormal decomposition network is composed of five modules: the device comprises an encoder, a characteristic embedding encoder, a decoder, an abnormal decoder and a partition guiding discriminator, wherein an abnormal negative sample is decomposed into a texture background image and an abnormal mask image, and the texture background accurate reconstruction and the defect area accurate partition can be realized through a partition counterstudy mechanism, so that the defect detection precision is greatly improved.
First, feature embedding encoding is performed by an encoderDecoder and decoder, defect-free texture background IpIs used to learn the texture feature representation of the implicit spatial nature. By means of an encoder, IpIs coded as a hidden spatial feature Zp:
Zp=fe(Ip;θe) (0-12)
Wherein,Wl,Hl,Clrespectively representing the width, height and number of channels of the hidden space feature, fe(. o) and θeRepresenting the functions and parameters of the encoder, respectively. The proposed feature embedding encoder will be ZpEncoding into texture features
Wherein,ft(. o) and θtRepresenting the function and parameter of the feature embedding encoder, respectively. The texture features are then decoded by a decoderA defect-free textured background I can be obtainedpIs reconstructed image of
Wherein,fd(. o) and θdRepresenting the functions and parameters of the decoder, respectively. The texture feature distribution can be learned through a reconstruction process formed by an encoder, a feature embedding encoder and a decoder:
at the same time, with a defect-free textured background IpCorresponding paired abnormal negative sample InEncoding into implicit spatial features Z by an encodernAnd abnormal feature Za:
Zn,Za=fe(In;θe) (0-16)
wherein Z belongs toAndunion of (1), NbRepresenting the batch size during random gradient descent training. Finally, the abnormal decoder and the decoder are respectively pairedAnd ZaDecoding, abnormal negative samples InI.e. decomposed into a textured background imageAnd abnormal foreground image
Wherein,fad(. o) and θadRespectively representing the function and parameters of the anomalous decoder.
Finally, an anomalous negative example InWith reconstructed textured background imageThe discriminator division defect input to the division guide:
wherein,is the output pixel-by-pixel prediction result. Pixel-by-pixel cross entropy loss function is used to perform segmentation bootThe antagonistic learning process of (1):
for decoding out abnormal foreground imageThe pixel-by-pixel cross-entropy penalty is used for the segmentation penalty:
by combining equations (0-15) (0-18) (0-21) (0-22), the anomaly decomposition network can be trained through a multitask antagonistic learning process:
wherein λ is1,λ2,λ3,λ4The weights of the four losses are respectively.
The purpose of the feature embedding encoder is to make the texture features of the defect-free positive sample image and the defect negative sample image share a common feature space, thereby forcing the model to efficiently separate the texture background image from the defect negative sample image.
The structure of the feature-embedded encoder is shown in fig. 2. Hidden spatial feature ZpAnd ZnDimension C that can be considered as an input imagelLocal feature set Z ofp={f1,...,fNWhere N ═ Wl×HlThe total number of local features. D ═ D1,...,dKDenotes a set of learnable word vectors of number K (K16 in this study). s ═ s1,...,sKAnd σ ═ σ }1,...,σKThe learnable smoothing factor and the diagonal covariance between the word vectors, respectively. The residual vector between the local features and the word vector can be calculated as: r isik=fi-dkWherein i is1, N, K1, K. The weight assigned to each local feature to each word vector may be calculated as:
the first and second order residual coding vectors for each word vector may be computed as:
wherein,andfirst and second order coded vectors, respectively.Andquilt L2The code vectors concatenated into each word vector after normalization:concatenating the code vectors of all word vectors, i.e.A residual coding vector can be obtained, which represents the unordered texture feature expression. Finally, e is input to a fully-connected layer to reduce its dimension to ClAnd the resultant vectors e' and Z are combinedpMultiplying by channels to obtain final texture feature expression:
(4) actual defect detection process
After the model optimization is completed, the abnormal decomposition network can be used for detecting texture defects. As shown in FIG. 1, in the defect detection stage, a texture defect image I is first obtaineddInputting the abnormal decomposition network to obtain a texture background imageAnd abnormal segmentation of imagesThen, the residual image and the abnormal segmentation image are fused to detect the texture surface defects:
wherein 0 < lambdafIs less than 1. The effect of defect detection is shown in fig. 3. The first column is the original image, the second column is the true value image, the third column is the detection result of the existing unsupervised defect detection method AE-SSIM which shows the best performance, and the fourth column is the semi-supervised mode detection result image.
The texture surface defect detection algorithm based on the abnormal synthesis and decomposition network provided by the invention utilizes the provided segmentation-guided defect generation network and a small amount of real defect training samples to generate a large amount of defect samples similar to real defects; the abnormal synthesis method based on Gaussian sampling can randomly synthesize abnormal negative samples by using defect-free positive samples; finally, the proposed abnormal decomposition network decomposes the abnormal negative sample into a texture background image and an abnormal mask image. When the detection is carried out, only an abnormal decomposition network is needed, and the defect image is decomposed into a texture background image and an abnormal image; and fusing a residual image between the defect image and the texture background image with the abnormal image, so that the defect can be accurately detected.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (7)
1. A texture surface defect detection method based on abnormal synthesis and decomposition is characterized by comprising the following steps:
s1, constructing a defect generation network for segmentation guidance; the defect generation network comprises a generator and a discriminator of segmentation guide; the generator is used for synthesizing the first type negative sample image IcnConversion into a first type abnormal negative sample image I more similar to a real defect imagen(ii) a The first type of synthetic negative sample image IcnFrom defect-free textured background IpAnd a defect image IdDefective label image I oflBy regional superposition synthesis; the partition-guided discriminator is used for respectively discriminating abnormal negative sample images InWith the real defect image IdCarrying out defect segmentation;
s2, aiming at defect segmentation, training a defect generation network by using a counterstudy method;
s3, converting the second type of synthetic negative sample image I 'by using a trained generator'cnObtaining a second abnormal negative sample image I'n(ii) a The second type of synthetic negative sample image is composed of a defect-free texture background IpAbnormal mask image I generated by random samplingaBy regional superposition synthesis;
s4, constructing an abnormal decomposition network and using a defect-free texture background IpAnd a second-type abnormal negative sample image I'nTraining an abnormal decomposition network as a training set; the abnormal decomposition network is used for decomposing the defect image into a texture background image and an abnormal image;
and S5, inputting the texture surface image to be detected into the trained abnormal decomposition network to obtain a corresponding texture background image and an abnormal image, and fusing a residual image between the defect image and the texture background image with the abnormal image to obtain a defect detection result and obtain a defect detection result.
2. The texture surface defect detection method based on abnormal synthesis and decomposition as claimed in claim 1, wherein the first type synthesized negative sample image IcnFrom defect-free textured background IpAnd defective label image IlPerforming regional superposition synthesis by using the following expression;
Icn(x,y)=(1-Il(x,y))·Ip(x,y)+λ·Il(x,y)
where λ represents a random variable, x 1., W, y 1., H, W, H represent the image width and height, respectively.
3. The texture surface defect detection method based on abnormal synthesis and decomposition as claimed in claim 1 or 2, characterized in that the defect generation network is trained by using the following loss function:
4. The texture surface defect detection method based on abnormal synthesis and decomposition as claimed in any one of claims 1-3, wherein the random sampling comprises Gaussian sampling, student's t-distribution sampling, and uniform sampling.
5. The texture surface defect detection method based on abnormal synthesis and decomposition as claimed in any one of claims 1-4, wherein the abnormal decomposition network training process is as follows:
defect-free textured background I using an encoderpCoded as corresponding implicit spatial features Zp(ii) a Using a feature embedding encoder to embed a spatial feature ZpEncoding into corresponding texture featuresDecoding texture features with a decoderObtaining a defect-free textured background IpIs reconstructed image of
Utilizing an encoder to obtain a second type abnormal negative sample image I'nCoded as corresponding implicit spatial features ZnAnd abnormal feature Za(ii) a Using a feature embedding encoder to embed a spatial feature ZnEncoding into corresponding texture featuresMake itAndsharing a feature distribution space; decoding texture features with a decoderCorrespondingly obtaining a second abnormal negative sample image I'nReconstructed textured background imageDecoding of exception features Z with exception decoderaObtaining an abnormal foreground image
Respectively processing the second type abnormal negative sample image I 'by a discriminator of segmentation guidance'nWith reconstructed textured background imageCarrying out defect segmentation;
an encoder, a feature embedding encoder, a decoder and a discriminator are trained using a counterlearning method.
6. The texture surface defect detection method based on abnormal synthesis and decomposition as claimed in claim 5, characterized in that the abnormal decomposition network is trained by using the following loss function:
Lrrepresenting reconstruction losses for training the encoder, the feature embedding encoder and the decoder; l iszRepresenting implicit spatial feature constraint loss; for training the encoder and the feature embedding encoder; l isaRepresenting an anomalous decoding loss for training the encoder and the anomalous decoder; l isadRepresents the discrimination loss of the discriminator, LagDiscriminator generation penalty for training an encoder, a feature embedding encoder and decoder, lambda1,λ2,λ3,λ4The weights of the four losses are respectively.
7. A texture surface defect detection system based on abnormal synthesis and decomposition, comprising: a computer-readable storage medium and a processor;
the computer-readable storage medium is used for storing executable instructions;
the processor is used for reading executable instructions stored in the computer-readable storage medium and executing the texture surface defect detection method based on abnormal synthesis and decomposition of any one of claims 1 to 6.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110033380.9A CN112700432B (en) | 2021-01-12 | 2021-01-12 | Texture surface defect detection method and system based on abnormal synthesis and decomposition |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110033380.9A CN112700432B (en) | 2021-01-12 | 2021-01-12 | Texture surface defect detection method and system based on abnormal synthesis and decomposition |
Publications (2)
Publication Number | Publication Date |
---|---|
CN112700432A true CN112700432A (en) | 2021-04-23 |
CN112700432B CN112700432B (en) | 2022-07-05 |
Family
ID=75513933
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202110033380.9A Active CN112700432B (en) | 2021-01-12 | 2021-01-12 | Texture surface defect detection method and system based on abnormal synthesis and decomposition |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN112700432B (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113658167A (en) * | 2021-08-24 | 2021-11-16 | 凌云光技术股份有限公司 | Defect-containing training image generation method and device |
CN113947567A (en) * | 2021-09-23 | 2022-01-18 | 西安电子科技大学广州研究院 | Defect detection method based on multi-task learning |
CN115409694A (en) * | 2022-11-03 | 2022-11-29 | 季华实验室 | Semantic-guided defect image generation method, device, equipment and storage medium |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5737072A (en) * | 1991-08-22 | 1998-04-07 | Kla Instruments Corporation | Automated photomask inspection apparatus and method |
US20050104020A1 (en) * | 2003-11-19 | 2005-05-19 | Honeywell International Inc. | Apparatus and method for identifying possible defect indicators for a valve |
CN106023158A (en) * | 2016-05-10 | 2016-10-12 | 浙江科技学院 | SD-OCT-image-based nacre layer defect identification method for fresh water non-nucleated pearl |
CN107563355A (en) * | 2017-09-28 | 2018-01-09 | 哈尔滨工程大学 | Hyperspectral abnormity detection method based on generation confrontation network |
CN110969606A (en) * | 2019-11-29 | 2020-04-07 | 华中科技大学 | Texture surface defect detection method and system |
CN112164033A (en) * | 2020-09-14 | 2021-01-01 | 华中科技大学 | Abnormal feature editing-based method for detecting surface defects of counternetwork texture |
-
2021
- 2021-01-12 CN CN202110033380.9A patent/CN112700432B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5737072A (en) * | 1991-08-22 | 1998-04-07 | Kla Instruments Corporation | Automated photomask inspection apparatus and method |
US20050104020A1 (en) * | 2003-11-19 | 2005-05-19 | Honeywell International Inc. | Apparatus and method for identifying possible defect indicators for a valve |
CN106023158A (en) * | 2016-05-10 | 2016-10-12 | 浙江科技学院 | SD-OCT-image-based nacre layer defect identification method for fresh water non-nucleated pearl |
CN107563355A (en) * | 2017-09-28 | 2018-01-09 | 哈尔滨工程大学 | Hyperspectral abnormity detection method based on generation confrontation network |
CN110969606A (en) * | 2019-11-29 | 2020-04-07 | 华中科技大学 | Texture surface defect detection method and system |
CN112164033A (en) * | 2020-09-14 | 2021-01-01 | 华中科技大学 | Abnormal feature editing-based method for detecting surface defects of counternetwork texture |
Non-Patent Citations (1)
Title |
---|
HUA YANG ET AL.: "An Anomaly Feature-Editing-Based Adversarial Network for Texture Defect Visual Inspection", 《IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS》 * |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113658167A (en) * | 2021-08-24 | 2021-11-16 | 凌云光技术股份有限公司 | Defect-containing training image generation method and device |
CN113658167B (en) * | 2021-08-24 | 2024-03-26 | 凌云光技术股份有限公司 | Training image generation method and device containing defects |
CN113947567A (en) * | 2021-09-23 | 2022-01-18 | 西安电子科技大学广州研究院 | Defect detection method based on multi-task learning |
CN113947567B (en) * | 2021-09-23 | 2024-03-22 | 西安电子科技大学广州研究院 | Defect detection method based on multitask learning |
CN115409694A (en) * | 2022-11-03 | 2022-11-29 | 季华实验室 | Semantic-guided defect image generation method, device, equipment and storage medium |
CN115409694B (en) * | 2022-11-03 | 2023-01-13 | 季华实验室 | Semantic-guided defect image generation method, device, equipment and storage medium |
Also Published As
Publication number | Publication date |
---|---|
CN112700432B (en) | 2022-07-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN112700432B (en) | Texture surface defect detection method and system based on abnormal synthesis and decomposition | |
Ulyanov et al. | It takes (only) two: Adversarial generator-encoder networks | |
CN108648197B (en) | Target candidate region extraction method based on image background mask | |
CN112070727B (en) | Metal surface defect detection method based on machine learning | |
CN114677346B (en) | Method for detecting end-to-end semi-supervised image surface defects based on memory information | |
CN109886970A (en) | The detection dividing method and computer storage medium of target object in terahertz image | |
CN112818764B (en) | Low-resolution image facial expression recognition method based on feature reconstruction model | |
CN110969606B (en) | Texture surface defect detection method and system | |
CN111861924B (en) | Cardiac magnetic resonance image data enhancement method based on evolutionary GAN | |
CN114463759A (en) | Lightweight character detection method and device based on anchor-frame-free algorithm | |
CN117409192B (en) | Data enhancement-based infrared small target detection method and device | |
CN110930378A (en) | Emphysema image processing method and system based on low data demand | |
CN114565594A (en) | Image anomaly detection method based on soft mask contrast loss | |
CN117079098A (en) | Space small target detection method based on position coding | |
CN116883933A (en) | Security inspection contraband detection method based on multi-scale attention and data enhancement | |
CN115731400A (en) | X-ray image foreign matter detection method based on self-supervision learning | |
CN117593243A (en) | Compressor appearance self-adaptive detection method guided by reliable pseudo tag | |
CN116740051A (en) | Steel surface defect detection method based on improved YOLO model | |
CN117095199A (en) | Industrial visual anomaly detection system based on simplex diffusion model | |
Saito et al. | Image processing method for automatic measurement of number of DNA breaks | |
CN116452408A (en) | Transparent liquid sensing method based on style migration | |
CN114666571A (en) | Video sensitive content detection method and system | |
CN111429436B (en) | Intrinsic image analysis method based on multi-scale attention and label loss | |
Rippel et al. | Anomaly detection for the automated visual inspection of pet preform closures | |
Cho et al. | Synthesizing industrial defect images under data imbalance |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |