CN112164033A - Abnormal feature editing-based method for detecting surface defects of counternetwork texture - Google Patents
Abnormal feature editing-based method for detecting surface defects of counternetwork texture Download PDFInfo
- Publication number
- CN112164033A CN112164033A CN202010962259.XA CN202010962259A CN112164033A CN 112164033 A CN112164033 A CN 112164033A CN 202010962259 A CN202010962259 A CN 202010962259A CN 112164033 A CN112164033 A CN 112164033A
- Authority
- CN
- China
- Prior art keywords
- image
- texture
- abnormal
- feature
- features
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 82
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 75
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000001514 detection method Methods 0.000 claims abstract description 36
- 238000012549 training Methods 0.000 claims abstract description 13
- 238000005457 optimization Methods 0.000 claims abstract description 7
- 238000012545 processing Methods 0.000 claims abstract description 6
- 230000003042 antagnostic effect Effects 0.000 claims abstract 4
- 238000004364 calculation method Methods 0.000 claims description 14
- 238000009499 grossing Methods 0.000 claims description 6
- 230000003321 amplification Effects 0.000 claims description 4
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 3
- 230000035945 sensitivity Effects 0.000 claims description 3
- 230000000877 morphologic effect Effects 0.000 claims description 2
- 230000002950 deficient Effects 0.000 abstract description 2
- 238000000605 extraction Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 6
- 238000009826 distribution Methods 0.000 description 6
- 230000005856 abnormality Effects 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 230000001788 irregular Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 230000002547 anomalous effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000002401 inhibitory effect Effects 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 239000002994 raw material Substances 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 239000004753 textile Substances 0.000 description 1
- 239000002023 wood Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/0008—Industrial image inspection checking presence/absence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/088—Non-supervised learning, e.g. competitive learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/40—Analysis of texture
- G06T7/41—Analysis of texture based on statistical description of texture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/40—Analysis of texture
- G06T7/41—Analysis of texture based on statistical description of texture
- G06T7/44—Analysis of texture based on statistical description of texture using image operators, e.g. filters, edge density metrics or local histograms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20032—Median filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Probability & Statistics with Applications (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of image processing, and particularly discloses a method for detecting surface defects of an anti-network texture based on abnormal feature editing, which comprises the following steps: acquiring a non-defective good product image and a corresponding defect image to jointly form an image data set; constructing an antagonistic network, wherein the antagonistic network comprises a generator and a discriminator, the generator is used for extracting image characteristics, detecting abnormal characteristics, and editing the abnormal characteristics by adopting normal characteristics to obtain a reconstructed image; the discriminator is used for discriminating the good product image and the reconstructed image; training the countermeasure network through an image data set according to a pre-constructed optimization target to obtain a reconstructed image generation model; and inputting the image to be detected into the reconstructed image generation model to obtain a corresponding reconstructed image, and further obtaining the texture surface defects according to the image to be detected and the corresponding reconstructed image. The invention has higher detection precision for defects with different shapes, sizes and contrasts on different texture surfaces.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a network texture resisting surface defect detection method based on abnormal feature editing.
Background
In the industrial manufacturing field, due to factors such as the quality of raw materials and complicated manufacturing processes, texture defects may be generated on the surface of products, such as mobile phone screens, wood, textiles, and tiles. Texture defects refer to local areas of irregular brightness variation or texture disruption. These texture defects directly affect the user experience, and in order to control the product quality, all types of texture defects should be effectively controlled in the manufacturing process, so defect detection is the basis and key for promoting the whole industrial manufacturing industry.
The image recognition technology only needs to acquire texture images through a CCD camera, then positions the texture images to the positions needing to be interested by utilizing an image processing algorithm, and detects defects accurately. Therefore, the defect detection method based on the image technology can meet the requirement of industrial automation.
At present, the detection of the texture surface defects has the following three difficulties: the texture surface defect detection is still a very challenging problem because the texture types are various, that is, various regular and irregular textures exist, the defects are various, that is, various defects may appear on the same texture surface, and the defect samples in the industry are few and difficult to collect and label. Some existing research results can only partially meet the requirements of precision and robustness of texture surface defect detection. Therefore, it is necessary to provide an unsupervised texture defect detection algorithm, which is suitable for detecting multiple defects on multiple texture surfaces.
Disclosure of Invention
Aiming at the defects or improvement requirements in the prior art, the invention provides an anti-network texture surface defect detection method based on abnormal feature editing, and aims to adopt a strategy of paired input, add abnormality in a training stage, and detect and edit the utilized abnormality, so that the reconstruction of the defects is inhibited, the detection rate of the defects is improved, and meanwhile, the precision of texture detail reconstruction is further improved by adopting a pixel-level anti-learning mode.
In order to achieve the above object, the present invention provides a method for detecting surface defects of an anti-network texture based on abnormal feature editing, which comprises the following steps:
s1, acquiring a defect-free good product image and a defect image corresponding to the good product image, wherein the defect image and the good product image jointly form an image data set;
s2, constructing a countermeasure network edited based on abnormal features, wherein the countermeasure network comprises a generator and a discriminator, the generator is used for extracting image features of a good product image and a defect image, detecting the abnormal features in the good product image and the defect image, editing the abnormal features by adopting normal features, and reconstructing the abnormal features according to the processed features to generate a reconstructed image; the discriminator is used for discriminating the good product image and the reconstructed image on each pixel;
s3, training the confrontation network edited based on abnormal features through an image data set according to a pre-constructed optimization target, and performing confrontation learning by a generator and a discriminator during training so as to obtain a reconstructed image generation model;
s4, inputting the image to be detected into the reconstructed image generation model to obtain a corresponding reconstructed image, and further obtaining texture surface defects according to the image to be detected and the corresponding reconstructed image to complete the texture surface defect detection of the image to be detected.
More preferably, in S2, the image features are extracted by 5 convolutional layers, where the step size of the first convolutional layer is 1, the number of channels is 16, the step sizes of the following three convolutional layers are 2, the number of channels is increased by a multiple of 2, and the step size of the fifth convolutional layer is 1.
Preferably, in S2, the method for detecting abnormal features by using a clustering method based on center constraint specifically includes the following steps:
(1) with C ═ C1,c2,…,cKDenotes a cluster center, calculates a cluster center ckAnd image feature fiResidual error R betweenikFurther obtaining the distance of each image feature to the clustering center and passing through the minimum distance diAnd obtaining a cluster to which the image features belong, wherein the specific calculation formula is as follows:
Rik=fi-ck
wherein i belongs to 1,2, …, N; k is 1,2, …, K; n represents the number of image features, and K represents the number of clustering centers;
(2) calculating the center boundary T of each clustercAnd will be spaced from the cluster center by more than TcAs an abnormal feature, the central boundary TcIs calculated as follows:
wherein σdIs the standard deviation of the distance of the image features from the cluster center.
Preferably, in S2, when editing the abnormal features, the abnormal features are convolved with the normal background features to obtain matching scores, the matching scores are normalized into a score map through softmax operation, corresponding normal features are obtained by combining the score map and the background features, and finally the abnormal features are replaced with the normal features to complete the editing of the abnormal features.
As a further preferred, the calculation formula of the optimization objective, i.e. the trained joint loss L, is as follows:
L=λ1Lrec+λ2Lclu+λ3Ladv
wherein L isrecFor texture reconstruction loss, LcluTo cluster the losses, LadvDetermining a loss for the pixel; lambda [ alpha ]1、λ2And λ3The weights are texture reconstruction loss, clustering loss, and pixel discrimination loss, respectively.
As a further preferred, the clustering penalty Lclu=Lc+LklWherein:
Lcfor the central constraint, the calculation formula is as follows:
Lklto aid in scoring, specifically based on the residual R between the cluster center and the image featureikCalculating a distance score SikFurther, a target score S 'is obtained'ikThen using KL divergence to obtain an auxiliary score LklThe specific calculation formula is as follows:
further preferably, the texture reconstruction lossLose Lrec=Lrec_o+Lrec_d+Lrec_mThe specific calculation formula is as follows:
wherein: i isoIs a good product picture IobIs a reconstructed image corresponding to the good image IdbFor reconstructed images corresponding to defect maps, ImIs a mask image; beta is the amplification factor, and beta is the amplification factor,w and H are the width and height of the image respectively.
More preferably, the pixel discrimination loss L isadvIs calculated as follows:
wherein, IoIs a good product picture IobIs a reconstructed image corresponding to the good image IdbA reconstructed image corresponding to the defect map; w and H are the width and height of the image respectively.
Preferably, in S4, the method for obtaining the texture surface defect according to the image to be detected and the reconstructed image includes the following steps:
(1) the image to be detected and the corresponding reconstructed image are differenced to obtain a residual error image Ires:
Ires=|Id-Idb|
Wherein, IdFor the image to be detected, IdbFor reconstitutionAn image;
(2) performing median filtering on the residual image, and then performing double-threshold processing and morphological closing operation on the residual image to obtain a binary image I containing texture surface defectsbin(i,j):
Wherein i belongs to (1, …, W), j belongs to (1, …, H), W and H are the width and height of the image respectively; t is1、T2Are two thresholds from the residual map.
Further preferably, the threshold value T is set to be lower than the threshold value T1、T2Is calculated as follows:
where μ and σ are the mean value and standard deviation of the residual map, respectively, and are the division sensitivity control coefficients.
Generally, compared with the prior art, the above technical solution conceived by the present invention mainly has the following technical advantages:
1. the invention provides an unsupervised defect detection method, which provides abnormity for model training by adopting artificially manufactured defect graphs and corresponding good product graphs as paired input in an off-line training stage, and is beneficial to the model to process abnormal characteristics in a testing stage; during online detection, only the images are input into a trained network model, corresponding texture background images are reconstructed, and the two images are subjected to subtraction and post-processing, so that defects can be detected; the method solves the problem of low detection precision caused by different types, large-scale change, low contrast, irregular brightness change, variable shape, lack of samples and the like in the texture defect detection algorithm.
2. The method for detecting the defects of the network texture surface based on abnormal characteristic editing is not limited by texture types and defect types, has higher detection precision on the defects with different sizes and different contrasts on different texture surfaces, and is suitable for detecting various defects on various texture surfaces.
3. The method adopts a clustering method based on central constraint to detect the abnormal features of the hidden space, enables the extracted features to be more discriminative while detecting the abnormal features, edits the detected abnormal features to inhibit the reconstruction of the defects and improve the detection rate of the defects.
4. The method utilizes a plurality of convolutional layers to extract the characteristics of the image, and extracts the characteristics of deeper images by increasing the number of channels of the convolutional layers and setting the step length of convolution operation; in addition, by adopting pixel level discrimination, the reconstruction precision of texture background details can be further improved, and the over-detection rate is reduced while the defect detection rate is improved.
5. In order to accurately reconstruct the texture background image, the texture reconstruction loss, the clustering loss and the pixel discrimination loss are jointly adopted, and the whole model is optimized through a multi-task loss function.
Drawings
FIG. 1 is a diagram of an anomaly feature editing-based countermeasure network model according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of an abnormal feature detection module according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an abnormal feature editing module according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a pixel-level decision module according to an embodiment of the invention;
FIG. 5 is a schematic diagram of a defect detection stage according to an embodiment of the present invention;
fig. 6 is a schematic diagram illustrating the effect of detecting surface defects of different textures in an embodiment of the present invention, where (a), (c), (e), (g), (i), and (k) are images to be detected, and (b), (d), (f), (h), (j), and (l) are corresponding detection results.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
The embodiment of the invention provides a method for detecting the surface defects of counternetwork textures based on abnormal feature editing, which comprises an off-line training stage and an on-line detection stage:
the offline training phase comprises:
s1, artificially generating defects:
on a defect-free good product chart IoGenerating a corresponding artificial defect map IdWherein, in the step (A),W,H,C0respectively representing the width, height and channel number of an input image; the defect map and the good map together form an image data set;
and S2, constructing the countermeasure network based on abnormal feature editing, wherein the countermeasure network comprises a generator and a discriminator as shown in figure 1. The generator comprises a feature extraction module FEM, an abnormal feature detection module AFDM, an abnormal feature editing module GCFEM and a feature decoding module FDM, wherein the feature extraction module is used for extracting the hidden spatial features of the input texture image; the abnormal feature detection module detects abnormal features by adopting a clustering method based on central constraint; the abnormal characteristic editing module uses the cosine similarity of the abnormal characteristic and the normal characteristic to carry out weighting construction to construct a normal characteristic to replace the abnormal characteristic, thereby inhibiting the reconstruction of the defect; the feature decoding module is used for reconstructing the texture background and reconstructing the hidden space features into the texture picture by using a plurality of convolution layers symmetrical to the feature extraction module. The discriminator comprises a pixel-level discrimination module PDM for discriminating the good images and the reconstructed images on each pixel, and the texture background reconstruction precision is further improved. Specifically, the method comprises the following steps:
s21, a feature extraction module:
obtaining the input hidden space characteristics by using 5 convolution layers, wherein the convolution kernels of all the layers except the last layer have the sizeAre all 3 × 3; the step size and the number of channels of the first convolutional layer are 1 and C, respectively1(in the present embodiment, C116), the step size of the following three convolutional layers is all 2, the number of channels is incremented by a multiple of 2, and finally, the convolutional kernel size of the fifth layer is 1 × 1, the step size is 1. Therefore, the feature extraction module will IoAnd IdAs input and to propose a hidden spatial feature F, whereinC is set to 10.
S22, an abnormal feature detection module:
the hidden spatial feature F extracted by the feature extraction module can be regarded as a set of local features, that is, F ═ F1,f2,…,fNAnd is andthe purpose of the abnormal feature editing module is to use K centers and an adaptive threshold value TcTo learn the distribution of the hidden spatial features, then those distances exceed TcIs considered to be a characteristic of the anomaly. Notably, K centers and an adaptive threshold TcAre learnable parameters whose values are affected only by non-defective good maps and are updated as clustering is lost. To achieve this goal, different types of local features in the texture image should be separated and clustered into K clusters. However, the original implicit spatial distribution of the texture image is cluttered, which is very difficult for clustering.
The invention adopts a clustering method based on center constraint to detect abnormal features, and specifically comprises the following steps as shown in figure 2: with C ═ C1,c2,…,cKDenotes the cluster center, where c1∈RC×1. Assuming that all normal hidden spatial features are distributed around the corresponding center, the abnormal features mapped from the defect are far from the center, cluster center ckAnd image feature fiResidual error R betweenikThe calculation is as follows:
Rik=fi-ck
wherein i belongs to 1,2, …, N; k is 1,2, …, K; n represents the number of image features. Then using the distance score SikTo measure each implicit spatial feature fiWhether the degree of belonging to the cluster center is normal or not, that is, the closer the normal feature is to the corresponding center, the farther the abnormal feature is from the center, and the calculation formula is as follows:
wherein α ═ { α ═ α1,α2,…,αKIs a learned smoothing factor, it is noted that the smoothing factor controls the distance, updating the parameters due to subsequent clustering losses. Each normal feature fiA distance d to C, and a minimum distance diIt is indicated to which cluster the feature belongs:
center boundary T for each clustercCalculated by the following equation:
wherein σdIs the standard deviation of the distances of the image features to the cluster centers, so those distances exceed TcIs considered to be an anomalous feature.
S23, an abnormal characteristic editing module:
for detected abnormal features, how to edit them to suppress reconstruction of defects is crucial for texture surface defect detection. The anomaly feature editing module replaces the feature that has been replaced with the ordered global semantic information from the normal background feature. As shown in fig. 3, the normal background features are sorted according to their similarity to the abnormal features to be edited, and then combined into a new feature to replace the abnormal features. Each detected abnormal feature can be regarded as a convolution kernel of 1 multiplied by 1, a matching score is obtained by convolution of the abnormal feature and a normal background feature, then the matching score is normalized into a score map by softmax operation, then the corresponding normal feature is obtained by combining the score map and the background feature, and finally the obtained normal feature is used for replacing the corresponding abnormal feature so as to achieve the purpose of editing the abnormal feature.
S24, a feature decoding module:
the feature decoding module reconstructs the hidden spatial features F into texture pictures by using a plurality of convolution layers which are symmetrical to the feature extraction module. Similarly, the convolution kernels of all layers are 3 × 3, except for the first layer, which has a convolution kernel size of 1 × 1. Similar to the U-net structure, the feature decoding module directly reconstructs the texture background of the features of each layer in the coding module through jumping connection.
As shown in FIG. 1, the model transforms a texture image IoOr IdAs input, further output the corresponding texture background image IobOr IdbWhereinThe FEM extracts the input convolution features as texture expressions, and then the FDM reconstructs these texture expressions out of the texture background, which can be expressed as follows:
wherein the content of the first and second substances,representing a function incorporating a feature extraction module, a feature decoding module, an abnormal feature detection module and a feature editing module, thetaTRepresenting the parameters of these modules.
S25, a pixel level judging module:
as shown in fig. 4, it is composed of 4 convolutional layers and 4 transposed convolutional layers, the convolution kernel of each convolutional layer is 3 × 3, the step size is 2, the number of filter kernels of the first layer is 32, the following convolution kernels sequentially increase by a multiple of 2, the convolution kernel of each transposed convolutional layer is 3 × 3, the step size is 2, the number of filter kernels of the first layer is 256, the following convolution kernels sequentially decrease by a multiple of 2, and the number of filter kernels of the last layer is 1.
When judging, the real good product is mapped IoReconstructed image Idb、IobThese three types of samples are used as input to discriminate between true and false at each pixel of the image, where IoIs a true sample, IdbAnd IobIs a false sample whose output is a discriminant score that predicts the likelihood that each pixel in the image is from the original image distribution, thereby achieving pixel-level discrimination.
And S3, training the confrontation network edited based on the abnormal features through the image data set according to the pre-constructed optimization target, and performing confrontation learning by the generator and the discriminator during training to obtain a reconstructed image generation model.
Specifically, for accurately reconstructing the texture background map, the calculation formula of the optimization target, i.e. the trained joint loss L, is as follows:
L=λ1Lrec+λ2Lclu+λ3Ladv
wherein L isrecFor texture reconstruction loss, LcluTo cluster the losses, LadvDetermining a loss for the pixel; lambda [ alpha ]1、λ2And λ3Three lost weights, respectively; more specifically:
(1) clustering loss Lclu=Lc+Lkl:
In the clustering process of step S22, the mainstream depth clustering method neglects to reduce the intra-class gap and increase the inter-class gap, which results in a less than ideal clustering effect. To solve this problem and accurately detect the abnormalityThe constant characteristic makes the original distribution in FIG. 2 approach the target distribution, and proposes an auxiliary score LklAnd a central constraint Lc。
The purpose of this auxiliary score is to make the distance d more compact, thereby increasing the discriminability of the hidden spatial features, so by calculating SikAnd normalized on each cluster center to get a target score S'ik:
Wherein the content of the first and second substances,the Kullback-Leibler (KL) divergence is then used to minimize the distance score and the auxiliary target score:
to further reduce intra-class similarity and increase inter-class similarity, the following central constraints are proposed:
central constraint LcThe hidden space features extracted by the feature extraction module can be more representative by enabling different centers to be more dispersed and similar features to be more compact, and meanwhile, the detection precision of the abnormal features is also improved.
(2) Texture reconstruction loss Lrec=Lrec_o+Lrec_d+Lrec_m:
For texture reconstruction, there are three penalties: l isrec_o、Lrec_dAnd Lrec_mThe purpose of these penalty functions is to minimize the gap between the input image and the reconstructed texture background image by mean square error:
wherein: II-2Is represented by L2A norm; i ismFor the mask image, β is the magnification factor,sum (-) is the summation operation.
(3) The pixel discrimination loss LadvIn the determination process performed in step S24, the calculation formula is as follows:
the online detection phase comprises:
s4, as shown in FIG. 5, the image I to be detecteddInputting the reconstructed image into a reconstructed image generation model, extracting hidden space features, detecting abnormal features, editing abnormal features, and decoding the edited features to obtain a corresponding reconstructed image IdbAnd further obtaining the texture surface defects according to the image to be detected and the corresponding reconstructed image.
Specifically, firstly, the image I to be detected is detecteddWith corresponding reconstructed image IdbMaking difference to obtain residual error image Ires:
Ires=|Id-Idb|
Then, carrying out median filtering operation on the residual image to remove noise therein, and then carrying out double-threshold processing and morphology closing operation on the residual image to obtain a binary image I of a final detection resultbin:
Wherein i ∈ (1, …, W), j ∈ (1, …, H), and T ∈1And T2Is from IresCan be regarded as a boundary of the residual map, the defect is outside this boundary. Assuming that the residuals follow a gaussian distribution, the threshold is calculated as follows:
where μ and σ are the mean value and standard deviation from the residual map, respectively, and are the division sensitivity control coefficients. The image to be detected and the detected binary image are shown in fig. 6.
According to the method for detecting the surface defects of the counternetwork texture based on the abnormal feature editing, which is provided by the invention, the strategy of paired input is adopted, the abnormality is added in the training stage, and the abnormality is processed by using the abnormal feature detection module and the abnormal feature editing module, so that the reconstruction of the defects is inhibited, and the detection rate of the defects is improved; the pixel-level confrontation learning mode is carried out by adopting the pixel-level discrimination module, so that the texture background reconstruction precision can be further improved, and the overdetection rate can be reduced while the detection rate of texture defects is improved.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. A method for detecting the surface defects of counternetwork textures based on abnormal feature editing is characterized by comprising the following steps:
s1, acquiring a defect-free good product image and a defect image corresponding to the good product image, wherein the defect image and the good product image jointly form an image data set;
s2, constructing a countermeasure network edited based on abnormal features, wherein the countermeasure network comprises a generator and a discriminator, the generator is used for extracting image features of a good product image and a defect image, detecting the abnormal features in the good product image and the defect image, editing the abnormal features by adopting normal features, and reconstructing the abnormal features according to the processed features to generate a reconstructed image; the discriminator is used for discriminating the good product image and the reconstructed image on each pixel;
s3, training the confrontation network edited based on abnormal features through an image data set according to a pre-constructed optimization target, and performing confrontation learning by a generator and a discriminator during training so as to obtain a reconstructed image generation model;
s4, inputting the image to be detected into the reconstructed image generation model to obtain a corresponding reconstructed image, and further obtaining texture surface defects according to the image to be detected and the corresponding reconstructed image to complete the texture surface defect detection of the image to be detected.
2. The method for detecting surface defects of anti-net texture based on abnormal feature editing as claimed in claim 1, wherein in S2, the image features are extracted by 5 convolutional layers, wherein the step size of the first convolutional layer is 1, the number of channels is 16, the step size of the following three convolutional layers is 2, the number of channels is increased by a multiple of 2, and the step size of the fifth convolutional layer is 1.
3. The method for detecting surface defects of antagonistic network textures based on abnormal feature editing as claimed in claim 1, wherein in S2, a clustering method based on central constraint is adopted for abnormal feature detection, specifically comprising the following steps:
(1) with C ═ C1,c2,…,cKDenotes a cluster center, calculates a cluster center ckAnd image feature fiResidual error R betweenikFurther obtaining the distance of each image feature to the clustering center and passing through the minimum distance diAnd obtaining a cluster to which the image features belong, wherein the specific calculation formula is as follows:
Rik=fi-ck
wherein i belongs to 1,2, …, N; k is 1,2, …, K; n represents the number of image features, and K represents the number of clustering centers;
(2) calculating the center boundary T of each clustercAnd will be spaced from the cluster center by more than TcAs an abnormal feature, the central boundary TcIs calculated as follows:
wherein σdIs the standard deviation of the distance of the image features from the cluster center.
4. The method for detecting surface defects of antagonistic network textures based on abnormal feature editing as claimed in claim 1, wherein in S2, when abnormal feature editing is performed, the abnormal feature is convolved with the normal background feature to obtain a matching score, the matching score is normalized into a score map by softmax operation, a corresponding normal feature is obtained by combining the score map and the background feature, and finally the abnormal feature is replaced by the normal feature to complete the editing of the abnormal feature.
5. The method for detecting the surface defects of the counternetwork textures based on the abnormal feature editing as claimed in claim 3, wherein the optimization objective, namely the calculation formula of the trained joint loss L, is as follows:
L=λ1Lrec+λ2Lclu+λ3Ladv
wherein L isrecFor texture reconstruction loss, LcluTo cluster the losses, LadvDetermining a loss for the pixel; lambda [ alpha ]1、λ2And λ3The weights are texture reconstruction loss, clustering loss, and pixel discrimination loss, respectively.
6. The method of claim 5, wherein the cluster loss L is a loss of similarity between the net texture surface defect and the cluster loss Lclu=Lc+LklWherein:
Lcfor the central constraint, the calculation formula is as follows:
Lklto aid in scoring, specifically based on the residual R between the cluster center and the image featureikCalculating a distance score SikFurther, a target score S 'is obtained'ikThen using KL divergence to obtain an auxiliary score LklThe specific calculation formula is as follows:
7. the method of claim 5, wherein the anomaly-based editing for detecting the surface defects of the network texture is performed by a computerThe texture reconstruction loss Lrec=Lrec_o+Lrec_d+Lrec_mThe specific calculation formula is as follows:
wherein: i isoIs a good product picture IobIs a reconstructed image corresponding to the good image IdbFor reconstructed images corresponding to defect maps, ImIs a mask image; beta is the amplification factor, and beta is the amplification factor,w and H are the width and height of the image respectively.
8. The method of claim 5, wherein the pixel discrimination loss L is a penalty L for detecting the surface defects of the network texture based on abnormal feature editingadvIs calculated as follows:
wherein, IoIs a good product picture IobIs a reconstructed image corresponding to the good image IdbA reconstructed image corresponding to the defect map; w and H are the width and height of the image respectively.
9. The method for detecting surface defects of counternetwork textures based on abnormal feature editing as claimed in claim 1, wherein in S4, texture surface defects are obtained according to the image to be detected and the reconstructed image, and the method specifically comprises the following steps:
(1) the image to be detected and the corresponding reconstructed image are differenced to obtain a residual error image Ires:
Ires=|Id-Idb|
Wherein, IdFor the image to be detected, IdbTo reconstruct an image;
(2) performing median filtering on the residual image, and then performing double-threshold processing and morphological closing operation on the residual image to obtain a binary image I containing texture surface defectsbin(i,j):
Wherein i belongs to (1, …, W), j belongs to (1, …, H), W and H are the width and height of the image respectively; t is1、T2Are two thresholds from the residual map.
10. The method of claim 9, wherein the threshold T is set as a threshold T for detecting surface defects of the texture of the resist web based on abnormal feature editing1、T2Is calculated as follows:
where μ and σ are the mean value and standard deviation of the residual map, respectively, and are the division sensitivity control coefficients.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010962259.XA CN112164033A (en) | 2020-09-14 | 2020-09-14 | Abnormal feature editing-based method for detecting surface defects of counternetwork texture |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010962259.XA CN112164033A (en) | 2020-09-14 | 2020-09-14 | Abnormal feature editing-based method for detecting surface defects of counternetwork texture |
Publications (1)
Publication Number | Publication Date |
---|---|
CN112164033A true CN112164033A (en) | 2021-01-01 |
Family
ID=73857448
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010962259.XA Pending CN112164033A (en) | 2020-09-14 | 2020-09-14 | Abnormal feature editing-based method for detecting surface defects of counternetwork texture |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN112164033A (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112700432A (en) * | 2021-01-12 | 2021-04-23 | 华中科技大学 | Texture surface defect detection method and system based on abnormal synthesis and decomposition |
CN113516650A (en) * | 2021-07-30 | 2021-10-19 | 深圳康微视觉技术有限公司 | Circuit board hole plugging defect detection method and device based on deep learning |
CN113838040A (en) * | 2021-09-29 | 2021-12-24 | 西安工程大学 | Detection method for defect area of color texture fabric |
CN113947567A (en) * | 2021-09-23 | 2022-01-18 | 西安电子科技大学广州研究院 | Defect detection method based on multi-task learning |
CN114757900A (en) * | 2022-03-31 | 2022-07-15 | 启东新朋莱纺织科技有限公司 | Artificial intelligence-based textile defect type identification method |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110969606A (en) * | 2019-11-29 | 2020-04-07 | 华中科技大学 | Texture surface defect detection method and system |
CN110992354A (en) * | 2019-12-13 | 2020-04-10 | 华中科技大学 | Abnormal region detection method for countering self-encoder based on introduction of automatic memory mechanism |
-
2020
- 2020-09-14 CN CN202010962259.XA patent/CN112164033A/en active Pending
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110969606A (en) * | 2019-11-29 | 2020-04-07 | 华中科技大学 | Texture surface defect detection method and system |
CN110992354A (en) * | 2019-12-13 | 2020-04-10 | 华中科技大学 | Abnormal region detection method for countering self-encoder based on introduction of automatic memory mechanism |
Non-Patent Citations (1)
Title |
---|
H. YANG等: ""An Anomaly Feature-Editing-Based Adversarial Network for Texture Defect Visual Inspection"", 《IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS》 * |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112700432A (en) * | 2021-01-12 | 2021-04-23 | 华中科技大学 | Texture surface defect detection method and system based on abnormal synthesis and decomposition |
CN112700432B (en) * | 2021-01-12 | 2022-07-05 | 华中科技大学 | Texture surface defect detection method and system based on abnormal synthesis and decomposition |
CN113516650A (en) * | 2021-07-30 | 2021-10-19 | 深圳康微视觉技术有限公司 | Circuit board hole plugging defect detection method and device based on deep learning |
CN113516650B (en) * | 2021-07-30 | 2023-08-25 | 深圳康微视觉技术有限公司 | Circuit board hole plugging defect detection method and device based on deep learning |
CN113947567A (en) * | 2021-09-23 | 2022-01-18 | 西安电子科技大学广州研究院 | Defect detection method based on multi-task learning |
CN113947567B (en) * | 2021-09-23 | 2024-03-22 | 西安电子科技大学广州研究院 | Defect detection method based on multitask learning |
CN113838040A (en) * | 2021-09-29 | 2021-12-24 | 西安工程大学 | Detection method for defect area of color texture fabric |
CN114757900A (en) * | 2022-03-31 | 2022-07-15 | 启东新朋莱纺织科技有限公司 | Artificial intelligence-based textile defect type identification method |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN112164033A (en) | Abnormal feature editing-based method for detecting surface defects of counternetwork texture | |
Gao et al. | A generative adversarial network based deep learning method for low-quality defect image reconstruction and recognition | |
CN108398268B (en) | Bearing performance degradation evaluation method | |
CN105224872B (en) | A kind of user's anomaly detection method based on neural network clustering | |
CN104992223A (en) | Intensive population estimation method based on deep learning | |
Markou et al. | A neural network-based novelty detector for image sequence analysis | |
CN107133943A (en) | A kind of visible detection method of stockbridge damper defects detection | |
CN109389171B (en) | Medical image classification method based on multi-granularity convolution noise reduction automatic encoder technology | |
CN103366180A (en) | Cell image segmentation method based on automatic feature learning | |
CN110287770B (en) | Water individual target matching identification method based on convolutional neural network | |
CN109815967A (en) | CNN ship seakeeping system and method based on Fusion Features | |
CN109191418B (en) | Remote sensing image change detection method based on feature learning of contraction self-encoder | |
CN110473231A (en) | A kind of method for tracking target of the twin full convolutional network with anticipation formula study more new strategy | |
CN108550163A (en) | Moving target detecting method in a kind of complex background scene | |
CN111145145B (en) | Image surface defect detection method based on MobileNet | |
CN107609588A (en) | A kind of disturbances in patients with Parkinson disease UPDRS score Forecasting Methodologies based on voice signal | |
CN108171710A (en) | A kind of image segmentation processing method | |
CN110969606A (en) | Texture surface defect detection method and system | |
CN114998602A (en) | Domain adaptive learning method and system based on low confidence sample contrast loss | |
CN114049305A (en) | Distribution line pin defect detection method based on improved ALI and fast-RCNN | |
CN113361346B (en) | Scale parameter self-adaptive face recognition method for replacing adjustment parameters | |
CN105160336B (en) | Face identification method based on Sigmoid functions | |
CN114821174B (en) | Content perception-based transmission line aerial image data cleaning method | |
CN113469977B (en) | Flaw detection device, method and storage medium based on distillation learning mechanism | |
Abazar et al. | An efficient ensemble of convolutional deep steganalysis based on clustering |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210101 |
|
RJ01 | Rejection of invention patent application after publication |