CN117876763A - Coating defect classification method and system based on self-supervision learning strategy - Google Patents
Coating defect classification method and system based on self-supervision learning strategy Download PDFInfo
- Publication number
- CN117876763A CN117876763A CN202311833505.1A CN202311833505A CN117876763A CN 117876763 A CN117876763 A CN 117876763A CN 202311833505 A CN202311833505 A CN 202311833505A CN 117876763 A CN117876763 A CN 117876763A
- Authority
- CN
- China
- Prior art keywords
- coating
- labeling
- sample data
- sample
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000011248 coating agent Substances 0.000 title claims abstract description 389
- 238000000576 coating method Methods 0.000 title claims abstract description 389
- 230000007547 defect Effects 0.000 title claims abstract description 133
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000002372 labelling Methods 0.000 claims abstract description 174
- 238000012549 training Methods 0.000 claims abstract description 93
- 238000001514 detection method Methods 0.000 claims abstract description 28
- 238000007781 pre-processing Methods 0.000 claims abstract description 10
- 239000011159 matrix material Substances 0.000 claims description 55
- 238000012545 processing Methods 0.000 claims description 53
- 230000004913 activation Effects 0.000 claims description 22
- 238000011176 pooling Methods 0.000 claims description 22
- 238000010606 normalization Methods 0.000 claims description 18
- 238000004364 calculation method Methods 0.000 claims description 15
- 230000006870 function Effects 0.000 claims description 13
- 238000005520 cutting process Methods 0.000 claims description 10
- 238000013527 convolutional neural network Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 8
- 238000013507 mapping Methods 0.000 claims description 6
- 230000003213 activating effect Effects 0.000 claims description 3
- 238000007499 fusion processing Methods 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 5
- 238000004519 manufacturing process Methods 0.000 description 5
- 230000002950 deficient Effects 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 239000006115 industrial coating Substances 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 229910015234 MoCo Inorganic materials 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Image Analysis (AREA)
Abstract
The application discloses a coating defect classification method and system based on a self-supervision learning strategy, wherein the method comprises the following steps: acquiring non-labeling coating sample data and performing data preprocessing to obtain first non-labeling coating enhancement sample data and second non-labeling coating enhancement sample data; introducing a spatial attention module, and constructing a coating defect classification self-supervision training network model; training the coating defect classification self-supervision training network model based on the first non-labeling coating enhancement sample data and the second non-labeling coating enhancement sample data to obtain a trained coating defect classification self-supervision training network model; and performing coating defect classification detection based on the trained coating defect classification self-supervision training network model to obtain a coating defect detection classification result. According to the method and the device for detecting the defect image classification, the detection defect image classification result with high accuracy can be achieved when large-scale labeling data are not available. The method and the device can be widely applied to the technical field of coating defect classification detection.
Description
Technical Field
The application relates to the technical field of coating defect classification detection, in particular to a coating defect classification method and system based on a self-supervision learning strategy.
Background
During industrial coating production, the reasons for defects are analyzed through an automatic and intelligent surface defect detection method, so that maintenance work of a production line is more targeted, and maintenance cost and overhaul cost are effectively reduced. Currently, coating defect detection based on deep learning generally adopts a supervised image classification or target detection method. When the production is started, due to the lack of a large number of NG images, a large number of marking data are not available even in the case of good production process, and it is difficult to perform defect detection, defect classification and the like by using a supervised deep learning method, and as for a target detection method, it is required to explain that self-supervised learning is very challenging for images with larger resolution, single and fixed background and variable defect size positions, such as coating images. Because the most common individual discrimination agent task in self-supervision learning can randomly cut out images, the images after random cutting out have high probability of not cutting in defective areas due to small defects and uneven distribution, so that positive samples are changed into negative samples and are treated as positive samples, ambiguity of model learning is caused, and the training process is influenced. Even if the defect area is cut out during random cutting, the defect area may only occupy a small part of the whole image, and the model is difficult to pay attention to the position with real defects, so that the problem of low defect classification detection precision exists in the target detection method.
In summary, the technical problems in the related art are to be improved.
Disclosure of Invention
The embodiment of the application mainly aims to provide a coating defect classification method and a coating defect classification system based on a self-supervision learning strategy, which can realize high-accuracy defect image detection classification results when large-scale labeling data is not available.
To achieve the above object, an aspect of an embodiment of the present application provides a coating defect classification method based on a self-supervised learning strategy, the method including:
acquiring non-labeling coating sample data and performing data preprocessing to obtain first non-labeling coating enhancement sample data and second non-labeling coating enhancement sample data;
introducing a spatial attention module, and constructing a coating defect classification self-supervision training network model;
training the coating defect classification self-supervision training network model based on the first non-labeling coating enhancement sample data and the second non-labeling coating enhancement sample data to obtain a trained coating defect classification self-supervision training network model;
and performing coating defect classification detection based on the trained coating defect classification self-supervision training network model to obtain a coating defect detection classification result.
In some embodiments, the obtaining the non-labeling coated sample data and performing data preprocessing to obtain the first non-labeling coated enhanced sample data and the second non-labeling coated enhanced sample data includes:
the method comprises the steps of obtaining non-marked coating sample data, and carrying out data enhancement processing to obtain enhanced non-marked coating sample data;
cutting the reinforced non-marked coating sample data through a multi-cutting strategy to obtain cut non-marked coating sample data;
calculating the variance of the enhanced non-marked coated sample data and the variance of the cut non-marked coated sample data;
constructing a variance judgment conditional expression, selecting the cut non-marked coating sample data which meets the variance judgment conditional expression as first non-marked coating enhancement positive sample data, and selecting the enhanced non-marked coating sample data which meets the variance judgment conditional expression as second non-marked coating enhancement negative sample data;
removing first sub-data from the first non-labeling coating enhancement positive sample data to obtain first non-labeling coating enhancement sample data;
and selecting first sub-data in the first non-labeling coating enhancement positive sample data and the second non-labeling coating enhancement negative sample data to fuse, so as to obtain second non-labeling coating enhancement sample data.
In some embodiments, the coating defect classification self-supervised training network model includes a first encoder, a first projection head, a second encoder, a second projection head, and a loss calculation module, wherein an output of the first encoder is connected to an input of the first projection head, an output of the second encoder is connected to an input of the second projection head, and a first output of the first projection head, a second output of the second projection head, and an input of the loss calculation module are connected.
In some embodiments, the first encoder and the second encoder each include a plurality of deep convolutional neural network modules, a plurality of spatial attention modules, and a global pooling module, the plurality of deep convolutional neural network modules are sequentially connected with the plurality of spatial attention modules, and the global pooling module is a final output layer of the first encoder and the second encoder, wherein a network parameter updating mode of the first encoder is gradient backhaul encoding, and a network parameter updating mode of the second encoder is momentum encoding.
In some embodiments, the training the coating defect classification self-supervision training network model based on the first and second unlabeled coating enhancement sample data to obtain a trained coating defect classification self-supervision training network model includes:
Inputting the first and second unlabeled coated enhanced sample data into the coating defect classification self-supervised training network model;
the first encoder is used for carrying out coding processing on the first non-labeling coating enhancement sample data based on the coating defect classification self-supervision training network model to obtain coded first non-labeling coating enhancement sample data;
performing projection processing on the encoded first non-labeling coating enhancement sample data based on the first projection head of the coating defect classification self-supervision training network model to obtain non-labeling coating positive sample characteristics;
the second encoder based on the coating defect classification self-supervision training network model carries out encoding processing on the second non-marked coating enhancement sample data to obtain encoded second non-marked coating enhancement sample data;
performing projection processing on the encoded second non-labeling coating enhancement sample data based on the second projection head of the coating defect classification self-supervision training network model to obtain non-labeling coating mixed sample characteristics, wherein the non-labeling coating mixed sample characteristics comprise non-labeling coating positive sample characteristics and non-labeling coating negative sample characteristics;
The loss calculation module is used for carrying out feature loss value calculation processing on the non-marked coating positive sample features and the non-marked coating mixed sample features based on the coating defect classification self-supervision training network model to obtain non-marked coating feature loss values;
if the unmarked coating feature loss value does not meet the preset feature threshold, repeating the step of training the coating defect classification self-supervision training network model until the unmarked coating feature loss value meets the preset feature threshold;
and if the unmarked coating feature loss value meets the preset feature threshold, outputting the trained coating defect classification self-supervision training network model.
In some embodiments, the encoding the first unlabeled coated enhanced sample data by the first encoder based on the coating defect classification self-supervised training network model to obtain encoded first unlabeled coated enhanced sample data includes:
inputting the first unlabeled coating enhancement sample data to a first encoder of the coating defect classification self-supervised training network model;
based on the deep convolutional neural network module of the first encoder, performing feature extraction processing on the first non-labeling coating enhancement sample data to obtain preliminary first non-labeling coating enhancement sample feature data;
Based on the spatial attention module of the first encoder, performing spatial attention feature extraction processing on the preliminary first non-labeling coating enhancement sample feature data to obtain first non-labeling coating enhancement sample feature data;
and carrying out global pooling processing on the first non-labeling coating enhancement sample characteristic data based on a global pooling module of the first encoder to obtain encoded first non-labeling coating enhancement sample data.
In some embodiments, the spatial attention module based on the first encoder performs spatial attention feature extraction processing on the preliminary first non-labeling coating enhancement sample feature data to obtain first non-labeling coating enhancement sample feature data, including:
inputting the preliminary first non-labeling coated enhanced sample feature data to a spatial attention module of the first encoder, the spatial attention module comprising a global average branch, a max pooling branch, a shared full-connection branch, and an active convolution layer;
based on the global average branch of the spatial attention module, performing global average processing on the preliminary first non-labeling coating enhancement sample feature data to obtain a first non-labeling coating enhancement sample feature matrix;
Performing maximum pooling processing on the preliminary first non-labeling coating enhancement sample feature data based on the maximum pooling branch of the spatial attention module to obtain a second non-labeling coating enhancement sample feature matrix;
based on the shared full-connection branch of the spatial attention module, performing full-connection processing on the initial first non-labeling coating enhancement sample feature data to obtain a third non-labeling coating enhancement sample feature matrix;
based on an activated convolution layer of the spatial attention module, performing convolution activation processing on the first non-labeling coating enhanced sample feature matrix, the second non-labeling coating enhanced sample feature matrix and the third non-labeling coating enhanced sample feature matrix respectively to obtain a corresponding first non-labeling coating enhanced sample spatial attention matrix, a corresponding second non-labeling coating enhanced sample spatial attention matrix and a corresponding third non-labeling coating enhanced sample spatial attention matrix;
multiplying the first non-labeling coating enhanced sample space attention matrix with the preliminary first non-labeling coating enhanced sample characteristic data to obtain a first non-labeling coating enhanced sample space attention matrix to be fused;
Multiplying the second non-marked coating enhancement sample space attention matrix with the preliminary first non-marked coating enhancement sample characteristic data to obtain a second non-marked coating enhancement sample space attention matrix to be fused;
multiplying the third non-marked coating enhancement sample space attention matrix with the preliminary first non-marked coating enhancement sample characteristic data to obtain a third non-marked coating enhancement sample space attention matrix to be fused;
and carrying out fusion processing on the first to-be-fused non-marked coating enhanced sample space attention matrix, the second to-be-fused non-marked coating enhanced sample space attention matrix and the third to-be-fused non-marked coating enhanced sample space attention moment matrix to obtain first non-marked coating enhanced sample characteristic data.
In some embodiments, the performing, by the first projection head of the self-supervised training network model based on the coating defect classification, projection processing on the encoded first non-labeled coating enhancement sample data to obtain non-labeled coating positive sample features includes:
inputting the encoded first label-free coating enhancement sample data to a first projection head of the coating defect classification self-supervision training network model, wherein the first projection head comprises a first full-connection layer, a first normalization layer, a first activation layer, a second full-connection layer, a second normalization layer and a second activation layer;
Mapping the encoded first non-labeling coating enhancement sample data based on a first full-connection layer of the first projection head to obtain primarily mapped first non-labeling coating enhancement sample data;
based on a first normalization layer of the first projection head, performing normalization processing on the primarily mapped first label-free coating enhancement sample data to obtain primarily normalized first label-free coating enhancement sample data;
based on a first activation layer of the first projection head, performing activation processing on the primarily normalized first non-labeling coating enhancement sample data to obtain primarily activated first non-labeling coating enhancement sample data;
mapping the primarily activated first non-labeling coating enhancement sample data based on a second full-connection layer of the first projection head to obtain mapped first non-labeling coating enhancement sample data;
based on a second normalization layer of the first projection head, performing normalization processing on the mapped first label-free coating enhancement sample data to obtain normalized first label-free coating enhancement sample data;
and activating the normalized first non-labeling coating enhancement sample data based on a second activation layer of the first projection head to obtain non-labeling coating positive sample characteristics.
In some embodiments, the loss calculation module employs a binary cross entropy loss function having the expression:
on the upper partWherein L represents a binary cross entropy loss function, q andrepresenting the characteristics of a non-marked coated positive sample, k j Representing non-annotated coated negative sample features, y j Representing binary labels, sigma representing a sigmoid activation function, P representing the number of positive samples after all screening, and M representing the number selected from N-1 negative samples.
To achieve the above object, another aspect of the embodiments of the present application proposes a coating defect classification system based on a self-supervised learning strategy, the system comprising:
the first module is used for acquiring the non-marked coating sample data and carrying out data preprocessing to obtain first non-marked coating enhancement sample data and second non-marked coating enhancement sample data;
the second module is used for introducing the spatial attention module and constructing a coating defect classification self-supervision training network model;
the third module is used for training the coating defect classification self-supervision training network model based on the first non-labeling coating enhancement sample data and the second non-labeling coating enhancement sample data to obtain a trained coating defect classification self-supervision training network model;
And a fourth module, configured to perform coating defect classification detection based on the trained coating defect classification self-supervision training network model, to obtain a coating defect detection classification result.
The embodiment of the application at least comprises the following beneficial effects: the application provides a coating defect classification method and a coating defect classification system based on a self-supervision learning strategy, wherein the scheme obtains first non-labeling coating enhancement sample data and second non-labeling coating enhancement sample data by acquiring non-labeling coating sample data and performing data preprocessing; introducing a spatial attention module, and constructing a coating defect classification self-supervision training network model, so that the coating defect classification self-supervision training network model can pay attention to a defect area in an image; training the coating defect classification self-supervision training network model based on the first non-labeling coating enhancement sample data and the second non-labeling coating enhancement sample data to obtain a trained coating defect classification self-supervision training network model; coating defect classification detection is carried out based on a trained coating defect classification self-supervision training network model to obtain a coating defect detection classification result, and the embodiment of the application can realize high-accuracy defect detection images when industrial production materials such as industrial coating and the like have no large-scale defect data and no large-scale marking data at the beginning of production, and can better process images with more background areas and few defects in the images.
Drawings
FIG. 1 is a flow chart of a coating defect classification method based on a self-supervised learning strategy provided in an embodiment of the present application;
FIG. 2 is a schematic diagram of a data processing flow of a coating defect classification self-monitoring training network model according to an embodiment of the present application;
FIG. 3 is a schematic diagram of an encoder constructed in accordance with an embodiment of the present application;
FIG. 4 is a schematic diagram of the structure of a spatial attention module constructed in accordance with an embodiment of the present application;
FIG. 5 is a schematic view of a projection head constructed in accordance with an embodiment of the present application;
fig. 6 is a schematic structural diagram of a coating defect classification system based on a self-supervised learning strategy according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the embodiments of the application, but are merely examples of apparatuses and methods consistent with some aspects of the embodiments of the application as detailed in the accompanying claims.
It will be understood that the terms "first," "second," and the like, as used herein, may be used to describe various concepts, but are not limited by these terms unless otherwise specified. These terms are only used to distinguish one concept from another. For example, the first information may also be referred to as second information, and similarly, the second information may also be referred to as first information, without departing from the scope of embodiments of the present application. The words "if", as used herein, may be interpreted as "at … …" or "at … …" or "in response to a determination", depending on the context.
The terms "at least one," "a plurality," "each," "any" and the like as used herein, wherein at least one includes one, two or more, and a plurality includes two or more, each referring to each of a corresponding plurality, and any one referring to any one of the plurality.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the present application.
Referring to fig. 1, fig. 1 is a flowchart of a coating defect classification method based on a self-supervised learning strategy according to an embodiment of the present invention, and referring to fig. 1, the method includes the following steps:
s100, acquiring non-labeling coating sample data and performing data preprocessing to obtain first non-labeling coating enhancement sample data and second non-labeling coating enhancement sample data;
it should be noted that, in some embodiments, step S100 may include: s110, obtaining non-labeling coating sample data, and performing data enhancement processing to obtain enhanced non-labeling coating sample data; s120, cutting the reinforced non-marked coating sample data through a multi-cutting strategy to obtain cut non-marked coating sample data; s130, calculating the variance of the reinforced unlabeled coated sample data and the variance of the cut unlabeled coated sample data; s140, constructing a variance judgment condition formula, selecting cut non-marked coating sample data which meet the variance judgment condition formula as first non-marked coating enhancement positive sample data, and selecting enhanced non-marked coating sample data which meet the variance judgment condition formula as second non-marked coating enhancement negative sample data; s150, removing the first sub-data from the first non-labeling coating enhancement positive sample data to obtain first non-labeling coating enhancement sample data; s160, selecting first sub-data in the first non-labeling coating enhancement positive sample data and second non-labeling coating enhancement negative sample data to fuse, and obtaining second non-labeling coating enhancement sample data.
In some embodiments, each unlabeled exemplar x in the dataset is first enhanced by two times to obtain x1 and x2. The data enhancement includes random clipping, random brightness contrast saturation tone enhancement, random gray scale map variation, random gaussian smoothing, random flipping, etc., in order to enhance the model learning of local information, the embodiment of the present application references and improves the multi crop strategy in SwAV, and the size of random clipping in the proxy task is changed from random clipping of two 224×224 sizes to random clipping of two 160×160 sizes and four 96×96 sized areas. So that the number of positive samples changes from two to six. However, the samples after the random data enhancement are not necessarily positive samples with the data before the enhancement, because many defects in the coating are very small and the position and shape are changeable, such as bright spots, dark spots and the like, and the images cut out during random cutting may have no defect areas, so that the positive samples become negative samples and are treated as positive samples. Therefore, we filter the samples coming out of the multi-crop and calculate the sample variance s before the multi-crop 1 If the variance s of the samples after the current multicrop 2 ≥0.5×s 1 And s is 2 ≤2×s 1 The samples after this multicdrop are treated as positive samples, otherwise as negative samples.
S200, introducing a spatial attention module, and constructing a coating defect classification self-supervision training network model;
it should be noted that in some embodiments, the coating defect classification self-supervision training network model includes a first encoder, a first projection head, a second encoder, a second projection head and a loss calculation module, where an output end of the first encoder is connected to an input end of the first projection head, an output end of the second encoder is connected to an input end of the second projection head, and a first output end of the first projection head and a second output end of the second projection head are connected to an input end of the loss calculation module.
The first encoder and the second encoder comprise a plurality of deep convolutional neural network modules, a plurality of spatial attention modules and a global pooling module, the plurality of deep convolutional neural network modules are sequentially connected with the plurality of spatial attention modules, the global pooling module is the last output layer of the first encoder and the second encoder, wherein the network parameter updating mode of the first encoder is gradient feedback coding, and the network parameter updating mode of the second encoder is momentum coding.
S300, training a coating defect classification self-supervision training network model based on the first non-labeling coating enhancement sample data and the second non-labeling coating enhancement sample data to obtain a trained coating defect classification self-supervision training network model;
It should be noted that, in some embodiments, step S300 may include: s310, inputting the first non-labeling coating enhancement sample data and the second non-labeling coating enhancement sample data into a coating defect classification self-supervision training network model; s320, a first encoder based on a coating defect classification self-supervision training network model encodes the first non-labeling coating enhancement sample data to obtain encoded first non-labeling coating enhancement sample data; s330, performing projection processing on the encoded first non-labeling coating enhancement sample data based on a first projection head of the coating defect classification self-supervision training network model to obtain non-labeling coating positive sample characteristics; s340, a second encoder based on the coating defect classification self-supervision training network model encodes the second non-marked coating enhancement sample data to obtain encoded second non-marked coating enhancement sample data; s350, performing projection processing on the encoded second non-labeling coating enhancement sample data based on a second projection head of the coating defect classification self-supervision training network model to obtain non-labeling coating mixed sample characteristics, wherein the non-labeling coating mixed sample characteristics comprise non-labeling coating positive sample characteristics and non-labeling coating negative sample characteristics; s360, carrying out feature loss value calculation processing on the non-marked coating positive sample features and the non-marked coating mixed sample features based on a loss calculation module of the coating defect classification self-supervision training network model to obtain non-marked coating feature loss values; s370, if the unmarked coating feature loss value does not meet the preset feature threshold, repeating the step of training the coating defect classification self-supervision training network model until the unmarked coating feature loss value meets the preset feature threshold; and S380, outputting the trained coating defect classification self-supervision training network model if the unmarked coating characteristic loss value meets the preset characteristic threshold.
Further, in some embodiments, step S320 may include: s321, inputting first label-free coating enhancement sample data to a first encoder of a coating defect classification self-supervision training network model; s322, performing feature extraction processing on the first non-labeling coating enhancement sample data based on the depth convolution neural network module of the first encoder to obtain preliminary first non-labeling coating enhancement sample feature data; s323, performing spatial attention feature extraction processing on the preliminary first non-labeling coating enhancement sample feature data based on the spatial attention module of the first encoder to obtain first non-labeling coating enhancement sample feature data; s324, performing global pooling processing on the first non-labeling coating enhancement sample characteristic data based on a global pooling module of the first encoder to obtain encoded first non-labeling coating enhancement sample data.
It should be noted that, in the embodiment of the present application, the network structure of the first encoder is the same as that of the second encoder, so that the data processing procedure is the same.
In some embodiments, x1 and x2 are encoded with two encoders, a first encoder and a second encoder, respectively, to obtain y1 and y2. The two encoder networks have the same structure and are composed of ResNet50 and the spatial attention module designed as described above, and one spatial attention module is added behind each layer of ResNet50, and the network structure is shown in figure 3. The two network parameters are different, one adopts gradient feedback update parameters, and the other uses a mode of a momentum encoder in MoCo to update parameters.
Further, in some embodiments, step S323 may include: s3231, inputting the initial first non-labeling coating enhancement sample characteristic data to a spatial attention module of a first encoder, wherein the spatial attention module comprises a global average branch, a maximum pooling branch, a shared full-connection branch and an activated convolution layer; s3232, performing global average processing on the preliminary first non-labeling coating enhancement sample feature data based on global average branches of the spatial attention module to obtain a first non-labeling coating enhancement sample feature matrix; s3233, carrying out maximum pooling treatment on the initial first non-labeling coating enhancement sample feature data based on the maximum pooling branch of the spatial attention module to obtain a second non-labeling coating enhancement sample feature matrix; s3234, carrying out full connection processing on the initial first non-labeling coating enhancement sample feature data based on the shared full connection branch of the spatial attention module to obtain a third non-labeling coating enhancement sample feature matrix; s3235, a convolution layer is activated based on the spatial attention module, and convolution activation processing is carried out on the first non-marked coating enhancement sample feature matrix, the second non-marked coating enhancement sample feature matrix and the third non-marked coating enhancement sample feature matrix respectively to obtain a corresponding first non-marked coating enhancement sample spatial attention matrix, a corresponding second non-marked coating enhancement sample spatial attention matrix and a corresponding third non-marked coating enhancement sample spatial attention matrix; s3236, multiplying the first non-labeling coating enhanced sample space attention matrix with the preliminary first non-labeling coating enhanced sample characteristic data to obtain a first non-labeling coating enhanced sample space attention matrix to be fused; s3237, multiplying the second non-marked coating enhanced sample space attention matrix with the initial first non-marked coating enhanced sample characteristic data to obtain a second non-marked coating enhanced sample space attention matrix to be fused; s3238, multiplying the third non-marked coating enhanced sample space attention matrix with the initial first non-marked coating enhanced sample characteristic data to obtain a third non-marked coating enhanced sample space attention matrix to be fused; s3239, carrying out fusion processing on the first to-be-fused non-labeling coating enhanced sample space attention matrix, the second to-be-fused non-labeling coating enhanced sample space attention matrix and the third to-be-fused non-labeling coating enhanced sample space attention moment matrix to obtain first non-labeling coating enhanced sample characteristic data.
In some embodiments, because the background area may be large in the sample, the defect area occupies only a small area and the positions are changeable, so that the network needs to pay more attention to the defect area during self-supervision training, and therefore, the embodiment of the application designs a spatial attention mechanism, so that the model pays more attention to the defect area to optimize the training process. The spatial attention mechanism of the design is shown in fig. 4. The spatial attention mechanism of the embodiment of the application has three branches, and three h×w×1 matrixes are obtained after the original features X are respectively subjected to global average, maximum pooling and shared full connection of channel dimensions. The three matrices are convolved with 1×1 and sigmoid activated to obtain three h×w×1 spatial attention matrices, respectively. The spatial attention matrix is a mask in which each location represents the degree of attention of the network in the input profile X. These three masks are multiplied by X to obtain three features A1, A2 and A3, respectively. And adding the three features and carrying out information fusion to finally obtain a module A, wherein compared with the initial X, the module A focuses on the defective area in the features.
Further, in some embodiments, step S330 may include: s331, inputting the encoded first label-free coating enhancement sample data to a first projection head of a coating defect classification self-supervision training network model, wherein the first projection head comprises a first full-connection layer, a first normalization layer, a first activation layer, a second full-connection layer, a second normalization layer and a second activation layer; s332, mapping the encoded first non-labeling coating enhancement sample data based on the first full-connection layer of the first projection head to obtain primarily mapped first non-labeling coating enhancement sample data; s333, based on a first normalization layer of the first projection head, performing normalization processing on the primarily mapped first label-free coating enhancement sample data to obtain primarily normalized first label-free coating enhancement sample data; s334, based on a first activation layer of the first projection head, performing activation processing on the primarily normalized first non-labeling coating enhancement sample data to obtain primarily activated first non-labeling coating enhancement sample data; s335, mapping the primarily activated first non-labeling coating enhancement sample data based on a second full-connection layer of the first projection head to obtain mapped first non-labeling coating enhancement sample data; s336, based on a second normalization layer of the first projection head, performing normalization processing on the mapped first label-free coating enhancement sample data to obtain normalized first label-free coating enhancement sample data; and S337, based on a second activation layer of the first projection head, activating the normalized first non-labeling coating enhancement sample data to obtain non-labeling coating positive sample characteristics.
It should be noted that, in the embodiment of the present application, the network structures of the first projection head and the second projection head are the same, so the flow of data processing is the same.
In some embodiments, y1 and y2 are then converted to 128-dimensional features from the 2048-dimensional features generated by ResNet50 via two projection heads, each of which is a positive sample, respectively, the projection heads being as shown in FIG. 5, and the features of the data set encoded by the momentum encoder for samples other than sample x being k 1 ,k 2 ,k 3 ,...,k N-1 Indicating that all are negative samples, N is the total number of samples in the data set, and all positive samples coming out of multi-crop are x 0 ,x 1 ...,x P-1 P represents the number of positive samples after all screens. Positive sample x 0 Processing by gradient back-pass encoder and projection head to obtain positive sample characteristic q and positive sample x 1 ...,x P-1 Processing with momentum encoder and projection head to obtain positive sample characteristicsAll negative samples are also encoded with a momentum encoderProcessing by a projection head to obtain a negative sample characteristic k 1 ,k 2 ,k 3 ,...,k N-1 . The overall network structure diagram for contrast learning is shown in fig. 2.
In some embodiments, for sample x, the positive samples are q andthe negative sample is k 1 ,k 2 ,k 3 ,...,k N-1 . To reduce the computational effort, M out of N-1 negative samples were randomly chosen, m=4096. Because point multiplication may represent the similarity between two vectors, embodiments of the present application use the binary cross entropy value of q multiplied by the feature points of all positive and negative samples as the loss function value. The loss function of the embodiment of the application is:
Wherein y is i ＝1，y j =0, the above formula can thus be reduced to:
in the above formula, L represents a binary cross entropy loss function, q andrepresenting the characteristics of a non-marked coated positive sample, k j Representing non-annotated coated negative sample features, y j Representing binary labels, sigma representing a sigmoid activation function, P representing the number of positive samples after all screening, and M representing the number selected from N-1 negative samples.
Wherein, y is as follows j Is a binary label 1 or 0; when q is dot multiplied with the positive sample, the binary label is 1, so y j 1 is shown in the specification; when q is dot multiplied by the negative sample, the binary label is 0, so y j Is 0; sigma is a sigmoid activation function to multiply two sample feature points between 0-1, convenient and binaryThe tag performs a loss calculation.
S400, performing coating defect classification detection based on the trained coating defect classification self-supervision training network model to obtain a coating defect detection classification result;
in some embodiments, after the training of the data enhancement, the network structure and the loss function is completed, the network parameters of the gradient backhaul encoder and the projection head are frozen, and all samples in the data set are sent to the gradient backhaul encoder and the projection head without data enhancement to obtain the characteristics of all samples. The features of the image samples which can be obtained after the training are similar, and the features are also gathered in the feature space. Dissimilar sample features are also dissimilar, which are also far apart in feature space. We therefore clustered the features of all samples into class 100. Similar features are grouped together and similar figures are grouped together accordingly. Next, a random image in each of the 100 groups is checked, and if the image is a normal defect-free OK image, all images in the group of the images are regarded as OK images; if the random images in the group are defective, then all images in the group are treated as NG images, thus achieving the effect of self-supervising classification of all unlabeled images.
Referring to fig. 6, an embodiment of the present application further provides a coating defect classification system based on a self-supervised learning strategy, which can implement the above-mentioned coating defect classification method based on the self-supervised learning strategy, where the system includes:
the first module is used for acquiring the non-marked coating sample data and carrying out data preprocessing to obtain first non-marked coating enhancement sample data and second non-marked coating enhancement sample data;
the second module is used for introducing the spatial attention module and constructing a coating defect classification self-supervision training network model;
the third module is used for training the coating defect classification self-supervision training network model based on the first non-labeling coating enhancement sample data and the second non-labeling coating enhancement sample data to obtain a trained coating defect classification self-supervision training network model;
and the fourth module is used for carrying out coating defect classification detection based on the trained coating defect classification self-supervision training network model to obtain a coating defect detection classification result.
It can be understood that the content in the above method embodiment is applicable to the system embodiment, and the functions specifically implemented by the system embodiment are the same as those of the above method embodiment, and the achieved beneficial effects are the same as those of the above method embodiment.
Preferred embodiments of the present application are described above with reference to the accompanying drawings, and thus do not limit the scope of the claims of the embodiments of the present application. Any modifications, equivalent substitutions and improvements made by those skilled in the art without departing from the scope and spirit of the embodiments of the present application shall fall within the scope of the claims of the embodiments of the present application.
Claims (10)
1. A coating defect classification method based on a self-supervised learning strategy, the method comprising:
acquiring non-labeling coating sample data and performing data preprocessing to obtain first non-labeling coating enhancement sample data and second non-labeling coating enhancement sample data;
introducing a spatial attention module, and constructing a coating defect classification self-supervision training network model;
training the coating defect classification self-supervision training network model based on the first non-labeling coating enhancement sample data and the second non-labeling coating enhancement sample data to obtain a trained coating defect classification self-supervision training network model;
and performing coating defect classification detection based on the trained coating defect classification self-supervision training network model to obtain a coating defect detection classification result.
2. The method of claim 1, wherein the obtaining and data preprocessing of the unlabeled coated sample data to obtain first unlabeled coated enhanced sample data and second unlabeled coated enhanced sample data comprises:
The method comprises the steps of obtaining non-marked coating sample data, and carrying out data enhancement processing to obtain enhanced non-marked coating sample data;
cutting the reinforced non-marked coating sample data through a multi-cutting strategy to obtain cut non-marked coating sample data;
calculating the variance of the enhanced non-marked coated sample data and the variance of the cut non-marked coated sample data;
constructing a variance judgment conditional expression, selecting the cut non-marked coating sample data which meets the variance judgment conditional expression as first non-marked coating enhancement positive sample data, and selecting the enhanced non-marked coating sample data which meets the variance judgment conditional expression as second non-marked coating enhancement negative sample data;
removing first sub-data from the first non-labeling coating enhancement positive sample data to obtain first non-labeling coating enhancement sample data;
and selecting first sub-data in the first non-labeling coating enhancement positive sample data and the second non-labeling coating enhancement negative sample data to fuse, so as to obtain second non-labeling coating enhancement sample data.
3. The method of claim 1, wherein the coating defect classification self-supervised training network model includes a first encoder, a first projection head, a second encoder, a second projection head, and a loss calculation module, wherein an output of the first encoder is coupled to an input of the first projection head, an output of the second encoder is coupled to an input of the second projection head, and a first output of the first projection head, a second output of the second projection head is coupled to an input of the loss calculation module.
4. The method of claim 3, wherein the first encoder and the second encoder each comprise a plurality of deep convolutional neural network modules, a plurality of spatial attention modules, and a global pooling module, the plurality of deep convolutional neural network modules are sequentially connected with the plurality of spatial attention modules, the global pooling module is a final output layer of the first encoder and the second encoder, wherein a network parameter updating mode of the first encoder is gradient backhaul encoding, and a network parameter updating mode of the second encoder is momentum encoding.
5. The method of claim 1, wherein the training the coating defect classification self-supervised training network model based on the first and second unlabeled coating enhancement sample data to obtain a trained coating defect classification self-supervised training network model comprises:
inputting the first and second unlabeled coated enhanced sample data into the coating defect classification self-supervised training network model;
the first encoder is used for carrying out coding processing on the first non-labeling coating enhancement sample data based on the coating defect classification self-supervision training network model to obtain coded first non-labeling coating enhancement sample data;
Performing projection processing on the encoded first non-labeling coating enhancement sample data based on the first projection head of the coating defect classification self-supervision training network model to obtain non-labeling coating positive sample characteristics;
the second encoder based on the coating defect classification self-supervision training network model carries out encoding processing on the second non-marked coating enhancement sample data to obtain encoded second non-marked coating enhancement sample data;
performing projection processing on the encoded second non-labeling coating enhancement sample data based on the second projection head of the coating defect classification self-supervision training network model to obtain non-labeling coating mixed sample characteristics, wherein the non-labeling coating mixed sample characteristics comprise non-labeling coating positive sample characteristics and non-labeling coating negative sample characteristics;
the loss calculation module is used for carrying out feature loss value calculation processing on the non-marked coating positive sample features and the non-marked coating mixed sample features based on the coating defect classification self-supervision training network model to obtain non-marked coating feature loss values;
if the unmarked coating feature loss value does not meet the preset feature threshold, repeating the step of training the coating defect classification self-supervision training network model until the unmarked coating feature loss value meets the preset feature threshold;
And if the unmarked coating feature loss value meets the preset feature threshold, outputting the trained coating defect classification self-supervision training network model.
6. The method of claim 5, wherein the encoding the first unlabeled coated enhanced sample data based on the first encoder of the coating defect classification self-supervised training network model to obtain encoded first unlabeled coated enhanced sample data comprises:
inputting the first unlabeled coating enhancement sample data to a first encoder of the coating defect classification self-supervised training network model;
based on the deep convolutional neural network module of the first encoder, performing feature extraction processing on the first non-labeling coating enhancement sample data to obtain preliminary first non-labeling coating enhancement sample feature data;
based on the spatial attention module of the first encoder, performing spatial attention feature extraction processing on the preliminary first non-labeling coating enhancement sample feature data to obtain first non-labeling coating enhancement sample feature data;
and carrying out global pooling processing on the first non-labeling coating enhancement sample characteristic data based on a global pooling module of the first encoder to obtain encoded first non-labeling coating enhancement sample data.
7. The method of claim 6, wherein the performing, by the spatial attention module of the first encoder, spatial attention feature extraction on the preliminary first non-labeling coated enhanced sample feature data to obtain first non-labeling coated enhanced sample feature data comprises:
inputting the preliminary first non-labeling coated enhanced sample feature data to a spatial attention module of the first encoder, the spatial attention module comprising a global average branch, a max pooling branch, a shared full-connection branch, and an active convolution layer;
based on the global average branch of the spatial attention module, performing global average processing on the preliminary first non-labeling coating enhancement sample feature data to obtain a first non-labeling coating enhancement sample feature matrix;
performing maximum pooling processing on the preliminary first non-labeling coating enhancement sample feature data based on the maximum pooling branch of the spatial attention module to obtain a second non-labeling coating enhancement sample feature matrix;
based on the shared full-connection branch of the spatial attention module, performing full-connection processing on the initial first non-labeling coating enhancement sample feature data to obtain a third non-labeling coating enhancement sample feature matrix;
Based on an activated convolution layer of the spatial attention module, performing convolution activation processing on the first non-labeling coating enhanced sample feature matrix, the second non-labeling coating enhanced sample feature matrix and the third non-labeling coating enhanced sample feature matrix respectively to obtain a corresponding first non-labeling coating enhanced sample spatial attention matrix, a corresponding second non-labeling coating enhanced sample spatial attention matrix and a corresponding third non-labeling coating enhanced sample spatial attention matrix;
multiplying the first non-labeling coating enhanced sample space attention matrix with the preliminary first non-labeling coating enhanced sample characteristic data to obtain a first non-labeling coating enhanced sample space attention matrix to be fused;
multiplying the second non-marked coating enhancement sample space attention matrix with the preliminary first non-marked coating enhancement sample characteristic data to obtain a second non-marked coating enhancement sample space attention matrix to be fused;
multiplying the third non-marked coating enhancement sample space attention matrix with the preliminary first non-marked coating enhancement sample characteristic data to obtain a third non-marked coating enhancement sample space attention matrix to be fused;
And carrying out fusion processing on the first to-be-fused non-marked coating enhanced sample space attention matrix, the second to-be-fused non-marked coating enhanced sample space attention matrix and the third to-be-fused non-marked coating enhanced sample space attention moment matrix to obtain first non-marked coating enhanced sample characteristic data.
8. The method of claim 5, wherein the performing, by the first projection head of the self-supervised training network model based on the coating defect classification, projection processing of the encoded first non-labeled coating enhancement sample data to obtain non-labeled coating positive sample features comprises:
inputting the encoded first label-free coating enhancement sample data to a first projection head of the coating defect classification self-supervision training network model, wherein the first projection head comprises a first full-connection layer, a first normalization layer, a first activation layer, a second full-connection layer, a second normalization layer and a second activation layer;
mapping the encoded first non-labeling coating enhancement sample data based on a first full-connection layer of the first projection head to obtain primarily mapped first non-labeling coating enhancement sample data;
Based on a first normalization layer of the first projection head, performing normalization processing on the primarily mapped first label-free coating enhancement sample data to obtain primarily normalized first label-free coating enhancement sample data;
based on a first activation layer of the first projection head, performing activation processing on the primarily normalized first non-labeling coating enhancement sample data to obtain primarily activated first non-labeling coating enhancement sample data;
mapping the primarily activated first non-labeling coating enhancement sample data based on a second full-connection layer of the first projection head to obtain mapped first non-labeling coating enhancement sample data;
based on a second normalization layer of the first projection head, performing normalization processing on the mapped first label-free coating enhancement sample data to obtain normalized first label-free coating enhancement sample data;
and activating the normalized first non-labeling coating enhancement sample data based on a second activation layer of the first projection head to obtain non-labeling coating positive sample characteristics.
9. The method of claim 5, wherein the loss calculation module employs a binary cross-entropy loss function having the expression:
In the above formula, L represents a binary cross entropy loss function, q andrepresenting the characteristics of a non-marked coated positive sample, k j Representing non-annotated coated negative sample features, y j Representing binary labels, sigma representing a sigmoid activation function, P representing the number of positive samples after all screening, and M representing the number selected from N-1 negative samples.
10. A coating defect classification system based on a self-supervised learning strategy, the system comprising:
the first module is used for acquiring the non-marked coating sample data and carrying out data preprocessing to obtain first non-marked coating enhancement sample data and second non-marked coating enhancement sample data;
the second module is used for introducing the spatial attention module and constructing a coating defect classification self-supervision training network model;
the third module is used for training the coating defect classification self-supervision training network model based on the first non-labeling coating enhancement sample data and the second non-labeling coating enhancement sample data to obtain a trained coating defect classification self-supervision training network model;
and a fourth module, configured to perform coating defect classification detection based on the trained coating defect classification self-supervision training network model, to obtain a coating defect detection classification result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311833505.1A CN117876763A (en) | 2023-12-27 | 2023-12-27 | Coating defect classification method and system based on self-supervision learning strategy |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311833505.1A CN117876763A (en) | 2023-12-27 | 2023-12-27 | Coating defect classification method and system based on self-supervision learning strategy |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117876763A true CN117876763A (en) | 2024-04-12 |
Family
ID=90591213
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311833505.1A Pending CN117876763A (en) | 2023-12-27 | 2023-12-27 | Coating defect classification method and system based on self-supervision learning strategy |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117876763A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110929512A (en) * | 2018-09-19 | 2020-03-27 | 北京京东尚科信息技术有限公司 | Data enhancement method and device |
| CN113920472A (en) * | 2021-10-15 | 2022-01-11 | 中国海洋大学 | Unsupervised target re-identification method and system based on attention mechanism |
| CN115393671A (en) * | 2022-08-25 | 2022-11-25 | 河海大学 | Rock class prediction method based on multi-teacher knowledge distillation and normalized attention |
| CN116206185A (en) * | 2023-02-27 | 2023-06-02 | 山东浪潮科学研究院有限公司 | Lightweight small target detection method based on improved YOLOv7 |
| CN116994047A (en) * | 2023-08-01 | 2023-11-03 | 北京工商大学 | Small sample image defect target detection method based on self-supervision pre-training |
-
2023
- 2023-12-27 CN CN202311833505.1A patent/CN117876763A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110929512A (en) * | 2018-09-19 | 2020-03-27 | 北京京东尚科信息技术有限公司 | Data enhancement method and device |
| CN113920472A (en) * | 2021-10-15 | 2022-01-11 | 中国海洋大学 | Unsupervised target re-identification method and system based on attention mechanism |
| CN115393671A (en) * | 2022-08-25 | 2022-11-25 | 河海大学 | Rock class prediction method based on multi-teacher knowledge distillation and normalized attention |
| CN116206185A (en) * | 2023-02-27 | 2023-06-02 | 山东浪潮科学研究院有限公司 | Lightweight small target detection method based on improved YOLOv7 |
| CN116994047A (en) * | 2023-08-01 | 2023-11-03 | 北京工商大学 | Small sample image defect target detection method based on self-supervision pre-training |
Non-Patent Citations (1)
| Title |
|---|
| KAIMING HE ET AL: "Momentum Contrast for Unsupervised Visual Representation Learning", 《ARXIV:1911.05722V3》, 23 March 2020 (2020-03-23), pages 1 - 12 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111259905B (en) | Feature fusion remote sensing image semantic segmentation method based on downsampling | |
| CN112308860B (en) | Earth observation image semantic segmentation method based on self-supervision learning | |
| CN109241972B (en) | Image semantic segmentation method based on deep learning | |
| CN114120102A (en) | Boundary-optimized remote sensing image semantic segmentation method, device, equipment and medium | |
| CN110570396A (en) | industrial product defect detection method based on deep learning | |
| CN110889449A (en) | Edge-enhanced multi-scale remote sensing image building semantic feature extraction method | |
| CN112541503A (en) | Real-time semantic segmentation method based on context attention mechanism and information fusion | |
| CN112070158B (en) | Facial flaw detection method based on convolutional neural network and bilateral filtering | |
| CN112488025B (en) | Double-temporal remote sensing image semantic change detection method based on multi-modal feature fusion | |
| JP6107531B2 (en) | Feature extraction program and information processing apparatus | |
| CN114022408A (en) | Remote sensing image cloud detection method based on multi-scale convolution neural network | |
| CN114663685B (en) | Pedestrian re-recognition model training method, device and equipment | |
| CN113255837A (en) | Improved CenterNet network-based target detection method in industrial environment | |
| CN116152611B (en) | Multistage multi-scale point cloud completion method, system, equipment and storage medium | |
| CN114037640A (en) | Image generation method and device | |
| CN115527072A (en) | Chip surface defect detection method based on sparse space perception and meta-learning | |
| CN116958827A (en) | Deep learning-based abandoned land area extraction method | |
| CN114283325A (en) | Underwater target identification method based on knowledge distillation | |
| CN116612283A (en) | Image semantic segmentation method based on large convolution kernel backbone network | |
| CN114581789A (en) | Hyperspectral image classification method and system | |
| CN117474796B (en) | Image generation method, device, equipment and computer readable storage medium | |
| CN117876763A (en) | Coating defect classification method and system based on self-supervision learning strategy | |
| CN115424276B (en) | Ship license plate number detection method based on deep learning technology | |
| CN114694031B (en) | Remote sensing image typical object extraction method based on multitasking attention mechanism | |
| CN113688783B (en) | Face feature extraction method, low-resolution face recognition method and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |