CN114648480A

CN114648480A - Surface defect detection method, device and system

Info

Publication number: CN114648480A
Application number: CN202011495050.3A
Authority: CN
Inventors: 张营营; 钟巧勇; 谢迪; 浦世亮
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2020-12-17
Filing date: 2020-12-17
Publication date: 2022-06-21
Also published as: EP4266244A4; WO2022127919A1; EP4266244A1

Abstract

The embodiment of the application discloses a surface defect detection method, device and system, and belongs to the technical field of deep learning. In the embodiment of the application, if it is determined that the surface defect of the known defect type does not exist in the image to be detected through the neural network defect segmentation model detection, the image feature of the image to be detected is extracted through the neural network defect feature extraction model to obtain the feature to be compared, and if the similarity between the feature to be compared and the normal data representation feature is small, the surface defect of the unknown defect type in the image to be detected is determined, that is, the scheme can detect the surface defect of the unknown defect type, and the detection accuracy is improved.

Description

Surface defect detection method, device and system

Technical Field

The embodiment of the application relates to the technical field of deep learning, in particular to a surface defect detection method, device and system.

Background

The surface defect refers to the flaw on the appearance of an object, has the characteristics of various types, variable forms, unfixed positions, diversified background textures and the like, is an important link for quality control in the industrial field, and requires that the higher the accuracy of the surface defect detection is, the better the surface defect detection is.

The deep learning neural network in the related art can only detect the surface defects of known defect types, and cannot effectively detect the new type of defects encountered in actual detection, namely the surface defects of unknown defect types.

Disclosure of Invention

The embodiment of the application provides a surface defect detection method, device and system, which can detect the surface defects of unknown defect types and improve the detection accuracy. The technical scheme is as follows:

in one aspect, a method for detecting surface defects is provided, the method comprising:

acquiring an image to be detected;

inputting the image to be detected into a neural network defect segmentation model and outputting a first detection result, wherein the neural network defect segmentation model is used for detecting the surface defects of known defect types;

if the first detection result is that the image to be detected does not have the surface defects of the known defect types, inputting the image to be detected into a neural network defect feature extraction model, and outputting the feature to be compared, wherein the feature to be compared is the image feature of the image to be detected;

and if the similarity between the features to be compared and the normal data representation features is smaller than a similarity threshold value, determining that the image to be detected has the surface defects of unknown defect types, wherein the normal data representation features are generated based on the image features of the image without the surface defects.

Optionally, after determining that the image to be detected has the surface defect of the unknown defect type, the method further includes:

and determining the defect position of the surface defect in the image to be detected according to the similarity between the feature to be compared and the normal data representation feature and the mapping relation between the image feature matrix and the image pixel matrix.

Optionally, before inputting the image to be detected into the neural network defect segmentation model and outputting the first detection result, the method further includes:

acquiring a first data set, wherein the first data set comprises a defect image of a known defect type and corresponding first labeling information, the first labeling information is labeling information representing the defect type, the first data set further comprises an image without surface defects and corresponding second labeling information, and the second labeling information is labeling information representing no surface defects;

and training to obtain the neural network defect segmentation model according to the first data set.

Optionally, before inputting the image to be detected into the neural network defect feature extraction model and outputting the feature to be compared, the method further includes:

acquiring a second data set, wherein the second data set comprises an image of a known object type and corresponding third labeling information, and the third labeling information is labeling information representing the object type;

and training to obtain the neural network defect feature extraction model according to the second data set.

Optionally, before determining that the image to be detected has the surface defect of the unknown defect type, if the similarity between the feature to be compared and the normal data representation feature is smaller than a similarity threshold, the method further includes:

acquiring at least one first sample image, the at least one first sample image being free of surface defects;

inputting the at least one first sample image into the neural network defect feature extraction model, and outputting the image features of the at least one first sample image;

and taking the image characteristics of the at least one first sample image as the normal data representation characteristics.

obtaining a plurality of second sample images, the plurality of second sample images being free of surface defects;

inputting the second sample images into the neural network defect feature extraction model, and outputting image features of the second sample images;

clustering the image characteristics of the second sample images to obtain multiple groups of normal data characteristics;

selecting at least one image feature from each group of normal data features in the multiple groups of normal data features to obtain the normal data representation features.

Optionally, after the inputting the image to be detected into the neural network defect feature extraction model and outputting the feature to be compared, the method further includes:

and if the similarity between the features to be compared and the normal data representation features is greater than or equal to the similarity threshold, determining that the image to be detected has no surface defects.

In another aspect, there is provided a surface defect detecting apparatus, the apparatus including:

the first acquisition module is used for acquiring an image to be detected;

the detection module is used for inputting the image to be detected into a neural network defect segmentation model and outputting a first detection result, and the neural network defect segmentation model is used for detecting the surface defects of known defect types;

the first processing module is used for inputting the image to be detected into a neural network defect feature extraction model and outputting a feature to be compared if the first detection result indicates that the image to be detected does not have the surface defect of the known defect type, wherein the feature to be compared is the image feature of the image to be detected;

the first determining module is used for determining that the image to be detected has the surface defect of the unknown defect type if the similarity between the feature to be compared and the normal data representation feature is smaller than a similarity threshold value, wherein the normal data representation feature is generated based on the image feature of the image without the surface defect.

Optionally, the apparatus further comprises:

and the second determining module is used for determining the defect position of the surface defect in the image to be detected according to the similarity between the feature to be compared and the normal data representation feature and the mapping relation between the image feature matrix and the image pixel matrix.

Optionally, the apparatus further comprises:

the system comprises a first acquisition module, a second acquisition module and a control module, wherein the first acquisition module is used for acquiring a first data set, the first data set comprises a defect image of a known defect type and corresponding first marking information, the first marking information is marking information representing the defect type, the first data set further comprises an image without surface defects and corresponding second marking information, and the second marking information is defect information representing no surface defects;

and the first training module is used for training to obtain the neural network defect segmentation model according to the first data set.

Optionally, the apparatus further comprises:

the third acquisition module is used for acquiring a second data set, wherein the second data set comprises an image of a known object type and corresponding third labeling information, and the third labeling information is labeling information representing the object type;

and the second training module is used for training to obtain the neural network defect feature extraction model according to the second data set.

Optionally, the apparatus further comprises:

a fourth acquisition module for acquiring at least one first sample image, the at least one first sample image being free of surface defects;

the second processing module is further used for inputting the at least one first sample image into the neural network defect feature extraction model and outputting the image features of the at least one first sample image;

and the third determining module is used for taking the image characteristics of the at least one first sample image as the normal data representation characteristics.

Optionally, the apparatus further comprises:

a fifth acquiring module, configured to acquire a plurality of second sample images, where the plurality of second sample images are free of surface defects;

the third processing module is used for inputting the second sample images into the neural network defect feature extraction model and outputting the image features of the second sample images;

the clustering module is used for clustering the image characteristics of the plurality of second sample images to obtain a plurality of groups of normal data characteristics;

and the fourth determining module is used for selecting at least one image feature from each group of normal data features in the multiple groups of normal data features to obtain the normal data representation features.

Optionally, the apparatus further comprises:

and the fifth determining module is used for determining that the image to be detected has no surface defect if the similarity between the feature to be compared and the normal data representation feature is greater than or equal to the similarity threshold.

In another aspect, a surface defect detection system is provided, the surface defect detection system comprising a camera and at least one processor;

the camera is used for shooting at least one surface of an object to be detected as an image to be detected;

and the at least one processor is used for acquiring the image to be detected and realizing the steps of the surface defect detection method.

Optionally, the surface defect detecting system further comprises a conveying device for conveying the object to be detected;

the camera is used for shooting the object to be detected in the process of transmitting the object to be detected by the conveying device.

In another aspect, a computer device is provided, where the computer device includes a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete mutual communication through the communication bus, the memory is used to store a computer program, and the processor is used to execute the program stored in the memory to implement the steps of the surface defect detection method.

In another aspect, a computer-readable storage medium is provided, in which a computer program is stored, which, when being executed by a processor, carries out the steps of the surface defect detection method described above.

In another aspect, a computer program product comprising instructions is provided, which when run on a computer, causes the computer to perform the steps of the surface defect detection method described above.

The technical scheme provided by the embodiment of the application can at least bring the following beneficial effects:

in the embodiment of the application, if it is determined that the surface defect of the known defect type does not exist in the image to be detected through the neural network defect segmentation model detection, the image feature of the image to be detected is extracted through the neural network defect feature extraction model to obtain the feature to be compared, and if the similarity between the feature to be compared and the normal data representation feature is small, the surface defect of the unknown defect type in the image to be detected is determined, that is, the scheme can detect the surface defect of the unknown defect type, and the detection accuracy is improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a system architecture diagram of a surface defect detection system provided by an embodiment of the present application;

FIG. 2 is a flow chart of a method for detecting surface defects according to an embodiment of the present disclosure;

FIG. 3 is a flow chart of another method for detecting surface defects according to an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of a method for training a detection model according to an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of a method for modeling normal data features according to an embodiment of the present application;

FIG. 6 is a schematic structural diagram of a surface defect detecting apparatus according to an embodiment of the present disclosure;

fig. 7 is a schematic structural diagram of a terminal according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a server according to an embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the embodiments of the present application more clear, the embodiments of the present application will be further described in detail with reference to the accompanying drawings.

First, a system architecture related to the surface defect detection method provided by the embodiment of the present application is introduced.

Fig. 1 is a system architecture diagram of a surface defect detecting system according to an embodiment of the present disclosure, which is used to implement a surface defect detecting method according to an embodiment of the present disclosure. Referring to fig. 1, the system includes a camera 101 and at least one processor 102.

The at least one processor 102 is a processor in a computer device, and the camera 101 and the at least one processor 102 in the computer device are connected in a wireless or wired manner for communication. The camera 101 is used to capture an image of an object to be inspected, which is sent to the at least one processor 102 as the apparatus to be inspected. The computer device is used as a device under inspection for surface defect detection by the at least one processor 102.

In some embodiments, the at least one processor 102 is a processor in the camera 101, and the camera 101 is used as a detection device to capture an image of an object to be detected as an image to be detected for surface defect detection.

Optionally, the system further comprises a transport device for transporting the object to be detected. In one scenario, the conveyor is in the process of transporting the object to be inspected, the camera 101 shoots the object to be inspected, and the surface defect is inspected by the inspection equipment. Conveying devices such as conveying belts, conveying vehicles, conveying machines and the like, and objects to be detected such as metal parts, glassware, paper and other products produced in factories. Therefore, in the process of conveying the object to be detected, the object with surface defects, namely defective products, is detected, and the product quality is ensured.

The surface defect detection method provided in the embodiments of the present application is explained in detail below.

Fig. 2 is a flowchart of a surface defect detection method provided in an embodiment of the present application, and is described by taking an example in which the method is applied to a surface defect detection apparatus (referred to as a detection apparatus). Referring to fig. 2, the method includes the following steps.

Step 201: and acquiring an image to be detected.

In the embodiment of the application, the detection equipment acquires an image to be detected, wherein the image to be detected refers to an image of an object needing to be detected whether surface defects exist. For example, images of objects such as industrially produced appliances, paper, and the like.

For example, the industrial camera sends the acquired image of the object to the detection device, and the detection device takes the received image as the image to be detected.

Step 202: and inputting the image to be detected into a neural network defect segmentation model, and outputting a first detection result, wherein the neural network defect segmentation model is used for detecting the surface defects of known defect types.

In the embodiment of the application, a neural network defect segmentation model is deployed in the detection device, and the neural network defect segmentation model is a deep learning model and is used for detecting surface defects of known defect types. The detection equipment inputs an image to be detected into the neural network defect segmentation model and outputs a first detection result.

Exemplarily, assuming that the image to be detected has surface defects of known defect types, the first detection result is that the image to be detected has surface defects of known defect types. And if the image to be detected does not have the surface defects of the known defect types, the first detection result is that the image to be detected does not have the surface defects of the known defect types.

Step 203: and if the first detection result is that the image to be detected does not have the surface defects of the known defect types, inputting the image to be detected into a neural network defect feature extraction model, and outputting the feature to be compared, wherein the feature to be compared is the image feature of the image to be detected.

In the embodiment of the application, a neural network defect feature extraction model is also deployed in the detection device, the neural network defect feature extraction model is a deep learning model, and the neural network defect feature extraction model is used for extracting image features. And if the first detection result is that the image to be detected does not have the surface defects of the known defect types, the detection equipment inputs the image to be detected into the neural network defect feature extraction model and outputs the feature to be compared, wherein the feature to be compared is the image feature of the image to be detected.

Step 204: and if the similarity between the features to be compared and the normal data representation features is smaller than a similarity threshold value, determining that the images to be detected have the surface defects of unknown defect types, wherein the normal data representation features are generated based on the image features of the images without the surface defects.

In the embodiment of the present application, the detection device stores therein normal data representing features generated based on image features of an image without surface defects, for example, the normal data representing features include image features obtained by processing the image without surface defects by a neural network defect feature extraction model. After extracting the features to be compared through the neural network defect feature extraction model, the detection equipment calculates the similarity between the features to be compared and the normal data representation features, and if the similarity between the features to be compared and the normal data representation features is smaller than a similarity threshold value, the detection equipment determines that the image to be detected has surface defects of unknown defect types.

There are many methods for calculating the similarity between the features to be compared and the normal data representation features, for example, an euclidean distance similarity calculation method between the features, a mahalanobis distance similarity calculation method between the features, a cosine distance similarity calculation method between the features, and the like are adopted, which is not limited in the embodiment of the present application.

In an embodiment of the application, the normal data representation features include image features of one or more images free of surface defects. The detection equipment calculates the similarity between the features to be compared and each image feature included in the normal data representation features to obtain one or more similarities, and if at least one of the one or more similarities is smaller than a first similarity threshold value, the detection equipment determines that the surface defects of the unknown defect types exist in the images to be detected. Or if the average value of the one or more similarity is smaller than the second similarity threshold, the detection equipment determines that the image to be detected has the surface defect of the unknown defect type. Wherein the first similarity threshold and the second similarity threshold are the same or different.

In the embodiment of the application, if the similarity between the feature to be compared and the normal data representation feature is greater than or equal to the similarity threshold, the detection device determines that the image to be detected has no surface defects.

Illustratively, the detection device calculates the similarity between the features to be compared and each image feature included in the normal data representation features to obtain one or more similarities, and if the one or more similarities are all larger than or equal to a first similarity threshold value, the detection device determines that the image to be detected has no surface defects. Or if the average value of the one or more similarity is larger than or equal to the second similarity threshold value, the detection equipment determines that the image to be detected has no surface defect.

Optionally, in this embodiment of the present application, if the image to be detected has surface defects of unknown defect types, the detection apparatus can also detect the positions of the surface defects of unknown defect types existing in the image to be detected, that is, detect and determine the defect positions of the surface defects existing in the image to be detected.

In the embodiment of the application, the detection equipment determines the defect position of the image to be detected by comparing the feature to be compared with the normal data representation feature. For example, the detection device determines the defect position of the surface defect existing in the image to be detected according to the similarity between the feature to be compared and the normal data representation feature and the mapping relation between the image feature matrix and the image pixel matrix.

In one implementation, a certain mapping relationship exists between an image feature matrix and an image pixel matrix, the image pixel matrix refers to a matrix formed by pixel values of an image, the image feature matrix is obtained by downsampling the image pixel matrix through a neural network defect feature extraction model, that is, image features extracted by the neural network defect feature extraction model are expressed as the image feature matrix in a matrix form.

Exemplarily, it is assumed that the neural network defect feature extraction model in the embodiment of the present application is constructed based on a convolutional neural network, the extracted image features include features of C channels, an image pixel matrix is a matrix of 100 × 100, the image pixel matrix is subjected to four-fold down-sampling by the neural network defect feature extraction model to obtain a feature array of 25 × 25C-dimensional features, the feature array is a third-order tensor, and it can be understood that each position of the 25 × 25 image feature matrix is a C-dimensional vector, and each position corresponds to a 4 × 4 region in the image pixel matrix. For example, the first position in the 25 x 25 image feature matrix corresponds to the first 4 x 4 region of the 100 x 100 image pixel matrix. Then, the similarity between the C-dimensional vector of each position in the 25 × 25 to-be-compared feature matrix and the C-dimensional vector of the corresponding position in the 25 × 25 normal data feature matrix determines whether the surface defect exists in the corresponding 4 × 4 region in the image pixel matrix.

The detection equipment calculates the similarity between the elements at the same position in the feature matrix to be compared and the normal data feature matrix, if the similarity between the elements at the same position is smaller than a similarity threshold value, the detection equipment detects the defect feature position, and the defect feature position refers to the position in the feature matrix to be compared, wherein the feature similarity is lower than the similarity threshold value. Then, the detection device determines an image pixel position corresponding to the defect characteristic position from the image pixel matrix corresponding to the image to be compared according to the mapping relation between the image characteristic matrix and the image pixel matrix, and the image pixel position is used as the defect position of the detected surface defect of the image to be detected. The characteristic matrix to be compared refers to an image characteristic matrix corresponding to an image to be detected, and the normal data characteristic matrix refers to an image characteristic matrix corresponding to normal data representation characteristics.

Exemplarily, assuming that the size of an image pixel matrix corresponding to an image to be detected is 100 × 100, three-order tensors of 25 × 25C-dimensional features are respectively used for the feature matrix to be compared and the normal data feature matrix, the detection device calculates the similarity between the C-dimensional vector at the first position in the feature matrix to be compared and the C-dimensional vector at the first position in the normal data feature matrix, and if the similarity is smaller than a similarity threshold, the detection device determines that the image position of the first 4 × 4 region in the image pixel matrix corresponding to the image to be detected is the defect position according to the mapping relationship between the image feature matrix and the image pixel matrix.

Fig. 3 is a flowchart of another surface defect detection method provided in an embodiment of the present application. Referring to fig. 3, taking an image to be detected as an image shot by an industrial camera in real time as an example, the detection device has two functions of detecting known defects and detecting unknown defects, and a trained model including a neural network defect segmentation model and a neural network defect feature extraction model is deployed in the detection device. The neural network defect segmentation model is used for realizing the function of detecting the known defects, namely detecting the surface defects of the known defect types, normal data representation features are stored in the detection equipment, and the neural network defect feature extraction model and the normal data representation features are combined for realizing the function of detecting the unknown defects, namely detecting the surface defects of the unknown defect types.

The process of defect detection is known to include: the detection equipment inputs the image shot by the industrial camera in real time into the neural network defect segmentation model, and outputs the detection result of the known defect if the surface defect is detected, namely outputs the defect type of the surface defect existing in the image to be detected. If no surface defects are detected, unknown defect detection is performed.

The process of unknown defect detection includes: the detection equipment inputs the image shot by the industrial camera in real time into the neural network defect feature extraction model and outputs real-time image features, namely the image features of the image to be detected. And then, the detection equipment compares the feature similarity of the real-time image features with the normal data representation features and outputs an unknown defect detection result. For example, if the similarity between the real-time image feature and the normal data representation feature is smaller than the similarity threshold, the unknown defect detection result indicates that the image has surface defects of unknown defect types, and the defect positions are marked on the image. If the similarity between the real-time image characteristics and the normal data representation characteristics is larger than or equal to the similarity threshold value, the unknown defect detection result is that the image has no surface defects, namely the image is a normal image without surface defects, and the object corresponding to the image is a normal object.

Therefore, in the embodiment of the application, the detection equipment can preliminarily detect the surface defects of the known defect types through the neural network defect segmentation model, and further detect whether the surface defects of the unknown defect types exist in the image to be detected through the neural network defect feature extraction model and the normal data representation feature, so that the scheme can effectively detect the surface defects of the unknown defect types, and the detection accuracy is improved.

The above describes a process of detecting a surface defect by a detection device according to a detection model and normal data representation characteristics, wherein the detection model comprises a neural network defect segmentation model and a neural network defect characteristic extraction model. It should be noted that, in the embodiment of the present application, the neural network defect segmentation model deployed in the detection device is a deep learning model after training, and an implementation manner of obtaining the neural network defect segmentation model through training is described next.

In the embodiment of the application, the detection device acquires a first data set, and a neural network defect segmentation model is obtained through training according to the first data set. The first data set comprises a defect image of a known defect type and corresponding first labeling information, the first labeling information is labeling information representing the defect type, the defect image is an image with surface defects, the first data set further comprises an image without surface defects and corresponding second labeling information, and the second labeling information is labeling information representing no surface defects.

That is, a part of defect images with known defect types are obtained, another part of normal images without surface defects are obtained, the normal images are images without surface defects, the known defect types of the defect images are labeled to obtain first labeling information corresponding to the defect images, the first labeling information is corresponding to the defect types, and the normal images are labeled without surface defects.

For example, assuming that the size of the image in the embodiment of the present application is 100 × 100 pixels, the labeling information corresponding to the defect image and the normal image is also represented by a matrix of 100 × 100, for example, an all-zero matrix of 100 × 100 represents that there is no surface defect, that is, the labeling information corresponding to the normal image is an all-zero matrix of 100 × 100, and a non-all-zero matrix of 100 × 100 represents that there is a surface defect, that is, the labeling information corresponding to the defect image is a non-all-zero matrix of 100 × 100. That is, in the embodiment of the present application, an all-zero matrix having the same size as an image is used as annotation information of a normal image, and a non-all-zero matrix having the same size as the image is used as annotation information of a defective image.

Different values are used for representing different defect types in the non-all-zero matrix, for example, a value of 1 represents a first type of known defect, a value of 2 represents a second type of known defect, and a value of 3 represents a third type of known defect. In addition, the position of the non-zero element in the non-all-zero matrix is the local position of the image with the surface defect, for example, the first row element of the non-all-zero matrix is non-zero, which indicates that the position of the first row pixel in the image has the surface defect. If the value of the first 3 elements in the first row of the non-all-zero matrix is 1, it indicates that the first 3 pixels in the first row of pixels in the image are located with the first type of known defects.

Based on the foregoing example, optionally, the first detection result is represented in a matrix of the same size as the image, and the first detection result may be referred to as a result matrix. The detection equipment inputs the image to be detected into the neural network defect feature extraction model, outputs a result matrix with the same size as the image, and can determine whether the surface defect of the known defect type exists in the image to be detected and the defect position according to the result matrix. For example, if the result matrix is a non-all-zero matrix, the detection device determines that the image to be detected has surface defects of known defect types, and determines the defect types and defect positions of the surface defects in the image to be detected according to the positions and values of the elements of non-all-zero in the result matrix.

In the embodiment of the present application, the neural network defect feature extraction model deployed in the detection device is also a deep learning model after training, and an implementation manner of obtaining the neural network defect feature extraction model through training is described next.

In the embodiment of the application, the detection device acquires the second data set, and trains to obtain the neural network defect feature extraction model according to the second data set. The second data set comprises images of known object types and corresponding third annotation information, and the third annotation information is annotation information representing the object types.

Illustratively, the detection device obtains a large number of public data sets with object class labels as a second data set, and trains a neural network defect feature extraction model according to the second data set comprising a large number of data, wherein the richer the data included in the second data set, the better the effect of the trained neural network defect feature extraction model in extracting image features is.

One implementation way for obtaining the neural network defect feature extraction model by the training of the detection equipment is as follows: firstly, the detection device builds a neural network classification model based on the convolutional neural network, the built neural network classification model comprises a convolutional layer, a pooling layer, a full-link layer, a softmax layer and the like, the detection device trains the neural network classification model according to a second data set, and the output of the trained neural network classification model is the object class. Because the neural network classification model calculates and extracts image features layer by layer in the forward reasoning process, and finally outputs object categories through the full connection layer and the softmax layer, in order to obtain the neural network defect feature extraction model capable of outputting the image features, the detection equipment removes the last full connection layer and the softmax layer of the trained neural network classification model, and the neural network defect feature extraction model capable of outputting the image features is obtained.

The embodiments of the present application do not limit the deep learning technique and the deep learning framework used for constructing the neural network defect feature extraction model.

Fig. 4 is a schematic diagram of a method for training a detection model according to an embodiment of the present application, where the detection model includes a neural network segmentation model and a neural network defect feature extraction model. Referring to fig. 4, the training process of the detection model includes collecting training data, constructing and training a neural network.

The process of collecting training data includes: the industrial camera sends the acquired image data to computer equipment, the computer equipment is detection equipment, the image data of the defective object is labeled with a defect label through the detection equipment, namely, the defect image of the known defect type is labeled, and labeling information corresponding to the defect image is obtained, wherein the labeling information is the corresponding defect type. The image data of the normal object is marked as a normal image without surface defects through the detection equipment, namely, the normal image without surface defects is marked, so that marking information corresponding to the normal image is obtained, and the marking information is free of surface defects.

The process of constructing and training the neural network comprises the following steps: a segmentation model (an initialized neural network defect segmentation model) is built through detection equipment, and the segmentation model is trained according to training data collected by an industrial camera to obtain the trained neural network defect segmentation model. A classification model (initialized neural network classification model) is established through detection equipment, the classification model is trained according to a large number of public data sets with class labels, and a trained neural network defect feature extraction model is obtained.

Two implementations of the detection device determining the normal data representation characteristic are described next.

The first implementation mode comprises the following steps: the detection device acquires at least one first sample image, the at least one first sample image is free of surface defects, the detection device inputs the at least one first sample image into a neural network defect feature extraction model and outputs image features of the at least one first sample image, and the detection device takes the image features of the at least one first sample image as normal data representation features.

That is, the detection device acquires image data of some normal objects, extracts image features of the image data through a neural network defect feature extraction model, and takes the extracted image features as normal data representation features.

Exemplarily, as can be seen from the foregoing, if the first data set includes normal images without surface defects, then the detection device may acquire at least one normal image from the first data set as at least one first sample image. For example, the detection device randomly selects a certain proportion or a certain number of images from the normal images included in the first data set as at least one first sample image, or the detection device takes the normal images included in the first data set as the first sample images.

That is, the detection device selects a normal image from the first data set, from which image features need to be extracted, and then extracts image features from the selected normal image as normal data representation features.

Optionally, the detection device inputs all normal images included in the first data set into the neural network defect feature extraction model, and outputs image features of all normal images included in the first data set. Then, the detection device randomly selects a certain proportion or a certain number of image features from the image features of all the normal images included in the first data set as the normal data representation features, or the detection device takes the image features of all the normal images included in the first data set as the normal data representation features.

That is, the detection device extracts image features from all normal images included in the first data set, and then randomly selects a certain proportion or all of the extracted image features as normal data representation features.

The second implementation mode comprises the following steps: the detection device acquires a plurality of second sample images, the second sample images are free of surface defects, the detection device inputs the second sample images into a neural network defect feature extraction model, and image features of the second sample images are output. And clustering the image characteristics of the plurality of second sample images by the detection equipment to obtain a plurality of groups of normal data characteristics, and then selecting at least one image characteristic from each group of normal data characteristics in the plurality of groups of normal data characteristics to obtain normal data representation characteristics.

That is, the detection device obtains image data of some normal objects, extracts image features of the image data through a neural network defect feature extraction model, clusters the extracted image features to obtain multiple groups of normal data features, and selects a certain proportion or quantity of image features from each group of normal data features as normal data representation features. Thus, the normal data representation characteristics obtained by the detection equipment are rich in types and are more representative.

Illustratively, as can be seen from the foregoing, the first data set includes normal images without surface defects, and then the detection apparatus may take all the normal images included in the first data set as a plurality of second sample images, extract image features from the plurality of second sample images, and cluster all the extracted image features to obtain a plurality of groups of normal representation features. And the detection equipment selects a certain number or a certain proportion of image characteristics from each group of normal data characteristics as normal data representation characteristics.

Of course, the detection device may also select a certain proportion or a certain number of normal images from the first data set as a plurality of second sample images, and then obtain the normal data representation features through feature extraction, clustering and screening.

As can be seen from the foregoing description of the first implementation manner and the second implementation manner for determining the normal data representing features, assuming that the detecting device is the normal data representing features determined according to the normal images included in the first data set, the detecting device may extract image features from all the normal images included in the first data set, and then, according to a screening mechanism, screen and determine the normal data representing features from all the extracted image features. Wherein, the screening mechanism is as follows: randomly selecting a certain proportion or a certain number of image features, or selecting all image features, or clustering all image features, and selecting image features of a certain proportion or certain data from each class of image features.

It should be noted that there are many clustering algorithms that are used by the detection device to cluster the image features, for example, a K-means clustering algorithm, a mean shift clustering algorithm, a density-based clustering algorithm, a hierarchy-based clustering algorithm, and the like, which are not limited in the embodiment of the present application.

In the embodiment of the present application, the process of determining the normal data representation feature by the detection device may be understood as a process of modeling the normal data feature, fig. 5 is a schematic diagram of a method for providing normal data feature modeling according to the embodiment of the present application, and the process is described again with reference to fig. 5.

Referring to fig. 5, the inspection apparatus inputs image data of a normal object into the neural network defect feature extraction model and outputs normal data features, for example, inputs all normal images included in the first data set into the neural network defect feature extraction model and outputs image features of all normal images. Then, the detection device selects a screening mechanism to screen the normal data characteristics from the normal data characteristics to obtain normal data representation characteristics.

In summary, in the embodiment of the present application, if it is determined that the surface defect of the known defect type does not exist in the image to be detected through the neural network defect segmentation model detection, the image feature of the image to be detected is extracted through the neural network defect feature extraction model to obtain the feature to be compared, and if the similarity between the feature to be compared and the normal data representation feature is small, it is determined that the surface defect of the unknown defect type exists in the image to be detected.

All the above optional technical solutions can be combined arbitrarily to form an optional embodiment of the present application, and the present application embodiment is not described in detail again.

Fig. 6 is a schematic structural diagram of a surface defect detecting apparatus 600 according to an embodiment of the present application, where the surface defect detecting apparatus 600 may be implemented as part or all of a computer device by software, hardware, or a combination of the two, and the computer device may be the detecting device in the foregoing embodiments. Referring to fig. 6, the apparatus 600 includes: a first obtaining module 601, a detecting module 602, a first processing module 603, and a first determining module 604.

A first obtaining module 601, configured to obtain an image to be detected;

the detection module 602 is configured to input the image to be detected into a neural network defect segmentation model, and output a first detection result, where the neural network defect segmentation model is used to detect a surface defect of a known defect type;

the first processing module 603 is configured to, if the first detection result is that the image to be detected does not have surface defects of a known defect type, input the image to be detected into the neural network defect feature extraction model, and output a feature to be compared, where the feature to be compared is an image feature of the image to be detected;

the first determining module 604 is configured to determine that the image to be detected has a surface defect of an unknown defect type if the similarity between the feature to be compared and the normal data representing feature is smaller than a similarity threshold, where the normal data representing feature is generated based on an image feature of an image without a surface defect.

Optionally, the apparatus 600 further comprises:

and the second determining module is used for determining the defect position of the surface defect in the image to be detected according to the similarity between the characteristic to be compared and the normal data representation characteristic and the mapping relation between the image characteristic matrix and the image pixel matrix.

Optionally, the apparatus 600 further comprises:

the second acquisition module is used for acquiring a first data set, wherein the first data set comprises a defect image of a known defect type and corresponding first marking information, the first marking information is marking information representing the defect type, the first data set also comprises an image without surface defects and corresponding second marking information, and the second marking information is marking information representing no surface defects;

and the first training module is used for training to obtain a neural network defect segmentation model according to the first data set.

Optionally, the apparatus further comprises:

the third acquisition module is used for acquiring a second data set, wherein the second data set comprises an image of a known object type and corresponding third annotation information, and the third annotation information is annotation information representing the object type;

and the second training module is used for training to obtain a neural network defect feature extraction model according to the second data set.

Optionally, the apparatus 600 further comprises:

the fourth acquisition module is used for acquiring at least one first sample image, and the at least one first sample image has no surface defect;

the second processing module is also used for inputting the at least one first sample image into the neural network defect feature extraction model and outputting the image features of the at least one first sample image;

and the third determining module is used for taking the image characteristics of at least one first sample image as normal data representation characteristics.

Optionally, the apparatus 600 further comprises:

the fifth acquisition module is used for acquiring a plurality of second sample images, and the second sample images have no surface defects;

the clustering module is used for clustering the image characteristics of the second sample images to obtain multiple groups of normal data characteristics;

and the fourth determining module is used for selecting at least one image feature from each group of normal data features in the multiple groups of normal data features to obtain normal data representation features.

Optionally, the apparatus 600 further comprises:

In the embodiment of the application, if the surface defects of the known defect types do not exist in the image to be detected through the neural network defect segmentation model detection, the image characteristics of the image to be detected are extracted through the neural network defect characteristic extraction model to obtain the characteristics to be compared, and if the similarity between the characteristics to be compared and the normal data representation characteristics is small, the surface defects of the unknown defect types exist in the image to be detected.

It should be noted that: in the surface defect detecting apparatus provided in the above embodiment, when detecting a surface defect, only the division of the functional modules is exemplified, and in practical applications, the function distribution may be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the functions described above. In addition, the surface defect detection apparatus provided in the above embodiments and the surface defect detection method embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments and are not described herein again.

Fig. 7 shows a block diagram of a terminal 700 according to an exemplary embodiment of the present application. The terminal 700 may be: a smartphone, a tablet, a laptop, or a desktop computer. Terminal 700 may also be referred to by other names such as user equipment, portable terminal, laptop terminal, desktop terminal, and so on. Optionally, the terminal 700 is the detection device in the above embodiment.

In general, terminal 700 includes: a processor 701 and a memory 702.

The processor 701 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so on. The processor 701 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 701 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 701 may be integrated with a GPU (Graphics Processing Unit) which is responsible for rendering and drawing the content required to be displayed by the display screen. In some embodiments, the processor 701 may further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

Memory 702 may include one or more computer-readable storage media, which may be non-transitory. Memory 702 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 702 is used to store at least one instruction for execution by processor 701 to implement a surface defect detection method provided by method embodiments herein.

In some embodiments, the terminal 700 may further optionally include: a peripheral interface 703 and at least one peripheral. The processor 701, the memory 702, and the peripheral interface 703 may be connected by buses or signal lines. Various peripheral devices may be connected to peripheral interface 703 via a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 704, a display screen 705, a camera assembly 706, an audio circuit 707, a positioning component 708, and a power source 709.

The peripheral interface 703 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 701 and the memory 702. In some embodiments, processor 701, memory 702, and peripheral interface 703 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 701, the memory 702, and the peripheral interface 703 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

The Radio Frequency circuit 704 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuitry 704 communicates with communication networks and other communication devices via electromagnetic signals. The rf circuit 704 converts an electrical signal into an electromagnetic signal to transmit, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 704 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth. The radio frequency circuitry 704 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocols include, but are not limited to: metropolitan area networks, various generation mobile communication networks (2G, 3G, 4G, and 5G), Wireless local area networks, and/or WiFi (Wireless Fidelity) networks. In some embodiments, the radio frequency circuit 704 may further include NFC (Near Field Communication) related circuits, which are not limited in this application.

The display screen 705 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display screen 705 is a touch display screen, the display screen 705 also has the ability to capture touch signals on or over the surface of the display screen 705. The touch signal may be input to the processor 701 as a control signal for processing. At this point, the display 705 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display 705 may be one, disposed on a front panel of the terminal 700; in other embodiments, the display 705 can be at least two, respectively disposed on different surfaces of the terminal 700 or in a folded design; in other embodiments, the display 705 may be a flexible display disposed on a curved surface or a folded surface of the terminal 700. Even more, the display 705 may be arranged in a non-rectangular irregular pattern, i.e. a shaped screen. The Display 705 may be made of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or the like.

The camera assembly 706 is used to capture images or video. Optionally, camera assembly 706 includes a front camera and a rear camera. Generally, a front camera is disposed at a front panel of the terminal, and a rear camera is disposed at a rear surface of the terminal. In some embodiments, the number of the rear cameras is at least two, and each rear camera is any one of a main camera, a depth-of-field camera, a wide-angle camera and a telephoto camera, so that the main camera and the depth-of-field camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize panoramic shooting and VR (Virtual Reality) shooting functions or other fusion shooting functions. In some embodiments, camera assembly 706 may also include a flash. The flash lamp can be a monochrome temperature flash lamp or a bicolor temperature flash lamp. The double-color-temperature flash lamp is a combination of a warm-light flash lamp and a cold-light flash lamp, and can be used for light compensation at different color temperatures.

The audio circuitry 707 may include a microphone and a speaker. The microphone is used for collecting sound waves of a user and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 701 for processing or inputting the electric signals to the radio frequency circuit 704 to realize voice communication. The microphones may be provided in plural numbers, respectively, at different portions of the terminal 700 for the purpose of stereo sound collection or noise reduction. The microphone may also be an array microphone or an omni-directional pick-up microphone. The speaker is used to convert electrical signals from the processor 701 or the radio frequency circuit 704 into sound waves. The loudspeaker can be a traditional film loudspeaker or a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, the speaker can be used for purposes such as converting an electric signal into a sound wave audible to a human being, or converting an electric signal into a sound wave inaudible to a human being to measure a distance. In some embodiments, the audio circuitry 707 may also include a headphone jack.

The positioning component 708 is used to locate the current geographic Location of the terminal 700 for navigation or LBS (Location Based Service). The Positioning component 708 can be a Positioning component based on the GPS (Global Positioning System) in the united states, the beidou System in china, the graves System in russia, or the galileo System in the european union.

Power supply 709 is provided to supply power to various components of terminal 700. The power source 709 may be alternating current, direct current, disposable batteries, or rechargeable batteries. When power source 709 includes a rechargeable battery, the rechargeable battery may support wired or wireless charging. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, terminal 700 also includes one or more sensors 710. The one or more sensors 710 include, but are not limited to: acceleration sensor 711, gyro sensor 712, pressure sensor 713, fingerprint sensor 714, optical sensor 715, and proximity sensor 716.

The acceleration sensor 711 can detect the magnitude of acceleration in three coordinate axes of a coordinate system established with the terminal 700. For example, the acceleration sensor 711 may be used to detect components of the gravitational acceleration in three coordinate axes. The processor 701 may control the display screen 705 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 711. The acceleration sensor 711 may also be used for acquisition of motion data of a game or a user.

The gyro sensor 712 may detect a body direction and a rotation angle of the terminal 700, and the gyro sensor 712 may cooperate with the acceleration sensor 711 to acquire a 3D motion of the terminal 700 by the user. From the data collected by the gyro sensor 712, the processor 701 may implement the following functions: motion sensing (such as changing the UI according to a user's tilting operation), image stabilization at the time of photographing, game control, and inertial navigation.

Pressure sensors 713 may be disposed on a side frame of terminal 700 and/or underneath display screen 705. When the pressure sensor 713 is disposed on a side frame of the terminal 700, a user's grip signal on the terminal 700 may be detected, and the processor 701 performs right-left hand recognition or shortcut operation according to the grip signal collected by the pressure sensor 713. When the pressure sensor 713 is disposed at a lower layer of the display screen 705, the processor 701 controls the operability control on the UI interface according to the pressure operation of the user on the display screen 705. The operability control comprises at least one of a button control, a scroll bar control, an icon control and a menu control.

The fingerprint sensor 714 is used for collecting a fingerprint of a user, and the processor 701 identifies the identity of the user according to the fingerprint collected by the fingerprint sensor 714, or the fingerprint sensor 714 identifies the identity of the user according to the collected fingerprint. When the user identity is identified as a trusted identity, the processor 701 authorizes the user to perform relevant sensitive operations, including unlocking a screen, viewing encrypted information, downloading software, paying, changing settings, and the like. The fingerprint sensor 714 may be disposed on the front, back, or side of the terminal 700. When a physical button or a vendor Logo is provided on the terminal 700, the fingerprint sensor 714 may be integrated with the physical button or the vendor Logo.

The optical sensor 715 is used to collect the ambient light intensity. In one embodiment, the processor 701 may control the display brightness of the display screen 705 based on the ambient light intensity collected by the optical sensor 715. Specifically, when the ambient light intensity is high, the display brightness of the display screen 705 is increased; when the ambient light intensity is low, the display brightness of the display screen 705 is adjusted down. In another embodiment, processor 701 may also dynamically adjust the shooting parameters of camera assembly 706 based on the ambient light intensity collected by optical sensor 715.

A proximity sensor 716, also referred to as a distance sensor, is typically disposed on a front panel of the terminal 700. The proximity sensor 716 is used to collect the distance between the user and the front surface of the terminal 700. In one embodiment, when the proximity sensor 716 detects that the distance between the user and the front surface of the terminal 700 gradually decreases, the processor 701 controls the display 705 to switch from the bright screen state to the dark screen state; when the proximity sensor 716 detects that the distance between the user and the front surface of the terminal 700 is gradually increased, the processor 701 controls the display 705 to switch from the breath-screen state to the bright-screen state.

Those skilled in the art will appreciate that the configuration shown in fig. 7 is not intended to be limiting of terminal 700 and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components may be used.

Fig. 8 is a schematic diagram illustrating a server structure of a surface defect detecting apparatus according to an exemplary embodiment. The server 800 may be a server in a background server cluster, and the server 800 may be the detection device in the above embodiment. Specifically, the method comprises the following steps:

the server 800 includes a Central Processing Unit (CPU)801, a system memory 804 including a Random Access Memory (RAM)802 and a Read Only Memory (ROM)803, and a system bus 805 connecting the system memory 804 and the central processing unit 801. The server 800 also includes a basic input/output system (I/O system) 806 for facilitating information transfer between various devices within the computer, and a mass storage device 807 for storing an operating system 813, application programs 814, and other program modules 815.

The basic input/output system 806 includes a display 808 for displaying information and an input device 809 such as a mouse, keyboard, etc. for user input of information. Wherein a display 808 and an input device 809 are connected to the central processing unit 801 through an input output controller 810 connected to the system bus 805. The basic input/output system 806 may also include an input/output controller 810 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, input-output controller 810 also provides output to a display screen, a printer, or other type of output device.

The mass storage device 807 is connected to the central processing unit 801 through a mass storage controller (not shown) connected to the system bus 805. The mass storage device 807 and its associated computer-readable media provide non-volatile storage for the server 800. That is, the mass storage device 807 may include a computer-readable medium (not shown) such as a hard disk or CD-ROM drive.

Without loss of generality, computer readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, EPROM, EEPROM, flash memory or other solid state memory technology, CD-ROM, DVD, or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media is not limited to the foregoing. The system memory 804 and mass storage 807 described above may be collectively referred to as memory.

According to various embodiments of the present application, server 800 may also operate as a remote computer connected to a network through a network, such as the Internet. That is, the server 800 may be connected to the network 812 through the network interface unit 811 coupled to the system bus 805, or may be connected to other types of networks or remote computer systems (not shown) using the network interface unit 811.

The memory further includes one or more programs, and the one or more programs are stored in the memory and configured to be executed by the CPU. The one or more programs include instructions for performing the surface defect detection methods provided by embodiments of the present application.

In some embodiments, a computer-readable storage medium is also provided, in which a computer program is stored, which, when being executed by a processor, carries out the steps of the surface defect detection method in the above-mentioned embodiments. For example, the computer readable storage medium may be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

It is noted that the computer-readable storage medium referred to in the embodiments of the present application may be a non-volatile storage medium, in other words, a non-transitory storage medium.

It should be understood that all or part of the steps to implement the above embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The computer instructions may be stored in the computer readable storage medium described above.

That is, in some embodiments, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the steps of the surface defect detection method described above.

It is to be understood that reference herein to "at least one" means one or more and "a plurality" means two or more. In the description of the embodiments of the present application, "/" means "or" unless otherwise specified, for example, a/B may mean a or B; "and/or" herein is merely an association relationship describing an associated object, and means that there may be three relationships, for example, a and/or B, and may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, in order to facilitate clear description of technical solutions of the embodiments of the present application, in the embodiments of the present application, terms such as "first" and "second" are used to distinguish the same items or similar items having substantially the same functions and actions. Those skilled in the art will appreciate that the terms "first," "second," etc. do not denote any order or quantity, nor do the terms "first," "second," etc. denote any order or importance.

The above-mentioned embodiments are provided not to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method of surface defect inspection, the method comprising:

acquiring an image to be detected;

2. The method according to claim 1, wherein after determining that the image to be detected has surface defects of unknown defect type, the method further comprises:

3. The method according to claim 1, wherein before inputting the image to be detected into the neural network defect segmentation model and outputting the first detection result, the method further comprises:

4. The method according to claim 1, wherein before inputting the image to be detected into a neural network defect feature extraction model and outputting the feature to be compared, the method further comprises:

5. The method according to any one of claims 1 to 4, wherein before determining that the image to be detected has the surface defect of the unknown defect type if the similarity between the feature to be compared and the normal data representation feature is smaller than the similarity threshold, the method further comprises:

6. The method according to any one of claims 1 to 4, wherein before determining that the image to be detected has the surface defect of the unknown defect type if the similarity between the feature to be compared and the normal data representation feature is smaller than the similarity threshold, the method further comprises:

inputting the plurality of second sample images into the neural network defect feature extraction model, and outputting image features of the plurality of second sample images;

7. The method according to any one of claims 1 to 4, wherein after inputting the image to be detected into a neural network defect feature extraction model and outputting the feature to be compared, the method further comprises:

8. A surface defect inspection apparatus, comprising:

the first acquisition module is used for acquiring an image to be detected;

9. The apparatus of claim 8, further comprising:

10. The apparatus of claim 8, further comprising:

the second acquisition module is used for acquiring a first data set, wherein the first data set comprises a defect image of a known defect type and corresponding first labeling information, the first labeling information is labeling information representing the defect type, the first data set further comprises an image without surface defects and corresponding second labeling information, the labeling information corresponding to the image without surface defects, which is included in the first data set, is surface defect-free, and the second labeling information is labeling information representing the surface defect-free;

11. A surface defect detection system, characterized in that the surface defect detection system comprises a camera and at least one processor;

the at least one processor is configured to obtain the image to be detected and to implement the steps of the method according to any one of claims 1 to 7.

12. The system of claim 11, wherein the surface defect inspection system further comprises a conveyor for transporting the object to be inspected