WO2022127919A1 - 表面缺陷检测方法、装置、***、存储介质及程序产品 - Google Patents

表面缺陷检测方法、装置、***、存储介质及程序产品 Download PDF

Info

Publication number
WO2022127919A1
WO2022127919A1 PCT/CN2021/139333 CN2021139333W WO2022127919A1 WO 2022127919 A1 WO2022127919 A1 WO 2022127919A1 CN 2021139333 W CN2021139333 W CN 2021139333W WO 2022127919 A1 WO2022127919 A1 WO 2022127919A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
defect
feature
detected
neural network
Prior art date
Application number
PCT/CN2021/139333
Other languages
English (en)
French (fr)
Inventor
张营营
钟巧勇
谢迪
浦世亮
Original Assignee
杭州海康威视数字技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州海康威视数字技术股份有限公司 filed Critical 杭州海康威视数字技术股份有限公司
Priority to EP21905842.7A priority Critical patent/EP4266244A4/en
Publication of WO2022127919A1 publication Critical patent/WO2022127919A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G06T7/001—Industrial image inspection using an image reference approach
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection

Definitions

  • the embodiments of the present application relate to the technical field of deep learning, and in particular, to a surface defect detection method, device, system, storage medium, and program product.
  • Surface defects refer to the defects on the appearance of objects. Surface defects have the characteristics of various types, changing shapes, unstable positions, and diverse background textures. Surface defect detection is an important part of quality control in the industrial field, requiring the accuracy of surface defect detection. The higher the better.
  • the deep learning neural network in the related art can only detect surface defects of known defect types, but cannot effectively detect new types of defects encountered in actual detection, that is, surface defects of unknown defect types.
  • Embodiments of the present application provide a surface defect detection method, device, system, storage medium and program product, which can detect surface defects of unknown defect types and improve detection accuracy.
  • the technical solution is as follows:
  • a surface defect detection method comprising:
  • the neural network defect segmentation model Inputting the to-be-detected image into a neural network defect segmentation model, and outputting a first detection result, the neural network defect segmentation model is used to detect surface defects of known defect categories;
  • the image to be inspected is input into a neural network defect feature extraction model, and the feature to be compared is output, and the feature to be compared is output. is the image feature of the image to be detected;
  • the similarity between the to-be-compared feature and the normal data representation feature is less than the similarity threshold, it is determined that the to-be-detected image has a surface defect of an unknown defect category, and the normal data representation feature is based on an image without surface defects generated from image features.
  • the method further includes:
  • the defect position of the surface defect existing in the image to be detected is determined.
  • the method before inputting the image to be detected into a neural network defect segmentation model and outputting the first detection result, the method further includes:
  • first data set includes defect images of known defect categories and corresponding first annotation information, where the first annotation information is annotation information representing the defect category, and the first data set further includes an image without surface defects and corresponding second annotation information, where the second annotation information is annotation information indicating no surface defects;
  • the neural network defect segmentation model is obtained by training.
  • the method before inputting the image to be detected into a neural network defect feature extraction model and outputting the feature to be compared, the method further includes:
  • the second data set includes images of known object categories and corresponding third annotation information, where the third annotation information is annotation information representing the object category;
  • the neural network defect feature extraction model is obtained by training.
  • the method before determining that the to-be-detected image has a surface defect of an unknown defect category if the similarity between the feature to be compared and the normal data representation feature is less than a similarity threshold, the method further includes:
  • the image features of the at least one first sample image are used as the normal data representation features.
  • the method before determining that the to-be-detected image has a surface defect of an unknown defect category if the similarity between the feature to be compared and the normal data representation feature is less than a similarity threshold, the method further includes:
  • At least one image feature is selected from each group of normal data features in the plurality of sets of normal data features to obtain the normal data representation feature.
  • the method further includes:
  • the similarity between the feature to be compared and the normal data representation feature is greater than or equal to the similarity threshold, it is determined that the image to be detected has no surface defects.
  • a surface defect detection device comprising:
  • a first acquisition module used for acquiring an image to be detected
  • a detection module configured to input the image to be detected into a neural network defect segmentation model, and output a first detection result, and the neural network defect segmentation model is used to detect surface defects of known defect categories;
  • a first processing module configured to input the to-be-detected image into a neural network defect feature extraction model if the first detection result is that the to-be-detected image does not have surface defects of the known defect category, and output the to-be-compared image feature, the feature to be compared is the image feature of the image to be detected;
  • the first determination module is configured to determine that the to-be-detected image has a surface defect of an unknown defect category if the similarity between the feature to be compared and the normal data representation feature is less than a similarity threshold, and the normal data representation feature is Generated based on image features of surface defect-free images.
  • the device further includes:
  • the second determination module is configured to determine the surface existing in the image to be detected according to the similarity between the feature to be compared and the feature represented by the normal data, and the mapping relationship between the image feature matrix and the image pixel matrix The defect location of the defect.
  • the device further includes:
  • the second acquisition module is configured to acquire a first data set, where the first data set includes defect images of known defect categories and corresponding first annotation information, where the first annotation information is annotation information indicating the defect category, and the The first data set further includes images without surface defects and corresponding second annotation information, where the second annotation information is defect information indicating no surface defects;
  • a first training module configured to obtain the neural network defect segmentation model by training according to the first data set.
  • the device further includes:
  • a third acquisition module configured to acquire a second data set, where the second data set includes images of known object categories and corresponding third annotation information, where the third annotation information is annotation information representing the object category;
  • the second training module is configured to obtain the neural network defect feature extraction model by training according to the second data set.
  • the device further includes:
  • a fourth acquisition module configured to acquire at least one first sample image, wherein the at least one first sample image has no surface defects
  • the second processing module is further configured to input the at least one first sample image into the neural network defect feature extraction model, and output the image feature of the at least one first sample image;
  • the third determining module is configured to use the image feature of the at least one first sample image as the normal data representation feature.
  • the device further includes:
  • a fifth acquisition module configured to acquire a plurality of second sample images, wherein the plurality of second sample images have no surface defects
  • a third processing module configured to input the plurality of second sample images into the neural network defect feature extraction model, and output the image features of the plurality of second sample images
  • a clustering module configured to perform clustering on the image features of the plurality of second sample images to obtain multiple sets of normal data features
  • the fourth determining module is configured to select at least one image feature from each group of normal data features in the multiple groups of normal data features to obtain the normal data representation feature.
  • the device further includes:
  • a fifth determination module configured to determine that the to-be-detected image has no surface defects if the similarity between the feature to be compared and the normal data representation feature is greater than or equal to the similarity threshold.
  • a surface defect detection system comprising a camera and at least one processor;
  • the camera is used for photographing at least one surface of the object to be detected as an image to be detected
  • the at least one processor is configured to acquire the to-be-detected image and implement the steps of the above-mentioned surface defect detection method.
  • the surface defect detection system further comprises a conveying device for conveying the object to be inspected;
  • the camera is used for photographing the object to be detected during the transmission of the object to be detected by the conveying device.
  • a computer device in another aspect, includes a processor, a communication interface, a memory and a communication bus, and the processor, the communication interface and the memory communicate with each other through the communication bus , the memory is used for storing a computer program, and the processor is used for executing the program stored in the memory, so as to realize the steps of the above-mentioned surface defect detection method.
  • a computer-readable storage medium is provided, and a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned surface defect detection method are implemented.
  • a computer program product comprising instructions which, when run on a computer, cause the computer to perform the steps of the above-described surface defect detection method.
  • the image features of the image to be inspected are extracted through the neural network defect feature extraction model to obtain the image to be compared. If the similarity between the feature to be compared and the feature represented by normal data is small, it is determined that there are surface defects of unknown defect category in the image to be detected, that is, this scheme can detect surface defects of unknown defect category, which improves the detection accuracy.
  • FIG. 1 is a system architecture diagram of a surface defect detection system provided by an embodiment of the present application.
  • FIG. 2 is a flowchart of a method for detecting surface defects provided by an embodiment of the present application
  • FIG. 3 is a flowchart of another surface defect detection method provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a method for training a detection model provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a method for modeling normal data features provided by an embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of a surface defect detection device provided by an embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of a terminal provided by an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of a server provided by an embodiment of the present application.
  • FIG. 1 is a system architecture diagram of a surface defect detection system provided by an embodiment of the present application.
  • the surface defect detection system is used to implement the surface defect detection method provided by the embodiment of the present application.
  • the system includes a camera 101 and at least one processor 102 .
  • the at least one processor 102 is a processor in a computer device, and the camera 101 and the at least one processor 102 in the computer device are connected wirelessly or wiredly for communication.
  • the camera 101 is configured to capture an image of the object to be detected, and send it to the at least one processor 102 as the image to be detected.
  • the computer device acts as an inspection device and performs surface defect inspection through the at least one processor 102 .
  • the at least one processor 102 is a processor in a camera 101, and the camera 101 is used as a detection device to capture an image of the object to be detected, and use it as an image to be detected to perform surface defect detection.
  • the system further includes a conveying device for conveying the object to be detected.
  • a conveying device for conveying the object to be detected.
  • the camera 101 photographs the object to be inspected, and detects surface defects through the inspection device.
  • Conveying devices such as conveyor belts, transfer vehicles, conveyors, etc., objects to be detected such as factory-produced metal parts, glassware, paper and other products. In this way, in the process of conveying the object to be detected, the object with surface defects is detected, that is, the defective product is detected, and the product quality is guaranteed.
  • FIG. 2 is a flow chart of a surface defect detection method provided by an embodiment of the present application, which is introduced by taking the application of the method to surface defect detection equipment (referred to as detection equipment for short) as an example. Please refer to FIG. 2 , the method includes the following steps.
  • Step 201 Acquire an image to be detected.
  • the detection device acquires an image to be detected, and the image to be detected refers to an image of an object that needs to be detected for surface defects. For example, images of industrially produced utensils, paper, etc.
  • the detection device stores the image to be detected, or the detection device receives the image to be detected sent by other devices, for example, the industrial camera sends the image of the collected object to the detection device, and the detection device uses the received image as the image to be detected. .
  • Step 202 Input the image to be detected into a neural network defect segmentation model, and output a first detection result, and the neural network defect segmentation model is used to detect surface defects of known defect categories.
  • a neural network defect segmentation model is deployed in the detection device.
  • the neural network defect segmentation model is a deep learning model, and the neural network defect segmentation model is used to detect surface defects of known defect categories.
  • the detection device inputs the image to be detected into the neural network defect segmentation model, and outputs the first detection result.
  • the image to be inspected has surface defects of known defect types
  • the first detection result is that the image to be inspected has surface defects of known defect types.
  • the first detection result is that the image to be inspected does not have surface defects of known defect types.
  • Step 203 If the first detection result is that the image to be detected does not have surface defects of known defect categories, the image to be detected is input into the neural network defect feature extraction model, and the feature to be compared is output, and the feature to be compared is the image feature of the image to be detected. .
  • a neural network defect feature extraction model is also deployed in the detection device.
  • the neural network defect feature extraction model is a deep learning model, and the neural network defect feature extraction model is used to extract image features. If the first detection result is that the image to be inspected does not have surface defects of known defect types, the inspection device inputs the image to be inspected into the neural network defect feature extraction model, and outputs the feature to be compared, and the feature to be compared is the image feature of the image to be inspected.
  • Step 204 If the similarity between the feature to be compared and the normal data representation feature is less than the similarity threshold, it is determined that the image to be inspected has surface defects of an unknown defect category, and the normal data representation feature is generated based on the image features of the image without surface defects. of.
  • the detection device stores normal data representation features, and the normal data representation features are generated based on image features of images without surface defects.
  • the normal data representation features include a neural network defect feature extraction model for Image features obtained from image processing of surface defects. After the detection device extracts the features to be compared through the neural network defect feature extraction model, it calculates the similarity between the features to be compared and the features represented by the normal data. If the similarity between the features to be compared and the features represented by the normal data is less than the similarity threshold, Then the inspection device determines that the image to be inspected has surface defects of unknown defect type.
  • the normal data representation features include one or more image features of images without surface defects.
  • the detection device calculates the similarity between the feature to be compared and each image feature included in the normal data representation feature, and obtains one or more similarities, if at least one of the one or more similarities is smaller than the first similarity If the threshold is set, the inspection device determines that the image to be inspected has surface defects of unknown defect types. Alternatively, if the average value of the one or more similarities is less than the second similarity threshold, the inspection device determines that the image to be inspected has surface defects of an unknown defect category. Wherein, the first similarity threshold and the second similarity threshold are the same or different.
  • the detection device determines that the image to be detected has no surface defects.
  • the detection device calculates the similarity between the feature to be compared and each image feature included in the normal data representation feature to obtain one or more similarities, if the one or more similarities are all greater than or equal to the first similarity If the degree threshold is exceeded, the inspection device determines that the image to be inspected has no surface defects. Alternatively, if the average value of the one or more similarities is greater than or equal to the second similarity threshold, the inspection apparatus determines that the image to be inspected has no surface defects.
  • the detection device can also detect the position of the unknown defect-type surface defect in the to-be-detected image, that is, the detection determines that the surface defect of the unknown defect type exists. Detect the defect location of surface defects present in the image.
  • the detection device determines the defect position of the image to be detected by comparing the feature to be compared with the normal data representation feature. For example, the detection device determines the defect position of the surface defect existing in the image to be detected according to the similarity between the feature to be compared and the normal data representation feature and the mapping relationship between the image feature matrix and the image pixel matrix.
  • the image pixel matrix refers to a matrix composed of pixel values of the image
  • the image feature matrix is the image pixel matrix obtained by the neural network defect feature extraction model.
  • the matrix obtained by down-sampling that is, the image features extracted by the neural network defect feature extraction model, is expressed as an image feature matrix in the form of a matrix.
  • the neural network defect feature extraction model in the embodiment of the present application is constructed based on a convolutional neural network
  • the extracted image features include the features of C channels
  • the image pixel matrix is a 100*100 matrix
  • the image pixel matrix passes through the neural network.
  • the defect feature extraction model performs quadruple downsampling
  • the image feature matrix is a feature array of 25*25 C-dimensional features.
  • the feature array is a third-order tensor, which can be understood as each position of the 25*25 image feature matrix. is a C-dimensional vector, each position corresponds to a 4*4 area in the image pixel matrix.
  • the first position in the 25*25 image feature matrix corresponds to the first 4*4 area of the 100*100 image pixel matrix. Then, the similarity between the C-dimensional vector of each position in the 25*25 feature matrix to be compared and the C-dimensional vector of the corresponding position in the 25*25 normal data feature matrix determines the corresponding 4* in the image pixel matrix 4 Whether there are surface defects in the area.
  • the detection device calculates the similarity between the elements at the same position in the feature matrix to be compared and the normal data feature matrix. If the similarity between elements at the same position is less than the similarity threshold, that is, the detection device detects the defect feature position, the defect The feature position refers to the position in the feature matrix to be compared where the feature similarity is lower than the similarity threshold. Then, according to the mapping relationship between the image feature matrix and the image pixel matrix, the detection device determines the image pixel position corresponding to the defect feature position from the image pixel matrix corresponding to the image to be compared, as the detected surface of the image to be detected. The defect location of the defect.
  • the feature matrix to be compared refers to the image feature matrix corresponding to the image to be detected
  • the normal data feature matrix refers to the image feature matrix corresponding to the normal data representation feature.
  • the detection device calculates Compare the similarity between the C-dimensional vector at the first position in the feature matrix and the C-dimensional vector at the first position in the normal data feature matrix. If the similarity is less than the similarity threshold, the detection device will compare the The mapping relationship between the pixel matrices determines that the image position where the first 4*4 area in the image pixel matrix corresponding to the image to be detected is located is the defect position.
  • FIG. 3 is a flowchart of another surface defect detection method provided by an embodiment of the present application.
  • the inspection equipment taking the image to be inspected as an example taken by an industrial camera in real time, the inspection equipment has two functions: known defect detection and unknown defect detection.
  • the trained models are deployed in the inspection equipment, including the neural network defect segmentation model and the neural network defect segmentation model.
  • Network defect feature extraction model is used to realize the function of known defect detection, that is, to detect surface defects of known defect categories.
  • the detection equipment also stores normal data representation features, neural network defect feature extraction model and normal data representation features. Combined with functions for the detection of unknown defects, that is, surface defects of unknown defect classes.
  • the process of known defect detection includes: the detection equipment inputs the image captured by the industrial camera in real time into the neural network defect segmentation model. If a surface defect is detected, the known defect detection result is output, that is, the defect category of the surface defect existing in the image to be detected is output. . If no surface defects are detected, an unknown defect detection is performed.
  • the process of unknown defect detection includes: the detection equipment inputs the image captured by the industrial camera in real time into the neural network defect feature extraction model, and outputs the real-time image feature, that is, the image feature of the image to be detected. After that, the detection equipment compares the feature similarity between the real-time image feature and the normal data representation feature, and outputs the unknown defect detection result. For example, if the similarity between the real-time image feature and the normal data representation feature is less than the similarity threshold, the unknown defect detection result is that the image has surface defects of unknown defect category, and the defect location is marked on the image.
  • the unknown defect detection result is that the image has no surface defects, that is, the image is a normal image without surface defects, and the object corresponding to the image for normal objects.
  • the detection device can initially detect the surface defects of known defect categories through the neural network defect segmentation model, and further detect whether the image to be detected is detected through the neural network defect feature extraction model and the normal data representation features.
  • the solution can effectively detect surface defects of unknown defect types, and the detection accuracy is improved.
  • the solution can detect both surface defects of known defect types and surface defects of unknown defect types, which improves the practicability of defect detection.
  • the detection model includes a neural network defect segmentation model and a neural network defect feature extraction model.
  • the neural network defect segmentation model deployed in the detection device is a trained deep learning model.
  • an implementation method for obtaining the neural network defect segmentation model by training is introduced.
  • the neural network defect segmentation model can be obtained by training in the above detection equipment, and the neural network defect segmentation model can also be obtained by training on other computer equipment, which is not limited in the embodiment of the present application.
  • the detection device obtains a first data set, and trains the neural network defect segmentation model according to the first data set.
  • the first dataset includes defect images of known defect categories and corresponding first annotation information
  • the first annotation information is annotation information indicating the defect category
  • the defect image refers to an image with surface defects
  • the first dataset further includes An image without surface defects and corresponding second annotation information, where the second annotation information is annotation information indicating no surface defects.
  • the detection equipment obtains a part of the defect images with known defect categories, and obtains another part of the normal images without surface defects.
  • the first labeling information corresponding to the defect image, the first labeling information is the corresponding defect category, and the normal image is labeled as having no surface defects. It should be noted that, the labeling process of the first data set may be performed on a detection device, or may be performed on other computer devices, which is not limited in this embodiment of the present application.
  • the size of the image in this embodiment of the present application is 100*100 pixels
  • the annotation information corresponding to the defective image and the normal image is also represented by a 100*100 matrix
  • a 100*100 all-zero matrix represents no Surface defects
  • the annotation information corresponding to the normal image is an all-zero matrix of 100*100
  • a non-all-zero matrix of 100*100 indicates the existence of surface defects
  • the annotation information corresponding to the defect image is a non-all-zero matrix of 100*100 matrix. That is, in the embodiment of the present application, an all-zero matrix with the same size as the image is used as the annotation information of the normal image, and the non-all-zero matrix with the same size as the image is used as the annotation information of the defective image.
  • a value of 1 indicates the first type of known defects
  • a value of 2 indicates a second type of known defects
  • a value of 3 indicates that the third type has been Know the flaws.
  • the position of the non-zero elements in the non-all-zero matrix is the local position of the image with surface defects.
  • the elements in the first row of the non-all-zero matrix are non-zero, indicating that the position of the first row of pixels in the image has surface defects. . If the value of the first three elements of the first row in the non-all-zero matrix is 1, it means that the first type of known defects exist at the positions of the first three pixels in the first row of pixels in the image.
  • the first detection result is represented by a matrix of the same size as the image, and the first detection result may be referred to as a result matrix.
  • the inspection equipment inputs the image to be inspected into the neural network defect feature extraction model, it outputs a result matrix with the same size as the image.
  • the inspection equipment can determine whether there are surface defects of known defect types in the image to be inspected, and the location of the defect. .
  • the result matrix is a non-all-zero matrix
  • the inspection device determines that the image to be inspected has surface defects of known defect types, and according to the position and value of the non-zero elements in the result matrix, determine the image to be inspected. Defect type and defect location of surface defects present.
  • the neural network defect feature extraction model deployed in the detection device is also a trained deep learning model.
  • an implementation method for obtaining the neural network defect feature extraction model by training is introduced. It should be noted that, the neural network defect segmentation model can be obtained by training in the above detection equipment, and the neural network defect segmentation model can also be obtained by training on other computer equipment, which is not limited in the embodiment of the present application.
  • the detection device obtains the second data set, and according to the second data set, the neural network defect feature extraction model is obtained by training.
  • the second data set includes images of known object categories and corresponding third annotation information, where the third annotation information is annotation information representing the object category.
  • the detection device obtains a large number of public data sets with object category annotations as the second data set, and obtains a neural network defect feature extraction model by training on the second data set including a large amount of data, and the second data set includes more abundant data. , the better the effect of the trained neural network defect feature extraction model to extract image features.
  • An implementation method of the neural network defect feature extraction model obtained by the detection equipment training is: first, the detection equipment builds a neural network classification model based on the convolutional neural network, and the constructed neural network classification model includes a convolution layer, a pooling layer, and a fully connected layer. , softmax layer, etc., the detection device trains the neural network classification model according to the second data set, and the output of the trained neural network classification model is the object category.
  • the detection equipment Since the neural network classification model calculates and extracts image features layer by layer in the forward reasoning process, and finally outputs the object category through the fully connected layer and the softmax layer, in order to obtain a neural network defect feature extraction model capable of outputting image features, the detection equipment will train The final fully connected layer and softmax layer of the obtained neural network classification model are removed, that is, a neural network defect feature extraction model capable of outputting image features is obtained.
  • the embodiments of the present application do not limit the deep learning technology, the deep learning framework, etc. used for constructing the neural network defect feature extraction model.
  • FIG. 4 is a schematic diagram of a method for training a detection model provided by an embodiment of the present application, wherein the detection model includes a neural network segmentation model and a neural network defect feature extraction model.
  • the training process of the detection model includes collecting training data, building and training a neural network.
  • the process of collecting training data includes: the industrial camera sends the collected image data to the computer equipment, the computer equipment is the detection equipment, and the image data of the defective object is marked with defect labels by the detection equipment, that is, the defects of known defect categories are marked. image to obtain the annotation information corresponding to the defect image, and the annotation information is the corresponding defect category.
  • the image data of the normal object is marked as having no surface defects by the detection device, that is, a normal image without surface defects is marked, and the annotation information corresponding to the normal image is obtained, and the annotation information is no surface defects.
  • the process of constructing and training the neural network is: constructing a segmentation model (initialized neural network defect segmentation model) through the detection equipment, training the segmentation model according to the training data collected by the industrial camera, and obtaining the trained neural network defect segmentation model.
  • the classification model (initialized neural network classification model) is constructed by the detection equipment, and the classification model is trained according to a large number of public data sets with category annotations, and the trained neural network defect feature extraction model is obtained.
  • the first implementation manner the detection device acquires at least one first sample image, the at least one first sample image has no surface defects, the detection device inputs the at least one first sample image into a neural network defect feature extraction model, and outputs the at least one first sample image. Image features of at least one first sample image, and the detection device uses the image features of the at least one first sample image as normal data to represent features.
  • the detection device acquires image data of some normal objects, extracts image features of these image data through a neural network defect feature extraction model, and uses the extracted image features as normal data representation features.
  • the detection device may acquire at least one normal image from the first data set as at least one first sample image. For example, the detection device randomly selects a certain proportion or a certain number of images from the normal images included in the first data set as at least one first sample image, or the detection device takes all the normal images included in the first data set as the first sample image this image.
  • the detection device first selects a normal image whose image features need to be extracted from the first data set, and then extracts image features from the selected normal images, and uses the extracted image features as normal data representation features.
  • the detection device inputs all normal images included in the first data set into the neural network defect feature extraction model, and outputs image features of all normal images included in the first data set.
  • the detection device randomly selects a certain proportion or a certain number of image features from the image features of all normal images included in the first data set, as the normal data representation features, or the detection device uses all normal images included in the first data set.
  • Image features represent features as normal data.
  • the detection device first extracts image features from all normal images included in the first data set, and then randomly selects a certain proportion or all of the extracted image features as the normal data representation features.
  • the second implementation manner the detection device acquires a plurality of second sample images, the plurality of second sample images have no surface defects, the detection device inputs the plurality of second sample images into the neural network defect feature extraction model, and outputs the plurality of second sample images.
  • the detection device clusters the image features of the plurality of second sample images to obtain multiple sets of normal data features, and then selects at least one image feature from each set of normal data features in the multiple sets of normal data features to obtain normal data represent features.
  • the detection equipment obtains the image data of some normal objects, extracts the image features of these image data through the neural network defect feature extraction model, and clusters the extracted image features to obtain multiple sets of normal data features, and then extracts the image features from each set of normal data features.
  • a certain proportion or a certain number of image features are selected from the data features as normal data representation features. In this way, the normal data obtained by the detection device represents a rich variety of features and is more representative.
  • the inspection device may use all the normal images included in the first data set as multiple second sample images, and for the multiple second sample images Extract image features, and cluster all the extracted image features to obtain multiple groups of normal representation features.
  • the detection device selects a certain number or a certain proportion of image features from each group of normal data features as the normal data representation features.
  • the detection device may first select a certain proportion or a certain number of normal images from the first data set as multiple second sample images, and then obtain the normal data representation features through feature extraction, clustering and screening.
  • the detection device can First, image features are extracted from all normal images included in the first data set, and then, according to a screening mechanism, normal data representation features are determined by screening all the extracted image features.
  • the screening mechanism is: randomly select a certain proportion or a certain number of image features, or select all image features, or cluster all image features, and select a certain proportion or certain data image features from each type of image features .
  • clustering algorithms used by detection equipment to cluster image features such as K-means clustering algorithm, mean-shift clustering algorithm, density-based clustering algorithm, and hierarchical-based clustering algorithm. etc., which are not limited in the embodiments of the present application.
  • FIG. 5 is a schematic diagram of a method for modeling the normal data feature provided by the embodiment of the present application. Next, refer to Figure 5 illustrates this process again.
  • the detection device inputs the image data of the normal object into the neural network defect feature extraction model, and outputs the normal data features, for example, inputs all normal images included in the first data set into the neural network defect feature extraction model, and outputs all normal images. image features. Afterwards, the detection device selects a screening mechanism to obtain normal data representation features by screening the normal data features.
  • the image of the image to be detected is extracted by the neural network defect feature extraction model.
  • feature to obtain the feature to be compared if the similarity between the feature to be compared and the normal data representation feature is small, it is determined that there are surface defects of unknown defect category in the image to be detected, that is, this scheme can detect unknown defect categories. Surface defects, improving the detection accuracy.
  • FIG. 6 is a schematic structural diagram of a surface defect detection apparatus 600 provided in an embodiment of the present application.
  • the surface defect detection apparatus 600 may be implemented by software, hardware, or a combination of the two as part or all of computer equipment.
  • the computer equipment may be The detection device in the above embodiment.
  • the apparatus 600 includes: a first acquisition module 601 , a detection module 602 , a first processing module 603 and a first determination module 604 .
  • the first acquisition module 601 is used to acquire the image to be detected
  • a detection module 602 configured to input the image to be detected into a neural network defect segmentation model, and output a first detection result, and the neural network defect segmentation model is used to detect surface defects of known defect categories;
  • the first processing module 603 is configured to input the image to be detected into the neural network defect feature extraction model if the first detection result is that the image to be detected does not have surface defects of known defect categories, and output the feature to be compared, and the feature to be compared is the feature to be compared. Detect image features of an image;
  • the first determination module 604 is configured to determine that the image to be detected has a surface defect of an unknown defect category if the similarity between the feature to be compared and the normal data representation feature is less than the similarity threshold, and the normal data representation feature is based on the absence of surface defects.
  • the image features of the image are generated.
  • the apparatus 600 further includes:
  • the second determination module is configured to determine the defect position of the surface defect existing in the image to be detected according to the similarity between the feature to be compared and the feature represented by the normal data, and the mapping relationship between the image feature matrix and the image pixel matrix.
  • the apparatus 600 further includes:
  • the second acquisition module is configured to acquire a first data set, where the first data set includes defect images of known defect categories and corresponding first annotation information, where the first annotation information is annotation information indicating the defect category, and the first data set also includes It includes an image without surface defects and corresponding second annotation information, where the second annotation information is annotation information indicating no surface defects;
  • the first training module is used for training a neural network defect segmentation model according to the first data set.
  • the device further includes:
  • a third acquisition module configured to acquire a second dataset, where the second dataset includes images of known object categories and corresponding third annotation information, where the third annotation information is annotation information representing the object category;
  • the second training module is used for training to obtain a neural network defect feature extraction model according to the second data set.
  • the apparatus 600 further includes:
  • a fourth acquisition module configured to acquire at least one first sample image, where the at least one first sample image has no surface defects
  • the second processing module is further configured to input at least one first sample image into the neural network defect feature extraction model, and output image features of at least one first sample image;
  • the third determining module is configured to use the image features of the at least one first sample image as normal data to represent the features.
  • the apparatus 600 further includes:
  • a fifth acquisition module configured to acquire a plurality of second sample images, and the plurality of second sample images have no surface defects
  • a third processing module configured to input multiple second sample images into the neural network defect feature extraction model, and output image features of multiple second sample images
  • a clustering module for clustering image features of multiple second sample images to obtain multiple sets of normal data features
  • the fourth determination module is used for selecting at least one image feature from each group of normal data features in the multiple groups of normal data features to obtain the normal data representation feature.
  • the apparatus 600 further includes:
  • a fifth determination module configured to determine that the image to be detected has no surface defects if the similarity between the feature to be compared and the normal data representation feature is greater than or equal to the similarity threshold.
  • the image feature of the image to be detected is extracted through the neural network defect feature extraction model to obtain the feature to be compared.
  • this scheme can detect surface defects of unknown defect categories, improving the detection performance. accuracy.
  • the surface defect detection device provided by the above embodiment detects surface defects
  • only the division of the above functional modules is used as an example for illustration.
  • the above functions can be allocated to different functional modules as required. , that is, dividing the internal structure of the device into different functional modules to complete all or part of the functions described above.
  • the surface defect detection device provided in the above embodiments and the surface defect detection method embodiments belong to the same concept, and the specific implementation process thereof is detailed in the method embodiments, which will not be repeated here.
  • FIG. 7 shows a structural block diagram of a terminal 700 provided by an exemplary embodiment of the present application.
  • the terminal 700 can be: a smart phone, a tablet computer, a notebook computer or a desktop computer. Terminal 700 may also be called user equipment, portable terminal, laptop terminal, desktop terminal, and the like by other names.
  • the terminal 700 is the detection device in the foregoing embodiment.
  • the terminal 700 includes: a processor 701 and a memory 702 .
  • the processor 701 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like.
  • the processor 701 can use at least one hardware form among DSP (Digital Signal Processing, digital signal processing), FPGA (Field-Programmable Gate Array, field programmable gate array), and PLA (Programmable Logic Array, programmable logic array).
  • the processor 701 may also include a main processor and a coprocessor.
  • the main processor is a processor used to process data in the wake-up state, also called CPU (Central Processing Unit, central processing unit); the coprocessor is A low-power processor for processing data in a standby state.
  • the processor 701 may be integrated with a GPU (Graphics Processing Unit, image processor), and the GPU is responsible for rendering and drawing the content that needs to be displayed on the display screen.
  • the processor 701 may further include an AI (Artificial Intelligence, artificial intelligence) processor, where the AI processor is used to process computing operations related to machine learning.
  • AI Artificial Intelligence, artificial intelligence
  • Memory 702 may include one or more computer-readable storage media, which may be non-transitory. Memory 702 may also include high-speed random access memory, as well as non-volatile memory, such as one or more disk storage devices, flash storage devices. In some embodiments, a non-transitory computer-readable storage medium in the memory 702 is used to store at least one instruction for being executed by the processor 701 to implement the surface defects provided by the method embodiments of the present application Detection method.
  • the terminal 700 may optionally further include: a peripheral device interface 703 and at least one peripheral device.
  • the processor 701, the memory 702 and the peripheral device interface 703 may be connected by a bus or a signal line.
  • Each peripheral device can be connected to the peripheral device interface 703 through a bus, a signal line or a circuit board.
  • the peripheral device includes: at least one of a radio frequency circuit 704 , a display screen 705 , a camera assembly 706 , an audio circuit 707 , a positioning assembly 708 and a power supply 709 .
  • the peripheral device interface 703 may be used to connect at least one peripheral device related to I/O (Input/Output) to the processor 701 and the memory 702 .
  • processor 701, memory 702, and peripherals interface 703 are integrated on the same chip or circuit board; in some other embodiments, any one of processor 701, memory 702, and peripherals interface 703 or The two can be implemented on a separate chip or circuit board, which is not limited in this embodiment.
  • the radio frequency circuit 704 is used for receiving and transmitting RF (Radio Frequency, radio frequency) signals, also called electromagnetic signals.
  • the radio frequency circuit 704 communicates with the communication network and other communication devices via electromagnetic signals.
  • the radio frequency circuit 704 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals.
  • the radio frequency circuit 704 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and the like.
  • the radio frequency circuit 704 may communicate with other terminals through at least one wireless communication protocol.
  • the wireless communication protocols include, but are not limited to, metropolitan area networks, mobile communication networks of various generations (2G, 3G, 4G and 5G), wireless local area networks and/or WiFi (Wireless Fidelity, wireless fidelity) networks.
  • the radio frequency circuit 704 may further include a circuit related to NFC (Near Field Communication, short-range wireless communication), which is not limited in this application.
  • the display screen 705 is used to display UI (User Interface, user interface).
  • the UI can include graphics, text, icons, video, and any combination thereof.
  • the display screen 705 also has the ability to acquire touch signals on or above the surface of the display screen 705 .
  • the touch signal may be input to the processor 701 as a control signal for processing.
  • the display screen 705 may also be used to provide virtual buttons and/or virtual keyboards, also referred to as soft buttons and/or soft keyboards.
  • the display screen 705 may be one display screen 705, which is arranged on the front panel of the terminal 700; in other embodiments, there may be at least two display screens 705, which are respectively arranged on different surfaces of the terminal 700 or in a folded design; In other embodiments, the display screen 705 may be a flexible display screen, which is disposed on a curved surface or a folding surface of the terminal 700 . Even, the display screen 705 can also be set as a non-rectangular irregular figure, that is, a special-shaped screen.
  • the display screen 705 can be prepared by using materials such as LCD (Liquid Crystal Display, liquid crystal display), OLED (Organic Light-Emitting Diode, organic light emitting diode).
  • the camera assembly 706 is used to capture images or video.
  • the camera assembly 706 includes a front camera and a rear camera.
  • the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal.
  • there are at least two rear cameras which are any one of a main camera, a depth-of-field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth-of-field camera to realize the background blur function, the main camera It is integrated with the wide-angle camera to achieve panoramic shooting and VR (Virtual Reality, virtual reality) shooting functions or other integrated shooting functions.
  • camera assembly 706 may also include a flash.
  • the flash can be a single color temperature flash or a dual color temperature flash. Dual color temperature flash refers to the combination of warm light flash and cold light flash, which can be used for light compensation under different color temperatures.
  • Audio circuitry 707 may include a microphone and speakers.
  • the microphone is used to collect the sound waves of the user and the environment, convert the sound waves into electrical signals, and input them to the processor 701 for processing, or to the radio frequency circuit 704 to realize voice communication.
  • the microphone may also be an array microphone or an omnidirectional collection microphone.
  • the speaker is used to convert the electrical signal from the processor 701 or the radio frequency circuit 704 into sound waves.
  • the loudspeaker can be a traditional thin-film loudspeaker or a piezoelectric ceramic loudspeaker.
  • the speaker When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves audible to humans, but also convert electrical signals into sound waves inaudible to humans for distance measurement and other purposes.
  • the audio circuit 707 may also include a headphone jack.
  • the positioning component 708 is used to locate the current geographic location of the terminal 700 to implement navigation or LBS (Location Based Service).
  • the positioning component 708 may be a positioning component based on the GPS (Global Positioning System, global positioning system) of the United States, the Beidou system of China, the Grenas system of Russia, or the Galileo system of the European Union.
  • the power supply 709 is used to power various components in the terminal 700 .
  • the power source 709 may be alternating current, direct current, disposable batteries or rechargeable batteries.
  • the rechargeable battery can support wired charging or wireless charging.
  • the rechargeable battery can also be used to support fast charging technology.
  • the terminal 700 also includes one or more sensors 710 .
  • the one or more sensors 710 include, but are not limited to, an acceleration sensor 711 , a gyro sensor 712 , a pressure sensor 713 , a fingerprint sensor 714 , an optical sensor 715 and a proximity sensor 716 .
  • the acceleration sensor 711 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal 700 .
  • the acceleration sensor 711 can be used to detect the components of the gravitational acceleration on the three coordinate axes.
  • the processor 701 may control the display screen 705 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 711 .
  • the acceleration sensor 711 can also be used for game or user movement data collection.
  • the gyroscope sensor 712 can detect the body direction and rotation angle of the terminal 700 , and the gyroscope sensor 712 can cooperate with the acceleration sensor 711 to collect 3D actions of the user on the terminal 700 .
  • the processor 701 can implement the following functions according to the data collected by the gyro sensor 712: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.
  • the pressure sensor 713 may be disposed on the side frame of the terminal 700 and/or the lower layer of the display screen 705 .
  • the processor 701 can perform left and right hand identification or shortcut operations according to the holding signal collected by the pressure sensor 713.
  • the processor 701 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 705 .
  • the operability controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.
  • the fingerprint sensor 714 is used to collect the user's fingerprint, and the processor 701 identifies the user's identity according to the fingerprint collected by the fingerprint sensor 714 , or the fingerprint sensor 714 identifies the user's identity according to the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 701 authorizes the user to perform related sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings.
  • the fingerprint sensor 714 may be provided on the front, back or side of the terminal 700 . When the terminal 700 is provided with physical buttons or a manufacturer's logo, the fingerprint sensor 714 may be integrated with the physical buttons or the manufacturer's logo.
  • Optical sensor 715 is used to collect ambient light intensity.
  • the processor 701 may control the display brightness of the display screen 705 according to the ambient light intensity collected by the optical sensor 715 . Specifically, when the ambient light intensity is high, the display brightness of the display screen 705 is increased; when the ambient light intensity is low, the display brightness of the display screen 705 is decreased.
  • the processor 701 may also dynamically adjust the shooting parameters of the camera assembly 706 according to the ambient light intensity collected by the optical sensor 715 .
  • a proximity sensor 716 also called a distance sensor, is usually provided on the front panel of the terminal 700 .
  • the proximity sensor 716 is used to collect the distance between the user and the front of the terminal 700 .
  • the processor 701 controls the display screen 705 to switch from the bright screen state to the off screen state; when the proximity sensor 716 detects When the distance between the user and the front of the terminal 700 gradually increases, the processor 701 controls the display screen 705 to switch from the closed screen state to the bright screen state.
  • FIG. 7 does not constitute a limitation on the terminal 700, and may include more or less components than the one shown, or combine some components, or adopt different component arrangements.
  • Fig. 8 is a schematic diagram showing a server structure of a surface defect detection apparatus according to an exemplary embodiment.
  • the server 800 may be a server in a background server cluster, and the server 800 may be the detection device in the above embodiment. Specifically:
  • Server 800 includes central processing unit (CPU) 801 , system memory 804 including random access memory (RAM) 802 and read only memory (ROM) 803 , and a system bus 805 connecting system memory 804 and central processing unit 801 .
  • Server 800 also includes a basic input/output system (I/O system) 806 that facilitates the transfer of information between various components within the computer, and a mass storage device 807 for storing operating system 813, application programs 814, and other program modules 815 .
  • I/O system basic input/output system
  • Basic input/output system 806 includes a display 808 for displaying information and input devices 809 such as a mouse, keyboard, etc., for user input of information. Both the display 808 and the input device 809 are connected to the central processing unit 801 through the input and output controller 810 connected to the system bus 805 .
  • the basic input/output system 806 may also include an input output controller 810 for receiving and processing input from various other devices such as a keyboard, mouse, or electronic stylus. Similarly, input output controller 810 also provides output to a display screen, printer, or other type of output device.
  • Mass storage device 807 is connected to central processing unit 801 through a mass storage controller (not shown) connected to system bus 805 .
  • Mass storage device 807 and its associated computer-readable media provide non-volatile storage for server 800 . That is, mass storage device 807 may include a computer-readable medium (not shown) such as a hard disk or a CD-ROM drive.
  • Computer-readable media can include computer storage media and communication media.
  • Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.
  • Computer storage media include RAM, ROM, EPROM, EEPROM, flash memory or other solid state storage technology, CD-ROM, DVD or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices.
  • RAM random access memory
  • ROM read only memory
  • EPROM Erasable programmable read-only memory
  • EEPROM electrically erasable programmable read-only memory
  • the server 800 may also run on a remote computer connected to a network through a network such as the Internet. That is, the server 800 can be connected to the network 812 through the network interface unit 811 connected to the system bus 805, or can also use the network interface unit 811 to connect to other types of networks or remote computer systems (not shown).
  • the above-mentioned memory also includes one or more programs, and the one or more programs are stored in the memory and configured to be executed by the CPU.
  • the one or more programs include instructions for performing the surface defect detection method provided by the embodiments of the present application.
  • a computer-readable storage medium is also provided, and a computer program is stored in the storage medium, and when the computer program is executed by a processor, the steps of the surface defect detection method in the above-mentioned embodiments are implemented.
  • the computer-readable storage medium may be ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.
  • the computer-readable storage medium mentioned in the embodiments of the present application may be a non-volatile storage medium, in other words, may be a non-transitory storage medium.
  • a computer program product comprising instructions which, when executed on a computer, cause the computer to perform the steps of the above-described surface defect detection method.
  • references herein to "at least one” refers to one or more, and “plurality” refers to two or more.
  • “/” means or means, for example, A/B can mean A or B;
  • "and/or” in this document is only an association that describes an associated object Relation, it means that there can be three kinds of relations, for example, A and/or B can mean that A exists alone, A and B exist at the same time, and B exists alone.
  • words such as “first” and “second” are used to distinguish the same or similar items with basically the same function and effect. Those skilled in the art can understand that the words “first”, “second” and the like do not limit the quantity and execution order, and the words “first”, “second” and the like are not necessarily different.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)

Abstract

本申请实施例公开了一种表面缺陷检测方法、装置、***、存储介质及程序产品，属于深度学习技术领域。在本申请实施例中，如果通过神经网络缺陷分割模型检测确定待检测图像中不存在已知缺陷类别的表面缺陷，那么通过神经网络缺陷特征提取模型来提取待检测图像的图像特征，得到待对比特征，如果待对比特征与正常数据表示特征之间的相似度较小，则确定待检测图像中存在未知缺陷类别的表面缺陷，也即是，本方案能够检测未知缺陷类别的表面缺陷，提高了检测的精准度。

Description

表面缺陷检测方法、装置、***、存储介质及程序产品
本申请实施例要求于2020年12月17日提交的申请号为202011495050.3、发明名称为“表面缺陷检测方法、装置及***”的中国专利申请的优先权，其全部内容通过引用结合在本申请实施例中。
技术领域
本申请实施例涉及深度学习技术领域，特别涉及一种表面缺陷检测方法、装置、***、存储介质及程序产品。
背景技术
表面缺陷是指物体外观上的瑕疵，表面缺陷具有种类繁多、形态多变、位置不固定以及背景纹理多样化等特点，表面缺陷检测是工业领域质量控制的重要环节，要求表面缺陷检测的精准度越高越好。
相关技术中的深度学习神经网络只能检测出已知缺陷类别的表面缺陷，对于实际检测中遇到的新类别的缺陷，也即对于未知缺陷类别的表面缺陷，则无法有效检测出。
发明内容
本申请实施例提供了一种表面缺陷检测方法、装置、***、存储介质及程序产品，能够检测未知缺陷类别的表面缺陷，提高检测的精准度。所述技术方案如下：
一方面，提供了一种表面缺陷检测方法，所述方法包括：
获取待检测图像；
将所述待检测图像输入神经网络缺陷分割模型，输出第一检测结果，所述神经网络缺陷分割模型用于检测已知缺陷类别的表面缺陷；
如果所述第一检测结果为所述待检测图像不存在所述已知缺陷类别的表面缺陷，则将所述待检测图像输入神经网络缺陷特征提取模型，输出待对比特征，所述待对比特征为所述待检测图像的图像特征；
如果所述待对比特征与正常数据表示特征之间的相似度小于相似度阈值，则确定所述待检测图像存在未知缺陷类别的表面缺陷，所述正常数据表示特征是基于无表面缺陷的图像的图像特征生成的。
可选地，所述确定所述待检测图像存在未知缺陷类别的表面缺陷之后，还包括：
根据所述待对比特征与所述正常数据表示特征之间的相似度，以及图像特征矩阵与图像像素矩阵之间的映射关系，确定所述待检测图像中存在的表面缺陷的缺陷位置。
可选地，所述将所述待检测图像输入神经网络缺陷分割模型，输出第一检测结果之前，还包括：
获取第一数据集，所述第一数据集包括已知缺陷类别的缺陷图像以及对应的第一标注信息，所述第一标注信息为表示缺陷类别的标注信息，所述第一数据集还包括无表面缺陷的图像以及对应的第二标注信息，所述第二标注信息为表示无表面缺陷的标注信息；
根据所述第一数据集，训练得到所述神经网络缺陷分割模型。
可选地，所述将所述待检测图像输入神经网络缺陷特征提取模型，输出待对比特征之前，还包括：
获取第二数据集，所述第二数据集包括已知物体类别的图像以及对应的第三标注信息，所述第三标注信息为表示物体类别的标注信息；
根据所述第二数据集，训练得到所述神经网络缺陷特征提取模型。
可选地，所述如果所述待对比特征与正常数据表示特征之间的相似度小于相似度阈值，则确定所述待检测图像存在未知缺陷类别的表面缺陷之前，还包括：
获取至少一个第一样本图像，所述至少一个第一样本图像无表面缺陷；
将所述至少一个第一样本图像输入所述神经网络缺陷特征提取模型，输出所述至少一个第一样本图像的图像特征；
将所述至少一个第一样本图像的图像特征作为所述正常数据表示特征。
可选地，所述如果所述待对比特征与正常数据表示特征之间的相似度小于相似度阈值，则确定所述待检测图像存在未知缺陷类别的表面缺陷之前，还包括：
获取多个第二样本图像，所述多个第二样本图像无表面缺陷；
将所述多个第二样本图像输入所述神经网络缺陷特征提取模型，输出所述多个第二样本图像的图像特征；
对所述多个第二样本图像的图像特征进行聚类，得到多组正常数据特征；
从所述多组正常数据特征中的每组正常数据特征中选择至少一个图像特征，得到所述正常数据表示特征。
可选地，所述将所述待检测图像输入神经网络缺陷特征提取模型，输出待对比特征之后，还包括：
如果所述待对比特征与所述正常数据表示特征之间的相似度大于或等于所述相似度阈值，则确定所述待检测图像无表面缺陷。
另一方面，提供了一种表面缺陷检测装置，所述装置包括：
第一获取模块，用于获取待检测图像；
检测模块，用于将所述待检测图像输入神经网络缺陷分割模型，输出第一检测结果，所述神经网络缺陷分割模型用于检测已知缺陷类别的表面缺陷；
第一处理模块，用于如果所述第一检测结果为所述待检测图像不存在所述已知缺陷类别的表面缺陷，则将所述待检测图像输入神经网络缺陷特征提取模型，输出待对比特征，所述待对比特征为所述待检测图像的图像特征；
第一确定模块，用于如果所述待对比特征与正常数据表示特征之间的相似度小于相似度阈值，则确定所述待检测图像存在未知缺陷类别的表面缺陷，所述正常数据表示特征是基于无表面缺陷的图像的图像特征生成的。
可选地，所述装置还包括：
第二确定模块，用于根据所述待对比特征与所述正常数据表示特征之间的相似度，以及图像特征矩阵与图像像素矩阵之间的映射关系，确定所述待检测图像中存在的表面缺陷的缺陷位置。
可选地，所述装置还包括：
第二获取模块，用于获取第一数据集，所述第一数据集包括已知缺陷类别的缺陷图像以及对应的第一标注信息，所述第一标注信息为表示缺陷类别的标注信息，所述第一数据集还包括无表面缺陷的图像以及对应的第二标注信息，所述第二标注信息为表示无表面缺陷的缺陷信息；
第一训练模块，用于根据所述第一数据集，训练得到所述神经网络缺陷分割模型。
可选地，所述装置还包括：
第三获取模块，用于获取第二数据集，所述第二数据集包括已知物体类别的图像以及对应的第三标注信息，所述第三标注信息为表示物体类别的标注信息；
第二训练模块，用于根据所述第二数据集，训练得到所述神经网络缺陷特征提取模型。
可选地，所述装置还包括：
第四获取模块，用于获取至少一个第一样本图像，所述至少一个第一样本图像无表面缺陷；
第二处理模块，还用于将所述至少一个第一样本图像输入所述神经网络缺陷特征提取模型，输出所述至少一个第一样本图像的图像特征；
第三确定模块，用于将所述至少一个第一样本图像的图像特征作为所述正常数据表示特征。
可选地，所述装置还包括：
第五获取模块，用于获取多个第二样本图像，所述多个第二样本图像无表面缺陷；
第三处理模块，用于将所述多个第二样本图像输入所述神经网络缺陷特征提取模型，输出所述多个第二样本图像的图像特征；
聚类模块，用于对所述多个第二样本图像的图像特征进行聚类，得到多组正常数据特征；
第四确定模块，用于从所述多组正常数据特征中的每组正常数据特征中选择至少一个图像特征，得到所述正常数据表示特征。
可选地，所述装置还包括：
第五确定模块，用于如果所述待对比特征与所述正常数据表示特征之间的相似度大于或等于所述相似度阈值，则确定所述待检测图像无表面缺陷。
另一方面，提供了一种表面缺陷检测***，所述表面缺陷检测***包括相机和至少一个处理器；
所述相机，用于拍摄待检测物体的至少一个表面，作为待检测图像；
所述至少一个处理器，用于获取所述待检测图像，实现上述所述表面缺陷检测方法的步骤。
可选地，所述表面缺陷检测***还包括传送装置，所述传送装置用于传输所述待检测物体；
所述相机用于在所述传送装置传输所述待检测物体的过程中，拍摄所述待检测物体。
另一方面，提供了一种计算机设备，所述计算机设备包括处理器、通信接口、存储器和通信总线，所述处理器、所述通信接口和所述存储器通过所述通信总线完成相互间的通信，所述存储器用于存放计算机程序，所述处理器用于执行所述存储器上所存放的程序，以实现上述所述表面缺陷检测方法的步骤。
另一方面，提供了一种计算机可读存储介质，所述计算机可读存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现上述所述表面缺陷检测方法的步骤。
另一方面，提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述所述的表面缺陷检测方法的步骤。
本申请实施例提供的技术方案至少可以带来以下有益效果：
在本申请实施例中，如果通过神经网络缺陷分割模型检测确定待检测图像中不存在已知缺陷类别的表面缺陷，那么通过神经网络缺陷特征提取模型来提取待检测图像的图像特征，得到待对比特征，如果待对比特征与正常数据表示特征之间的相似度较小，则确定待检测图像中存在未知缺陷类别的表面缺陷，也即是，本方案能够检测未知缺陷类别的表面缺陷，提高了检测的精准度。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的一种表面缺陷检测***的***架构图；
图2是本申请实施例提供的一种表面缺陷检测方法的流程图；
图3是本申请实施例提供的另一种表面缺陷检测方法的流程图；
图4是本申请实施例提供的一种训练检测模型的方法示意图；
图5是本申请实施例提供的一种正常数据特征建模的方法示意图；
图6是本申请实施例提供的一种表面缺陷检测装置的结构示意图；
图7是本申请实施例提供的一种终端的结构示意图；
图8是本申请实施例提供的一种服务器的结构示意图。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。
首先对本申请实施例提供的表面缺陷检测方法所涉及的***架构进行介绍。
图1是本申请实施例提供的一种表面缺陷检测***的***架构图，该表面缺陷检测***用于实现本申请实施例提供的表面缺陷检测方法。参见图1，该***包括相机101和至少一个处理器102。
该至少一个处理器102为计算机设备中的处理器，相机101和该计算机设备中的该至少一个处理器102以无线或有线方式连接以进行通信。相机101用于拍摄待检测物体的图像，作为待检测图像，发送给该至少一个处理器102。该计算机设备作为检测设备，通过该至少一个处理器102进行表面缺陷检测。
在一些实施例中，该至少一个处理器102为相机101中的处理器，相机101作为检测设备，拍摄待检测物体的图像，作为待检测图像，进行表面缺陷检测。
可选地，该***还包括传送装置，传送装置用于传输待检测物体。在一种场景中，传送装置在传输待检测物体的过程中，相机101拍摄待检测物体，通过检测设备检测表面缺陷。传送装置如传送带、传输车、传输机等，待检测物体如工厂生产的金属零件、玻璃器具、纸张等产品。这样，在传输待检测物体的过程中，检测存在表面缺陷的物体，也即检出残次品，保证产品质量。
接下来对本申请实施例提供的表面缺陷检测方法进行详细的解释说明。
图2是本申请实施例提供的一种表面缺陷检测方法的流程图，以该方法应用于表面缺陷检测设备(简称为检测设备)为例进行介绍。请参考图2，该方法包括如下步骤。
步骤201：获取待检测图像。
在本申请实施例中，检测设备获取待检测图像，待检测图像是指需要检测是否存在表面缺陷的物体的图像。例如，工业生产的器具、纸张等物体的图像。
示例性地，检测设备存储有待检测图像，或者，检测设备接收其他设备发送的待检测图像，例如，工业相机将采集的物体的图像发送给检测设备，检测设备将接收到的图像作为待检测图像。
步骤202：将待检测图像输入神经网络缺陷分割模型，输出第一检测结果，神经网络缺陷分割模型用于检测已知缺陷类别的表面缺陷。
在本申请实施例中，检测设备中部署有神经网络缺陷分割模型，神经网络缺陷分割模型是一种深度学习模型，神经网络缺陷分割模型用于检测已知缺陷类别的表面缺陷。检测设备将待检测图像输入神经网络缺陷分割模型，输出第一检测结果。
示例性地，假设待检测图像存在已知缺陷类别的表面缺陷，则第一检测结果为该待检测图像存在已知缺陷类别的表面缺陷。假设待检测图像不存在已知缺陷类别的表面缺陷，则第一检测结果为该待检测图像不存在已知缺陷类别的表面缺陷。
步骤203：如果第一检测结果为待检测图像不存在已知缺陷类别的表面缺陷，则将待检测图像输入神经网络缺陷特征提取模型，输出待对比特征，待对比特征为待检测图像的图像特征。
在本申请实施例中，检测设备中还部署有神经网络缺陷特征提取模型，神经网络缺陷特征提取模型是一种深度学习模型，神经网络缺陷特征提取模型用于提取图像特征。如果第一检测结果为待检测图像不存在已知缺陷类别的表面缺陷，则检测设备将待检测图像输入神经网络缺陷特征提取模型，输出待对比特征，待对比特征为待检测图像的图像特征。
步骤204：如果待对比特征与正常数据表示特征之间的相似度小于相似度阈值，则确定待检测图像存在未知缺陷类别的表面缺陷，正常数据表示特征是基于无表面缺陷的图像的图像特征生成的。
在本申请实施例中，检测设备中存储有正常数据表示特征，正常数据表示特征是基于无表面缺陷的图像的图像特征生成的，例如，正常数据表示特征包括通过神经网络缺陷特征提取模型对无表面缺陷的图像处理得到的图像特征。检测设备通过神经网络缺陷特征提取模型提取出待对比特征之后，计算待对比 特征与正常数据表示特征之间的相似度，如果待对比特征与正常数据表示特征之间的相似度小于相似度阈值，则检测设备确定该待检测图像存在未知缺陷类别的表面缺陷。
其中，计算待对比特征与正常数据表示特征之间的相似度的方法有很多，例如采用特征之间的欧式距离相似度计算方法、采用特征之间的马氏距相似度计算方法、采用特征之间的余弦距离相似度计算方法等，本申请实施例对此不作限定。
在本申请实施例中，正常数据表示特征包括一个或多个无表面缺陷的图像的图像特征。检测设备计算待对比特征与正常数据表示特征包括的每个图像特征之间的相似度，得到一个或多个相似度，如果该一个或多个相似度中的至少一个相似度小于第一相似度阈值，则检测设备确定待检测图像存在未知缺陷类别的表面缺陷。或者，如果该一个或多个相似度的平均值小于第二相似度阈值，则检测设备确定待检测图像存在未知缺陷类别的表面缺陷。其中，第一相似度阈值和第二相似度阈值相同或不同。
在本申请实施例中，如果待对比特征与正常数据表示特征之间的相似度大于或等于相似度阈值，则检测设备确定待检测图像无表面缺陷。
示例性地，检测设备计算待对比特征与正常数据表示特征包括的每个图像特征之间的相似度，得到一个或多个相似度，如果该一个或多个相似度均大于或等于第一相似度阈值，则检测设备确定待检测图像无表面缺陷。或者，如果该一个或多个相似度的平均值大于或等于第二相似度阈值，则检测设备确定待检测图像无表面缺陷。
可选地，在本申请实施例中，如果待检测图像存在未知缺陷类别的表面缺陷，那么检测设备还能够检测出待检测图像中存在的未知缺陷类别的表面缺陷的位置，也即检测确定待检测图像中存在的表面缺陷的缺陷位置。
在本申请实施例中，检测设备通过对比待对比特征与正常数据表示特征，确定待检测图像的缺陷位置。例如，检测设备根据待对比特征与正常数据表示特征之间的相似度，以及图像特征矩阵与图像像素矩阵之间的映射关系，确定待检测图像中存在的表面缺陷的缺陷位置。
在一种实现方式中，图像特征矩阵与图像像素矩阵之间存在一定的映射关系，图像像素矩阵是指图像的像素值组成的矩阵，图像特征矩阵是通过神经网络缺陷特征提取模型对图像像素矩阵下采样得到的矩阵，也即神经网络缺陷特 征提取模型提取出的图像特征以矩阵形式表示为图像特征矩阵。
示例性地，假设本申请实施例中神经网络缺陷特征提取模型基于卷积神经网络构建，提取的图像特征包括C个通道的特征，图像像素矩阵为100*100的矩阵，图像像素矩阵通过神经网络缺陷特征提取模型进行四倍下采样后得到图像特征矩阵为25*25个C维特征的特征阵列，该特征阵列为一个三阶张量，可以理解为25*25的图像特征矩阵的每个位置为一个C维向量，每个位置对应图像像素矩阵中的一个4*4区域。例如，25*25的图像特征矩阵中第一个位置对应100*100的图像像素矩阵的第一个4*4区域。那么，25*25的待对比特征矩阵中每个位置的C维向量与25*25的正常数据特征矩阵中对应位置的C维向量之间的相似度，决定了图像像素矩阵中对应的4*4区域是否存在表面缺陷。
检测设备计算待对比特征矩阵与正常数据特征矩阵中相同位置的元素之间的相似度，如果某相同位置的元素之间的相似度小于相似度阈值，也即检测设备检测到缺陷特征位置，缺陷特征位置是指待对比特征矩阵中特征相似度低于相似度阈值的位置。然后，检测设备根据图像特征矩阵与图像像素矩阵之间的映射关系，从待对比图像对应的图像像素矩阵中，确定与缺陷特征位置对应的图像像素位置，作为检测到的待检测图像存在的表面缺陷的缺陷位置。其中，待对比特征矩阵是指待检测图像对应的图像特征矩阵，正常数据特征矩阵是指正常数据表示特征对应的图像特征矩阵。
示例性地，假设待检测图像对应的图像像素矩阵的大小为100*100，待对比特征矩阵与正常数据特征矩阵的均为25*25个C维特征的三阶张量，检测设备计算出待对比特征矩阵中第一个位置的C维向量与正常数据特征矩阵中第一个位置的C维向量之间的相似度，如果该相似度小于相似度阈值，则检测设备根据图像特征矩阵与图像像素矩阵之间的映射关系，确定待检测图像对应的图像像素矩阵中第一个4*4区域所在的图像位置即为缺陷位置。
图3是本申请实施例提供的另一种表面缺陷检测方法的流程图。参见图3，以待检测图像为工业相机实时拍摄的图像为例，检测设备具有已知缺陷检测和未知缺陷检测两个功能，检测设备中部署训练好的模型，包括神经网络缺陷分割模型和神经网络缺陷特征提取模型。其中，神经网络缺陷分割模型用于实现已知缺陷检测的功能，也即检测已知缺陷类别的表面缺陷，检测设备中还存储有正常数据表示特征，神经网络缺陷特征提取模型和正常数据表示特征结合用于实现未知缺陷检测的功能，也即检测未知缺陷类别的表面缺陷。
已知缺陷检测的过程包括：检测设备将工业相机实时拍摄的图像输入神经网络缺陷分割模型，如果检测到表面缺陷，输出已知缺陷检测结果，也即输出待检测图像存在的表面缺陷的缺陷类别。如果未检测到表面缺陷，则进行未知缺陷检测。
未知缺陷检测的过程包括：检测设备将工业相机实时拍摄的该图像输入神经网络缺陷特征提取模型，输出实时图像特征，也即输出待检测图像的图像特征。之后，检测设备将实时图像特征与正常数据表示特征进行特征相似度对比，输出未知缺陷检测结果。例如，如果实时图像特征与正常数据表示特征之间的相似度小于相似度阈值，则未知缺陷检测结果为该图像存在未知缺陷类别的表面缺陷，并在该图像上标出缺陷位置。如果实时图像特征与正常数据表示特征之间的相似度大于或等于相似度阈值，则未知缺陷检测结果为该图像无表面缺陷，也即该图像为无表面缺陷的正常图像，该图像对应的物体为正常物体。
由上述可知，在本申请实施例中，检测设备能够通过神经网络缺陷分割模型初步检测已知缺陷类别的表面缺陷，通过神经网络缺陷特征提取模型以及正常数据表示特征来进一步检测待检测图像中是否存在未知缺陷类别的表面缺陷，也即本方案能够有效检测未知缺陷类别的表面缺陷，提高了检测的精准度。且本方案即能检测已知缺陷类别的表面缺陷，也能检测未知缺陷类别的表面缺陷，提高了缺陷检测的实用性。
以上介绍了检测设备根据检测模型以及正常数据表示特征来检测表面缺陷的过程，其中，检测模型包括神经网络缺陷分割模型和神经网络缺陷特征提取模型。需要说明的是，在本申请实施例中，检测设备中部署的神经网络缺陷分割模型为经过训练的一个深度学习模型，接下来介绍训练得到神经网络缺陷分割模型的一种实现方式。需要说明的是，可以在上述检测设备中训练得到神经网络缺陷分割模型，也可以在其他的计算机设备上训练得到神经网络缺陷分割模型，本申请实施例对此不作限定。
在本申请实施例中，以在检测设备中训练得到神经网络缺陷分割模型为例，检测设备获取第一数据集，根据第一数据集，训练得到神经网络缺陷分割模型。其中，第一数据集包括已知缺陷类别的缺陷图像以及对应的第一标注信息，第一标注信息为表示缺陷类别的标注信息，缺陷图像是指存在表面缺陷的图像，第一数据集还包括无表面缺陷的图像以及对应的第二标注信息，第二标注信息为表示无表面缺陷的标注信息。
也即是，检测设备获取一部分存在已知缺陷类别的缺陷图像，获取另一部分无表面缺陷的正常图像，正常图像是指无表面缺陷的图像，对缺陷图像存在的已知缺陷类别进行标注，得到缺陷图像对应的第一标注信息，第一标注信息为对应的缺陷类别，将正常图像标注为无表面缺陷。需要说明的是，第一数据集的标注过程可以在检测设备上进行，也可以在其他的计算机设备上进行，本申请实施例对此不作限定。
示例性地，假设本申请实施例中的图像大小为100*100像素，缺陷图像和正常图像对应的标注信息也用100*100的矩阵进行表示，例如，以100*100的全零矩阵表示无表面缺陷，也即正常图像对应的标注信息为100*100的全零矩阵，以100*100的非全零矩阵表示存在表面缺陷，也即缺陷图像对应的标注信息为100*100的非全零矩阵。也即是，在本申请实施例中，用与图像大小相同的全零矩阵作为正常图像的标注信息，用与图像大小相同的非全零矩阵作为缺陷图像的标注信息。
其中，在非全零矩阵中以不同的数值表示不同的缺陷类别，例如，数值为1表示第一类已知缺陷、数值为2表示第二类已知缺陷，数值为3表示第三类已知缺陷。另外，非全零矩阵中非零元素所在的位置即为存在表面缺陷的图像局部位置，例如，非全零矩阵的第一行元素非零，表示图像中的第一行像素所在位置存在表面缺陷。如果非全零矩阵中第一行前3个元素的数值为1，表示图像中第一行像素中的前3个像素所在位置存在第一类已知缺陷。
基于前述示例，可选地，第一检测结果以与图像大小相同的矩阵表示，第一检测结果可以称为结果矩阵。检测设备将待检测图像输入神经网络缺陷特征提取模型后，输出与图像大小相同的结果矩阵，检测设备根据该结果矩阵即能确定待检测图像中是否存在已知缺陷类别的表面缺陷，以及缺陷位置。例如，如果该结果矩阵为非全零矩阵，则检测设备确定待检测图像存在已知缺陷类别的表面缺陷，且根据该结果矩阵中非全零的元素所在的位置以及数值，确定待检测图像中存在的表面缺陷的缺陷类别以及缺陷位置。
在本申请实施例中，检测设备中部署的神经网络缺陷特征提取模型也为经过训练的一个深度学习模型，接下来介绍训练得到神经网络缺陷特征提取模型的一种实现方式。需要说明的是，可以在上述检测设备中训练得到神经网络缺陷分割模型，也可以在其他的计算机设备上训练得到神经网络缺陷分割模型，本申请实施例对此不作限定。
在本申请实施例中，以在检测设备中训练得到神经网络缺陷特征提取模型为例，检测设备获取第二数据集，根据第二数据集，训练得到神经网络缺陷特征提取模型。其中，第二数据集包括已知物体类别的图像以及对应的第三标注信息，第三标注信息为表示物体类别的标注信息。
示例性地，检测设备获取大量具有物体类别标注的公开数据集，作为第二数据集，根据包括大量数据的第二数据集训练得到神经网络缺陷特征提取模型，第二数据集包括的数据越丰富，训练得到的神经网络缺陷特征提取模型提取图像特征的效果越好。
检测设备训练得到神经网络缺陷特征提取模型的一种实现方式为：首先，检测设备基于卷积神经网络构建神经网络分类模型，构建的神经网络分类模型包括卷积层、池化层、全连接层、softmax层等，检测设备根据第二数据集训练神经网络分类模型，训练得到的神经网络分类模型的输出是物体类别。由于神经网络分类模型在前向推理过程中逐层计算提取图像特征，最后通过全连接层和softmax层输出物体类别，因此，为了得到能够输出图像特征的神经网络缺陷特征提取模型，检测设备将训练得到的神经网络分类模型的最后的全连接层、softmax层拆除，即得到能够输出图像特征的神经网络缺陷特征提取模型。
需要说明的是，本申请实施例并不限定构建神经网络缺陷特征提取模型所采用的深度学习技术、深度学习框架等。
图4是本申请实施例提供的一种训练检测模型的方法示意图，其中，检测模型包括神经网络分割模型和神经网络缺陷特征提取模型。参见图4，检测模型的训练过程包括采集训练数据、构建及训练神经网络。
采集训练数据的过程包括：工业相机将采集的图像数据发送给计算机设备，计算机设备为检测设备，通过检测设备对有缺陷物体的图像数据标注缺陷标签，也即是，标注已知缺陷类别的缺陷图像，得到缺陷图像对应的标注信息，标注信息为对应的缺陷类别。通过检测设备对正常物体的图像数据标注为无表面缺陷，也即是，标注无表面缺陷的正常图像，得到正常图像对应的标注信息，标注信息为无表面缺陷。
构建及训练神经网络的过程为：通过检测设备构建分割模型(初始化的神经网络缺陷分割模型)，根据工业相机采集的训练数据训练分割模型，得到经过训练的神经网络缺陷分割模型。通过检测设备构建分类模型(初始化的神经网络分类模型)，根据大量具有类别标注的公开数据集训练分类模型，得到经过训 练的神经网络缺陷特征提取模型。
接下来介绍检测设备确定正常数据表示特征两种实现方式。
第一种实现方式：检测设备获取至少一个第一样本图像，该至少一个第一样本图像无表面缺陷，检测设备将该至少一个第一样本图像输入神经网络缺陷特征提取模型，输出该至少一个第一样本图像的图像特征，检测设备将该至少一个第一样本图像的图像特征作为正常数据表示特征。
也即是，检测设备获取一些正常物体的图像数据，通过神经网络缺陷特征提取模型提取这些图像数据的图像特征，将提取的图像特征作为正常数据表示特征。
示例性地，由前述可知，第一数据集中包括无表面缺陷的正常图像，那么检测设备可以从第一数据集中获取至少一个正常图像，作为至少一个第一样本图像。例如，检测设备从第一数据集包括的正常图像中随机选择一定比例或一定数量的图像作为至少一个第一样本图像，或者，检测设备将第一数据集包括的正常图像均作为第一样本图像。
也即是，检测设备先从第一数据集中选择需要提取图像特征的正常图像，再对选择的正常图像提取图像特征，将提取的图像特征作为正常数据表示特征。
可选地，检测设备将第一数据集包括的所有正常图像输入神经网络缺陷特征提取模型，输出第一数据集包括的所有正常图像的图像特征。之后，检测设备从第一数据集包括的所有正常图像的图像特征中随机选择一定比例或一定数量的图像特征，作为正常数据表示特征，或者，检测设备将第一数据集包括的所有正常图像的图像特征作为正常数据表示特征。
也即是，检测设备先对第一数据集包括的所有正常图像提取图像特征，再从提取的所有图像特征中随机选择一定比例或者选择全部，作为正常数据表示特征。
第二种实现方式：检测设备获取多个第二样本图像，该多个第二样本图像无表面缺陷，检测设备将该多个第二样本图像输入神经网络缺陷特征提取模型，输出该多个第二样本图像的图像特征。检测设备对该多个第二样本图像的图像特征进行聚类，得到多组正常数据特征，之后，从该多组正常数据特征中的每组正常数据特征中选择至少一个图像特征，得到正常数据表示特征。
也即是，检测设备获取一些正常物体的图像数据，通过神经网络缺陷特征提取模型提取这些图像数据的图像特征，对提取的图像特征进行聚类，得到多 组正常数据特征，再从每组正常数据特征中选择一定比例或一定数量的图像特征，作为正常数据表示特征。这样，检测设备得到的正常数据表示特征的种类丰富，且更加具有代表性。
示例性地，由前述可知，第一数据集包括无表面缺陷的正常图像，那么检测设备可以将第一数据集包括的所有正常图像作为多个第二样本图像，对该多个第二样本图像提取图像特征，对提取的所有图像特征进行聚类，得到多组正常表示特征。检测设备再从每组正常数据特征中选择一定数量或一定比例的图像特征，作为正常数据表示特征。
当然，检测设备也可以先从第一数据集中选择一定比例或一定数量的正常图像，作为多个第二样本图像，之后再通过特征提取、聚类和筛选，得到正常数据表示特征。
由前述对确定正常数据表示特征的第一种实现方式和第二种实现方式的介绍可知，假设检测设备是根据第一数据集包括的正常图像来确定的正常数据表示特征，那么检测设备均可以先对第一数据集包括的所有正常图像提取图像特征，之后，根据一种筛选机制，从提取的所有图像特征中筛选确定正常数据表示特征。其中，筛选机制为：随机选择一定比例或一定数量的图像特征，或者，选择全部图像特征，或者，对所有图像特征进行聚类，从每一类图像特征中选择一定比例或一定数据的图像特征。
需要说明的是，检测设备对图像特征进行聚类所采用的聚类算法有很多，例如K-均值聚类算法、均值偏移聚类算法、基于密度的聚类算法、基于层次的聚类算法等，本申请实施例对此不作限定。
在本申请实施例中，检测设备确定正常数据表示特征的过程可以理解为正常数据特征建模的过程，图5是本申请实施例提供的一种正常数据特征建模的方法示意图，接下来参照图5对该过程再次进行说明。
参见图5，检测设备将正常物体的图像数据输入神经网络缺陷特征提取模型，输出正常数据特征，例如，将第一数据集包括的所有正常图像输入神经网络缺陷特征提取模型，输出所有正常图像的图像特征。之后，检测设备选择一种筛选机制从正常数据特征中筛选得到正常数据表示特征。
综上所述，在本申请实施例中，如果通过神经网络缺陷分割模型检测确定待检测图像中不存在已知缺陷类别的表面缺陷，那么通过神经网络缺陷特征提取模型来提取待检测图像的图像特征，得到待对比特征，如果待对比特征与正 常数据表示特征之间的相似度较小，则确定待检测图像中存在未知缺陷类别的表面缺陷，也即是，本方案能够检测未知缺陷类别的表面缺陷，提高了检测的精准度。
上述所有可选技术方案，均可按照任意结合形成本申请的可选实施例，本申请实施例对此不再一一赘述。
图6是本申请实施例提供的一种表面缺陷检测装置600的结构示意图，该表面缺陷检测装置600可以由软件、硬件或者两者的结合实现成为计算机设备的部分或者全部，该计算机设备可以为上述实施例中的检测设备。请参考图6，该装置600包括：第一获取模块601、检测模块602、第一处理模块603和第一确定模块604。
第一获取模块601，用于获取待检测图像；
检测模块602，用于将待检测图像输入神经网络缺陷分割模型，输出第一检测结果，神经网络缺陷分割模型用于检测已知缺陷类别的表面缺陷；
第一处理模块603，用于如果第一检测结果为待检测图像不存在已知缺陷类别的表面缺陷，则将待检测图像输入神经网络缺陷特征提取模型，输出待对比特征，待对比特征为待检测图像的图像特征；
第一确定模块604，用于如果待对比特征与正常数据表示特征之间的相似度小于相似度阈值，则确定待检测图像存在未知缺陷类别的表面缺陷，正常数据表示特征是基于无表面缺陷的图像的图像特征生成的。
可选地，该装置600还包括：
第二确定模块，用于根据待对比特征与正常数据表示特征之间的相似度，以及图像特征矩阵与图像像素矩阵之间的映射关系，确定待检测图像中存在的表面缺陷的缺陷位置。
可选地，该装置600还包括：
第二获取模块，用于获取第一数据集，第一数据集包括已知缺陷类别的缺陷图像以及对应的第一标注信息，第一标注信息为表示缺陷类别的标注信息，第一数据集还包括无表面缺陷的图像以及对应的第二标注信息，第二标注信息为表示无表面缺陷的标注信息；
第一训练模块，用于根据第一数据集，训练得到神经网络缺陷分割模型。
可选地，装置还包括：
第三获取模块，用于获取第二数据集，第二数据集包括已知物体类别的图像以及对应的第三标注信息，第三标注信息为表示物体类别的标注信息；
第二训练模块，用于根据第二数据集，训练得到神经网络缺陷特征提取模型。
可选地，该装置600还包括：
第四获取模块，用于获取至少一个第一样本图像，至少一个第一样本图像无表面缺陷；
第二处理模块，还用于将至少一个第一样本图像输入神经网络缺陷特征提取模型，输出至少一个第一样本图像的图像特征；
第三确定模块，用于将至少一个第一样本图像的图像特征作为正常数据表示特征。
可选地，该装置600还包括：
第五获取模块，用于获取多个第二样本图像，多个第二样本图像无表面缺陷；
第三处理模块，用于将多个第二样本图像输入神经网络缺陷特征提取模型，输出多个第二样本图像的图像特征；
聚类模块，用于对多个第二样本图像的图像特征进行聚类，得到多组正常数据特征；
第四确定模块，用于从多组正常数据特征中的每组正常数据特征中选择至少一个图像特征，得到正常数据表示特征。
可选地，该装置600还包括：
第五确定模块，用于如果待对比特征与正常数据表示特征之间的相似度大于或等于相似度阈值，则确定待检测图像无表面缺陷。
在本申请实施例中，如果通过神经网络缺陷分割模型检测确定待检测图像中无已知缺陷类别的表面缺陷，那么通过神经网络缺陷特征提取模型来提取待检测图像的图像特征，得到待对比特征，如果待对比特征与正常数据表示特征之间的相似度较小，则确定待检测图像中存在未知缺陷类别的表面缺陷，也即是，本方案能够检测未知缺陷类别的表面缺陷，提高了检测的精准度。
需要说明的是：上述实施例提供的表面缺陷检测装置在检测表面缺陷时，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将装置的内部结构划分成不同的功能模 块，以完成以上描述的全部或者部分功能。另外，上述实施例提供的表面缺陷检测装置与表面缺陷检测方法实施例属于同一构思，其具体实现过程详见方法实施例，这里不再赘述。
图7示出了本申请一个示例性实施例提供的终端700的结构框图。该终端700可以是：智能手机、平板电脑、笔记本电脑或台式电脑。终端700还可能被称为用户设备、便携式终端、膝上型终端、台式终端等其他名称。可选地，该终端700为上述实施例中的检测设备。
通常，终端700包括有：处理器701和存储器702。
处理器701可以包括一个或多个处理核心，比如4核心处理器、8核心处理器等。处理器701可以采用DSP(Digital Signal Processing，数字信号处理)、FPGA(Field－Programmable Gate Array，现场可编程门阵列)、PLA(Programmable Logic Array，可编程逻辑阵列)中的至少一种硬件形式来实现。处理器701也可以包括主处理器和协处理器，主处理器是用于对在唤醒状态下的数据进行处理的处理器，也称CPU(Central Processing Unit，中央处理器)；协处理器是用于对在待机状态下的数据进行处理的低功耗处理器。在一些实施例中，处理器701可以集成有GPU(Graphics Processing Unit，图像处理器)，GPU用于负责显示屏所需要显示的内容的渲染和绘制。一些实施例中，处理器701还可以包括AI(Artificial Intelligence，人工智能)处理器，该AI处理器用于处理有关机器学习的计算操作。
存储器702可以包括一个或多个计算机可读存储介质，该计算机可读存储介质可以是非暂态的。存储器702还可包括高速随机存取存储器，以及非易失性存储器，比如一个或多个磁盘存储设备、闪存存储设备。在一些实施例中，存储器702中的非暂态的计算机可读存储介质用于存储至少一个指令，该至少一个指令用于被处理器701所执行以实现本申请中方法实施例提供的表面缺陷检测方法。
在一些实施例中，终端700还可选包括有：***设备接口703和至少一个***设备。处理器701、存储器702和***设备接口703之间可以通过总线或信号线相连。各个***设备可以通过总线、信号线或电路板与***设备接口703相连。具体地，***设备包括：射频电路704、显示屏705、摄像头组件706、音频电路707、定位组件708和电源709中的至少一种。
***设备接口703可被用于将I/O(Input/Output，输入/输出)相关的至少一个***设备连接到处理器701和存储器702。在一些实施例中，处理器701、存储器702和***设备接口703被集成在同一芯片或电路板上；在一些其他实施例中，处理器701、存储器702和***设备接口703中的任意一个或两个可以在单独的芯片或电路板上实现，本实施例对此不加以限定。
射频电路704用于接收和发射RF(Radio Frequency，射频)信号，也称电磁信号。射频电路704通过电磁信号与通信网络以及其他通信设备进行通信。射频电路704将电信号转换为电磁信号进行发送，或者，将接收到的电磁信号转换为电信号。可选地，射频电路704包括：天线***、RF收发器、一个或多个放大器、调谐器、振荡器、数字信号处理器、编解码芯片组、用户身份模块卡等等。射频电路704可以通过至少一种无线通信协议来与其它终端进行通信。该无线通信协议包括但不限于：城域网、各代移动通信网络(2G、3G、4G及5G)、无线局域网和/或WiFi(Wireless Fidelity，无线保真)网络。在一些实施例中，射频电路704还可以包括NFC(Near Field Communication，近距离无线通信)有关的电路，本申请对此不加以限定。
显示屏705用于显示UI(User Interface，用户界面)。该UI可以包括图形、文本、图标、视频及其它们的任意组合。当显示屏705是触摸显示屏时，显示屏705还具有采集在显示屏705的表面或表面上方的触摸信号的能力。该触摸信号可以作为控制信号输入至处理器701进行处理。此时，显示屏705还可以用于提供虚拟按钮和/或虚拟键盘，也称软按钮和/或软键盘。在一些实施例中，显示屏705可以为一个，设置在终端700的前面板；在另一些实施例中，显示屏705可以为至少两个，分别设置在终端700的不同表面或呈折叠设计；在另一些实施例中，显示屏705可以是柔性显示屏，设置在终端700的弯曲表面上或折叠面上。甚至，显示屏705还可以设置成非矩形的不规则图形，也即异形屏。显示屏705可以采用LCD(Liquid Crystal Display，液晶显示屏)、OLED(Organic Light-Emitting Diode,有机发光二极管)等材质制备。
摄像头组件706用于采集图像或视频。可选地，摄像头组件706包括前置摄像头和后置摄像头。通常，前置摄像头设置在终端的前面板，后置摄像头设置在终端的背面。在一些实施例中，后置摄像头为至少两个，分别为主摄像头、景深摄像头、广角摄像头、长焦摄像头中的任意一种，以实现主摄像头和景深摄像头融合实现背景虚化功能、主摄像头和广角摄像头融合实现全景拍摄以及 VR(Virtual Reality，虚拟现实)拍摄功能或者其它融合拍摄功能。在一些实施例中，摄像头组件706还可以包括闪光灯。闪光灯可以是单色温闪光灯，也可以是双色温闪光灯。双色温闪光灯是指暖光闪光灯和冷光闪光灯的组合，可以用于不同色温下的光线补偿。
音频电路707可以包括麦克风和扬声器。麦克风用于采集用户及环境的声波，并将声波转换为电信号输入至处理器701进行处理，或者输入至射频电路704以实现语音通信。出于立体声采集或降噪的目的，麦克风可以为多个，分别设置在终端700的不同部位。麦克风还可以是阵列麦克风或全向采集型麦克风。扬声器则用于将来自处理器701或射频电路704的电信号转换为声波。扬声器可以是传统的薄膜扬声器，也可以是压电陶瓷扬声器。当扬声器是压电陶瓷扬声器时，不仅可以将电信号转换为人类可听见的声波，也可以将电信号转换为人类听不见的声波以进行测距等用途。在一些实施例中，音频电路707还可以包括耳机插孔。
定位组件708用于定位终端700的当前地理位置，以实现导航或LBS(Location Based Service，基于位置的服务)。定位组件708可以是基于美国的GPS(Global Positioning System，全球定位***)、中国的北斗***、俄罗斯的格雷纳斯***或欧盟的伽利略***的定位组件。
电源709用于为终端700中的各个组件进行供电。电源709可以是交流电、直流电、一次性电池或可充电电池。当电源709包括可充电电池时，该可充电电池可以支持有线充电或无线充电。该可充电电池还可以用于支持快充技术。
在一些实施例中，终端700还包括有一个或多个传感器710。该一个或多个传感器710包括但不限于：加速度传感器711、陀螺仪传感器712、压力传感器713、指纹传感器714、光学传感器715以及接近传感器716。
加速度传感器711可以检测以终端700建立的坐标系的三个坐标轴上的加速度大小。比如，加速度传感器711可以用于检测重力加速度在三个坐标轴上的分量。处理器701可以根据加速度传感器711采集的重力加速度信号，控制显示屏705以横向视图或纵向视图进行用户界面的显示。加速度传感器711还可以用于游戏或者用户的运动数据的采集。
陀螺仪传感器712可以检测终端700的机体方向及转动角度，陀螺仪传感器712可以与加速度传感器711协同采集用户对终端700的3D动作。处理器701根据陀螺仪传感器712采集的数据，可以实现如下功能：动作感应(比如根 据用户的倾斜操作来改变UI)、拍摄时的图像稳定、游戏控制以及惯性导航。
压力传感器713可以设置在终端700的侧边框和/或显示屏705的下层。当压力传感器713设置在终端700的侧边框时，可以检测用户对终端700的握持信号，由处理器701根据压力传感器713采集的握持信号进行左右手识别或快捷操作。当压力传感器713设置在显示屏705的下层时，由处理器701根据用户对显示屏705的压力操作，实现对UI界面上的可操作性控件进行控制。可操作性控件包括按钮控件、滚动条控件、图标控件、菜单控件中的至少一种。
指纹传感器714用于采集用户的指纹，由处理器701根据指纹传感器714采集到的指纹识别用户的身份，或者，由指纹传感器714根据采集到的指纹识别用户的身份。在识别出用户的身份为可信身份时，由处理器701授权该用户执行相关的敏感操作，该敏感操作包括解锁屏幕、查看加密信息、下载软件、支付及更改设置等。指纹传感器714可以被设置终端700的正面、背面或侧面。当终端700上设置有物理按键或厂商Logo时，指纹传感器714可以与物理按键或厂商Logo集成在一起。
光学传感器715用于采集环境光强度。在一个实施例中，处理器701可以根据光学传感器715采集的环境光强度，控制显示屏705的显示亮度。具体地，当环境光强度较高时，调高显示屏705的显示亮度；当环境光强度较低时，调低显示屏705的显示亮度。在另一个实施例中，处理器701还可以根据光学传感器715采集的环境光强度，动态调整摄像头组件706的拍摄参数。
接近传感器716，也称距离传感器，通常设置在终端700的前面板。接近传感器716用于采集用户与终端700的正面之间的距离。在一个实施例中，当接近传感器716检测到用户与终端700的正面之间的距离逐渐变小时，由处理器701控制显示屏705从亮屏状态切换为息屏状态；当接近传感器716检测到用户与终端700的正面之间的距离逐渐变大时，由处理器701控制显示屏705从息屏状态切换为亮屏状态。
本领域技术人员可以理解，图7中示出的结构并不构成对终端700的限定，可以包括比图示更多或更少的组件，或者组合某些组件，或者采用不同的组件布置。
图8是根据一示例性实施例示出的一种表面缺陷检测装置的服务器结构示意图。该服务器800可以是后台服务器集群中的服务器，该服务器800可以是 上述实施例中的检测设备。具体来讲：
服务器800包括中央处理单元(CPU)801、包括随机存取存储器(RAM)802和只读存储器(ROM)803的***存储器804，以及连接***存储器804和中央处理单元801的***总线805。服务器800还包括帮助计算机内的各个器件之间传输信息的基本输入/输出***(I/O***)806，和用于存储操作***813、应用程序814和其他程序模块815的大容量存储设备807。
基本输入/输出***806包括有用于显示信息的显示器808和用于用户输入信息的诸如鼠标、键盘之类的输入设备809。其中显示器808和输入设备809都通过连接到***总线805的输入输出控制器810连接到中央处理单元801。基本输入/输出***806还可以包括输入输出控制器810以用于接收和处理来自键盘、鼠标、或电子触控笔等多个其他设备的输入。类似地，输入输出控制器810还提供输出到显示屏、打印机或其他类型的输出设备。
大容量存储设备807通过连接到***总线805的大容量存储控制器(未示出)连接到中央处理单元801。大容量存储设备807及其相关联的计算机可读介质为服务器800提供非易失性存储。也就是说，大容量存储设备807可以包括诸如硬盘或者CD-ROM驱动器之类的计算机可读介质(未示出)。
不失一般性，计算机可读介质可以包括计算机存储介质和通信介质。计算机存储介质包括以用于存储诸如计算机可读指令、数据结构、程序模块或其他数据等信息的任何方法或技术实现的易失性和非易失性、可移动和不可移动介质。计算机存储介质包括RAM、ROM、EPROM、EEPROM、闪存或其他固态存储其技术，CD-ROM、DVD或其他光学存储、磁带盒、磁带、磁盘存储或其他磁性存储设备。当然，本领域技术人员可知计算机存储介质不局限于上述几种。上述的***存储器804和大容量存储设备807可以统称为存储器。
根据本申请的各种实施例，服务器800还可以通过诸如因特网等网络连接到网络上的远程计算机运行。也即服务器800可以通过连接在***总线805上的网络接口单元811连接到网络812，或者说，也可以使用网络接口单元811来连接到其他类型的网络或远程计算机***(未示出)。
上述存储器还包括一个或者一个以上的程序，一个或者一个以上程序存储于存储器中，被配置由CPU执行。所述一个或者一个以上程序包含用于进行本申请实施例提供的表面缺陷检测方法的指令。
在一些实施例中，还提供了一种计算机可读存储介质，该存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现上述实施例中表面缺陷检测方法的步骤。例如，所述计算机可读存储介质可以是ROM、RAM、CD-ROM、磁带、软盘和光数据存储设备等。
值得注意的是，本申请实施例提到的计算机可读存储介质可以为非易失性存储介质，换句话说，可以是非瞬时性存储介质。
应当理解的是，实现上述实施例的全部或部分步骤可以通过软件、硬件、固件或者其任意结合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。所述计算机指令可以存储在上述计算机可读存储介质中。
也即是，在一些实施例中，还提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述所述的表面缺陷检测方法的步骤。
应当理解的是，本文提及的“至少一个”是指一个或多个，“多个”是指两个或两个以上。在本申请实施例的描述中，除非另有说明，“/”表示或的意思，例如，A/B可以表示A或B；本文中的“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，为了便于清楚描述本申请实施例的技术方案，在本申请的实施例中，采用了“第一”、“第二”等字样对功能和作用基本相同的相同项或相似项进行区分。本领域技术人员可以理解“第一”、“第二”等字样并不对数量和执行次序进行限定，并且“第一”、“第二”等字样也并不限定一定不同。
以上所述为本申请提供的实施例，并不用以限制本申请，凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (14)

  1. 一种表面缺陷检测方法，其特征在于，所述方法包括：
    获取待检测图像；
    将所述待检测图像输入神经网络缺陷分割模型，输出第一检测结果，所述神经网络缺陷分割模型用于检测已知缺陷类别的表面缺陷；
    如果所述第一检测结果为所述待检测图像不存在所述已知缺陷类别的表面缺陷，则将所述待检测图像输入神经网络缺陷特征提取模型，输出待对比特征，所述待对比特征为所述待检测图像的图像特征；
    如果所述待对比特征与正常数据表示特征之间的相似度小于相似度阈值，则确定所述待检测图像存在未知缺陷类别的表面缺陷，所述正常数据表示特征是基于无表面缺陷的图像的图像特征生成的。
  2. 根据权利要求1所述的方法，其特征在于，所述确定所述待检测图像存在未知缺陷类别的表面缺陷之后，还包括：
    根据所述待对比特征与所述正常数据表示特征之间的相似度，以及图像特征矩阵与图像像素矩阵之间的映射关系，确定所述待检测图像中存在的表面缺陷的缺陷位置。
  3. 根据权利要求1所述的方法，其特征在于，所述将所述待检测图像输入神经网络缺陷分割模型，输出第一检测结果之前，还包括：
    获取第一数据集，所述第一数据集包括已知缺陷类别的缺陷图像以及对应的第一标注信息，所述第一标注信息为表示缺陷类别的标注信息，所述第一数据集还包括无表面缺陷的图像以及对应的第二标注信息，所述第二标注信息为表示无表面缺陷的标注信息；
    根据所述第一数据集，训练得到所述神经网络缺陷分割模型。
  4. 根据权利要求1所述的方法，其特征在于，所述将所述待检测图像输入神经网络缺陷特征提取模型，输出待对比特征之前，还包括：
    获取第二数据集，所述第二数据集包括已知物体类别的图像以及对应的第三标注信息，所述第三标注信息为表示物体类别的标注信息；
    根据所述第二数据集，训练得到所述神经网络缺陷特征提取模型。
  5. 根据权利要求1-4任一所述的方法，其特征在于，所述如果所述待对比特征与正常数据表示特征之间的相似度小于相似度阈值，则确定所述待检测图像存在未知缺陷类别的表面缺陷之前，还包括：
    获取至少一个第一样本图像，所述至少一个第一样本图像无表面缺陷；
    将所述至少一个第一样本图像输入所述神经网络缺陷特征提取模型，输出所述至少一个第一样本图像的图像特征；
    将所述至少一个第一样本图像的图像特征作为所述正常数据表示特征。
  6. 根据权利要求1-4任一所述的方法，其特征在于，所述如果所述待对比特征与正常数据表示特征之间的相似度小于相似度阈值，则确定所述待检测图像存在未知缺陷类别的表面缺陷之前，还包括：
    获取多个第二样本图像，所述多个第二样本图像无表面缺陷；
    将所述多个第二样本图像输入所述神经网络缺陷特征提取模型，输出所述多个第二样本图像的图像特征；
    对所述多个第二样本图像的图像特征进行聚类，得到多组正常数据特征；
    从所述多组正常数据特征中的每组正常数据特征中选择至少一个图像特征，得到所述正常数据表示特征。
  7. 根据权利要求1-4任一所述的方法，其特征在于，所述将所述待检测图像输入神经网络缺陷特征提取模型，输出待对比特征之后，还包括：
    如果所述待对比特征与所述正常数据表示特征之间的相似度大于或等于所述相似度阈值，则确定所述待检测图像无表面缺陷。
  8. 一种表面缺陷检测装置，其特征在于，所述装置包括：
    第一获取模块，用于获取待检测图像；
    检测模块，用于将所述待检测图像输入神经网络缺陷分割模型，输出第一检测结果，所述神经网络缺陷分割模型用于检测已知缺陷类别的表面缺陷；
    第一处理模块，用于如果所述第一检测结果为所述待检测图像不存在所述 已知缺陷类别的表面缺陷，则将所述待检测图像输入神经网络缺陷特征提取模型，输出待对比特征，所述待对比特征为所述待检测图像的图像特征；
    第一确定模块，用于如果所述待对比特征与正常数据表示特征之间的相似度小于相似度阈值，则确定所述待检测图像存在未知缺陷类别的表面缺陷，所述正常数据表示特征是基于无表面缺陷的图像的图像特征生成的。
  9. 根据权利要求8所述的装置，其特征在于，所述装置还包括：
    第二确定模块，用于根据所述待对比特征与所述正常数据表示特征之间的相似度，以及图像特征矩阵与图像像素矩阵之间的映射关系，确定所述待检测图像中存在的表面缺陷的缺陷位置。
  10. 根据权利要求8所述的装置，其特征在于，所述装置还包括：
    第二获取模块，用于获取第一数据集，所述第一数据集包括已知缺陷类别的缺陷图像以及对应的第一标注信息，所述第一标注信息为表示缺陷类别的标注信息，所述第一数据集还包括无表面缺陷的图像以及对应的第二标注信息，所述第一数据集包括的无表面缺陷的图像对应的标注信息为无表面缺陷，所述第二标注信息为表示无表面缺陷的标注信息；
    第一训练模块，用于根据所述第一数据集，训练得到所述神经网络缺陷分割模型。
  11. 一种表面缺陷检测***，其特征在于，所述表面缺陷检测***包括相机和至少一个处理器；
    所述相机，用于拍摄待检测物体的至少一个表面，作为待检测图像；
    所述至少一个处理器，用于获取所述待检测图像，实现权利要求1-7任一所述方法的步骤。
  12. 根据权利要求11所述的***，其特征在于，所述表面缺陷检测***还包括传送装置，所述传送装置用于传输所述待检测物体；
    所述相机用于在所述传送装置传输所述待检测物体的过程中，拍摄所述待检测物体。
  13. 一种计算机可读存储介质，所述计算机可读存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现权利要求1-7任一所述方法的步骤。
  14. 一种计算机程序产品，其特征在于，所述计算机程序产品包含计算机指令，所述计算机指令在计算机上运行时实现权利要求1-7任一所述方法的步骤。
PCT/CN2021/139333 2020-12-17 2021-12-17 表面缺陷检测方法、装置、***、存储介质及程序产品 WO2022127919A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP21905842.7A EP4266244A4 (en) 2020-12-17 2021-12-17 METHOD, DEVICE, SYSTEM, STORAGE MEDIUM AND PROGRAM PRODUCT FOR DETECTING SURFACE DEFECTS

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011495050.3A CN114648480A (zh) 2020-12-17 2020-12-17 表面缺陷检测方法、装置及***
CN202011495050.3 2020-12-17

Publications (1)

Publication Number Publication Date
WO2022127919A1 true WO2022127919A1 (zh) 2022-06-23

Family

ID=81990954

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/139333 WO2022127919A1 (zh) 2020-12-17 2021-12-17 表面缺陷检测方法、装置、***、存储介质及程序产品

Country Status (3)

Country Link
EP (1) EP4266244A4 (zh)
CN (1) CN114648480A (zh)
WO (1) WO2022127919A1 (zh)

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114862845A (zh) * 2022-07-04 2022-08-05 深圳市瑞桔电子有限公司 手机触摸屏的缺陷检测方法、装置、设备及存储介质
CN115564779A (zh) * 2022-12-08 2023-01-03 东莞先知大数据有限公司 一种零件缺陷检测方法、装置及存储介质
CN115578377A (zh) * 2022-11-14 2023-01-06 成都数之联科技股份有限公司 一种面板缺陷的检测方法、训练方法、装置、设备及介质
CN115631186A (zh) * 2022-11-08 2023-01-20 哈尔滨工业大学 一种基于双分支神经网络的工业元件表面缺陷检测方法
CN115661140A (zh) * 2022-12-13 2023-01-31 深圳思谋信息科技有限公司 缺陷检测方法、装置、计算机设备及计算机可读存储介质
CN115839957A (zh) * 2023-02-20 2023-03-24 深圳新视智科技术有限公司 显示模组夹层缺陷的检测方法、装置、设备及存储介质
CN116071309A (zh) * 2022-12-27 2023-05-05 中国电子产品可靠性与环境试验研究所（（工业和信息化部电子第五研究所）（中国赛宝实验室）） 元器件的声扫缺陷检测方法、装置、设备和存储介质
CN116245846A (zh) * 2023-03-08 2023-06-09 华院计算技术(上海)股份有限公司 带钢的缺陷检测方法及装置、存储介质、计算设备
CN116399874A (zh) * 2023-06-08 2023-07-07 华东交通大学 剪切散斑干涉无损检测缺陷尺寸的方法和程序产品
CN116500042A (zh) * 2023-05-09 2023-07-28 哈尔滨工业大学重庆研究院 缺陷检测方法、装置、***及存储介质
CN116703874A (zh) * 2023-06-15 2023-09-05 小米汽车科技有限公司 目标检测方法、装置及存储介质
CN116824271A (zh) * 2023-08-02 2023-09-29 上海互觉科技有限公司 基于三模态向量空间对齐的smt芯片缺陷检测***及方法
CN117333383A (zh) * 2023-09-07 2024-01-02 广东奥普特科技股份有限公司 一种表面缺陷检测方法、装置和设备
CN118038151A (zh) * 2024-02-19 2024-05-14 广州里工实业有限公司 晶圆缺陷分类检测方法、***、电子设备及存储介质
CN118096774A (zh) * 2024-04-29 2024-05-28 常州爱克赛伦自动化设备有限公司 结合深度特征的传感器封装过程缺陷检测装置及方法

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117957438A (zh) * 2022-08-30 2024-04-30 宁德时代新能源科技股份有限公司 缺陷检测的方法、装置和计算机可读存储介质
CN115661160B (zh) * 2022-12-29 2023-03-10 成都数之联科技股份有限公司 一种面板缺陷检测方法及***及装置及介质
CN116883416B (zh) * 2023-09-08 2023-11-24 腾讯科技（深圳）有限公司 工业产品缺陷的检测方法、装置、设备及介质
CN117274180A (zh) * 2023-09-13 2023-12-22 广州诺芯软件科技有限公司 应用于产品质量评估模型的数据处理方法及***
CN117351010B (zh) * 2023-12-04 2024-03-01 中科慧远视觉技术（洛阳）有限公司 一种基于深度学习的金属凹陷结构缺陷检测方法及装置

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104751198A (zh) * 2013-12-27 2015-07-01 华为技术有限公司 图像中的目标物的识别方法及装置
CN105719291A (zh) * 2016-01-20 2016-06-29 江苏省沙钢钢铁研究院有限公司 品种可选择的表面缺陷图像分类***
CN110533058A (zh) * 2018-05-24 2019-12-03 株式会社捷太格特 信息处理方法、信息处理装置及程序
CN110619618A (zh) * 2018-06-04 2019-12-27 杭州海康威视数字技术股份有限公司 一种表面缺陷检测方法、装置及电子设备
US20200111217A1 (en) * 2017-06-06 2020-04-09 Denso Corporation Appearance inspection device, transformation data generation device, and program
CN111652319A (zh) * 2020-06-09 2020-09-11 创新奇智(广州)科技有限公司 一种布匹缺陷检测方法及装置
US20210279858A1 (en) * 2018-06-12 2021-09-09 Carl Zeiss Jena Gmbh Material testing of optical test pieces

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10650508B2 (en) * 2014-12-03 2020-05-12 Kla-Tencor Corporation Automatic defect classification without sampling and feature selection
IL259285B2 (en) * 2018-05-10 2023-07-01 Inspekto A M V Ltd A system and method for detecting defects on objects in an image

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104751198A (zh) * 2013-12-27 2015-07-01 华为技术有限公司 图像中的目标物的识别方法及装置
CN105719291A (zh) * 2016-01-20 2016-06-29 江苏省沙钢钢铁研究院有限公司 品种可选择的表面缺陷图像分类***
US20200111217A1 (en) * 2017-06-06 2020-04-09 Denso Corporation Appearance inspection device, transformation data generation device, and program
CN110533058A (zh) * 2018-05-24 2019-12-03 株式会社捷太格特 信息处理方法、信息处理装置及程序
CN110619618A (zh) * 2018-06-04 2019-12-27 杭州海康威视数字技术股份有限公司 一种表面缺陷检测方法、装置及电子设备
US20210279858A1 (en) * 2018-06-12 2021-09-09 Carl Zeiss Jena Gmbh Material testing of optical test pieces
CN111652319A (zh) * 2020-06-09 2020-09-11 创新奇智(广州)科技有限公司 一种布匹缺陷检测方法及装置

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
LI, CHAO ET AL.: "Effective Method of Weld Defect Detection and Classification based on Machine Vision", COMPUTER ENGINEERING AND APPLICATIONS, vol. 54, no. 6, 31 December 2018 (2018-12-31), pages 264 - 270, XP055944608 *
See also references of EP4266244A4

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114862845B (zh) * 2022-07-04 2022-09-06 深圳市瑞桔电子有限公司 手机触摸屏的缺陷检测方法、装置、设备及存储介质
CN114862845A (zh) * 2022-07-04 2022-08-05 深圳市瑞桔电子有限公司 手机触摸屏的缺陷检测方法、装置、设备及存储介质
CN115631186B (zh) * 2022-11-08 2023-10-03 哈尔滨工业大学 一种基于双分支神经网络的工业元件表面缺陷检测方法
CN115631186A (zh) * 2022-11-08 2023-01-20 哈尔滨工业大学 一种基于双分支神经网络的工业元件表面缺陷检测方法
CN115578377A (zh) * 2022-11-14 2023-01-06 成都数之联科技股份有限公司 一种面板缺陷的检测方法、训练方法、装置、设备及介质
CN115564779A (zh) * 2022-12-08 2023-01-03 东莞先知大数据有限公司 一种零件缺陷检测方法、装置及存储介质
CN115661140A (zh) * 2022-12-13 2023-01-31 深圳思谋信息科技有限公司 缺陷检测方法、装置、计算机设备及计算机可读存储介质
CN116071309A (zh) * 2022-12-27 2023-05-05 中国电子产品可靠性与环境试验研究所（（工业和信息化部电子第五研究所）（中国赛宝实验室）） 元器件的声扫缺陷检测方法、装置、设备和存储介质
CN116071309B (zh) * 2022-12-27 2024-05-17 中国电子产品可靠性与环境试验研究所（（工业和信息化部电子第五研究所）（中国赛宝实验室）） 元器件的声扫缺陷检测方法、装置、设备和存储介质
CN115839957A (zh) * 2023-02-20 2023-03-24 深圳新视智科技术有限公司 显示模组夹层缺陷的检测方法、装置、设备及存储介质
CN116245846B (zh) * 2023-03-08 2023-11-21 华院计算技术(上海)股份有限公司 带钢的缺陷检测方法及装置、存储介质、计算设备
CN116245846A (zh) * 2023-03-08 2023-06-09 华院计算技术(上海)股份有限公司 带钢的缺陷检测方法及装置、存储介质、计算设备
CN116500042A (zh) * 2023-05-09 2023-07-28 哈尔滨工业大学重庆研究院 缺陷检测方法、装置、***及存储介质
CN116500042B (zh) * 2023-05-09 2024-01-26 哈尔滨工业大学重庆研究院 缺陷检测方法、装置、***及存储介质
CN116399874B (zh) * 2023-06-08 2023-08-22 华东交通大学 剪切散斑干涉无损检测缺陷尺寸的方法和程序产品
CN116399874A (zh) * 2023-06-08 2023-07-07 华东交通大学 剪切散斑干涉无损检测缺陷尺寸的方法和程序产品
CN116703874A (zh) * 2023-06-15 2023-09-05 小米汽车科技有限公司 目标检测方法、装置及存储介质
CN116824271A (zh) * 2023-08-02 2023-09-29 上海互觉科技有限公司 基于三模态向量空间对齐的smt芯片缺陷检测***及方法
CN116824271B (zh) * 2023-08-02 2024-02-09 上海互觉科技有限公司 基于三模态向量空间对齐的smt芯片缺陷检测***及方法
CN117333383A (zh) * 2023-09-07 2024-01-02 广东奥普特科技股份有限公司 一种表面缺陷检测方法、装置和设备
CN117333383B (zh) * 2023-09-07 2024-05-24 广东奥普特科技股份有限公司 一种表面缺陷检测方法、装置和设备
CN118038151A (zh) * 2024-02-19 2024-05-14 广州里工实业有限公司 晶圆缺陷分类检测方法、***、电子设备及存储介质
CN118096774A (zh) * 2024-04-29 2024-05-28 常州爱克赛伦自动化设备有限公司 结合深度特征的传感器封装过程缺陷检测装置及方法

Also Published As

Publication number Publication date
EP4266244A1 (en) 2023-10-25
CN114648480A (zh) 2022-06-21
EP4266244A4 (en) 2024-05-15

Similar Documents

Publication Publication Date Title
WO2022127919A1 (zh) 表面缺陷检测方法、装置、***、存储介质及程序产品
WO2019101021A1 (zh) 图像识别方法、装置及电子设备
WO2020048308A1 (zh) 多媒体资源分类方法、装置、计算机设备及存储介质
WO2020019873A1 (zh) 图像处理方法、装置、终端及计算机可读存储介质
WO2020221012A1 (zh) 图像特征点的运动信息确定方法、任务执行方法和设备
WO2021077989A1 (zh) 推荐方法、装置、计算机设备及存储介质
CN110555839A (zh) 缺陷检测识别方法、装置、计算机设备及存储介质
CN111680697B (zh) 实现领域自适应的方法、装置、电子设备及介质
CN109360222B (zh) 图像分割方法、装置及存储介质
CN111127509B (zh) 目标跟踪方法、装置和计算机可读存储介质
WO2022042425A1 (zh) 视频数据处理方法、装置、计算机设备及存储介质
US11386586B2 (en) Method and electronic device for adding virtual item
CN111192262A (zh) 基于人工智能的产品缺陷分类方法、装置、设备及介质
WO2020108041A1 (zh) 耳部关键点检测方法、装置及存储介质
CN111027490A (zh) 人脸属性识别方法及装置、存储介质
CN110705614A (zh) 模型训练方法、装置、电子设备及存储介质
CN110853124B (zh) 生成gif动态图的方法、装置、电子设备及介质
KR20230071720A (ko) 얼굴 이미지의 랜드마크 좌표 예측 방법 및 장치
CN110675473B (zh) 生成gif动态图的方法、装置、电子设备及介质
CN110728167A (zh) 文本检测方法、装置及计算机可读存储介质
CN112819103A (zh) 基于图神经网络的特征识别方法、装置、存储介质及终端
WO2023066373A1 (zh) 确定样本图像的方法、装置、设备及存储介质
CN111898535A (zh) 目标识别方法、装置及存储介质
CN110163192B (zh) 字符识别方法、装置及可读介质
CN111488895B (zh) 对抗数据生成方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21905842

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021905842

Country of ref document: EP

Effective date: 20230717