CN110148130B

CN110148130B - Method and device for detecting part defects

Info

Publication number: CN110148130B
Application number: CN201910446993.8A
Authority: CN
Inventors: 黄特辉; 刘明浩; 聂磊; 郭江亮; 苏业; 邹建法
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2019-05-27
Filing date: 2019-05-27
Publication date: 2022-02-11
Anticipated expiration: 2039-05-27
Also published as: CN110148130A

Abstract

The embodiment of the application discloses a method and a device for detecting part defects. One embodiment of the method comprises: acquiring a to-be-detected surface image set of a to-be-detected part acquired at a plurality of specific angles of the to-be-detected part; extracting a surface feature set to be detected of the part to be detected from the surface image set to be detected by using a feature extraction algorithm; and detecting the surface feature set to be detected by using a pre-trained defect detection model set to obtain a defect detection result of the part to be detected, wherein the defect detection model set is used for detecting defects of the part, and the detection result comprises defect categories of the part to be detected. The embodiment relates to the field of cloud computing, and improves the detection accuracy of part defects.

Description

Method and device for detecting part defects

Technical Field

The embodiment of the application relates to the technical field of computers, in particular to a method and a device for detecting part defects.

Background

With the increasing degree of industrial modernization, machines are ubiquitous in life. For example, machines have been widely used in various industries such as electronics, chemical engineering, aerospace, and the like. The parts are basic components of the machine and basic units in the mechanical manufacturing process. Since the contour, shape and size of the part must be consistent with the precision of the initial design to meet the assembly requirement of the machine, detecting the defect of the part is one of indispensable links in the machining industry in the high-speed development industrial environment.

At present, the part defect detection mode is mainly manual quality inspection. That is, the surface of the part is observed by the naked human eye to determine whether the part has defects.

Disclosure of Invention

The embodiment of the application provides a method and a device for detecting part defects.

In a first aspect, an embodiment of the present application provides a method for detecting a defect of a part, including: acquiring a to-be-detected surface image set of a to-be-detected part acquired at a plurality of specific angles of the to-be-detected part; extracting a surface feature set to be detected of the part to be detected from the surface image set to be detected by using a feature extraction algorithm; and detecting the surface feature set to be detected by using a pre-trained defect detection model set to obtain a defect detection result of the part to be detected, wherein the defect detection model set is used for detecting defects of the part, and the detection result comprises defect categories of the part to be detected.

In some embodiments, before extracting the set of surface features to be detected of the part to be detected from the set of surface images to be detected by using the feature extraction algorithm, the method further includes: and preprocessing the surface image set to be detected by utilizing a preprocessing method.

In some embodiments, the detection result further includes a position of a defect type of the part to be detected and a confidence level of the defect type of the part to be detected in the predetermined defect type set.

In some embodiments, detecting the surface image feature set to be detected by using a pre-trained defect detection model set to obtain a defect detection result of the part to be detected includes: for the defect type in the defect type set, if the defect detection models in the defect detection model set detect that the part to be detected has the defect type, the confidence coefficient that the part to be detected has the defect type is generated based on the accuracy of the defect detection models and the confidence coefficient that the part to be detected has the defect type.

In some embodiments, after detecting the surface image feature set to be detected by using a defect detection model set trained in advance to obtain a defect detection result of the part to be detected, the method further includes: and sending display information to the terminal equipment based on the defect detection result of the part to be detected.

In some embodiments, sending the display information to the terminal device based on the defect detection result of the part to be detected includes: acquiring a defect marking result of a part to be detected; generating the accuracy and the recall rate of a defect detection model set based on the defect detection result and the defect labeling result of the part to be detected; and sending at least one of the accuracy, the recall rate, the recall-ready rate curve and the subject working characteristic curve of the defect detection model set to the terminal equipment.

In some embodiments, sending the display information to the terminal device based on the defect detection result of the part to be detected, further includes: generating the yield and the defect rate of the parts to be detected based on the defect detection result and the defect marking result of the parts to be detected; and sending at least one of the yield curve and the defect rate curve of the part to be detected to the terminal equipment.

In some embodiments, after sending at least one of the yield curve and the defect rate curve of the part to be detected to the terminal device, the method includes: determining whether a yield curve is in a first preset range or determining whether a defect rate curve is in a second preset range; and if the yield curve exceeds a first preset range or the defect rate curve exceeds a second preset range, sending an alarm command.

In some embodiments, the set of defect classes is determined by: acquiring a sample surface image set of a sample part acquired at a plurality of preset angles of the sample part; extracting a sample surface feature set of the sample part from the sample surface image set by using a feature extraction algorithm; clustering the sample surface feature set by using a clustering algorithm to obtain a first clustering result; and generating a defect category set based on the first clustering result.

In some embodiments, the plurality of specific angles is determined by: dividing the sample surface feature set into a plurality of sample surface feature subsets corresponding to a plurality of preset angles; clustering the surface feature subsets of the samples respectively by using a clustering algorithm to obtain a plurality of second clustering results corresponding to a plurality of preset angles; a plurality of specific angles are determined from a plurality of preset angles based on the plurality of second clustering results.

In some embodiments, the clustering algorithm comprises at least one of a k-means clustering algorithm, a mean shift clustering algorithm, a pixel clustering algorithm, a hierarchical clustering algorithm, a spectral clustering algorithm, and a deep embedded clustering algorithm based on a deep convolutional neural network.

In some embodiments, the set of defect detection models is trained by: selecting a sample surface image subset corresponding to the defect detection model from the sample surface image set for the defect detection model in the defect detection model set; extracting a sample surface feature subset from a sample surface image subset corresponding to the defect detection model by using a feature extraction algorithm; and training to obtain the defect detection model by taking the sample surface features in the sample surface feature subset corresponding to the defect detection model as input and taking the defect types of the sample parts corresponding to the input sample surface features as output.

In some embodiments, the sample surface image subset corresponding to one of the set of defect detection models is a sample surface image subset of the sample part acquired at one of a plurality of specific angles of the sample part.

In some embodiments, the feature extraction algorithm comprises at least one of a candidate area generation network algorithm, a feature pyramid network algorithm, and a deep network algorithm.

In some embodiments, before extracting the sample surface feature subset from the sample surface image subset corresponding to the defect detection model by using the feature extraction algorithm, the method further includes: and preprocessing the sample surface image subset corresponding to the defect detection model by using a preprocessing method.

In some embodiments, the pre-treatment method comprises at least one of: adjusting at least one of brightness, gray scale and contrast; adjusting at least one of a size and an offset; and (4) self-adaptive map cutting.

In some embodiments, the preprocessing method is used to preprocess the sample surface image subset corresponding to the defect detection model, and includes: for a sample surface image in the sample surface image subset, converting the sample surface image into a corresponding grayscale surface image; binarizing the gray surface image corresponding to the sample surface image to obtain a binarized surface image corresponding to the sample surface image; determining the outline of a sample part in a binarization surface image corresponding to the sample surface image; and cutting the sample surface image based on the outline of the sample part in the binarization surface image corresponding to the sample surface image.

In some embodiments, the preprocessing method is used to preprocess the sample surface image subset corresponding to the defect detection model, and includes: for a sample surface image in the sample surface image subset, if a sample part in the sample surface image is inclined, calculating an affine transformation matrix and a rotation matrix of the sample surface image; and carrying out affine transformation on the sample surface image by utilizing the affine transformation matrix and the rotation matrix of the sample surface image to generate a new sample surface image corresponding to the sample surface image.

In some embodiments, the preprocessing method is used to preprocess the sample surface image subset corresponding to the defect detection model, and includes: for a sample surface image in the sample surface image subset, adjusting the number of decision regions of the sample surface image if the defect type of the sample part in the sample surface image is edge scratch or dirty.

In some embodiments, the preprocessing method is used to preprocess the sample surface image subset corresponding to the defect detection model, and includes: and for the sample surface image in the sample surface image subset, if the defect type of the sample part in the sample surface image is peeling, adjusting the brightness and the contrast of the sample surface image.

In some embodiments, the preprocessing method is used to preprocess the sample surface image subset corresponding to the defect detection model, and includes: and for the sample surface images in the sample surface image subset, if the defect type of the sample part in the sample surface images is deformation, adjusting the contrast of the sample surface images, expanding the detection frame of the sample surface images, and performing pixel clustering on the detection frame of the sample surface images.

In some embodiments, the preprocessing method is used to preprocess the sample surface image subset corresponding to the defect detection model, and includes: for a sample surface image in a sample surface image subset, if a defect type of a sample part in the sample surface image is a window burr, extracting a window outline of the sample part in the sample surface image, storing a vertical coordinate of pixel values of two horizontal sides and a horizontal coordinate of pixel values of two vertical sides of the window outline of the sample part in the sample surface image into a one-dimensional array, normalizing the pixel values of the window outline of the sample part in the sample surface image, and generating a line graph of the window outline based on the normalized pixel values.

In some embodiments, the preprocessing method is used to preprocess the sample surface image subset corresponding to the defect detection model, and includes: and for the sample surface images in the sample surface image subset, if the defect type of the sample part in the sample surface images is the flow mark, expanding the detection frame of the sample surface images, and performing pixel clustering on the detection frame of the sample surface images.

In a second aspect, an embodiment of the present application provides an apparatus for detecting a defect of a part, including: an acquisition unit configured to acquire a to-be-detected surface image set of a to-be-detected part acquired at a plurality of specific angles of the to-be-detected part; the extraction unit is configured to extract a to-be-detected surface feature set of a to-be-detected part from the to-be-detected surface image set by using a feature extraction algorithm; the detection unit is configured to detect the surface feature set to be detected by using a defect detection model set trained in advance to obtain a defect detection result of the part to be detected, wherein the defect detection model set is used for detecting defects of the part, and the detection result comprises defect categories of the part to be detected.

In some embodiments, the apparatus further comprises: and the preprocessing unit is configured to preprocess the surface image set to be detected by utilizing a preprocessing method.

In some embodiments, the detection unit is further configured to: for the defect type in the defect type set, if the defect detection models in the defect detection model set detect that the part to be detected has the defect type, the confidence coefficient that the part to be detected has the defect type is generated based on the accuracy of the defect detection models and the confidence coefficient that the part to be detected has the defect type.

In some embodiments, the apparatus further comprises: and the sending unit is configured to send the display information to the terminal equipment based on the defect detection result of the part to be detected.

In some embodiments, the sending unit is further configured to: acquiring a defect marking result of a part to be detected; generating the accuracy and the recall rate of a defect detection model set based on the defect detection result and the defect labeling result of the part to be detected; and sending at least one of the accuracy, the recall rate, the recall-ready rate curve and the subject working characteristic curve of the defect detection model set to the terminal equipment.

In some embodiments, the sending unit is further configured to: generating the yield and the defect rate of the parts to be detected based on the defect detection result and the defect marking result of the parts to be detected; and sending at least one of the yield curve and the defect rate curve of the part to be detected to the terminal equipment.

In some embodiments, the apparatus comprises: a determining unit configured to determine whether the yield curve is in a first preset range or determine whether the defect rate curve is in a second preset range; and the alarm unit is configured to send an alarm command if the yield curve exceeds a first preset range or the defect rate curve exceeds a second preset range.

In a third aspect, an embodiment of the present application provides a server, where the server includes: one or more processors; a storage device having one or more programs stored thereon; when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method as described in any implementation of the first aspect.

In a fourth aspect, the present application provides a computer-readable medium, on which a computer program is stored, which, when executed by a processor, implements the method as described in any implementation manner of the first aspect.

According to the method and the device for detecting the part defects, firstly, a to-be-detected surface image set of the to-be-detected part collected at a plurality of specific angles of the to-be-detected part is obtained; then extracting a surface feature set to be detected of the part to be detected from the surface image set to be detected by using a feature extraction algorithm; and finally, detecting the surface feature set to be detected by using a pre-trained defect detection model set to obtain a defect detection result of the part to be detected. The defect detection model set is used for automatically detecting the defects of the parts, the whole detection process does not need manual participation, the labor cost is reduced, and the detection efficiency of the defects of the parts is improved. In addition, the defect detection models detect the defects of the parts together, so that the false detection rate and the missing detection rate of the defects of the parts are reduced, and the detection accuracy of the defects of the parts is improved.

Drawings

Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:

FIG. 1 is an exemplary system architecture to which the present application may be applied;

FIG. 2 is a flow diagram of one embodiment of a method for detecting a defect in a part according to the present application;

FIG. 3 is a flow chart of yet another embodiment of a method for detecting a defect in a part according to the present application;

FIG. 4 is a flow diagram of one embodiment of a method for determining a set of defect classes according to the present application;

FIG. 5 is a flow diagram of one embodiment of a method for determining a particular angle according to the present application;

FIG. 6 is a flow diagram of one embodiment of a method for training a set of defect detection models, according to the present application;

FIG. 7 is a schematic diagram of an embodiment of an apparatus for detecting part defects according to the present application;

FIG. 8 is a schematic block diagram of a computer system suitable for use in implementing a server according to embodiments of the present application.

Detailed Description

The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.

It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.

Fig. 1 illustrates an exemplary system architecture 100 to which embodiments of the present method for detecting a part defect or apparatus for detecting a part defect may be applied.

As shown in fig. 1, the system architecture 100 may include

image capturing devices

101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the

image capturing devices

101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The

image capturing devices

101, 102, 103 may send the surface images of the parts they capture to the server 105 over the network 104. The

image capturing devices

101, 102, 103 may be hardware or software. When the

image capturing devices

101, 102, 103 are hardware, they may be various electronic devices that support an image capturing function. Including but not limited to cameras, video cameras, smart phones, and the like. When the

image capturing devices

101, 102, and 103 are software, they may be installed in the electronic devices. It may be implemented as multiple pieces of software or software modules, or as a single piece of software or software module. And is not particularly limited herein.

The server 105 may be a server that provides various services. Such as a part defect detection server. The part defect detection server may analyze and otherwise process the acquired data such as the set of to-be-detected surface images of the to-be-detected part acquired at the plurality of specific angles of the to-be-detected part, and generate a processing result (e.g., a defect detection result of the to-be-detected part).

The server 105 may be hardware or software. When the server 105 is hardware, it may be implemented as a distributed server cluster composed of a plurality of servers, or may be implemented as a single server. When the server 105 is software, it may be implemented as multiple pieces of software or software modules (e.g., to provide distributed services), or as a single piece of software or software module. And is not particularly limited herein.

It should be noted that the method for detecting a part defect provided in the embodiment of the present application is generally performed by the server 105, and accordingly, the apparatus for detecting a part defect is generally disposed in the server 105.

It should be understood that the number of image capturing devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of image capture devices, networks, and servers, as desired for implementation.

With continued reference to FIG. 2, a flow 200 of one embodiment of a method for detecting a part defect according to the present application is shown. The method for detecting the part defects comprises the following steps:

step 201, acquiring a to-be-detected surface image set of a to-be-detected part acquired at a plurality of specific angles of the to-be-detected part.

In the present embodiment, an executing body (e.g., the server 105 shown in fig. 1) of the method for detecting a part defect may acquire a set of surface images to be detected from a plurality of image capturing apparatuses (e.g., the

image capturing apparatuses

101, 102, 103 shown in fig. 1) disposed in the vicinity of the part to be detected. The image acquisition equipment and the part to be detected form a specific angle and is used for acquiring the surface image to be detected of the part to be detected at the specific angle of the part to be detected. For example, three image acquisition devices are arranged at the front, side and back facing the part to be detected, and the three image acquisition devices respectively acquire the front image, the side image and the back image of the part to be detected and add the images to the surface image set to be detected as the surface image to be detected of the part to be detected.

Step 202, extracting a surface feature set to be detected of the part to be detected from the surface image set to be detected by using a feature extraction algorithm.

In this embodiment, for each surface image to be detected in the surface image set to be detected, the executing body may extract the surface feature to be detected of the part to be detected from the surface image to be detected by using a feature extraction algorithm, and add the extracted surface feature to the surface feature set to be detected of the part to be detected. Wherein the feature extraction algorithm may include, but is not limited to, at least one of: RPN (Region pro-social Network, candidate area generation Network) algorithm, FPN (Feature Pyramid Networks) algorithm, and DN (Deep Net) algorithm, and the like. The surface feature to be detected may be information for describing a feature possessed by the surface of the part to be detected in the surface image to be detected, including but not limited to various basic elements (e.g., surface shape, surface color, surface smoothness, etc.) relating to the surface of the part to be detected. In general, the surface features to be detected may be represented by a multi-dimensional vector.

In some optional implementation manners of this embodiment, the executing body may first perform preprocessing on the set of surface images to be detected by using a preprocessing method, and then perform step 202. Generally, the preprocessing method for preprocessing the surface image set to be detected is the same as the preprocessing method for preprocessing the sample surface image set used for training the defect detection model set, and specifically refer to the embodiment shown in fig. 6.

And 203, detecting the surface feature set to be detected by using the pre-trained defect detection model set to obtain a defect detection result of the part to be detected.

In this embodiment, the executing body may utilize the defect detection model set to detect the surface feature set to be detected, so as to obtain a defect detection result of the part to be detected. The defect detection model set can be used for detecting defects of the part and representing the corresponding relation between the surface feature set of the part and the defect detection result of the part. The detection result of the part to be detected can comprise the defect type of the part to be detected. Optionally, the detection result of the part to be detected may further include a position of a defect type of the part to be detected and a confidence level of the defect type of the part to be detected in the predetermined defect type set. The defect category set may include defect categories such as cracks, scratches, bulges, sticky materials, sand holes, edge scratches, dirt, peeling, deformation, window burrs, and flow lines.

In general, one of the set of defect detection models may be used to detect the presence of defects in a part in a surface image acquired at a particular angle. Therefore, for each specific angle in the plurality of specific angles, the executing body may first input the surface feature to be detected corresponding to the specific angle in the surface feature set to be detected to the defect detection model corresponding to the specific angle, so as to obtain the defect detection result corresponding to the specific angle. And then summarizing a plurality of defect detection results corresponding to a plurality of specific angles to obtain the defect detection result of the part to be detected.

In this embodiment, the defect detection model set may be trained in various ways. For example, for each defect detection model in the defect detection model set, the execution body may collect in advance the sample surface image acquired at a specific angle and the defect type of the sample part in the sample surface image, and generate the correspondence table in correspondence with the sample surface image, as the defect detection model. After the surface feature set to be detected is obtained, the execution main body may first select the surface feature to be detected corresponding to the specific angle from the surface feature set to be detected, and then calculate the similarity between the surface feature to be detected corresponding to the specific angle and the sample surface features of each sample surface image in the correspondence table; then determining a defect detection result corresponding to the specific angle from the corresponding relation table based on the calculated similarity; and finally, summarizing a plurality of defect detection results corresponding to the specific angles to obtain the defect detection result of the part to be detected.

In some optional implementation manners of this embodiment, for each defect type in the defect type set, if the defect detection models in the defect detection model set detect that the part to be detected has the defect type, the execution main body may generate the confidence that the part to be detected has the defect type based on the accuracy of the defect detection models and the confidence that the part to be detected has the defect type detected by the defect detection models. For example, the execution subject may select, as the confidence that the defect type exists in the part to be detected, which is detected by the defect detection model with high accuracy. For another example, the executing entity may perform normalization processing according to the accuracy of the plurality of defect detection models, and use the score value after the normalization processing as the corresponding weight factor.

In some optional implementation manners of this embodiment, the execution main body may send display information to the terminal device based on a defect detection result of the part to be detected. For example, the execution body may directly send the defect detection result of the part to be detected to the terminal device. Therefore, the terminal equipment can display the defect detection result of the part to be detected.

The method for detecting the part defects, provided by the embodiment of the application, comprises the steps of firstly obtaining a to-be-detected surface image set of a to-be-detected part, which is acquired at a plurality of specific angles of the to-be-detected part; then extracting a surface feature set to be detected of the part to be detected from the surface image set to be detected by using a feature extraction algorithm; and finally, detecting the surface feature set to be detected by using a pre-trained defect detection model set to obtain a defect detection result of the part to be detected. The defect detection model set is used for automatically detecting the defects of the parts, the whole detection process does not need manual participation, the labor cost is reduced, and the detection efficiency of the defects of the parts is improved. In addition, the defect detection models detect the defects of the parts together, so that the false detection rate and the missing detection rate of the defects of the parts are reduced, and the detection accuracy of the defects of the parts is improved.

With further reference to FIG. 3, a flow 300 of yet another embodiment of a method for detecting part defects according to the present application is shown. The method for detecting the part defects comprises the following steps:

step 301, acquiring a to-be-detected surface image set of a to-be-detected part acquired at a plurality of specific angles of the to-be-detected part.

Step 302, extracting a to-be-detected surface feature set of the to-be-detected part from the to-be-detected surface image set by using a feature extraction algorithm.

And 303, detecting the surface feature set to be detected by using the pre-trained defect detection model set to obtain a defect detection result of the part to be detected.

In the present embodiment, the specific operations of steps 301-.

And step 304, acquiring a defect marking result of the part to be detected.

In the embodiment, an executive body (for example, the server 105 shown in fig. 1) of the method for detecting the part defect may obtain a defect labeling result of the part to be detected. The defect marking result of the part to be detected can be a marking result of the part to be detected after manual quality inspection, and comprises the defect type of the part to be detected by human working medium. Optionally, the defect labeling result of the part to be detected may further include a position of a defect type of the part to be detected, which is detected by the human working medium.

And 305, generating the accuracy and the recall rate of the defect detection model set based on the defect detection result and the defect labeling result of the part to be detected.

In this embodiment, the execution body may calculate the accuracy and the recall rate of the defect detection model set based on the defect detection result and the defect labeling result of the part to be detected. For example, the execution body may use a ratio of the number of the parts to be detected, where the defect detection result and the defect labeling result are consistent, to the number of all the parts to be detected as the accuracy of the defect detection model set. The execution main body can take the ratio of the number of the parts to be detected with positive defect detection results and positive defect marking results to the number of the parts to be detected with positive defect marking results as the recall rate of the defect detection model set.

And step 306, sending at least one of the accuracy, the recall rate, the recall-ready rate curve and the subject working characteristic curve of the defect detection model set to the terminal equipment.

In this embodiment, the execution body may transmit at least one of an accuracy, a recall rate, a recall-to-recall curve (PRC curve), and a receiver operating characteristic curve (ROC curve) of the defect detection model set to the terminal device. Thus, the terminal device can display the received display information. Typically, the PRC curve and the ROC curve are plotted based on the accuracy and recall of the set of defect detection models.

And 307, generating the yield and the defect rate of the parts to be detected based on the defect detection result and the defect marking result of the parts to be detected.

In this embodiment, the execution body may calculate a yield and a defect rate of the part to be detected based on the defect detection result and the defect labeling result of the part to be detected. For example, the execution main body may use a ratio of the number of the parts to be detected with the positive defect detection result to the number of all the parts to be detected as the yield of the parts to be detected. The execution main body can use the ratio of the number of the parts to be detected with the negative defect detection result to the number of all the parts to be detected as the defect rate of the parts to be detected. And if the defect detection result is positive, the part to be detected does not have defects, and if the defect detection result is negative, the part to be detected has defects.

And 308, sending at least one of the yield curve and the defect rate curve of the part to be detected to the terminal equipment.

In this embodiment, the execution body may send at least one of a yield curve and a defect rate curve of the part to be detected to the terminal device. Thus, the terminal device can display the received display information.

Step 309, determining whether the yield curve is in a first preset range, or determining whether the defect rate curve is in a second preset range.

In this embodiment, the execution body may determine whether the yield curve is within a first preset range, or determine whether the defect rate curve is within a second preset range. If the yield curve exceeds the first predetermined range or the defect rate curve exceeds the second predetermined range, go to step 310.

Step 310, an alarm command is sent.

In this embodiment, if the yield curve exceeds a first preset range or the defect curve exceeds a second preset range, the execution main body may send an alarm command to the alarm device. Therefore, the alarm device can send out alarm sound to prompt that abnormal conditions occur in the machine for machining the parts or the production line for machining the parts.

As can be seen from fig. 3, compared with the embodiment corresponding to fig. 2, the flow 300 of the method for detecting a part defect in the present embodiment adds the steps of sending the display information and the alarm command. Therefore, according to the scheme described in the embodiment, the user can more visually see the defect condition of the part to be detected by displaying at least one of the accuracy, the recall rate, the PRC curve and the ROC curve of the defect detection model set and at least one of the yield and the defect rate of the part to be detected through the terminal equipment. And when the yield curve exceeds a first preset range or the defect rate curve exceeds a second preset range, the alarm device gives an alarm sound, so that abnormal conditions in the machine for machining the parts or the production line for machining the parts can be found in time according to prompts.

With further reference to FIG. 4, a flow 400 of one embodiment of a method for determining a set of defect classes is shown. The method for determining the defect class set comprises the following steps:

step 401, a sample surface image set of a sample part acquired at a plurality of preset angles of the sample part is obtained.

In this embodiment, the performing agent (e.g., server 105 shown in FIG. 1) of the method for determining a set of defect classes may acquire a set of sample surface images from a large number of image acquisition devices disposed in the vicinity of a sample part. The image acquisition equipment and the sample part form a preset angle and are used for acquiring a sample surface image of the sample part at the preset angle of the sample part. For example, one image capturing device is disposed at every certain angle (e.g., 10 degrees). The sample part may be a part having various defect categories.

Step 402, extracting a sample surface feature set of the sample part from the sample surface image set by using a feature extraction algorithm.

In this embodiment, for each sample surface image in the sample surface image set, the executing entity may extract the sample surface features from the sample surface image by using a feature extraction algorithm and add the sample surface features to the sample surface feature set. The sample surface feature may be information describing a feature of the surface of the sample part in the sample surface image, including but not limited to various basic elements related to the surface of the sample part (e.g., surface shape, surface color, surface smoothness, etc.). In general, the sample surface features may be represented by a multi-dimensional vector.

It should be noted that the feature extraction algorithm here is the same as the feature extraction algorithm in the embodiment shown in fig. 2, and is not described here again.

And 403, clustering the sample surface feature set by using a clustering algorithm to obtain a first clustering result.

In this embodiment, the executing entity may cluster the sample surface feature set by using a clustering algorithm to obtain a first clustering result. Clustering, among other things, is generally the process of dividing a large number of physical or abstract objects into multiple sets of similar objects. These objects are similar to objects in the same collection and are distinct from objects in other collections. The clustering algorithm may include, but is not limited to, at least one of: a k-means (k-means) Clustering algorithm, a mean shift Clustering algorithm, a pixel Clustering algorithm, a hierarchical Clustering algorithm, a spectral Clustering algorithm, and a DEC _ DCNN (Deep Embedded Clustering based on Deep probabilistic Neural Network) algorithm. The first clustering result may include a plurality of sample surface feature clusters. The sample surface features in the same sample surface feature cluster are similar, and the sample surface features in different sample surface feature clusters are different.

And step 404, generating a defect category set based on the first clustering result.

In this embodiment, the execution subject may generate a defect class set based on the first clustering result. In practice, the surface features of parts with defects of the same defect type are similar, and the surface features of parts with defects of different defect types are different, so that one sample surface feature cluster in the first clustering result can correspond to one defect type. Here, for each sample surface feature cluster in the first clustering result, the executing entity may analyze the sample surface feature cluster to determine a defect type corresponding to the sample surface feature cluster.

In some optional implementation manners of this embodiment, for each preset angle in the plurality of preset angles, the executing body may further determine a defect type corresponding to a sample surface feature cluster in which the sample surface features of the sample surface image at the preset angle are distributed as the defect type that can be acquired by the image acquisition device at the preset angle.

With further reference to FIG. 5, a flow 500 of one embodiment of a method for determining a particular angle is illustrated. The method for determining a specific angle comprises the following steps:

step 501, dividing a sample surface feature set into a plurality of sample surface feature subsets corresponding to a plurality of preset angles.

In this embodiment, an implementation subject of the method for determining a specific angle (e.g., the server 105 shown in fig. 1) may divide the sample surface feature set into a plurality of sample surface feature subsets corresponding to a plurality of preset angles. Typically, sample surface images of a sample part taken at the same preset angle of the sample part belong to the same sample surface feature subset.

And 502, clustering the surface feature subsets of the samples respectively by using a clustering algorithm to obtain a plurality of second clustering results corresponding to a plurality of preset angles.

In this embodiment, the executing entity may respectively cluster the plurality of sample surface feature subsets by using a clustering algorithm to obtain a plurality of second clustering results corresponding to a plurality of preset angles. Wherein the second clustering result corresponding to each preset angle may include a plurality of sample surface feature clusters.

It should be noted that the clustering algorithm is the same as that in the embodiment shown in fig. 4, and is not described herein again.

Step 503, determining a plurality of specific angles from a plurality of preset angles based on a plurality of second clustering results.

In this embodiment, the executing body may determine the specific angles from the preset angles based on the second clustering results. For example, the executing entity may select, based on the clustering effect of the second clustering result corresponding to each preset angle in the plurality of preset angles, a plurality of specific angles of all defect classes in the defect class set that have a good clustering effect and can be collected from the plurality of preset angles. Generally, the higher the similarity of the surface features of the samples in the same sample surface feature cluster is, the lower the similarity of the surface features of the samples in different sample surface feature clusters is, and the better the clustering effect is.

With further reference to FIG. 6, a flow 600 of one embodiment of a method for training a set of defect detection models is illustrated. The method for training the defect detection model set comprises the following steps:

step 601, for the defect detection model in the defect detection model set, selecting a sample surface image subset corresponding to the defect detection model from the sample surface image set.

In this embodiment, for each defect detection model in the defect detection model set, an executive (e.g., the server 105 shown in fig. 1) of the method for training the defect detection model set may select a sample surface image subset corresponding to the defect detection model from the sample surface image set. In general, the sample surface image subset corresponding to one of the set of defect detection models may be a sample surface image subset of the sample part acquired at one of a plurality of specific angles of the sample part.

Step 602, a sample surface feature subset is extracted from a sample surface image subset corresponding to the defect detection model by using a feature extraction algorithm.

In this embodiment, for each sample surface image in the sample surface image subset corresponding to the defect detection model, the executing entity may extract the sample surface features from the sample surface image by using a feature extraction algorithm, and add the sample surface features to the sample surface feature subset corresponding to the defect detection model.

In some optional implementations of this embodiment, the executing body may first perform preprocessing on the sample surface image subset corresponding to the defect detection model by using a preprocessing method, and then perform step 602.

In some optional implementations of this embodiment, the preprocessing method may include, but is not limited to, at least one of: adjusting at least one of brightness, gray scale and contrast; adjusting at least one of a size and an offset; and (4) self-adaptive map cutting.

In some optional implementations of this embodiment, for a sample surface image in the sample surface image subset, the executing entity may first convert the sample surface image into a corresponding grayscale surface image; then binarizing the gray surface image corresponding to the sample surface image to obtain a binarized surface image corresponding to the sample surface image; then determining the outline of the sample part in the binarization surface image corresponding to the sample surface image; and finally, cutting the sample surface image based on the outline of the sample part in the binarization surface image corresponding to the sample surface image.

In some optional implementations of this embodiment, for a sample surface image in the sample surface image subset, if a sample part in the sample surface image is tilted, the execution subject may first calculate an affine transformation matrix and a rotation matrix of the sample surface image; and then carrying out affine transformation on the sample surface image by using the affine transformation matrix and the rotation matrix of the sample surface image to generate a new sample surface image corresponding to the sample surface image. The translation, the scaling, the rotation and the like of the image are realized through affine transformation to generate a new sample surface image, so that the amplification of a training sample is realized, and the generalization capability of the trained defect detection model is improved.

In some optional implementations of this embodiment, for a sample surface image in the sample surface image subset, if the defect type of the sample part in the sample surface image is edge scratch or dirt, the executing entity may adjust the number of decision regions of the sample surface image. Generally, the similarity of the edge bruising or smudging on the image visual characteristics is high, the executing subject can appropriately adjust the number of decision areas of the sample surface image, and then the false detection rate can be effectively reduced through a threshold deletion method. For example, the RFCN (Region-based full convolution Networks) algorithm divides the target Region into k × k decision regions, calculates the similarity of the k × k decision regions, respectively, and takes the average value as the overall similarity to participate in the defect type similarity calculation. The greatest difference in visual characteristics on images due to edge bruising and smudging is: the edges that bruise such defects are irregularly jagged, while the dirty edges are smoothly curved. Therefore, more edge areas participate in decision making by properly increasing the k value, edge bruising and dirt can be distinguished in similarity, and finally the dirt can be effectively prevented from being falsely detected as edge bruising by a threshold deletion method.

In some optional implementations of this embodiment, for a sample surface image in the sample surface image subset, if the defect type existing in the sample part in the sample surface image is peeling, the executing subject may adjust brightness and contrast of the sample surface image. Because peeling is sensitive to brightness change characteristics, missing detection and false detection are easily caused if brightness changes in the image acquisition process. Because the only image characteristic of skinning is darker than the color of a normal part and lighter than the color of a dirty part, the skinning detection accuracy is improved by properly adjusting the brightness and contrast of the image to highlight the image characteristic of skinning and by using an image segmentation method.

In some optional implementations of this embodiment, for a sample surface image in the sample surface image subset, if the defect type of the sample part in the sample surface image is deformation, the executing entity may adjust the contrast of the sample surface image, enlarge the detection frame of the sample surface image, and perform pixel clustering on the detection frame of the sample surface image. In general, the deformed image visual characteristics are very insignificant, and the requirements on the image acquisition angle and the polishing are very high. By capturing the image from 10 degrees from the deformed surface, the deformation has image features with varying brightness on the image. By properly adjusting the contrast, the image characteristics of the light and shade change are more obvious, and then the deformed part is detected by using a deep learning method. In addition, because the deformation generally appears in the middle of a certain surface of a part, the area is larger than other defects, and the position change range is smaller, in order to improve the accuracy, the detection frame is enlarged according to a certain proportion, so that the detection range is wider, and the accuracy is higher. And carrying out pixel clustering on the detection frame after expansion, and determining whether deformation exists through analysis of clustering results.

In some optional implementations of this embodiment, for a sample surface image in the sample surface image subset, if the defect type existing in the sample part in the sample surface image is a window burr, the executing entity may extract a window contour of the sample part in the sample surface image, store vertical coordinates of pixel values of two horizontal sides and horizontal coordinates of pixel values of two vertical sides of the window contour of the sample part in the sample surface image into a one-dimensional array, normalize the pixel values of the window contour of the sample part in the sample surface image, and generate a line graph of the window contour based on the normalized pixel values. Generally, the normal window outline is rectangular, the window outline with burrs has concave-convex phenomena, the vertical coordinates of the pixel values of two transverse edges and the horizontal coordinates of the pixel values of two vertical edges of the window outline are taken to be stored into a one-dimensional array, all the pixel values of the window outline are converted into the range of 0 to 1 through normalization processing, and a line graph is drawn. The line graph of the normal window is a four-segment straight line, and the line graph of the window with burrs has abnormal values. For example, higher or lower than the surrounding pixel values, or a curved distribution. The detection of window burrs is realized by detecting the abnormal values.

In some optional implementations of this embodiment, for a sample surface image in the sample surface image subset, if the defect type existing in the sample part in the sample surface image is a flow mark, the execution subject may expand the detection frame of the sample surface image and perform pixel clustering on the detection frame of the sample surface image. Generally, the image features of the flow marks are obvious, and if the deep learning method is directly used for detection, the defect of high false detection rate exists. In order to reduce the false detection rate, the detection frame is expanded, the pixel clustering is carried out on the expanded detection frame, and the flow pattern passes through the obvious boundary with the normal area after the pixel clustering, so that the false detection rate of the flow pattern is effectively reduced.

Step 603, taking the sample surface features in the sample surface feature subset corresponding to the defect detection model as input, taking the defect type of the sample part corresponding to the input sample surface features as output, and training to obtain the defect detection model.

In this embodiment, the executing entity may obtain the defect detection model by training with a deep learning method, with the sample surface features in the sample surface feature subset corresponding to the defect detection model as input, and the defect type of the sample part corresponding to the input sample surface features as output.

In general, the execution body may input the surface features of the sample from the input side of the target detection model, and output the detection result of the sample part from the output side through the processing of the target detection model. Subsequently, the execution body may target the detection accuracy of the detection model based on the detection result of the sample part and the defect classification of the sample part. And if the detection accuracy does not meet the preset constraint condition, adjusting the parameters of the target detection model, and then inputting the surface characteristics of the sample to continue the model training. And if the detection accuracy meets the preset constraint condition, finishing the model training, wherein the surface characteristic of the sample at the moment is the defect detection model.

With further reference to fig. 7, as an implementation of the method shown in the above figures, the present application provides an embodiment of an apparatus for detecting a defect of a part, which corresponds to the embodiment of the method shown in fig. 2, and which is particularly applicable to various electronic devices.

As shown in fig. 7, the apparatus 700 for detecting a part defect of the present embodiment may include: an acquisition unit 701, an extraction unit 702, and a detection unit 703. The acquiring unit 701 is configured to acquire a to-be-detected surface image set of a to-be-detected part acquired at a plurality of specific angles of the to-be-detected part; an extraction unit 702 configured to extract a to-be-detected surface feature set of a to-be-detected part from a to-be-detected surface image set by using a feature extraction algorithm; the detecting unit 703 is configured to detect the surface feature set to be detected by using a defect detection model set trained in advance, so as to obtain a defect detection result of the part to be detected, where the defect detection model set is used to detect defects existing in the part, and the detection result includes defect categories existing in the part to be detected.

In the present embodiment, in the apparatus 700 for detecting a part defect: the specific processing of the obtaining unit 701, the extracting unit 702, and the detecting unit 703 and the technical effects thereof can refer to the related descriptions of step 201, step 202, and step 203 in the corresponding embodiment of fig. 2, which are not repeated herein.

In some optional implementations of the present embodiment, the apparatus 700 for detecting a part defect further includes: a preprocessing unit (not shown in the figures) configured to preprocess the set of surface images to be detected by a preprocessing method.

In some optional implementation manners of this embodiment, the detection result further includes a position of a defect type of the part to be detected and a confidence level of the defect type of the part to be detected in the predetermined defect type set.

In some optional implementations of this embodiment, the detection unit 703 is further configured to: for the defect type in the defect type set, if the defect detection models in the defect detection model set detect that the part to be detected has the defect type, the confidence coefficient that the part to be detected has the defect type is generated based on the accuracy of the defect detection models and the confidence coefficient that the part to be detected has the defect type.

In some optional implementations of the present embodiment, the apparatus 700 for detecting a part defect further includes: and a sending unit (not shown in the figure) configured to send the display information to the terminal equipment based on the defect detection result of the part to be detected.

In some optional implementations of this embodiment, the sending unit is further configured to: acquiring a defect marking result of a part to be detected; generating the accuracy and the recall rate of a defect detection model set based on the defect detection result and the defect labeling result of the part to be detected; and sending at least one of the accuracy, the recall rate, the recall-ready rate curve and the subject working characteristic curve of the defect detection model set to the terminal equipment.

In some optional implementations of this embodiment, the sending unit is further configured to: generating the yield and the defect rate of the parts to be detected based on the defect detection result and the defect marking result of the parts to be detected; and sending at least one of the yield curve and the defect rate curve of the part to be detected to the terminal equipment.

In some optional implementations of the present embodiment, the apparatus 700 for detecting a part defect includes: a determining unit (not shown in the figure) configured to determine whether the yield curve is in a first preset range or whether the defect rate curve is in a second preset range; and the alarm unit (not shown in the figure) is configured to send an alarm command if the yield curve exceeds a first preset range or the defect rate curve exceeds a second preset range.

In some optional implementations of this embodiment, the defect class set is determined by: acquiring a sample surface image set of a sample part acquired at a plurality of preset angles of the sample part; extracting a sample surface feature set of the sample part from the sample surface image set by using a feature extraction algorithm; clustering the sample surface feature set by using a clustering algorithm to obtain a first clustering result; and generating a defect category set based on the first clustering result.

In some optional implementations of this embodiment, the plurality of specific angles are determined by: dividing the sample surface feature set into a plurality of sample surface feature subsets corresponding to a plurality of preset angles; clustering the surface feature subsets of the samples respectively by using a clustering algorithm to obtain a plurality of second clustering results corresponding to a plurality of preset angles; a plurality of specific angles are determined from a plurality of preset angles based on the plurality of second clustering results.

In some optional implementations of this embodiment, the clustering algorithm includes at least one of a k-means clustering algorithm, a mean shift clustering algorithm, a pixel clustering algorithm, a hierarchical clustering algorithm, a spectral clustering algorithm, and a deep-embedded clustering algorithm based on a deep convolutional neural network.

In some optional implementations of this embodiment, the defect detection model set is trained by the following steps: selecting a sample surface image subset corresponding to the defect detection model from the sample surface image set for the defect detection model in the defect detection model set; extracting a sample surface feature subset from a sample surface image subset corresponding to the defect detection model by using a feature extraction algorithm; and training to obtain the defect detection model by taking the sample surface features in the sample surface feature subset corresponding to the defect detection model as input and taking the defect types of the sample parts corresponding to the input sample surface features as output.

In some optional implementations of the embodiment, the sample surface image subset corresponding to one of the defect detection models in the defect detection model set is a sample surface image subset of the sample part acquired at one of a plurality of specific angles of the sample part.

In some optional implementations of this embodiment, the feature extraction algorithm includes at least one of a candidate area generation network algorithm, a feature pyramid network algorithm, and a deep network algorithm.

In some optional implementations of this embodiment, before extracting, by using a feature extraction algorithm, a sample surface feature subset from a sample surface image subset corresponding to the defect detection model, the method further includes: and preprocessing the sample surface image subset corresponding to the defect detection model by using a preprocessing method.

In some optional implementations of this embodiment, the preprocessing method includes at least one of: adjusting at least one of brightness, gray scale and contrast; adjusting at least one of a size and an offset; and (4) self-adaptive map cutting.

In some optional implementations of this embodiment, the preprocessing the sample surface image subset corresponding to the defect detection model by using a preprocessing method includes: for a sample surface image in the sample surface image subset, converting the sample surface image into a corresponding grayscale surface image; binarizing the gray surface image corresponding to the sample surface image to obtain a binarized surface image corresponding to the sample surface image; determining the outline of a sample part in a binarization surface image corresponding to the sample surface image; and cutting the sample surface image based on the outline of the sample part in the binarization surface image corresponding to the sample surface image.

In some optional implementations of this embodiment, the preprocessing the sample surface image subset corresponding to the defect detection model by using a preprocessing method includes: for a sample surface image in the sample surface image subset, if a sample part in the sample surface image is inclined, calculating an affine transformation matrix and a rotation matrix of the sample surface image; and carrying out affine transformation on the sample surface image by utilizing the affine transformation matrix and the rotation matrix of the sample surface image to generate a new sample surface image corresponding to the sample surface image.

In some optional implementations of this embodiment, the preprocessing the sample surface image subset corresponding to the defect detection model by using a preprocessing method includes: for a sample surface image in the sample surface image subset, adjusting the number of decision regions of the sample surface image if the defect type of the sample part in the sample surface image is edge scratch or dirty.

In some optional implementations of this embodiment, the preprocessing the sample surface image subset corresponding to the defect detection model by using a preprocessing method includes: and for the sample surface image in the sample surface image subset, if the defect type of the sample part in the sample surface image is peeling, adjusting the brightness and the contrast of the sample surface image.

In some optional implementations of this embodiment, the preprocessing the sample surface image subset corresponding to the defect detection model by using a preprocessing method includes: and for the sample surface images in the sample surface image subset, if the defect type of the sample part in the sample surface images is deformation, adjusting the contrast of the sample surface images, expanding the detection frame of the sample surface images, and performing pixel clustering on the detection frame of the sample surface images.

In some optional implementations of this embodiment, the preprocessing the sample surface image subset corresponding to the defect detection model by using a preprocessing method includes: for a sample surface image in a sample surface image subset, if a defect type of a sample part in the sample surface image is a window burr, extracting a window outline of the sample part in the sample surface image, storing a vertical coordinate of pixel values of two horizontal sides and a horizontal coordinate of pixel values of two vertical sides of the window outline of the sample part in the sample surface image into a one-dimensional array, normalizing the pixel values of the window outline of the sample part in the sample surface image, and generating a line graph of the window outline based on the normalized pixel values.

In some optional implementations of this embodiment, the preprocessing the sample surface image subset corresponding to the defect detection model by using a preprocessing method includes: and for the sample surface images in the sample surface image subset, if the defect type of the sample part in the sample surface images is the flow mark, expanding the detection frame of the sample surface images, and performing pixel clustering on the detection frame of the sample surface images.

Referring now to FIG. 8, a block diagram of a computer system 800 suitable for use in implementing a server (e.g., server 105 of FIG. 1) of an embodiment of the present application is shown. The server shown in fig. 8 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.

As shown in fig. 8, the computer system 800 includes a Central Processing Unit (CPU)801 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)802 or a program loaded from a storage section 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data necessary for the operation of the system 800 are also stored. The CPU 801, ROM 802, and RAM 803 are connected to each other via a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.

The following components are connected to the I/O interface 805: an input portion 806 including a keyboard, a mouse, and the like; an output section 807 including a signal such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 808 including a hard disk and the like; and a communication section 809 including a network interface card such as a LAN card, a modem, or the like. The communication section 809 performs communication processing via a network such as the internet. A drive 810 is also connected to the I/O interface 805 as necessary. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as necessary, so that a computer program read out therefrom is mounted on the storage section 808 as necessary.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 809 and/or installed from the removable medium 811. The computer program, when executed by a Central Processing Unit (CPU)701, performs the above-described functions defined in the method of the present application.

It should be noted that the computer readable medium described herein can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in the embodiments of the present application may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor includes an acquisition unit, an extraction unit, and a detection unit. The names of the units do not in some cases constitute a limitation to the unit itself, and for example, the acquiring unit may also be described as a unit that acquires a set of surface images to be detected of a part to be detected acquired at a plurality of specific angles of the part to be detected.

As another aspect, the present application also provides a computer-readable medium, which may be contained in the server described in the above embodiments; or may exist separately and not be assembled into the server. The computer readable medium carries one or more programs which, when executed by the server, cause the server to: acquiring a to-be-detected surface image set of a to-be-detected part acquired at a plurality of specific angles of the to-be-detected part; extracting a surface feature set to be detected of the part to be detected from the surface image set to be detected by using a feature extraction algorithm; and detecting the surface feature set to be detected by using a pre-trained defect detection model set to obtain a defect detection result of the part to be detected, wherein the defect detection model set is used for detecting defects of the part, and the detection result comprises defect categories of the part to be detected.

The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the invention. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.

Claims

1. A method for detecting a defect in a part, comprising:

acquiring a to-be-detected surface image set of a to-be-detected part, which is acquired at a plurality of specific angles of the to-be-detected part;

preprocessing the surface image set to be detected by utilizing a preprocessing method;

extracting a to-be-detected surface feature set of the to-be-detected part from the to-be-detected surface image set by using a feature extraction algorithm;

detecting the surface feature set to be detected by using a pre-trained defect detection model set to obtain a defect detection result of the part to be detected, wherein the defect detection model set is used for detecting defects of the part, and the detection result comprises defect types of the part to be detected;

the method comprises the following steps of utilizing a preprocessing method to preprocess the surface image set to be detected, wherein the preprocessing comprises at least one of the following steps:

increasing the number of decision areas of the surface image to be detected in the surface image set to be detected, wherein the decision areas are areas formed by dividing the area of a part to be detected in the surface image to be detected through an area-based full convolution network RFCN algorithm;

adjusting the contrast of the surface image to be detected in the surface image set to be detected, expanding the detection frame of the surface image to be detected, and performing pixel clustering on the detection frame of the surface image to be detected;

extracting a window contour of a part to be detected in a surface image to be detected in the surface image set to be detected, storing the ordinate of the pixel values of two transverse edges and the abscissa of the pixel values of two vertical edges of the window contour of the part to be detected in the surface image to be detected into a one-dimensional array, normalizing the pixel values of the window contour of the part to be detected in the surface image to be detected, and generating a line drawing of the window contour based on the normalized pixel values;

and expanding the detection frame of the surface image to be detected in the surface image set to be detected, and carrying out pixel clustering on the detection frame of the surface image to be detected.

2. The method according to claim 1, wherein the inspection result further comprises a location of a defect class present in the part to be inspected and a confidence level of the defect class present in the predetermined set of defect classes.

3. The method according to claim 2, wherein the detecting the set of surface features to be detected by using a pre-trained defect detection model set to obtain a defect detection result of the part to be detected comprises:

for the defect type in the defect type set, if the defect detection models in the defect detection model set detect that the part to be detected has the defect type, generating a confidence coefficient that the part to be detected has the defect type based on the accuracy of the defect detection models and the confidence coefficient that the part to be detected has the defect type.

4. The method according to claim 1, wherein after the detecting the set of surface features to be detected by using the pre-trained defect detection model set to obtain a defect detection result of the part to be detected, the method further comprises:

and sending display information to the terminal equipment based on the defect detection result of the part to be detected.

5. The method according to claim 4, wherein the sending of the display information to the terminal device based on the defect detection result of the part to be detected comprises:

acquiring a defect marking result of the part to be detected;

generating the accuracy and the recall rate of the defect detection model set based on the defect detection result and the defect labeling result of the part to be detected;

and sending at least one of the accuracy, the recall rate, the recall-ready rate curve and the subject working characteristic curve of the defect detection model set to the terminal equipment.

6. The method according to claim 5, wherein the sending of the display information to the terminal device based on the defect detection result of the part to be detected further comprises:

generating the yield and the defect rate of the parts to be detected based on the defect detection result and the defect marking result of the parts to be detected;

and sending at least one of the yield curve and the defect rate curve of the part to be detected to the terminal equipment.

7. The method according to claim 6, wherein after said sending at least one of a yield curve and a defect rate curve of the part to be detected to the terminal device, comprising:

determining whether the yield curve is in a first preset range or determining whether the defect rate curve is in a second preset range;

and if the yield curve exceeds the first preset range or the defect rate curve exceeds the second preset range, sending an alarm command.

8. The method of claim 3, wherein the set of defect classes is determined by:

acquiring a sample surface image set of a sample part acquired at a plurality of preset angles of the sample part;

extracting a sample surface feature set of the sample part from the sample surface image set using the feature extraction algorithm;

clustering the sample surface feature set by using a clustering algorithm to obtain a first clustering result;

and generating the defect category set based on the first clustering result.

9. The method of claim 8, wherein the plurality of specific angles are determined by:

dividing the sample surface feature set into a plurality of sample surface feature subsets corresponding to the preset angles;

clustering the sample surface feature subsets respectively by using the clustering algorithm to obtain a plurality of second clustering results corresponding to the preset angles;

determining the plurality of specific angles from the plurality of preset angles based on the plurality of second clustering results.

10. The method of claim 8 or 9, wherein the clustering algorithm comprises at least one of a k-means clustering algorithm, a mean shift clustering algorithm, a pixel clustering algorithm, a hierarchical clustering algorithm, a spectral clustering algorithm, and a deep-embedded clustering algorithm based on a deep convolutional neural network.

11. The method of claim 8, wherein the set of defect detection models is trained by:

selecting a sample surface image subset corresponding to the defect detection model from the sample surface image set for the defect detection model in the defect detection model set;

extracting a sample surface feature subset from a sample surface image subset corresponding to the defect detection model by using the feature extraction algorithm;

and training to obtain the defect detection model by taking the sample surface features in the sample surface feature subset corresponding to the defect detection model as input and taking the defect types of the sample parts corresponding to the input sample surface features as output.

12. The method of claim 11, wherein the subset of sample surface images corresponding to one of the set of defect detection models is a subset of sample surface images of the sample part acquired at one of the plurality of particular angles of the sample part.

13. The method of claim 11, wherein the feature extraction algorithm comprises at least one of a candidate area generation network algorithm, a feature pyramid network algorithm, and a deep mesh algorithm.

14. The method of claim 11, wherein prior to said extracting a subset of sample surface features from a corresponding subset of sample surface images of the defect inspection model using the feature extraction algorithm, further comprising:

and preprocessing the sample surface image subset corresponding to the defect detection model by using the preprocessing method.

15. The method of claim 14, wherein the pre-processing method comprises at least one of:

adjusting at least one of brightness, gray scale and contrast;

adjusting at least one of a size and an offset;

and (4) self-adaptive map cutting.

16. The method of claim 15, wherein the preprocessing the sample surface image subset corresponding to the defect detection model by the preprocessing method comprises:

for a sample surface image in the subset of sample surface images, converting the sample surface image into a corresponding grayscale surface image;

binarizing the gray surface image corresponding to the sample surface image to obtain a binarized surface image corresponding to the sample surface image;

determining the outline of a sample part in a binarization surface image corresponding to the sample surface image;

and cutting the sample surface image based on the outline of the sample part in the binarization surface image corresponding to the sample surface image.

17. The method of claim 15, wherein the preprocessing the sample surface image subset corresponding to the defect detection model by the preprocessing method comprises:

for the sample surface images in the sample surface image subset, if the sample parts in the sample surface images are inclined, calculating an affine transformation matrix and a rotation matrix of the sample surface images;

and carrying out affine transformation on the sample surface image by utilizing the affine transformation matrix and the rotation matrix of the sample surface image to generate a new sample surface image corresponding to the sample surface image.

18. The method of claim 15, wherein the preprocessing the sample surface image subset corresponding to the defect detection model by the preprocessing method comprises:

for the sample surface images in the sample surface image subset, if the defect type of the sample parts in the sample surface images is edge damage or dirt, the number of decision areas of the sample surface images is increased, wherein the decision areas are areas formed by dividing the sample part areas in the sample surface images through an area-based full convolution network (RFCN) algorithm.

19. The method of claim 15, wherein the preprocessing the sample surface image subset corresponding to the defect detection model by the preprocessing method comprises:

and for the sample surface image in the sample surface image subset, if the defect type of the sample part in the sample surface image is peeling, adjusting the brightness and the contrast of the sample surface image.

20. The method of claim 15, wherein the preprocessing the sample surface image subset corresponding to the defect detection model by the preprocessing method comprises:

and for the sample surface images in the sample surface image subset, if the defect type of the sample part in the sample surface images is deformation, adjusting the contrast of the sample surface images, expanding the detection frame of the sample surface images, and performing pixel clustering on the detection frame of the sample surface images.

21. The method of claim 15, wherein the preprocessing the sample surface image subset corresponding to the defect detection model by the preprocessing method comprises:

for the sample surface images in the sample surface image subset, if the defect type of the sample part in the sample surface image is window burr, extracting the window outline of the sample part in the sample surface image, storing the ordinate of the pixel values of the two transverse sides and the abscissa of the pixel values of the two vertical sides of the window outline of the sample part in the sample surface image into a one-dimensional array, normalizing the pixel values of the window outline of the sample part in the sample surface image, and generating a line graph of the window outline based on the normalized pixel values.

22. The method of claim 15, wherein the preprocessing the sample surface image subset corresponding to the defect detection model by the preprocessing method comprises:

and for the sample surface images in the sample surface image subset, if the defect type of the sample part in the sample surface images is the flow pattern, expanding the detection frame of the sample surface images, and performing pixel clustering on the detection frame of the sample surface images.

23. An apparatus for detecting defects in a part, comprising:

an acquisition unit configured to acquire a set of to-be-detected surface images of a to-be-detected part acquired at a plurality of specific angles of the to-be-detected part;

the preprocessing unit is configured to preprocess the surface image set to be detected by utilizing a preprocessing method;

an extraction unit configured to extract a to-be-detected surface feature set of the to-be-detected part from the to-be-detected surface image set by using a feature extraction algorithm;

the detection unit is configured to detect the surface feature set to be detected by using a defect detection model set trained in advance to obtain a defect detection result of the part to be detected, wherein the defect detection model set is used for detecting defects of the part, and the detection result comprises defect categories of the part to be detected;

wherein the pre-processing unit is further configured to perform at least one of:

24. The apparatus of claim 23, wherein the apparatus further comprises:

and the sending unit is configured to send display information to the terminal equipment based on the defect detection result of the part to be detected.

25. The apparatus of claim 23, wherein the set of defect classes is determined by:

and generating the defect category set based on the first clustering result.

26. The apparatus of claim 25, wherein the plurality of specific angles are determined by:

27. The apparatus of claim 25, wherein the set of defect detection models is trained by:

28. The apparatus of claim 27, wherein prior to said extracting a subset of sample surface features from a corresponding subset of sample surface images of the defect inspection model using said feature extraction algorithm, further comprising:

29. A server, comprising:

one or more processors;

a storage device having one or more programs stored thereon,

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-22.

30. A computer-readable medium, on which a computer program is stored, wherein the computer program, when being executed by a processor, carries out the method according to any one of claims 1-22.