CN117392042A - Defect detection method, defect detection apparatus, and storage medium - Google Patents

Defect detection method, defect detection apparatus, and storage medium Download PDF

Info

Publication number
CN117392042A
CN117392042A CN202210762871.1A CN202210762871A CN117392042A CN 117392042 A CN117392042 A CN 117392042A CN 202210762871 A CN202210762871 A CN 202210762871A CN 117392042 A CN117392042 A CN 117392042A
Authority
CN
China
Prior art keywords
defect
target
image
images
target image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210762871.1A
Other languages
Chinese (zh)
Inventor
严鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ZTE Corp
Original Assignee
ZTE Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ZTE Corp filed Critical ZTE Corp
Priority to CN202210762871.1A priority Critical patent/CN117392042A/en
Priority to PCT/CN2023/103337 priority patent/WO2024002187A1/en
Publication of CN117392042A publication Critical patent/CN117392042A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Quality & Reliability (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
  • Image Analysis (AREA)

Abstract

The embodiment of the invention provides a defect detection method, defect detection equipment and a storage medium, and belongs to the technical field of artificial intelligence. The defect detection method comprises the following steps: acquiring a target image set of a target to be detected, and determining whether a target image in the target image set has a suspected defect area or not; the suspected defect area is a defect area possibly existing on the target to be detected; obtaining a target image with a suspected defect area to obtain a defect image set; inputting the defect image set into a trained classification network model; and identifying suspected defect areas on the target image in the defect image set through the trained classification network model to obtain the real defect areas and corresponding real defect types of the target image. According to the technical scheme provided by the embodiment of the invention, the suspected defect area is firstly obtained, and then the classification network model is used for classifying, so that a large number of image samples of the target to be detected are not required to be obtained, and the defect area and the defect type of the target to be detected can still be accurately identified.

Description

Defect detection method, defect detection apparatus, and storage medium
Technical Field
The present invention relates to the field of artificial intelligence, and in particular, to a defect detection method, a defect detection apparatus, and a storage medium.
Background
With the continuous progress of society, users and manufacturers have increasingly high requirements on the quality of various products. Many users and manufacturing enterprises not only require the service performance of the product to reach or even exceed the standard, but also require the product to have good appearance. Good appearance, i.e. good surface quality, and defects on the product surface are often unavoidable in practical product manufacturing processes.
Surface defects of different products are defined and typed differently, and in general, surface defects are areas of non-uniform physical or chemical properties of the product surface locally; such as scratches, spots, holes on the metal surface, chromatic aberration on the paper surface, indentation, inclusions, breakage, stains, etc. on the non-metal surface of glass, etc. The surface defects of the product not only affect the beauty and comfort of the product, indirectly lead to the reduction of sales, but also can bring adverse effects to the service performance of the product. For this reason, many manufacturers are very concerned with surface defect detection of products.
The traditional product surface defect detection method is manual spot inspection. However, the manual spot check has the defects of low spot check rate, low accuracy, poor real-time performance, low efficiency, high labor intensity, large influence by manual experience and subjective factors and the like. Therefore, some manufacturers adopt a defect detection method based on a deep learning model, but the method needs to collect a large number of samples in the early stage, and the deep learning model is trained through the large number of samples so as to ensure the detection accuracy.
The collection of a large number of samples in the early stage consumes manpower and material resources, and the effective samples in some production industries are fewer and difficult to collect in a large quantity, so that a certain difficulty is brought to the implementation of a defect detection method based on a deep learning model.
Disclosure of Invention
The embodiment of the invention provides a defect detection method, defect detection equipment and a storage medium, which aim to avoid collecting a large number of samples and detect defects of various targets.
In a first aspect, an embodiment of the present invention provides a defect detection method, including:
acquiring a target image set of a target to be detected, and determining whether a target image in the target image set has a suspected defect area or not; the suspected defect area is a defect area possibly existing on the target to be detected;
obtaining a target image with the suspected defect area to obtain a defect image set;
inputting the defect image set into a trained classification network model;
and identifying suspected defect areas on the target image in the defect image set through the trained classification network model to obtain real defect areas and corresponding real defect types of the target image.
According to the scheme, the suspected defect areas are obtained firstly, and then the suspected defect areas are classified by using the classification network model, and the real defect areas and the corresponding defect types of each real defect area can be accurately detected only by identifying the suspected defect areas because the suspected defect areas are obtained in advance. At this time, a large number of image samples of the target to be detected are not required to be acquired, and the defect area and the defect type of the target to be detected can still be accurately identified.
In a second aspect, an embodiment of the present invention further provides a defect detection apparatus, the defect detection apparatus including a processor, a memory, a computer program stored on the memory and executable by the processor, and a data bus for enabling connection communication between the processor and the memory, wherein the computer program, when executed by the processor, implements any of the defect detection methods as provided in the present specification.
In a third aspect, embodiments of the present invention further provide a storage medium for computer-readable storage, where the storage medium stores one or more programs executable by one or more processors to implement any one of the defect detection methods provided in the present specification.
The embodiment of the invention provides a defect detection method, defect detection equipment and a storage medium, which are used for accurately detecting real defect areas and corresponding defect types of each real defect area by acquiring suspected defect areas and classifying the suspected defect areas by using a classification network model and only identifying the suspected defect areas because the suspected defect areas are acquired in advance. At this time, a large number of image samples of the target to be detected are not required to be acquired, and the defect area and the defect type of the target to be detected can still be accurately identified.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a defect detection method according to an embodiment of the present invention;
FIG. 2 is a flow chart illustrating sub-steps of the defect detection method of FIG. 1;
FIG. 3 is a flow chart illustrating sub-steps of the defect detection method of FIG. 2;
FIG. 4 is a flowchart of a tool for measuring a halcon caliper in a defect detection method according to an embodiment of the present invention;
FIG. 5 is a schematic alignment diagram of a target image to be detected and a standard template object in a defect detection method according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a software interface for detecting defects of a power module according to an embodiment of the present invention;
fig. 7 is a schematic block diagram of a defect detecting apparatus according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The flow diagrams depicted in the figures are merely illustrative and not necessarily all of the elements and operations/steps are included or performed in the order described. For example, some operations/steps may be further divided, combined, or partially combined, so that the order of actual execution may be changed according to actual situations.
It is to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
The embodiment of the invention provides a defect detection method, defect detection equipment and a storage medium. The defect detection method can be applied to mobile terminals, and the mobile terminals can be mobile phones, tablet computers, notebook computers, desktop computers, personal digital assistants, wearable devices and other electronic devices.
Some embodiments of the invention are described in detail below with reference to the accompanying drawings. The following embodiments and features of the embodiments may be combined with each other without conflict.
Hereinafter, a defect detection method provided by an embodiment of the present invention will be described in detail with reference to the scenario in fig. 1. It should be noted that, the scenario in fig. 1 is only used to explain the defect detection method provided by the embodiment of the present invention, but does not constitute limitation of the application scenario of the defect detection method provided by the embodiment of the present invention.
Referring to fig. 1, fig. 1 is a flowchart of a defect detection method according to an embodiment of the invention, where the defect detection method includes steps S101 to S104.
Step S101, acquiring a target image set of a target to be detected, and determining whether a target image in the target image set has a suspected defect area or not; the suspected defect area is a defect area possibly existing on the target to be detected, for example, a scratch, a pit and other defects possibly existing on the target to be detected.
In the embodiment of the invention, an industrial camera interface can be called, the industrial camera is started in a soft triggering mode, and the sample image of the target to be detected is acquired. Of course, the industrial camera may be any industrial camera that images normally.
Preferably, in an embodiment of the present invention, the resolution of the image may be 2592x1944, an industrial camera such as a basler industrial camera, or the like.
Further, the object to be detected may be a device on a production line, for example, a power module on a power production line.
Furthermore, since the production line is to transport the object to be detected, the production line is generally moving. In this case, the photographing time of the camera needs to be set to ensure that all the images of the objects to be detected on the production line are collected. Therefore, in the embodiment of the present invention, the time interval for photographing by the industrial camera may be set according to the transmission speed of the production line, for example, the interval may be 2s.
In step S101, it may be determined whether a suspected defect area exists in the target image set by some image acquisition technique.
Preferably, in an embodiment of the present invention, referring to fig. 2, fig. 2 is a schematic flow chart of sub-steps of the defect detection method in fig. 1, and step S101 may include:
s1011, acquiring a standard template image by utilizing a plurality of target images, wherein the standard template image is formed by mixing Gaussian background modeling through the plurality of target images.
The target images can be selected from the target image set, and the target images can be acquired from the production line through an industrial camera.
It should be emphasized that several target images need to be acquired at different angles and different light intensities.
It can be understood that the mixed gaussian background modeling is a background representation method based on statistical information of pixel samples, uses statistical information (such as the number of modes, the mean value and standard deviation of each mode) such as probability density of a large number of sample values of pixels in a long time to represent the background, and then uses statistical difference to judge the target pixels, so that the complex dynamic background can be modeled.
In the mixed gaussian background model, the color information between pixels is considered to be uncorrelated with each other, so that the processing of each pixel point is independent of each other. For each pixel point in the target image, the change of the value in the sequence image can be regarded as a random process of continuously generating the pixel value, namely, describing the color rendering rule of each pixel point by using Gaussian distribution. For a multi-peak gaussian distribution model, each pixel of an image is modeled as a superposition of a plurality of gaussian distributions of different weights, each gaussian distribution corresponding to a state that may produce the color exhibited by the pixel, the weights and distribution parameters of each gaussian distribution being updated over time.
In summary, when processing a color image, it is assumed that the three color channels of the image pixel R, G, B are independent of each other and have the same variance. Observation dataset { X for random variable X 1 ,x 2 ,...x N }，x t ＝(r t ,g t ,b t ) For the sampling point of the pixel at the time t, a single sampling point x t Is subject to a mixed gaussian distributed probability density function.
Wherein x is N Observation data representing an nth random variable X, N being a positive integer, r t ,g t ,b t And respectively represent the pixel points on the three-color channels at the time t R, G, B.
The process of obtaining the standard template image through Gaussian mixture modeling is mainly as follows:
11 Firstly, under different angles and different light intensities, a plurality of qualified sample images of the target to be detected, preferably 20 images, are collected, and the 20 target images are preprocessed.
The pretreatment mode can comprise: the brightness, contrast, tone and the like of the target image are corrected, and the target image is preprocessed, so that the result of the follow-up detection of the defect area can be more accurate.
12 Performing mixed Gaussian modeling on the preprocessed target object, and establishing a standard template image with high universality.
In a specific Gaussian mixture modeling process, firstly, initializing some parameters such as variance, mean value, weight and the like in a Gaussian mixture model, and solving data required by modeling through the parameters. In addition, the variance can be set larger, and the weight can be set smaller, so that the range of the updating parameter value can be continuously reduced, and as many pixels as possible are contained in the Gaussian mixture model, so that the Gaussian mixture model can comprehensively identify the defect areas of all target images.
In the embodiment of the invention, at the initialization stage of the mixed Gaussian model, 5 mixed Gaussian models can be used for representing the characteristics of each pixel point in the target image, the gray value of each pixel of the acquired current target image is taken as a mean value, the variance can be set to be 15, and the weight can be set to be 0.001.
The next target image is then input into the mixture gaussian model to update the mixture gaussian model. That is, each pixel point in the next target image is matched with the Gaussian mixture model, if the pixel point is successful, the pixel point is judged to be a background point, otherwise, the pixel point is judged to be a foreground point, and the background and the foreground are separated according to the judgment of the background and the foreground, so that the standard template image is obtained.
And step S1012, aligning the residual target images in the target image set with the standard template images respectively, and positioning suspected defect areas on the residual target images in the target image set.
Generally, the target images are respectively aligned with the standard template images, so as to be beneficial to identifying suspected defect areas on the target images.
Specifically, referring to fig. 3, fig. 3 is a schematic flow chart of sub-steps of the defect detection method in fig. 2; sub-step S1012 may include: substeps S10121-S10123.
In the substep S10121, the remaining target images in the target image set (i.e. all target objects to be detected) may be aligned with the standard template images, respectively, and the difference value calculation may be performed on the remaining target images in the target image set and the standard template images, to obtain a difference value image.
It is worth mentioning that the alignment of the target image to be detected and the standard template object is beneficial to finding out all suspected defect areas of the target image to be detected. In order to facilitate alignment of the target image to be measured and the standard template object, in the scheme of the embodiment of the invention, a halcon measuring caliper can be used for aligning the target image to be measured and the standard template object. Among them, halcon germany developed a very sophisticated set of machine vision algorithms.
The specific alignment procedure may include:
13 Referring to fig. 4, fig. 4 is a flowchart of a tool for measuring a halon caliper in the defect detection method according to the embodiment of the present invention. Firstly, a plurality of small edge detection rectangles are utilized, then, a measuring caliper is arranged on an image to be detected, and edge points on the image to be detected are detected one by one. And after the edge points are extracted, obtaining the plane gray data of the edge points, smoothing the plane gray data, and finally, calculating the derivative of the edge points, and fitting the edge to obtain the target contour.
14 Referring to fig. 5, fig. 5 is an alignment schematic diagram of a target image to be detected and a standard template object in the defect detection method according to the embodiment of the present invention; in the target contour area, two straight lines, straight line 1 and straight line 2, are constructed. The intersection of line 1 and line 2, i.e., at the point "X" in the upper left corner of FIG. 5.
15 Finally, using a halcon measuring caliper to align the intersection point and the straight line of the target image to be measured and the standard template image, namely aligning the intersection point of two straight lines and the two straight lines, so that the target image to be measured and the standard template image can be accurately aligned.
It is emphasized that before the difference value calculation is performed on the target images remaining in the target image set and the standard template image respectively, the method further includes:
and optimizing the residual target images in the target image set with the standard template images respectively by using the Gaussian pyramid, wherein the residual target images in the target image set are aligned with the standard template images respectively. And a Gaussian pyramid is established for the target image, and calculation is carried out from high to low, so that the subsequent speed of identifying the suspected defect area is convenient.
It will be appreciated that gaussian pyramid is one of the multi-scale representations in images, most notably for segmentation of images. The Gaussian pyramid obtains some downsampled images through Gaussian smoothing and subsampling, namely the K+1 layer Gaussian pyramid can obtain K+1 layer Gaussian images through smoothing and subsampling. The gaussian pyramid contains a series of low pass filters whose cut-off frequency can be increased gradually by a factor of 2 from the previous layer to the next, so that the gaussian pyramid can span a large frequency range, the higher the level, the smaller the image and the lower the resolution. Wherein K is a positive integer.
Step S10122, binarizing the difference image to obtain a binarized image, and threshold segmentation is performed on the binarized image to obtain a segmented image.
In one embodiment of the invention, the binarized image may be thresholded using a halcon's own operator.
And step S10123, filtering the segmented image, and extracting the outline of the filtered segmented image to obtain a suspected defect area.
In one embodiment of the invention, the segmented image may be further filtered by an open operation. The main steps of the open operation can comprise two steps of corrosion treatment and expansion treatment.
The segmented image is subjected to erosion treatment, for example, the edge of the segmented image can be reduced, so that the distinguishing degree of the defect area can be enhanced. Then, the segmented image after the etching is subjected to an expansion process. The expansion process can significantly enhance the probability that the defective area is identified.
In order to facilitate extraction of the contours of the filtered segmented image, a preset contour extraction operator may be used, so that all contours of the filtered segmented image are extracted, and all suspected defect areas in the image to be detected are located.
Step S102, obtaining a target image with a suspected defect area to obtain a defect image set.
Through step S101, it may be determined which target images have suspected defect areas and which target images do not have suspected defect areas in the acquired target image set. The target image does not have a suspected defect area, namely that the corresponding target product has no flaws, so that the target image without the suspected defect area can be removed. The remaining target images all have a suspected defect region, so the remaining target images can be combined into a defect image set.
In summary, the embodiment of the invention can realize the primary detection of the target image set of the target to be detected through the steps S101 and S102, and determine which target images have suspected defect areas.
Step S103, inputting the defect image set into a trained classification network model, and identifying suspected defect areas on the target image in the defect image set through the trained classification network model to obtain real defect areas and corresponding real defect types of the target image.
The classification network model may be obtained by training some lightweight network model structures. Taking an object to be detected as a power module as an example, a training process of the classification network model is as follows:
21 The user can call the industrial camera to collect the power module image on the production line through QT, make the label according to the defect type, and take the collected power module image and the corresponding label as training data. Wherein QT is a cross-platform c++ graphical user interface application development framework.
It is assumed that the power module has four main defects, such as scratch, missing component, recess, and component position not within a preset range. Correspondingly, the user can make 4 kinds of labels with corresponding defect types { defect1, defect2, defect3, defect4}, so as to collect training data.
22 A classification network model may be trained based on a lightweight network model structure, for example, may be trained based on a lightweight network mobilenet_v3. The weights of the lightweight network mobilenet_v3 are initialized and then the number of iterations is set.
23 Training data is input into a lightweight network mobilenet_v3 for training, and a classification network model is derived after iteration is completed.
When the power module is actually detected, the running software interface can be shown with reference to fig. 6, and fig. 6 is a schematic diagram of the software interface detection for the defect detection of the power module, which is provided by the embodiment of the invention.
The lightweight network is used for training the classification model, so that the instantaneity of the defect area identification process can be improved. In one embodiment of the present invention, the lightweight network may include a first convolution layer set and a second convolution layer set that are in communication. The method comprises the steps of extracting defect characteristics of training data by using a first convolution layer group in a lightweight network model;
and identifying the defect characteristics through a second convolution layer group in the lightweight network model so as to output a defect area of the target image in the training data and a corresponding defect type.
Wherein the first convolution layer group may comprise a plurality of convolution layers in communication connection, the convolution kernel of each convolution layer being 3x3. Lightweight network model. The second convolution layer group may include two convolution layers with a convolution kernel of 1x1.
To sum up, the lightweight network model extracts features through convolution of 3x3, and passes through a plurality of convolution layers in the middle. And finally, replacing full connection by two 1x1 convolution layers, and outputting a defect area and a corresponding defect category.
24 Using the test set prepared in advance, inputting the classification network model, and testing and calculating the accuracy of the classification network model on the new test set. If the accuracy meets the predetermined requirement, the classification network model may be saved as the last classification network model, otherwise the training of step 23) is continued to be repeated.
The defect image set obtained in step S102 may be input into the classification network model trained in step 24), and then analyzed and compared with the defect image set to obtain an accurate defect region.
And simultaneously, taking the target image with the accurate defect area and the defect type as new training data, executing the same operation in the step 3), training the classification network model, and achieving the effect of continuously iterating and optimizing the model, thereby greatly improving the accuracy of the detection result. If the classification network model locates a defective area on the target image that does not correspond to the label defined in step 21), the defective area located at that time may not be a true defective area, so the target image with the defective area may be discarded.
In summary, according to the embodiment of the invention, the suspected defect area of the target image is acquired first, and then the suspected defect area is classified by using the classification network model, so that the real defect area and the corresponding defect type of each real defect area can be accurately detected only by identifying the suspected defect area because the suspected defect area is acquired in advance. At this time, a large number of image samples of the target to be detected are not required to be acquired, and the defect area and the defect type of the target to be detected can still be accurately identified.
In an embodiment of the present invention, before inputting the defect image set into the trained classification network model, the method may further include:
and obtaining a target image sample with defects of the target to be detected and type labels corresponding to the types of the defects to obtain training data.
And inputting training data into a preset network model for iterative training to obtain a classified network model.
In an embodiment of the present invention, the defect detection method may be applied to the defect detection field of the power module, and mainly includes the following steps:
31 Collecting power module images on a production line through an industrial camera, taking the power module images as a power module image set (namely a target object set), and determining whether the power module images have suspected defect areas or not;
20 power module images can be selected from the power module image set, or 20 power module images are collected again from the production line, and the 20 power module images are collected under different angles and different light intensities. Then constructing a standard template image by using 20 power module images;
and (3) aligning the power module image with the standard template image by using a Halcon tool, and then sequentially performing difference value calculation, binarization processing, segmentation processing, filtering and extracting all suspected defect areas.
32 Eliminating the power module images without suspected defect areas, and forming a power module defect image set by the rest power module images.
33 Inputting the defect image set forming the power supply module into a trained classification network model, and outputting the defect area of the power supply module and the corresponding defect type through the classification network model.
The defect detection method provided by the embodiment can quickly detect the defect area of the power module, does not need to collect a large number of power module image samples in advance for training a model, and is beneficial to popularization and application of the defect detection method.
Referring to fig. 7, fig. 7 is a schematic block diagram illustrating a defect detecting apparatus according to an embodiment of the present invention. As shown in fig. 7, the defect detection apparatus 300 includes a processor 301 and a memory 302, the processor 301 and the memory 302 being connected by a bus 303, such as an I2C (Inter-integrated Circuit) bus.
In particular, the processor 301 is configured to provide computing and control capabilities to support the operation of the entire defect detection device. The processor 301 may be a central processing unit (Central Processing Unit, CPU), the processor 301 may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field-programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. Wherein the general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Specifically, the Memory 302 may be a Flash chip, a Read-Only Memory (ROM) disk, an optical disk, a U-disk, a removable hard disk, or the like.
It will be appreciated by those skilled in the art that the structure shown in fig. 7 is merely a block diagram of a portion of the structure associated with an embodiment of the present invention and is not intended to limit the defect detection apparatus to which an embodiment of the present invention is applied, and that a particular defect detection apparatus may include more or fewer components than shown, or may combine certain components, or may have a different arrangement of components.
The processor 301 is configured to execute a computer program stored in the memory 302, and implement any one of the defect detection methods provided in the embodiments of the present invention when the computer program is executed.
In one embodiment of the present invention, the processor 301 is configured to execute a computer program stored in a memory, and implement the steps of the defect detection method as described above when executing the computer program, including the following steps:
acquiring a target image set of a target to be detected, and determining whether a target image in the target image set has a suspected defect area or not; the suspected defect area is a defect area possibly existing on the target to be detected;
obtaining a target image with a suspected defect area to obtain a defect image set;
inputting the defect image set into a trained classification network model;
and identifying suspected defect areas on the target image in the defect image set through the trained classification network model to obtain the real defect areas and corresponding real defect types of the target image.
In one embodiment of the present invention, the processor 301, when implemented, is configured to implement the defect detection method as described above, including the following steps:
acquiring a target image set of a target to be detected, and determining whether a target image in the target image set has a suspected defect area or not; the suspected defect area is a defect area possibly existing on the target to be detected;
obtaining a target image with a suspected defect area to obtain a defect image set;
inputting the defect image set into a trained classification network model;
and identifying suspected defect areas on the target image in the defect image set through the trained classification network model to obtain the real defect areas and corresponding real defect types of the target image.
It should be noted that, for convenience and brevity of description, specific working procedures of the defect detection apparatus described above may refer to corresponding procedures in the foregoing defect detection method embodiments, and are not described herein again.
The embodiment of the invention also provides a storage medium for computer readable storage, the storage medium storing one or more programs, the one or more programs being executable by one or more processors to implement any one of the defect detection methods provided in the embodiments of the invention.
The storage medium may be an internal storage unit of the defect detecting apparatus of the foregoing embodiment, for example, a hard disk or a memory of the defect detecting apparatus. The storage medium may also be an external storage device of the defect detecting device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the defect detecting device.
Those of ordinary skill in the art will appreciate that all or some of the steps, systems, functional modules/units in the apparatus, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware embodiment, the division between the functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed cooperatively by several physical components. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
It should be understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations. It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments. While the invention has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the protection scope of the invention is subject to the protection scope of the claims.

Claims (10)

1. A defect detection method, comprising:
acquiring a target image set of a target to be detected, and determining whether a target image in the target image set has a suspected defect area or not; the suspected defect area is a defect area possibly existing on the target to be detected;
obtaining a target image with the suspected defect area to obtain a defect image set;
inputting the defect image set into a trained classification network model;
and identifying suspected defect areas on the target image in the defect image set through the trained classification network model to obtain real defect areas and corresponding real defect types of the target image.
2. The defect detection method of claim 1, wherein the determining whether a target image in the target image set has a suspected defect region comprises:
obtaining a standard template image by utilizing a plurality of target images in the target image set, wherein the standard template image is formed by modeling the plurality of target images through mixed Gaussian background;
and aligning the remaining target images in the target image set with the standard template images respectively, and positioning suspected defect areas on the remaining target images in the target image set.
3. The defect detection method of claim 2, wherein aligning the remaining target images in the set of target images with the standard template images, respectively, locates suspected defect areas on the remaining target images in the set of target images, comprises:
aligning the residual target images in the target image set with the standard template images respectively, and carrying out difference calculation on the residual target images in the target image set and the standard template images respectively to obtain difference images;
performing binarization processing on the difference image to obtain a binarized image, and performing threshold segmentation on the binarized image to obtain a segmented image;
and filtering the segmented image, and extracting the outline of the filtered segmented image to obtain a suspected defect area.
4. A defect detection method according to claim 3, wherein said filtering the segmented image and extracting the contours of the filtered segmented image comprises:
filtering the segmented image by an open operation;
and extracting the contour of the filtered segmented image by using a preset contour extraction operator.
5. A defect detection method according to claim 3, further comprising, before performing difference calculation on the target images remaining in the target image set and the standard template image, respectively:
and optimizing the residual target images in the target image set with the standard template images respectively by using a Gaussian pyramid, wherein the residual target images in the target image set are aligned with the standard template images respectively.
6. The defect detection method of claim 1, wherein prior to inputting the set of defect images into the trained classification network model, further comprising:
obtaining a target image sample with defects of the target to be detected and type labels corresponding to the types of the defects to obtain training data;
and inputting the training data into a preset network model for iterative training to obtain the classification network model.
7. The defect detection method of claim 6, wherein the predetermined network model comprises a first convolution group and a second convolution group connected in sequence; the step of inputting the training data into a preset classification network for iterative training comprises the following steps:
extracting defect characteristics of the training data by using a first convolution layer group in the lightweight network model;
and identifying the defect characteristics through a second convolution layer group in the lightweight network model so as to output a defect area of the target image in the training data and a corresponding defect type.
8. The defect detection method of any of claims 1-7, wherein the method further comprises:
inputting the real defect area and the corresponding real defect type into the classification network model for training so as to optimize the classification network model.
9. A defect detection apparatus comprising a processor, a memory, a computer program stored on the memory and executable by the processor, and a data bus for enabling a connected communication between the processor and the memory, wherein the computer program, when executed by the processor, implements the steps of the defect detection method according to any of claims 1 to 8.
10. A storage medium for computer-readable storage, wherein the storage medium stores one or more programs executable by one or more processors to implement the steps of the defect detection method of any of claims 1 to 8.
CN202210762871.1A 2022-06-30 2022-06-30 Defect detection method, defect detection apparatus, and storage medium Pending CN117392042A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202210762871.1A CN117392042A (en) 2022-06-30 2022-06-30 Defect detection method, defect detection apparatus, and storage medium
PCT/CN2023/103337 WO2024002187A1 (en) 2022-06-30 2023-06-28 Defect detection method, defect detection device, and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210762871.1A CN117392042A (en) 2022-06-30 2022-06-30 Defect detection method, defect detection apparatus, and storage medium

Publications (1)

Publication Number Publication Date
CN117392042A true CN117392042A (en) 2024-01-12

Family

ID=89383124

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210762871.1A Pending CN117392042A (en) 2022-06-30 2022-06-30 Defect detection method, defect detection apparatus, and storage medium

Country Status (2)

Country Link
CN (1) CN117392042A (en)
WO (1) WO2024002187A1 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117576105B (en) * 2024-01-17 2024-03-29 高科建材（咸阳）管道科技有限公司 Pipeline production control method and system based on artificial intelligence
CN117745715B (en) * 2024-02-06 2024-04-23 中科院南京耐尔思光电仪器有限公司 Large-caliber telescope lens defect detection method based on artificial intelligence

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4254330A3 (en) * 2016-01-08 2023-11-29 SCREEN Holdings Co., Ltd. Flaw detection device and flaw detection method
CN111640091B (en) * 2020-05-14 2023-08-22 阿丘机器人科技(苏州)有限公司 Product defect detection method and computer storage medium
CN112288727A (en) * 2020-10-30 2021-01-29 创新奇智(重庆)科技有限公司 Magnetic ring surface defect detection method and device, electronic equipment and storage medium
CN113658121A (en) * 2021-08-09 2021-11-16 国能榆林能源有限责任公司 Line defect detection method, system and computer storage medium

Also Published As

Publication number Publication date
WO2024002187A1 (en) 2024-01-04

Similar Documents

Publication Publication Date Title
CN110060237B (en) Fault detection method, device, equipment and system
CN106875381B (en) Mobile phone shell defect detection method based on deep learning
JP7138753B2 (en) Image defect detection method, device, electronic device, storage medium and product
CN107543828B (en) Workpiece surface defect detection method and system
CN110276754B (en) Surface defect detection method, terminal device and storage medium
CN110148130B (en) Method and device for detecting part defects
US20230419472A1 (en) Defect detection method, device and system
CN111693534B (en) Surface defect detection method, model training method, device, equipment and medium
CN114266773B (en) Display panel defect positioning method, device, equipment and storage medium
JP2011214903A (en) Appearance inspection apparatus, and apparatus, method and program for generating appearance inspection discriminator
CN117392042A (en) Defect detection method, defect detection apparatus, and storage medium
CN111815564B (en) Method and device for detecting silk ingots and silk ingot sorting system
CN116188475B (en) Intelligent control method, system and medium for automatic optical detection of appearance defects
CN111340796B (en) Defect detection method and device, electronic equipment and storage medium
CN111680750B (en) Image recognition method, device and equipment
CN110706224B (en) Optical element weak scratch detection method, system and device based on dark field image
CN115661160B (en) Panel defect detection method, system, device and medium
CN115131283A (en) Defect detection and model training method, device, equipment and medium for target object
CN116128839A (en) Wafer defect identification method, device, electronic equipment and storage medium
CN114494780A (en) Semi-supervised industrial defect detection method and system based on feature comparison
CN115690670A (en) Intelligent identification method and system for wafer defects
CN115937101A (en) Quality detection method, device, equipment and storage medium
CN116152242A (en) Visual detection system of natural leather defect for basketball
TW201512649A (en) Method of chip detects inspecting, system therefor, and computer program product thereof
CN112070762A (en) Mura defect detection method and device for liquid crystal panel, storage medium and terminal

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication