CN114881994A - Product defect detection method, device and storage medium - Google Patents
Product defect detection method, device and storage medium Download PDFInfo
- Publication number
- CN114881994A CN114881994A CN202210577469.6A CN202210577469A CN114881994A CN 114881994 A CN114881994 A CN 114881994A CN 202210577469 A CN202210577469 A CN 202210577469A CN 114881994 A CN114881994 A CN 114881994A
- Authority
- CN
- China
- Prior art keywords
- defect type
- classification result
- type classification
- image block
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000007547 defect Effects 0.000 title claims abstract description 318
- 238000001514 detection method Methods 0.000 title claims description 29
- 238000013145 classification model Methods 0.000 claims abstract description 36
- 238000000034 method Methods 0.000 claims abstract description 32
- 230000002159 abnormal effect Effects 0.000 claims abstract description 12
- 238000004364 calculation method Methods 0.000 claims description 30
- 238000004519 manufacturing process Methods 0.000 claims description 7
- 238000012795 verification Methods 0.000 claims description 7
- 229910000831 Steel Inorganic materials 0.000 description 7
- 239000010959 steel Substances 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 238000000605 extraction Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000004590 computer program Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 239000000725 suspension Substances 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000003628 erosive effect Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/0008—Industrial image inspection checking presence/absence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Quality & Reliability (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The present disclosure provides a method, an apparatus and a storage medium for detecting product defects, including: acquiring an image of a product to be detected; extracting an abnormal first image block from the image, and caching the first image block to a first buffer pool; performing parallel computation on the first image block in the first buffer pool by adopting a plurality of first thread execution feature classification models to obtain a primary defect type classification result; caching the first-level defect type classification result and the corresponding first image block to a second buffer pool; determining a second image block from the first image block cached in the second buffer pool according to the first-level defect type classification result and a preset rule; performing parallel computation on a second image block in a second buffer pool by adopting a plurality of second thread execution depth classification models to obtain a secondary defect type classification result; and comparing the primary defect type classification result corresponding to the second image block with the secondary defect type classification result, and determining the target defect type of the product to be detected according to the comparison result.
Description
Technical Field
The present disclosure relates to the field of product inspection technologies, and in particular, to a method and an apparatus for inspecting product defects, and a storage medium.
Background
With the continuous development of the neural network technology, the neural network technology has been successfully applied to various fields such as medical treatment, military, transportation and the like, and has been developed greatly in the steel industry in recent years, and complex defect identification models such as galvanized sheets with spangles, color-coated sheets and the like based on the CNN neural network also become research hotspots. At present, in the related technology, a feature model is mainly used for defect classification and identification, and a classification and identification method based on deep learning is partially used, so that although the classification result of deep learning is possibly higher in accuracy, the requirements on sample data quantity and calculation power are higher during model training and operation, the adjustment period of the model is longer, and the response efficiency is lower. In addition, in the identification process, a model is generally adopted to detect a plurality of defects in sequence, and the defects cannot be detected in parallel, so that the product defect detection efficiency is low.
Disclosure of Invention
The present disclosure provides a method, an apparatus and a storage medium for detecting product defects, which are intended to solve at least one of the technical problems in the related art to some extent.
An embodiment of a first aspect of the present disclosure provides a product defect detection method, including: acquiring an image of a product to be detected; extracting an abnormal first image block from the image, and caching the first image block to a first buffer pool; performing parallel calculation on the first image blocks in the first buffer pool by adopting a plurality of first threads to execute a feature classification model so as to obtain a primary defect type classification result; caching the first-level defect type classification result and the corresponding first image block to a second buffer pool; determining a second image block from the first image block cached in the second buffer pool according to the first-level defect type classification result and a preset rule; performing parallel computation on a second image block in a second buffer pool by adopting a plurality of second thread execution depth classification models to obtain a secondary defect type classification result; comparing the primary defect type classification result corresponding to the second image block with the secondary defect type classification result to obtain a comparison result, wherein the comparison result belongs to the primary defect type classification result or the secondary defect type classification result; and determining the target defect type of the product to be detected according to the comparison result.
An embodiment of a second aspect of the present disclosure provides a product defect detecting apparatus, including: the acquisition module is used for acquiring an image of a product to be detected; the first cache module is used for extracting an abnormal first image block from an image and caching the first image block to a first cache pool; the first calculation module is used for performing parallel calculation on the first image blocks in the first buffer pool by adopting a plurality of first thread execution feature classification models to obtain a primary defect type classification result; the second cache module is used for caching the first-level defect type classification result and the corresponding first image block into a second cache pool; the first determining module is used for determining a second image block from the first image block cached in the second buffer pool according to the primary defect type classification result and a preset rule; the second calculation module is used for performing parallel calculation on a second image block in the second buffer pool by adopting a plurality of second thread execution depth classification models to obtain a secondary defect type classification result; the comparison module is used for comparing the primary defect type classification result corresponding to the second image block with the secondary defect type classification result to obtain a comparison result, wherein the comparison result belongs to the primary defect type classification result or the secondary defect type classification result; and the determining module is used for determining the target defect type of the product to be detected according to the comparison result.
An embodiment of a third aspect of the present disclosure provides a computer device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of product defect detection of the embodiments of the present disclosure.
A fourth aspect of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute a method for detecting product defects disclosed in an embodiment of the present disclosure.
In the embodiment, an image of a product to be detected is obtained, an abnormal first image block is extracted from the image, the first image block is cached in a first buffer pool, a plurality of first thread execution feature classification models are adopted to perform parallel calculation on the first image block in the first buffer pool to obtain a first-level defect type classification result, the first-level defect type classification result and the corresponding first image block are cached in a second buffer pool, a second image block is determined from the first image block cached in the second buffer pool according to the first-level defect type classification result and a preset rule, a plurality of second thread execution depth classification models are adopted to perform parallel calculation on a second image block in the second buffer pool to obtain a second-level defect type classification result, and the first-level defect type classification result corresponding to the second image block is compared with the second-level defect type classification result, obtaining a comparison result, wherein the comparison result belongs to a primary defect type classification result or a secondary defect type classification result, determining a target defect type of a product to be detected according to the comparison result, performing deep classification only when the classification result obtained by the feature classification model meets a preset rule, and not requiring all data to perform deep classification, so that the method can adapt to changes of different scenes, save calculation, improve the defect detection efficiency of the product, and improve the accuracy by comparing the feature classification result with the deep classification result to obtain a final target defect type; in addition, calculation power can be reasonably distributed through parallel detection in a multithreading mode, and detection efficiency is improved.
Additional aspects and advantages of the disclosure will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the disclosure.
Drawings
The foregoing and/or additional aspects and advantages of the present disclosure will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a schematic flow chart diagram of a product defect detection method provided according to an embodiment of the present disclosure;
FIG. 2 is a schematic flow chart of product defect detection provided according to an embodiment of the present disclosure;
FIG. 3 is a schematic flow chart diagram of a product defect detection method according to another embodiment of the present disclosure;
FIG. 4 is a schematic diagram of a product defect detection apparatus provided in accordance with another embodiment of the present disclosure;
FIG. 5 illustrates a block diagram of an exemplary computer device suitable for use in implementing embodiments of the present disclosure.
Detailed Description
Reference will now be made in detail to the embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present disclosure, and are not to be construed as limiting the present disclosure. On the contrary, the embodiments of the disclosure include all changes, modifications and equivalents coming within the spirit and terms of the claims appended hereto.
It should be noted that an execution main body of the product defect detecting method of this embodiment may be a product defect detecting apparatus, the apparatus may be implemented by software and/or hardware, the apparatus may be configured in an electronic device, and the electronic device may include, but is not limited to, a terminal, a server, and the like.
Fig. 1 is a schematic flowchart of a product defect detection method according to an embodiment of the present disclosure, as shown in fig. 1, the method includes:
s101: and acquiring an image of the product to be detected.
The product to be detected can be called a product to be detected, and the product to be detected can be products in any fields such as medical treatment, military, traffic, steel and the like, for example: the product to be detected may be a steel strip, a steel plate, etc., without limitation.
Fig. 2 is a schematic flow chart of the product defect detection provided according to the embodiment of the present disclosure, and as shown in fig. 2, the embodiment of the present disclosure may first acquire an image of a product to be detected, where the image may be, for example, an image (i.e., real-time image data) acquired in real time while the product to be detected is produced. In practical application, the image may be an image obtained by directly capturing the product to be detected by using a single camera, or may be an image obtained by splicing images captured by a plurality of cameras, or may be an image obtained by preprocessing an image, which is not limited herein.
S102: and extracting abnormal first image blocks from the image, and caching the first image blocks to a first buffer pool.
The image area where an abnormality (defect) exists in the image of the product to be detected may be referred to as a first image block (or referred to as a defect block), the first image block may exist in each frame of image obtained in real time, for example, and one or more defect areas may exist in each frame of image, that is, one or more first image blocks may be extracted from each frame of image. The embodiment of the present disclosure may extract the first image block from the image, that is: and extracting a defect block.
In some embodiments, for example, the first image block may be extracted by using a threshold segmentation method, an edge extraction method, an adaptive singular image extraction method, and any other possible algorithm, which is not limited herein.
In other embodiments, after the first image block is preliminarily extracted by using the above algorithm, adjacent image blocks may be further fused according to a certain erosion principle to obtain a final first image block.
For example, an abnormal first image block is extracted from the image, for example, based on a block singular density decision method. The image is first partitioned. The average BAVG of the gray levels in each image block is then calculated K (y) and the standard deviation of the gray scale BSDV K (y) where k is the block number, if BAVG K (y)+3*BSDV K (y)>255 or BAVG K (y)-3*BSDV K (y)<0, then the block is regarded as an invalid block, and other blocks are continuously analyzed, otherwise, the backward execution is continuously carried out. Further, a highlight singular target boundary HLB and a low highlight singular target boundary LLB are determined, wherein HLB ═ min (254, BAVG) K (y)+4*BSDV K (y))，LLB＝Max(20,BAVG K (y)-4*BSDV K (y)) the image point at the region image having a higher HLB or lower than LLB is set as a singular point, and the image point at the singular point on the singular point image is set as 255, and the image point at the non-singular point is set as 0. And carrying out Gaussian mean processing of 3x3 or 5x5 on the whole singular point image, wherein the gray level of each processed pixel point is the singular density of the point. The singular density on the singular point image is larger than a singular threshold value C Step The pixel points on the collected image corresponding to the points are determined as defect pixel points, and the connected defect points are defined as defect images. Further, the segmentation is corroded outwards again to generate a defect block containing a defect image, and 20 pixels can be taken as a corrosion scale in an online application. Further, fusing the connected defect blocks, and performing rectangular filling to obtain a final defect block, namely: a first image block.
After the first tile is fetched, as shown in FIG. 2, the first tile may be further buffered in a first buffer pool to wait for a subsequent fetch. The first buffer pool is, for example, a first-in first-out queue, and the extracted first image block is immediately deleted.
S103: and performing parallel calculation on the first image blocks in the first buffer pool by adopting a plurality of first threads to execute a feature classification model so as to obtain a primary defect type classification result.
Specifically, after the first image block is cached in the first buffer pool, the first thread pool is first established, as shown in fig. 2, the first thread pool includes a plurality of first threads, for example: thread 1, thread 2,. and thread N; furthermore, each first thread is used for exclusively extracting a first image block from the first buffer pool, and a feature classification model is executed to calculate the first image block in the first buffer pool to obtain a primary defect type classification result, that is, each thread can calculate one first image block to obtain a corresponding primary defect type classification result. After the operation of each thread is completed, the first image block in the first buffer pool is circularly extracted, and the feature classification model operation is continuously and circularly executed, so that the parallel calculation of a plurality of threads can be realized.
In some embodiments, the feature classification model may be, for example, an expert system model, a decision tree model, or the like, which may predict a defect type and a reliability to which the first image block may belong, and correspondingly, the primary defect type classification result may include a defect type and a reliability.
For example, known defect types for steel strip are 5, including: defect type a, defect type B, defect type C, defect type D, defect type E. In this embodiment, the first image block may be calculated through the feature classification model, and several defect types and reliabilities that the first image block may belong to are predicted, that is: and (5) classifying the primary defect type. For example, the first-level defect type classification result of a certain first image block is: defect type a (confidence 20%), defect type C (confidence 50%), defect type E (confidence 30%).
In some embodiments, the number of first threads is related to the number of processor (CPU) blocks, the number of processor cores, and a thread running coefficient (e.g., a first thread pool running coefficient k, k is greater than or equal to 1), such as: the first thread count is processor block count and processor core count and thread run coefficient k. Thus, computational resources may be allocated reasonably based on actual parameters of the processor.
S104: and caching the first-level defect type classification result and the corresponding first image block to a second buffer pool.
As shown in fig. 2, after the first-level defect type classification result is obtained, the first-level classification suspension is further performed, that is, the first-level defect type classification result and the corresponding first image block are cached in the second buffer pool, and the second-level classification extraction is waited.
S105: and determining a second image block from the first image blocks cached in the second buffer pool according to the primary defect type classification result and a preset rule.
The preset rule may be related to the primary defect type classification result, for example, and the number of the second image blocks may be one or more, which is not limited in this respect.
In some embodiments, the preset rule may be a threshold of maximum reliability in the primary defect type classification result, and if the maximum reliability in the primary defect type classification result is less than the set threshold, the first image block calculated by the primary defect type classification result is used as the second image block to wait for subsequent extraction calculation; if the maximum reliability in the primary defect type classification result is greater than or equal to the set threshold, the defects of the product to be detected can be determined directly according to the primary defect type classification result.
For example, the first image block includes, for example, a first image block 1, a first image block 2, and a first image block 3, wherein the first-level defect type classification result corresponding to the first image block 1 is: and if the maximum reliability threshold is 80% (more than 50%), taking the first image block 1 as the second image block 1 and waiting for subsequent calling.
In other embodiments, the preset rule may be a threshold of the number of defect types in the primary defect type classification result, and if the number of defect types in the primary defect type classification result is less than the set threshold, the first image block calculated by the primary defect type classification result is used as the second image block to wait for subsequent extraction calculation; if the number of the defect types in the primary defect type classification result is greater than or equal to a set threshold value, the defects of the product to be detected can be determined directly according to the primary defect type classification result.
For example, if the first image block 2 has 3 defect types as the first-level defect type classification result, the first image block 3 has 2 defect types as the first-level defect type classification result, and the threshold of the number of defect types is 4, the first image block 2 and the first image block 3 are both used as the second image block to wait for the subsequent retrieval.
S106: and performing parallel computation on the second image blocks in the second buffer pool by adopting a plurality of second thread execution depth classification models to obtain a secondary defect type classification result.
After the second image block is determined, a second thread pool may be established, as shown in fig. 2, where the second thread pool includes a plurality of second threads, for example: thread 1, thread 2.. thread N; further, each second thread is used for exclusively extracting a second image block from the second buffer pool, and a depth classification model is executed to calculate the second image block in the second buffer pool to obtain a secondary defect type classification result, that is, each second thread can calculate one second image block to obtain a corresponding secondary defect type classification result. After the operation of each second thread is finished, circularly extracting a second image block in the second buffer pool and continuously circularly executing the feature classification model operation, thereby realizing the parallel calculation of a plurality of threads.
In some embodiments, the depth classification model may be, for example, various neural network models such as ANN, CNN, SVM, RNN, etc., which may predict a defect type and a confidence level to which the second image block may belong, and correspondingly, the secondary defect type classification result may include a defect type and a confidence level.
For example, known defect types for steel strip are 5, including: defect type a, defect type B, defect type C, defect type D, defect type E. The implementation can calculate the second image block through the depth classification model, and predict the credibility of each type of the second image block, namely: and (5) classifying the secondary defect types. For example, the secondary defect type classification result of the second image block 1 is: defect type a (confidence 20%), defect type B (confidence 15%), defect type C (confidence 15%), defect type D (confidence 10%), defect type E (confidence 40%).
In some embodiments, the number of second threads may be related to the number of first threads, for example: the number of second threads is half the number of first threads. In practical applications, the GPU may increase the number of threads if the hardware is configured.
In practical applications, as shown in fig. 2, the timeliness (i.e., the time for the preset rule to judge) of the suspension of the primary classification may also be set, for example: and N seconds, wherein the secondary classification (namely, the depth classification model calculation is executed) is not carried out within N seconds, and the defects of the product to be detected are determined directly according to the primary defect type classification result.
S107: and comparing the primary defect type classification result corresponding to the second image block with the secondary defect type classification result to obtain a comparison result, wherein the comparison result belongs to the primary defect type classification result or the secondary defect type classification result.
In some embodiments, the confidence level may be used as an alignment condition. Specifically, the maximum value P1 of the first class reliability in the primary defect type classification result of the second image block is first determined, for example: the first-level defect type classification result corresponding to the second image block 1 is as follows: defect type a (confidence 20%), defect type C (confidence 50%), defect type E (confidence 30%), then the maximum value of first class confidence P1 is 50%.
Further, whether the sum of the credibility of the known defect types in the secondary defect type classification result is greater than a set threshold value is judged. In practical application, the depth classification model can predict unknown defect types in addition to known defect types such as defect type a, defect type B, defect type C, defect type D, defect type E and the like. For example, in the secondary defect type classification result corresponding to the second image block 1, the defect type a (reliability 5%), the defect type B (reliability 15%), the defect type C (reliability 10%), the defect type D (reliability 10%), the defect type E (reliability 30%), and the unknown defect (reliability 30%) are included. The set threshold PS may be set randomly, for example, the set threshold PS is 40%, the sum of the reliabilities of the known defect types is equal to 70%, and is greater than the set threshold 40%. In this case, the second-class reliability maximum P2 in the secondary defect type classification result is determined, and the second-class reliability maximum P2 is 30%.
Further, comparing the first-class reliability maximum value P1 with the second-class reliability maximum value P2, in some embodiments, if P1 is greater than or equal to P2, the comparison result is a primary defect type classification result; if P1 is smaller than P2, the comparison result is the classification result of the secondary defect type.
It should be noted that, if the sum of the credibility of the known defect types is not greater than the set threshold, the comparison is not performed, and the defects of the product to be detected are determined directly according to the primary defect type classification result.
S108: and determining the target defect type of the product to be detected according to the comparison result.
That is, the target defect type of the product to be detected is determined according to the primary defect type classification result or the secondary defect type classification result. For example: and taking the defect type with the highest reliability in the primary defect type classification result or the secondary defect type classification result as the target defect type.
It is understood that each second image block may have a corresponding defect type, and thus the product to be detected may have a plurality of target defect types.
In the embodiment, an image of a product to be detected is obtained, an abnormal first image block is extracted from the image, the first image block is cached in a first buffer pool, a plurality of first thread execution feature classification models are adopted to perform parallel calculation on the first image block in the first buffer pool to obtain a first-level defect type classification result, the first-level defect type classification result and the corresponding first image block are cached in a second buffer pool, a second image block is determined from the first image block cached in the second buffer pool according to the first-level defect type classification result and a preset rule, a plurality of second thread execution depth classification models are adopted to perform parallel calculation on a second image block in the second buffer pool to obtain a second-level defect type classification result, and the first-level defect type classification result corresponding to the second image block is compared with the second-level defect type classification result, obtaining a comparison result, wherein the comparison result belongs to a primary defect type classification result or a secondary defect type classification result, determining a target defect type of a product to be detected according to the comparison result, performing deep classification only when the classification result obtained by the feature classification model meets a preset rule, and not requiring all data to perform deep classification, so that the method can adapt to changes of different scenes, save calculation, improve the defect detection efficiency of the product, and improve the accuracy by comparing the feature classification result with the deep classification result to obtain a final target defect type; in addition, computational power can be reasonably distributed through parallel detection in a multi-thread mode, and detection efficiency is improved.
Fig. 3 is a schematic flowchart of a product defect detection method according to another embodiment of the present disclosure, as shown in fig. 3, the method includes:
s301: and acquiring an image of the product to be detected.
S302: and extracting abnormal first image blocks from the image, and caching the first image blocks to a first buffer pool.
S303: and performing parallel calculation on the first image blocks in the first buffer pool by adopting a plurality of first threads to execute a feature classification model so as to obtain a primary defect type classification result.
S304: and caching the first-level defect type classification result and the corresponding first image block to a second buffer pool.
S305: and determining a second image block from the first image blocks cached in the second buffer pool according to the primary defect type classification result and a preset rule.
S306: and performing parallel computation on the second image blocks in the second buffer pool by adopting a plurality of second thread execution depth classification models to obtain a secondary defect type classification result.
S307: and comparing the primary defect type classification result corresponding to the second image block with the secondary defect type classification result to obtain a comparison result.
For specific descriptions of S301 to S307, reference may be made to the above embodiments, which are not described herein again.
S308: and verifying the reliability of the comparison result, and determining the target defect type of the product to be detected according to the verification result.
In the operation of determining the target defect type of the product to be detected according to the comparison result, the reliability of the comparison result can be checked.
Wherein, different comparison results can correspond to different credibility checking modes.
In some embodiments, when the comparison result belongs to the first-level defect type classification result, the defect type satisfying the condition that the reliability is greater than or equal to the set threshold and the reliability is the highest in the first-level defect type classification result is taken as the target defect type.
The first-level defect type classification result may have different set thresholds for different defect types, for example: the set thresholds for defect type a to defect type E are 10%, 30%, 60%, 30%, and 20%, respectively. In practical application, each defect type in the primary defect type classification result can be sorted according to the reliability from high to low, and then the defect types are sequentially compared with the set threshold corresponding to each defect type until the defect type which meets the condition that the reliability is greater than or equal to the set threshold and has the highest reliability is found.
For example, the classification result of the first-level defect type corresponding to the second image block 1 is as follows: the defect type A (reliability 20%), the defect type C (reliability 50%) and the defect type E (reliability 30%) are sorted into the defect type C, the defect type E and the defect type A according to the reliability, the set threshold corresponding to the defect type C is 60% which is larger than the reliability 50%, the defect type E is continuously judged, the set threshold corresponding to the defect type E is 20% which is smaller than the reliability 30%, and the defect type E is taken as the target defect type.
In other embodiments, the defect type with the highest reliability in the secondary defect type classification result is used as the target defect type when the comparison result belongs to the secondary defect type classification result.
In other embodiments, after the target defect type of the product to be detected is determined, the target defect type can be determined according to the production characteristic parameters of the product to be detected.
Among these, production characteristic parameters are inherent characteristics on a production line, such as: the circular hole with fixed size at the position of the middle strip head of the strip steel is not a defect. The present embodiment may determine the target defect type according to the production characteristic parameters of the product to be detected, and if the determined target defect type is the same as the production characteristic parameters, it indicates that the target defect type is not a defect.
Therefore, the embodiment of the disclosure can also perform the reliability verification and the feature verification again on the comparison result, thereby improving the accuracy of the defect detection result.
In the embodiment, an image of a product to be detected is obtained, an abnormal first image block is extracted from the image, the first image block is cached in a first buffer pool, a plurality of first thread execution feature classification models are adopted to perform parallel calculation on the first image block in the first buffer pool to obtain a first-level defect type classification result, the first-level defect type classification result and the corresponding first image block are cached in a second buffer pool, a second image block is determined from the first image block cached in the second buffer pool according to the first-level defect type classification result and a preset rule, a plurality of second thread execution depth classification models are adopted to perform parallel calculation on a second image block in the second buffer pool to obtain a second-level defect type classification result, and the first-level defect type classification result corresponding to the second image block is compared with the second-level defect type classification result, obtaining a comparison result, wherein the comparison result belongs to a primary defect type classification result or a secondary defect type classification result, determining a target defect type of a product to be detected according to the comparison result, performing deep classification only when the classification result obtained by the feature classification model meets a preset rule, and not requiring all data to perform deep classification, so that the method can adapt to changes of different scenes, save calculation, improve the defect detection efficiency of the product, and improve the accuracy by comparing the feature classification result with the deep classification result to obtain a final target defect type; in addition, computational power can be reasonably distributed through parallel detection in a multi-thread mode, and detection efficiency is improved. In addition, the embodiment of the disclosure can perform a second reliability check and a second feature check on the comparison result, so that the accuracy of the defect detection result can be improved.
Fig. 4 is a schematic diagram of a product defect detecting apparatus according to another embodiment of the present disclosure. As shown in fig. 4, the product defect detecting apparatus 40 includes:
an obtaining module 401, configured to obtain an image of a product to be detected;
a first caching module 402, configured to extract an abnormal first image block from an image, and cache the first image block to a first cache pool;
a first calculating module 403, configured to perform parallel calculation on a first image block in the first buffer pool by using a plurality of first threads to execute a feature classification model, so as to obtain a first-level defect type classification result;
a second cache module 404, configured to cache the first-level defect type classification result and the corresponding first image block to a second buffer pool;
a first determining module 405, configured to determine a second image block from the first image block cached in the second buffer pool according to the primary defect type classification result and a preset rule;
a second calculating module 406, configured to perform parallel calculation on a second image block in the second buffer pool by using a plurality of second thread execution depth classification models to obtain a secondary defect type classification result;
a comparison module 407, configured to compare the primary defect type classification result corresponding to the second image block with the secondary defect type classification result to obtain a comparison result, where the comparison result belongs to the primary defect type classification result or the secondary defect type classification result; and
and the determining module 408 is configured to determine the target defect type of the product to be detected according to the comparison result.
In some embodiments, alignment module 407 is specifically configured to:
determining the maximum value of the first class reliability in the first-class defect type classification result of the second image block;
determining the maximum value of the second class reliability in the secondary defect type classification result under the condition that the sum of the reliability of the known defect types in the secondary defect type classification result is greater than a set threshold value; and
and comparing the first category maximum confidence value with the second category maximum confidence value.
In some embodiments, the determining module 408 is specifically configured to:
and verifying the reliability of the comparison result, and determining the target defect type of the product to be detected according to the verification result.
In some embodiments, the determining module 408 is specifically configured to:
under the condition that the comparison result belongs to a first-level defect type classification result, taking the defect type which satisfies that the reliability is greater than or equal to a set threshold and has the maximum reliability in the first-level defect type classification result as a target defect type;
or alternatively
And taking the defect type with the highest reliability in the secondary defect type classification results as the target defect type under the condition that the comparison result belongs to the secondary defect type classification results.
In some embodiments, the determining module 408 is specifically configured to:
and judging the type of the target defect according to the production characteristic parameters of the product to be detected.
In some embodiments, the number of first threads is related to the number of processor blocks, the number of processor cores, and the thread running coefficient, and the number of second threads is related to the number of first threads.
In the embodiment, the deep classification can be performed only when the classification result obtained by the feature classification model meets the preset rule, and the deep classification of all data is not required, so that the method can adapt to the change of different scenes, save the calculation, improve the product defect detection efficiency, and improve the accuracy by comparing the feature classification result with the deep classification result to obtain the final target defect type; in addition, computational power can be reasonably distributed through parallel detection in a multi-thread mode, and detection efficiency is improved.
The present disclosure also provides a computer device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.
In order to implement the foregoing embodiments, the present disclosure further provides a computer program product, which when executed by an instruction processor in the computer program product, performs the product defect detection method as set forth in the foregoing embodiments of the present disclosure.
FIG. 5 illustrates a block diagram of an exemplary computer device suitable for use in implementing embodiments of the present disclosure. The computer device 12 shown in fig. 5 is only one example and should not bring any limitations to the functionality or scope of use of the embodiments of the present disclosure.
As shown in FIG. 5, computer device 12 is in the form of a general purpose computing device. The components of computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.
Computer device 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by computer device 12 and includes both volatile and nonvolatile media, removable and non-removable media.
Although not shown in FIG. 5, a disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a Compact disk Read Only Memory (CD-ROM), a Digital versatile disk Read Only Memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the disclosure.
A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally perform the functions and/or methodologies of the embodiments described in this disclosure.
Computer device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), with one or more devices that enable a user to interact with computer device 12, and/or with any devices (e.g., network card, modem, etc.) that enable computer device 12 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 22. Moreover, computer device 12 may also communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public Network such as the Internet) via Network adapter 20. As shown, network adapter 20 communicates with the other modules of computer device 12 via bus 18. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
The processing unit 16 executes various functional applications, such as implementing the product defect detection method mentioned in the foregoing embodiments, by executing programs stored in the system memory 28.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
It should be noted that, in the description of the present disclosure, the terms "first", "second", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. Further, in the description of the present disclosure, "a plurality" means two or more unless otherwise specified.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and the scope of the preferred embodiments of the present disclosure includes other implementations in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present disclosure.
It should be understood that portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present disclosure may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Although embodiments of the present disclosure have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure, and that changes, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present disclosure.
Claims (10)
1. A method of product defect detection, comprising:
acquiring an image of a product to be detected;
extracting an abnormal first image block from the image, and caching the first image block to a first buffer pool;
performing parallel computation on the first image blocks in the first buffer pool by adopting a plurality of first threads to execute a feature classification model so as to obtain a primary defect type classification result;
caching the first-level defect type classification result and the corresponding first image block to a second buffer pool;
determining a second image block from the first image blocks cached in the second buffer pool according to the primary defect type classification result and a preset rule;
performing parallel computation on the second image blocks in the second buffer pool by adopting a plurality of second thread execution depth classification models to obtain a secondary defect type classification result;
comparing the primary defect type classification result corresponding to the second image block with the secondary defect type classification result to obtain a comparison result, wherein the comparison result belongs to the primary defect type classification result or the secondary defect type classification result; and
and determining the target defect type of the product to be detected according to the comparison result.
2. The method of claim 1, wherein comparing the primary defect type classification result and the secondary defect type classification result corresponding to the second image block comprises:
determining the maximum value of the first class reliability in the first-class defect type classification result of the second image block;
determining the maximum value of the reliability of the second class in the secondary defect type classification result under the condition that the sum of the reliability of the known defect types in the secondary defect type classification result is greater than a set threshold value; and
and comparing the first category maximum confidence value with the second category maximum confidence value.
3. The method of claim 2, wherein determining the target defect type of the product to be detected according to the comparison result comprises:
and verifying the reliability of the comparison result, and determining the target defect type of the product to be detected according to the verification result.
4. The method of claim 3, wherein verifying the reliability of the comparison result and determining the defects of the product to be detected according to the verification result comprises:
taking the defect type which satisfies that the reliability is greater than or equal to a set threshold and has the maximum reliability in the primary defect type classification results as the target defect type under the condition that the comparison result belongs to the primary defect type classification result;
or
And taking the defect type with the highest reliability in the secondary defect type classification results as the target defect type under the condition that the comparison result belongs to the secondary defect type classification results.
5. The method of claim 1, wherein after determining the target defect type of the product to be detected according to the comparison result, the method further comprises:
and judging the type of the target defect according to the production characteristic parameters of the product to be detected.
6. The method of claim 1, wherein the number of first threads is related to a number of processor blocks, a number of processor cores, and a thread run coefficient, and wherein the number of second threads is related to the number of first threads.
7. A product defect detecting apparatus, comprising:
the acquisition module is used for acquiring an image of a product to be detected;
the first cache module is used for extracting an abnormal first image block from the image and caching the first image block to a first cache pool;
the first calculation module is used for performing parallel calculation on the first image blocks in the first buffer pool by adopting a plurality of first thread execution feature classification models to obtain a primary defect type classification result;
the second cache module is used for caching the first-level defect type classification result and the corresponding first image block to a second cache pool;
the first determining module is used for determining a second image block from the first image blocks cached in the second buffer pool according to the primary defect type classification result and a preset rule;
the second calculation module is used for performing parallel calculation on the second image blocks in the second buffer pool by adopting a plurality of second thread execution depth classification models to obtain a secondary defect type classification result;
a comparison module, configured to compare the primary defect type classification result corresponding to the second image block with the secondary defect type classification result to obtain a comparison result, where the comparison result belongs to the primary defect type classification result or the secondary defect type classification result; and
and the determining module is used for determining the target defect type of the product to be detected according to the comparison result.
8. The apparatus of claim 7, wherein the comparison module is specifically configured to:
determining the maximum value of the first class reliability in the first-class defect type classification result of the second image block;
determining the maximum value of the reliability of the second class in the secondary defect type classification result under the condition that the sum of the reliability of the known defect types in the secondary defect type classification result is greater than a set threshold value; and
and comparing the first category maximum confidence value with the second category maximum confidence value.
9. The apparatus of claim 8, wherein the determination module is specifically configured to:
and verifying the reliability of the comparison result, and determining the target defect type of the product to be detected according to the verification result.
10. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210577469.6A CN114881994B (en) | 2022-05-25 | 2022-05-25 | Product defect detection method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210577469.6A CN114881994B (en) | 2022-05-25 | 2022-05-25 | Product defect detection method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114881994A true CN114881994A (en) | 2022-08-09 |
| CN114881994B CN114881994B (en) | 2024-07-23 |
Family
ID=82677625
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210577469.6A Active CN114881994B (en) | 2022-05-25 | 2022-05-25 | Product defect detection method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114881994B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6148099A (en) * | 1997-07-03 | 2000-11-14 | Neopath, Inc. | Method and apparatus for incremental concurrent learning in automatic semiconductor wafer and liquid crystal display defect classification |
| CN109741295A (en) * | 2018-11-28 | 2019-05-10 | 歌尔股份有限公司 | Product quality detection method and device |
| CN111160451A (en) * | 2019-12-27 | 2020-05-15 | 中山德著智能科技有限公司 | Flexible material detection method and storage medium thereof |
| CN113537414A (en) * | 2021-09-15 | 2021-10-22 | 深圳新视智科技术有限公司 | Lithium battery defect detection method, device, equipment and storage medium |
| CN114494103A (en) * | 2020-11-11 | 2022-05-13 | 艾聚达信息技术(苏州)有限公司 | Defect detection method and device |
| CN114494260A (en) * | 2022-04-18 | 2022-05-13 | 深圳思谋信息科技有限公司 | Object defect detection method and device, computer equipment and storage medium |
-
2022
- 2022-05-25 CN CN202210577469.6A patent/CN114881994B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6148099A (en) * | 1997-07-03 | 2000-11-14 | Neopath, Inc. | Method and apparatus for incremental concurrent learning in automatic semiconductor wafer and liquid crystal display defect classification |
| CN109741295A (en) * | 2018-11-28 | 2019-05-10 | 歌尔股份有限公司 | Product quality detection method and device |
| CN111160451A (en) * | 2019-12-27 | 2020-05-15 | 中山德著智能科技有限公司 | Flexible material detection method and storage medium thereof |
| CN114494103A (en) * | 2020-11-11 | 2022-05-13 | 艾聚达信息技术(苏州)有限公司 | Defect detection method and device |
| CN113537414A (en) * | 2021-09-15 | 2021-10-22 | 深圳新视智科技术有限公司 | Lithium battery defect detection method, device, equipment and storage medium |
| CN114494260A (en) * | 2022-04-18 | 2022-05-13 | 深圳思谋信息科技有限公司 | Object defect detection method and device, computer equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 赵庆林, 陈湘萍: "面向图像的绝缘子自爆缺陷检测算法综述", 智能计算机与应用, vol. 12, no. 3, 1 March 2022 (2022-03-01), pages 33 - 38 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114881994B (en) | 2024-07-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110047095B (en) | Tracking method and device based on target detection and terminal equipment | |
| CN109671071B (en) | Underground pipeline defect positioning and grade judging method based on deep learning | |
| CN110348522B (en) | Image detection and identification method and system, electronic equipment, and image classification network optimization method and system | |
| CN111681240A (en) | Bridge surface crack detection method based on YOLO v3 and attention mechanism | |
| WO2020156409A1 (en) | Data processing method, defect detection method, computing apparatus, and storage medium | |
| CN110264444B (en) | Damage detection method and device based on weak segmentation | |
| US8811750B2 (en) | Apparatus and method for extracting edge in image | |
| CN112329881B (en) | License plate recognition model training method, license plate recognition method and device | |
| KR100868884B1 (en) | Flat glass defect information system and classification method | |
| WO2020007119A1 (en) | Display screen peripheral circuit detection method and device, electronic device and storage medium | |
| CN111625683B (en) | Automatic video abstract generation method and system based on graph structure difference analysis | |
| CN116485779B (en) | Adaptive wafer defect detection method and device, electronic equipment and storage medium | |
| CN115471487A (en) | Insulator defect detection model construction and insulator defect detection method and device | |
| CN115995056A (en) | Automatic bridge disease identification method based on deep learning | |
| CN115294332A (en) | Image processing method, device, equipment and storage medium | |
| CN115471476A (en) | Method, device, equipment and medium for detecting component defects | |
| CN111881775B (en) | Real-time face recognition method and device | |
| CN113283396A (en) | Target object class detection method and device, computer equipment and storage medium | |
| KR20220102506A (en) | Apparatus and method for detecting of faults in surfaces of metal tube | |
| CN114170642A (en) | Image detection processing method, device, equipment and storage medium | |
| CN114881994A (en) | Product defect detection method, device and storage medium | |
| KR102525491B1 (en) | Method of providing structure damage detection report | |
| CN111163332A (en) | Video pornography detection method, terminal and medium | |
| CN116486166A (en) | Power transmission line foreign matter identification detection method based on edge calculation | |
| CN115700758A (en) | Sperm activity detection method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |