CN113240579A - Intelligent industrial product defect detection method and device and computer storage medium thereof - Google Patents
Intelligent industrial product defect detection method and device and computer storage medium thereof Download PDFInfo
- Publication number
- CN113240579A CN113240579A CN202110361220.7A CN202110361220A CN113240579A CN 113240579 A CN113240579 A CN 113240579A CN 202110361220 A CN202110361220 A CN 202110361220A CN 113240579 A CN113240579 A CN 113240579A
- Authority
- CN
- China
- Prior art keywords
- image data
- target
- contour
- detection
- characteristic parameters
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 47
- 230000007547 defect Effects 0.000 title claims abstract description 22
- 238000003860 storage Methods 0.000 title claims abstract description 13
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 20
- 230000011218 segmentation Effects 0.000 claims abstract description 16
- 238000000605 extraction Methods 0.000 claims abstract description 10
- 238000007619 statistical method Methods 0.000 claims abstract description 10
- 238000005520 cutting process Methods 0.000 claims abstract description 7
- 238000004364 calculation method Methods 0.000 claims abstract description 4
- 238000000034 method Methods 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 10
- 238000010606 normalization Methods 0.000 claims description 9
- 238000010586 diagram Methods 0.000 description 10
- 238000013135 deep learning Methods 0.000 description 8
- 238000010801 machine learning Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000008719 thickening Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4023—Scaling of whole images or parts thereof, e.g. expanding or contracting based on decimating pixels or lines of pixels; based on inserting pixels or lines of pixels
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20092—Interactive image processing based on input by user
- G06T2207/20104—Interactive definition of region of interest [ROI]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
Abstract
An industrial product defect intelligent detection method, a device and a computer storage medium thereof comprise: zooming the image data by using a bilinear interpolation zooming algorithm to obtain zoomed image data; normalizing the zoomed image data; carrying out primary coarse detection on the normalized image data by using a segmentation model to obtain a target position ROI rectangular frame; cutting out image data of the ROI rectangular frame, and performing fine inference detection by using a segmentation model to obtain segmented target image data; using a contour extraction algorithm to the segmented target image data to obtain all target contour coordinate sequences; and carrying out statistical analysis on target data in the contour in the image data according to the multi-target contour coordinate sequence to obtain target characteristic parameters, carrying out contour characteristic parameter calculation according to the multi-target contour coordinate sequence, and further carrying out defect identification on the product image. The invention is suitable for low-cost large-scale, high-accuracy and high-efficiency detection.
Description
[ technical field ] A method for producing a semiconductor device
The invention belongs to the technical field of industrial product detection, and particularly relates to an intelligent industrial product defect detection method and device and a computer storage medium thereof.
[ background of the invention ]
The machine vision system in the field of industrial product defect detection is mainly divided into two parts: the machine vision detection system comprises an image acquisition unit consisting of a traditional camera, a lens, a light source, a camera fixing and moving mechanism, and an image processing unit consisting of a PC host, an image acquisition card and the like. The machine vision has high ductility and plasticity, and can be developed by customized hardware and customized software aiming at different product defects.
In the prior art, detection methods are divided into two types, one type is pure deep learning target detection and target segmentation, and the method cannot acquire accurate characteristic parameters of a target, namely cannot meet the requirements of clients on accurate and high-speed detection; the other type is detection in a traditional machine learning mode, an expert mode is needed to adjust parameters, and the accuracy rate cannot stably reach the recognition rate of more than 99%.
[ summary of the invention ]
The invention aims to provide an industrial product defect intelligent detection method and device based on combination of traditional machine learning and deep learning and a computer storage medium.
In a first aspect, the present invention provides, for example, an intelligent industrial product defect detection method based on a combination of traditional machine learning and deep learning, including the following steps:
step 1: zooming the image data by using a bilinear interpolation zooming algorithm to obtain zoomed image data;
step 2: normalizing the scaled image data, the normalization algorithm comprising: calculating the mean value and the variance of image data, subtracting the mean value from each data, and dividing the difference by the square difference;
and step 3: carrying out primary coarse detection on the normalized image data by using a segmentation model to obtain a target position ROI rectangular frame;
and 4, step 4: cutting out image data of the ROI rectangular frame, and performing fine inference detection by using a segmentation model to obtain segmented target image data;
and 5: using a contour extraction algorithm to the segmented target image data to obtain all target contour coordinate sequences;
step 6: carrying out statistical analysis on target data in the contour in the image data according to the multi-target contour coordinate sequence to obtain target characteristic parameters, wherein the target characteristic parameters comprise: contrast, brightness, and average gray scale and category information;
and 7: calculating profile characteristic parameters according to the multi-target profile coordinate sequence, wherein the method comprises the following steps: and obtaining target contour characteristic parameters through the area, the perimeter, the length, the width and the circularity, and further identifying the defects of the product image.
In a second aspect, the present invention provides an intelligent industrial product defect detection apparatus based on a combination of traditional machine learning and deep learning, for implementing the method according to the first aspect, specifically including:
the image scaling module is used for scaling the image data by using a bilinear interpolation scaling algorithm to obtain the scaled image data;
a normalization module for normalizing the scaled image data, the normalization algorithm comprising: calculating the mean value and the variance of image data, subtracting the mean value from each data, and dividing the difference by the square difference;
the rough detection module is used for carrying out first rough detection on the normalized image data by using a segmentation model to obtain a target position ROI rectangular frame;
the fine reasoning detection module is used for cutting the image data of the ROI rectangular frame and performing fine reasoning detection by using a segmentation model to obtain segmented target image data;
the contour extraction module is used for obtaining all target contour coordinate sequences by using a contour extraction algorithm on the segmented target image data;
the target data statistical analysis module is used for performing statistical analysis on target data in the contour in the image data according to the multi-target contour coordinate sequence to obtain target characteristic parameters, and comprises the following steps: contrast, brightness, and average gray scale and category information;
the contour characteristic parameter calculation module is used for calculating contour characteristic parameters according to the multi-target contour coordinate sequence and comprises the following steps: and obtaining target contour characteristic parameters through the area, the perimeter, the length, the width and the circularity, and further identifying the defects of the product image.
In a third aspect, the present invention provides a computer storage medium having stored thereon a computer program which, when executed by a processor, performs the method of the first aspect.
The invention has the advantages that: the method has the characteristics of low cost, large scale, high accuracy and high efficiency detection, and solves the problem that the accurate characteristic parameters of the target cannot be obtained by pure deep learning target detection and target segmentation, namely the requirements of customers on accuracy and high-speed detection cannot be met; the problem that the detection in the traditional machine learning mode needs an expert mode to adjust parameters, and the accuracy rate cannot stably reach the recognition rate of more than 99 percent is solved; the detection requirement on a high-speed assembly line is met by a mode of firstly thickening and then thinning in deep learning.
[ description of the drawings ]
The invention will be further described with reference to the following examples with reference to the accompanying drawings.
FIG. 1 is a flow chart of a method according to an embodiment of the present invention.
Fig. 2 is a schematic structural diagram of a second apparatus according to an embodiment of the present invention.
Fig. 3 is a schematic structural diagram of a computer-readable storage medium according to a third embodiment of the present invention.
[ detailed description ] embodiments
The first embodiment is as follows:
the embodiment provides an intelligent industrial product defect detection method based on combination of traditional machine learning and deep learning, and during specific operation, firstly, a picture file on a local hard disk is opened at an X86PC terminal to obtain image data (or obtained from an industrial camera), the image data is transmitted to a memory of a PC terminal, and a processor of the PC terminal processes the image data in the memory for an X86 processor; then, an X86 processor (processor includes but is not limited to CPU, GPU, FPGA, ASIC processor) processes the image data, as shown in fig. 1, specifically including the following steps:
step 1: zooming the image data by using a bilinear interpolation zooming algorithm to obtain zoomed image data;
step 2: normalizing the scaled image data, the normalization algorithm comprising: calculating the mean value and the variance of image data, subtracting the mean value from each data, and dividing the difference by the square difference;
and step 3: performing first coarse detection on the normalized image data by using a segmentation model (for example, a UNet + + model) to obtain a target position ROI rectangular frame;
and 4, step 4: cutting out image data of the ROI rectangular frame, and performing fine inference detection by using a segmentation model (UNet + + model) to obtain segmented target image data;
and 5: using a contour extraction algorithm to the segmented target image data to obtain all target contour coordinate sequences;
step 6: carrying out statistical analysis on target data in the contour in the image data according to the multi-target contour coordinate sequence to obtain target characteristic parameters, wherein the target characteristic parameters comprise: contrast, brightness, and average gray scale and category information;
and 7: calculating profile characteristic parameters according to the multi-target profile coordinate sequence, wherein the method comprises the following steps: and obtaining target contour characteristic parameters through the area, the perimeter, the length, the width and the circularity, and further identifying the defects of the product image.
And processing the accurate detection result, wherein the processing mode comprises but is not limited to local display or network transmission to external equipment or serial transmission to external equipment, and the external equipment comprises but is not limited to a PC terminal or a PLC.
Based on the same inventive concept, the invention also provides a device corresponding to the method in the first embodiment, which is detailed in the second embodiment.
Example two:
the invention provides an industrial product defect intelligent detection device based on the combination of traditional machine learning and deep learning, as shown in figure 2, comprising:
the image scaling module is used for scaling the image data by using a bilinear interpolation scaling algorithm to obtain the scaled image data;
a normalization module for normalizing the scaled image data, the normalization algorithm comprising: calculating the mean value and the variance of image data, subtracting the mean value from each data, and dividing the difference by the square difference;
the rough detection module is used for carrying out first rough detection on the normalized image data by using a segmentation model (for example, a UNet + + model) to obtain a target position ROI rectangular frame;
the fine inference detection module is used for cutting the image data of the ROI rectangular frame and performing fine inference detection by using a segmentation model (UNet + + model) to obtain segmented target image data;
the contour extraction module is used for obtaining all target contour coordinate sequences by using a contour extraction algorithm on the segmented target image data;
the target data statistical analysis module is used for performing statistical analysis on target data in the contour in the image data according to the multi-target contour coordinate sequence to obtain target characteristic parameters, and comprises the following steps: contrast, brightness, and average gray scale and category information;
the contour characteristic parameter calculation module is used for calculating contour characteristic parameters according to the multi-target contour coordinate sequence and comprises the following steps: and obtaining target contour characteristic parameters through the area, the perimeter, the length, the width and the circularity, and further identifying the defects of the product image.
Since the apparatus described in the second embodiment of the present invention is an apparatus used for implementing the method of the first embodiment of the present invention, based on the method described in the first embodiment of the present invention, a person skilled in the art can understand the specific structure and the deformation of the apparatus, and thus the details are not described herein. All the devices adopted in the method of the first embodiment of the present invention belong to the protection scope of the present invention.
Based on the same inventive concept, the application provides a computer-readable storage medium corresponding to the third embodiment.
Example three:
the invention provides a computer-readable storage medium, as shown in fig. 3, on which a computer program is stored which, when being executed by a processor, carries out the method according to the first embodiment.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only an example of the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (3)
1. An intelligent industrial product defect detection method is characterized by comprising the following steps: the method comprises the following steps:
step 1: zooming the image data by using a bilinear interpolation zooming algorithm to obtain zoomed image data;
step 2: normalizing the scaled image data, the normalization algorithm comprising: calculating the mean value and the variance of image data, subtracting the mean value from each data, and dividing the difference by the square difference;
and step 3: carrying out primary coarse detection on the normalized image data by using a segmentation model to obtain a target position ROI rectangular frame;
and 4, step 4: cutting out image data of the ROI rectangular frame, and performing fine inference detection by using a segmentation model to obtain segmented target image data;
and 5: using a contour extraction algorithm to the segmented target image data to obtain all target contour coordinate sequences;
step 6: carrying out statistical analysis on target data in the contour in the image data according to the multi-target contour coordinate sequence to obtain target characteristic parameters, wherein the target characteristic parameters comprise: contrast, brightness, and average gray scale and category information;
and 7: calculating profile characteristic parameters according to the multi-target profile coordinate sequence, wherein the method comprises the following steps: and obtaining target contour characteristic parameters through the area, the perimeter, the length, the width and the circularity, and further identifying the defects of the product image.
2. An intelligent industrial product defect detection device for realizing the intelligent industrial product defect detection method according to claim 1, which is characterized in that: the method comprises the following steps:
the image scaling module is used for scaling the image data by using a bilinear interpolation scaling algorithm to obtain the scaled image data;
a normalization module for normalizing the scaled image data, the normalization algorithm comprising: calculating the mean value and the variance of image data, subtracting the mean value from each data, and dividing the difference by the square difference;
the rough detection module is used for carrying out first rough detection on the normalized image data by using a segmentation model to obtain a target position ROI rectangular frame;
the fine reasoning detection module is used for cutting the image data of the ROI rectangular frame and performing fine reasoning detection by using a segmentation model to obtain segmented target image data;
the contour extraction module is used for obtaining all target contour coordinate sequences by using a contour extraction algorithm on the segmented target image data;
the target data statistical analysis module is used for performing statistical analysis on target data in the contour in the image data according to the multi-target contour coordinate sequence to obtain target characteristic parameters, and comprises the following steps: contrast, brightness, and average gray scale and category information;
the contour characteristic parameter calculation module is used for calculating contour characteristic parameters according to the multi-target contour coordinate sequence and comprises the following steps: and obtaining target contour characteristic parameters through the area, the perimeter, the length, the width and the circularity, and further identifying the defects of the product image.
3. A computer storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements a method for intelligent detection of defects in an industrial product as claimed in claim 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110361220.7A CN113240579A (en) | 2021-04-02 | 2021-04-02 | Intelligent industrial product defect detection method and device and computer storage medium thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110361220.7A CN113240579A (en) | 2021-04-02 | 2021-04-02 | Intelligent industrial product defect detection method and device and computer storage medium thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113240579A true CN113240579A (en) | 2021-08-10 |
Family
ID=77130973
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110361220.7A Pending CN113240579A (en) | 2021-04-02 | 2021-04-02 | Intelligent industrial product defect detection method and device and computer storage medium thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113240579A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118134907A (en) * | 2024-04-29 | 2024-06-04 | 金洲精工科技(昆山)有限公司 | Control method and device for drill point type and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105447512A (en) * | 2015-11-13 | 2016-03-30 | 中国科学院自动化研究所 | Coarse-fine optical surface defect detection method and coarse-fine optical surface defect detection device |
| JP2018169710A (en) * | 2017-03-29 | 2018-11-01 | 株式会社富士通アドバンストエンジニアリング | Program, information process system and information processing method |
| CN109115812A (en) * | 2018-08-23 | 2019-01-01 | 中国石油大学(北京) | A kind of weld seam egative film defect identification method and system |
| CN110223296A (en) * | 2019-07-08 | 2019-09-10 | 山东建筑大学 | A kind of screw-thread steel detection method of surface flaw and system based on machine vision |
| CN112541922A (en) * | 2020-12-04 | 2021-03-23 | 北京科技大学 | Test paper layout segmentation method based on digital image, electronic equipment and storage medium |
-
2021
- 2021-04-02 CN CN202110361220.7A patent/CN113240579A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105447512A (en) * | 2015-11-13 | 2016-03-30 | 中国科学院自动化研究所 | Coarse-fine optical surface defect detection method and coarse-fine optical surface defect detection device |
| JP2018169710A (en) * | 2017-03-29 | 2018-11-01 | 株式会社富士通アドバンストエンジニアリング | Program, information process system and information processing method |
| CN109115812A (en) * | 2018-08-23 | 2019-01-01 | 中国石油大学(北京) | A kind of weld seam egative film defect identification method and system |
| CN110223296A (en) * | 2019-07-08 | 2019-09-10 | 山东建筑大学 | A kind of screw-thread steel detection method of surface flaw and system based on machine vision |
| CN112541922A (en) * | 2020-12-04 | 2021-03-23 | 北京科技大学 | Test paper layout segmentation method based on digital image, electronic equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 张宏钊;周松斌;刘伟鑫;: "基于机器视觉的圆形马口铁罐罐口缺陷检测", 自动化与信息工程, no. 01, 15 February 2016 (2016-02-15) * |
| 李荣骏: "基于机器视觉的光缆表面缺陷识别研究", 《中国优秀博硕士学位论文全文数据库(电子期刊) 信息科技辑》, 15 February 2018 (2018-02-15), pages 56 - 72 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118134907A (en) * | 2024-04-29 | 2024-06-04 | 金洲精工科技(昆山)有限公司 | Control method and device for drill point type and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Dong et al. | PGA-Net: Pyramid feature fusion and global context attention network for automated surface defect detection | |
| CN110264416B (en) | Sparse point cloud segmentation method and device | |
| CN111462120A (en) | Defect detection method, device, medium and equipment based on semantic segmentation model | |
| JP7393472B2 (en) | Display scene recognition method, device, electronic device, storage medium and computer program | |
| CN111179233B (en) | Self-adaptive deviation rectifying method based on laser cutting of two-dimensional parts | |
| CN109726195B (en) | Data enhancement method and device | |
| CN111462140B (en) | Real-time image instance segmentation method based on block stitching | |
| CN112613384B (en) | Gesture recognition method, gesture recognition device and control method of interactive display equipment | |
| CN113674421A (en) | 3D target detection method, model training method, related device and electronic equipment | |
| CN112802037A (en) | Portrait extraction method, device, electronic equipment and storage medium | |
| CN116071327A (en) | Workpiece defect detection method based on deep neural network | |
| CN113570606A (en) | Target segmentation method and device and electronic equipment | |
| CN113487538A (en) | Multi-target segmentation defect detection method and device and computer storage medium thereof | |
| CN112967315A (en) | Target tracking method and device and electronic equipment | |
| CN114037684B (en) | Defect detection method based on yolov and attention mechanism model | |
| CN113240579A (en) | Intelligent industrial product defect detection method and device and computer storage medium thereof | |
| CN111275758A (en) | Hybrid 3D visual positioning method and device, computer equipment and storage medium | |
| CN113223037B (en) | Unsupervised semantic segmentation method and unsupervised semantic segmentation system for large-scale data | |
| CN110807416A (en) | Digital instrument intelligent recognition device and method suitable for mobile detection device | |
| CN114037710A (en) | Image segmentation method and device, electronic equipment and storage medium | |
| CN109816710B (en) | Parallax calculation method for binocular vision system with high precision and no smear | |
| CN115937492A (en) | Transformer equipment infrared image identification method based on feature identification | |
| Deshmukh et al. | Analysis of distance measurement system of leading vehicle | |
| CN112991308A (en) | Image quality determination method and device, electronic equipment and medium | |
| CN114120011B (en) | Method for improving image matching precision and speed by using group intelligent algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |