CN110992317B - PCB defect detection method based on semantic segmentation - Google Patents
PCB defect detection method based on semantic segmentation Download PDFInfo
- Publication number
- CN110992317B CN110992317B CN201911134198.1A CN201911134198A CN110992317B CN 110992317 B CN110992317 B CN 110992317B CN 201911134198 A CN201911134198 A CN 201911134198A CN 110992317 B CN110992317 B CN 110992317B
- Authority
- CN
- China
- Prior art keywords
- layer
- pixels
- convolution
- kernel size
- semantic segmentation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
- G01N21/956—Inspecting patterns on the surface of objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8854—Grading and classifying of flaws
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8883—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges involving the calculation of gauges, generating models
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
- G01N21/956—Inspecting patterns on the surface of objects
- G01N2021/95638—Inspecting patterns on the surface of objects for PCB's
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30141—Printed circuit board [PCB]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Pathology (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- Chemical & Material Sciences (AREA)
- Immunology (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Quality & Reliability (AREA)
- Signal Processing (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Multimedia (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a PCB defect detection method based on semantic segmentation, which comprises the following steps: step S1: data acquisition and defect definition; step S2: labeling and cleaning data; step S3: preprocessing an image and preparing a data set; step S4: constructing a neural network model and performing data training; step S5: testing a model; step S6: and (5) defect judgment. The invention provides a PCB defect detection algorithm based on a built semantic segmentation network, which can directly send PCB picture data collected on a production site into a model for judgment after pretreatment without manual intervention.
Description
Technical Field
The invention relates to the field of PCB defect detection, in particular to a PCB defect detection method based on semantic segmentation.
Background
In the production process, the quality problem of the PCB products caused by collision, dirt and the like is concerned by manufacturers, and the PCB defect detection work and importance thereof are paid attention to, and the defect detection task becomes harder along with the continuous expansion of the market demands of the PCB.
In the past, traditional PCB defect detection methods have been developed from manual screening to an automatic detection stage in the industrial field, such as through machine vision and traditional image processing technology, but the methods have the problems of large consumption of manpower, manual misjudgment, missed judgment of automatic detection equipment, low detection efficiency and the like.
Accordingly, there is a need in the art for further improvements and perfection.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a PCB defect detection method based on semantic segmentation.
The aim of the invention is achieved by the following technical scheme:
a PCB defect detection method based on semantic segmentation mainly comprises the following specific steps:
step S1: data acquisition and defect definition.
Specifically, the step S1 specifically includes: and collecting a large number of PCB pictures and a corresponding number of template pictures which cannot be judged to be defective by a detection device under the traditional automatic process flow from a factory production site, and defining the template pictures as the standard of the defects to be detected of the PCB.
Specifically, the step S1 further includes: and determining the defect type of the PCB by combining the manufacturer requirements and expert knowledge of the industry, and performing data marking by using semantic segmentation marking software as an image defect marking tool.
Specifically, the step S1 further includes: the defect types include: open circuit, short circuit, kong Po, burrs, gaps, copper slag, wire young, pinholes.
Specifically, the step S1 further includes: the acquired pictures are unified in size of 572 x 572 (long x wide), and corresponding template pictures of each picture are prepared.
Step S2: labeling and cleaning data: and generating Mask pictures from the marked picture files, and storing the Mask pictures as a true value of a model prediction error in a model training process.
Specifically, the step S2 of cleaning the image data is to classify the actual image acquired in the step S1 into the true defect and the false defect according to the definitions of the manufacturer and the expert in the industry on the defects.
Step S3: image preprocessing and data set manufacturing: unifying the image size, enhancing the data, enlarging the number of pictures, and completing the production of a PCB data set { original image (trace_1), mask image (trace_2) and test set (test) }. The Mask image refers to a Mask image, and is generally marked as a target area. The method is used for carrying out operation with the original image and marking the target area from the original image.
Step S4: building a neural network model and performing data training: and constructing a full-convolution semantic segmentation network Model, setting initialization parameters, a loss function and an optimizer of the full-convolution semantic segmentation network Model, inputting training data sets { train_1 and train_2} into the constructed semantic segmentation network to start training, and obtaining a segmentation Model (Model) after multiple iterative convergence.
Further, in the step S4, an image semantic segmentation algorithm based on deep learning is adopted, and the network is a neural network with a full convolution layer, and the specific structure is as follows:
part1: feature extraction
The convolution layer is used for extracting characteristics of the multi-channel composite image;
the pooling layer reduces the size of the image, increases the receptive field, reduces the parameters and prevents over fitting;
layers 1 and 2, a convolution layer, a convolution kernel size of 3 pixels, a step length of 1 pixel and outputting 64 feature images;
layer 3, pooling layer, pooling kernel size of 2 pixels, step length of 2 pixels;
layers 4 and 5, a convolution layer, a convolution kernel size of 3 pixels, a step length of 1 pixel and 128 feature images output;
layer 6, pooling layer, pooling kernel size of 2 pixels, step length of 2 pixels;
the 7 th layer, the 8 th layer, the convolution kernel size is 3 pixels, the step length is 1 pixel, and 256 feature images are output;
layer 9, pooling layer, pooling kernel size of 2 pixels, step length of 2 pixels;
layers 10 and 11, a convolution layer, a convolution kernel size of 3 pixels, a step length of 1 pixel and 512 feature images output;
layer 12, pooling layer, pooling kernel size of 2 pixels, step length of 2 pixels;
layers 13 and 14, a convolution layer, a convolution kernel size of 3 pixels, a step length of 1 pixel and 1024 feature images output;
part2: upsampling
Restoring the image size to be the size of the input image so that the deep learning network structure model achieves the semantic segmentation effect of the pixel level;
layer 15, deconvolution layer, convolution kernel size of 2 pixels, step length of 2 pixels, and output combination after layer 14 convolution;
layers 16 and 17, a convolution layer, a convolution kernel size of 3 pixels, a step length of 1 pixel and 512 feature images output;
layer 18, deconvolution layer, convolution kernel size of 2 pixels, step length of 2 pixels, and output combination after convolution of layer 17;
the 19 th and 20 th layers, the convolution kernel size is 3 pixels, the step length is 1 pixel, and 256 feature images are output;
layer 21, deconvolution layer, convolution kernel size of 2 pixels, step length of 2 pixels, and output combination after convolution with layer 20;
the 22 nd layer, the 23 rd layer, the convolution kernel size is 3 pixels, the step length is 1 pixel, and 128 feature images are output;
layer 24, deconvolution layer, convolution kernel size of 2 pixels, step length of 2 pixels, and output combination after layer 23 convolution;
the 25 th and 26 th layers, the convolution kernel size is 3 pixels, the step length is 1 pixel, and 64 feature images are output;
and the 27 th layer, the convolution kernel size is 1 pixel, the step length is 1 pixel, and 2 feature images are output.
Step S5: model test: inputting the test data set { test } into a semantic segmentation network, performing semantic segmentation on the picture by utilizing the segmentation Model (Model) obtained in the step S4, comparing the obtained PCB segmentation map { Mask } with the template map, judging the segmentation precision of the Model, if the precision requirement is met, finishing Model training, and otherwise, continuing iteration or adding the training data set (modification parameters).
Step S6: defect judgment: and (5) deriving a model file in the step (S5), and classifying defects of the PCB picture of which the defects need to be judged by using the model.
The working process and principle of the invention are as follows: the invention provides a PCB defect detection algorithm based on a built semantic segmentation network, which can directly send PCB picture data collected on a production site into a model for judgment after pretreatment without manual intervention.
Compared with the prior art, the invention has the following advantages:
(1) The PCB defect detection method based on semantic segmentation provided by the invention fully utilizes the advantages of high accuracy of a deep learning algorithm and rapid detection of a large amount of image data in the PCB defect detection in process production, and promotes the intelligent development of the traditional industry.
(2) The PCB defect detection method based on the semantic segmentation provided by the invention is based on the deep learning, so that the manual intervention can be greatly reduced, and the work can be completed more quickly and efficiently.
(3) The PCB defect detection method based on semantic segmentation overcomes the defects of the traditional image processing detection method, combines the current mature deep learning research result, utilizes the semantic segmentation model to detect the defects of the PCB, and can efficiently detect in batches. The PCB image collected by description is manufactured into a training data set under a semantic segmentation network, a trained network model is obtained after the image is preprocessed, and the prediction of the rear end of the model is combined, so that intelligent, high-efficiency and high-precision defect detection is realized, and the whole defect detection workflow can greatly reduce manual intervention.
Drawings
Fig. 1 is a schematic flow chart of a method for detecting defects of a PCB board based on semantic segmentation.
Fig. 2 is a schematic diagram of a defect type of a PCB board in an actual production process according to the present invention.
FIG. 3 is a graph of the detection results generated when the semantic segmentation model is used for detecting the defect classification.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more clear and clear, the present invention will be further described below with reference to the accompanying drawings and examples.
Example 1:
as shown in fig. 1, 2 and 3, the present embodiment discloses a method for detecting defects of a PCB board based on semantic segmentation, which mainly includes the following specific steps:
step S1: data acquisition and defect definition.
Specifically, the step S1 specifically includes: and collecting a large number of PCB pictures and a corresponding number of template pictures which cannot be judged to be defective by a detection device under the traditional automatic process flow from a factory production site, and defining the template pictures as the standard of the defects to be detected of the PCB.
Specifically, the step S1 further includes: and determining the defect type of the PCB by combining the manufacturer requirements and expert knowledge of the industry, and performing data marking by using semantic segmentation marking software as an image defect marking tool.
Specifically, the step S1 further includes: the defect types include: open circuit, short circuit, kong Po, burrs, gaps, copper slag, wire young, pinholes.
Specifically, the step S1 further includes: the acquired pictures are unified in size of 572 x 572 (long x wide), and corresponding template pictures of each picture are prepared.
Step S2: labeling and cleaning data: and generating Mask pictures from the marked picture files, and storing the Mask pictures as a true value of a model prediction error in a model training process.
Specifically, the step S2 of cleaning the image data is to classify the actual image acquired in the step S1 into the true defect and the false defect according to the definitions of the manufacturer and the expert in the industry on the defects.
Step S3: image preprocessing and data set manufacturing: unifying the image size, enhancing the data, enlarging the number of pictures, and completing the production of a PCB data set { original image (trace_1), mask image (trace_2) and test set (test) }.
Step S4: building a neural network model and performing data training: and constructing a full-convolution semantic segmentation network Model, setting initialization parameters, a loss function and an optimizer of the full-convolution semantic segmentation network Model, inputting training data sets { train_1 and train_2} into the constructed semantic segmentation network to start training, and obtaining a segmentation Model (Model) after multiple iterative convergence.
Further, in the step S4, an image semantic segmentation algorithm based on deep learning is adopted, and the network is a neural network with a full convolution layer, and the specific structure is as follows:
part1: feature extraction
The convolution layer is used for extracting characteristics of the multi-channel composite image;
the pooling layer reduces the size of the image, increases the receptive field, reduces the parameters and prevents over fitting;
layers 1 and 2, a convolution layer, a convolution kernel size of 3 pixels, a step length of 1 pixel and outputting 64 feature images;
layer 3, pooling layer, pooling kernel size of 2 pixels, step length of 2 pixels;
layers 4 and 5, a convolution layer, a convolution kernel size of 3 pixels, a step length of 1 pixel and 128 feature images output;
layer 6, pooling layer, pooling kernel size of 2 pixels, step length of 2 pixels;
the 7 th layer, the 8 th layer, the convolution kernel size is 3 pixels, the step length is 1 pixel, and 256 feature images are output;
layer 9, pooling layer, pooling kernel size of 2 pixels, step length of 2 pixels;
layers 10 and 11, a convolution layer, a convolution kernel size of 3 pixels, a step length of 1 pixel and 512 feature images output;
layer 12, pooling layer, pooling kernel size of 2 pixels, step length of 2 pixels;
layers 13 and 14, a convolution layer, a convolution kernel size of 3 pixels, a step length of 1 pixel and 1024 feature images output;
part2: upsampling
Restoring the image size to be the size of the input image so that the deep learning network structure model achieves the semantic segmentation effect of the pixel level;
layer 15, deconvolution layer, convolution kernel size of 2 pixels, step length of 2 pixels, and output combination after layer 14 convolution;
layers 16 and 17, a convolution layer, a convolution kernel size of 3 pixels, a step length of 1 pixel and 512 feature images output;
layer 18, deconvolution layer, convolution kernel size of 2 pixels, step length of 2 pixels, and output combination after convolution of layer 17;
the 19 th and 20 th layers, the convolution kernel size is 3 pixels, the step length is 1 pixel, and 256 feature images are output;
layer 21, deconvolution layer, convolution kernel size of 2 pixels, step length of 2 pixels, and output combination after convolution with layer 20;
the 22 nd layer, the 23 rd layer, the convolution kernel size is 3 pixels, the step length is 1 pixel, and 128 feature images are output;
layer 24, deconvolution layer, convolution kernel size of 2 pixels, step length of 2 pixels, and output combination after layer 23 convolution;
the 25 th and 26 th layers, the convolution kernel size is 3 pixels, the step length is 1 pixel, and 64 feature images are output;
and the 27 th layer, the convolution kernel size is 1 pixel, the step length is 1 pixel, and 2 feature images are output.
Step S5: model test: inputting the test data set { test } into a semantic segmentation network, performing semantic segmentation on the picture by utilizing the segmentation Model (Model) obtained in the step S4, comparing the obtained PCB segmentation map { Mask } with the template map, judging the segmentation precision of the Model, if the precision requirement is met, finishing Model training, and otherwise, continuing iteration or adding the training data set (modification parameters).
Step S6: defect judgment: and (5) deriving a model file in the step (S5), and classifying defects of the PCB picture of which the defects need to be judged by using the model.
The working process and principle of the invention are as follows: the invention provides a PCB defect detection algorithm based on a built semantic segmentation network, which can directly send PCB picture data collected on a production site into a model for judgment after pretreatment without manual intervention.
Compared with the prior art, the invention has the following advantages:
(1) The PCB defect detection method based on semantic segmentation provided by the invention fully utilizes the advantages of high accuracy of a deep learning algorithm and rapid detection of a large amount of image data in the PCB defect detection in process production, and promotes the intelligent development of the traditional industry.
(2) The PCB defect detection method based on the semantic segmentation provided by the invention is based on the deep learning, so that the manual intervention can be greatly reduced, and the work can be completed more quickly and efficiently.
The PCB defect detection method based on semantic segmentation overcomes the defects of the traditional image processing detection method, combines the current mature deep learning research result, utilizes the semantic segmentation model to detect the defects of the PCB, and can efficiently detect in batches. The PCB image collected by description is manufactured into a training data set under a semantic segmentation network, a trained network model is obtained after the image is preprocessed, and the prediction of the rear end of the model is combined, so that intelligent, high-efficiency and high-precision defect detection is realized, and the whole defect detection workflow can greatly reduce manual intervention.
Example 2:
referring to fig. 1 to 3, the embodiment discloses a method for detecting defects of a PCB board based on semantic segmentation, which is characterized by comprising the following implementation steps:
step 1: image data acquisition and defect type definition.
Step 1.1: collecting a large number of PCB images, uniformly sizing the collected images to 572 x 572 (length x width), and preparing corresponding template images of each image;
step 1.2: and defining PCB defect type standards by combining manufacturer requirements and industry knowledge thereof, wherein the defect types comprise: open circuit, short circuit, kong Po, burrs, gaps, copper slag, wire young, pinholes.
Step 2: and (5) cleaning and labeling the image data.
Step 2.1: cleaning the image data acquired in the step 1, marking defect characteristics by using semantic segmentation marking software, and generating a json file for storage after marking;
step 2.2: and converting the json file into a Mask picture to be stored, and calculating a true value { Y } of the model prediction error as a model training process.
Step 3: image processing and data set making.
And (3) preprocessing the original PCB board image obtained in the step (2) and the Mask image with the label after marking, such as image translation, size conversion, rotation conversion and the like, unifying image parameters, and enhancing a rich image training set through data so as to prevent the model from being fitted excessively. Finally, dividing the picture into an original picture, a Mask picture and a test, and completing the manufacture of a PCB data set { track_1, track_2 and test };
step 4: semantic segmentation network model training.
Constructing a full-convolution semantic segmentation network Model, setting initialization parameters, a loss function and an optimizer of the full-convolution semantic segmentation network Model, inputting training data sets { train_1 and train_2} in the constructed semantic segmentation network for training, and obtaining a segmentation Model (Model) after multiple iterative convergence;
and 5, testing a semantic segmentation network model. Inputting a test data set { test } into a semantic segmentation network, carrying out semantic segmentation on the picture by utilizing the segmentation Model (Model) obtained in the step 4, comparing the obtained PCB segmentation graph { Mask } with a template graph, judging the segmentation precision of the Model, if the precision requirement is met, finishing Model training, otherwise, continuing iteration or adding a training data set (modification parameters);
and 6, judging the defects of the PCB. And (5) deriving a model file in the step (5), and classifying defects of the PCB picture of which the defects need to be judged by using the model.
Further, the step 2 of cleaning the image data is to classify the real defects and the false defects of the actual image acquired in the step 1 according to the definitions of the manufacturer and the industry expert on the defects.
Further, a semantic segmentation network model is constructed in the step 4. Modeling analysis is carried out on the PCB by using the actual demands of PCB manufacturers, and then a full-convolution neural network model is built by combining a deep learning semantic segmentation algorithm, wherein the specific model structure is as follows:
the size of the input artwork is 224 x 224.
Layers 1 and 2, a convolution layer, a convolution kernel size of 3, a step length of 1 and outputting 64 feature graphs;
layer 3, pooling layer, pooling core size of 2, step length of 2;
layers 4 and 5, a convolution layer, a convolution kernel size of 3, a step length of 1 and 128 feature graphs output;
layer 6, pooling layer, pooling core size of 2, step length of 2;
the 7 th and 8 th layers, the convolution kernel size is 3, the step length is 1, and 256 feature maps are output;
layer 9, pooling layer, pooling core size of 2, step length of 2;
layers 10 and 11, a convolution layer, a convolution kernel size of 3, a step length of 1 and 512 feature graphs output;
layer 12, pooling layer, pooling core size of 2, step length of 2;
layers 13 and 14, a convolution layer, a convolution kernel size of 3, a step length of 1 and 1024 feature images output;
layer 15, deconvolution layer, convolution kernel size 2, step length 2, and three-dimensional combination with the output after layer 14 convolution;
layers 16 and 17, a convolution layer, a convolution kernel size of 3, a step length of 1 and 512 feature graphs output;
layer 18, deconvolution layer, convolution kernel size 2, step length 2, and three-dimensional combination with the output after layer 17 convolution;
layers 19 and 20, a convolution layer, a convolution kernel size of 3, a step length of 1 and 256 feature images output;
layer 21, deconvolution layer, convolution kernel size 2, step length 2, and three-dimensional combination with the output after layer 20 convolution;
layers 22 and 23, a convolution layer, a convolution kernel size of 3, a step length of 1 and 128 feature graphs output;
layer 24, deconvolution layer, convolution kernel size is 2, step length is 2, and three-dimensional combination is carried out with the output after layer 23 convolution;
layers 25 and 26, a convolution layer, a convolution kernel size of 3, a step length of 1 and 64 feature graphs output;
layer 27, convolution layer, convolution kernel size 3, step length 1, 2 feature graphs output;
layer 28, the convolution layer (output layer), convolution kernel size 1, step size 1, output 1 feature map.
The above examples are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above examples, and any other changes, modifications, substitutions, combinations, and simplifications that do not depart from the spirit and principle of the present invention should be made in the equivalent manner, and the embodiments are included in the protection scope of the present invention.
Claims (6)
1. A PCB defect detection method based on semantic segmentation is characterized by comprising the following steps:
step S1: data acquisition and defect definition;
step S2: labeling and cleaning data: generating Mask pictures from the marked picture files, and storing the Mask pictures as a true value of a model prediction error in a model training process;
step S3: image preprocessing and data set manufacturing: unifying the size of the image, enhancing the data, enlarging the number of pictures, and completing the manufacture of a PCB data set, wherein the PCB data set comprises an original image, a Mask image and a test set;
step S4: building a neural network model and performing data training: constructing a full-convolution semantic segmentation network model, setting initialization parameters, a loss function and an optimizer of the full-convolution semantic segmentation network model, inputting a training data set in the constructed semantic segmentation network to start training, and obtaining a segmentation model after multiple iteration convergence;
step S5: model test: inputting the test data set into a semantic segmentation network, performing semantic segmentation on the picture by utilizing the segmentation model obtained in the step S4, comparing the obtained PCB segmentation graph with the template graph, judging the segmentation precision of the model, if the precision requirement is met, finishing model training, and otherwise, continuing iteration or adding the training data set;
step S6: defect judgment: the model file in the step S5 is exported, and the PCB picture needing to be judged for defects is subjected to defect classification by using the model;
in the step S4, an image semantic segmentation algorithm based on deep learning is adopted, and the network is a neural network with a full convolution layer, and the specific structure is as follows:
part1: feature extraction
The convolution layer is used for extracting characteristics of the multi-channel composite image;
the pooling layer reduces the size of the image, increases the receptive field, reduces the parameters and prevents over fitting;
layers 1 and 2, a convolution layer, a convolution kernel size of 3 pixels, a step length of 1 pixel and outputting 64 feature images;
layer 3, pooling layer, pooling kernel size of 2 pixels, step length of 2 pixels;
layers 4 and 5, a convolution layer, a convolution kernel size of 3 pixels, a step length of 1 pixel and 128 feature images output;
layer 6, pooling layer, pooling kernel size of 2 pixels, step length of 2 pixels;
the 7 th layer, the 8 th layer, the convolution kernel size is 3 pixels, the step length is 1 pixel, and 256 feature images are output;
layer 9, pooling layer, pooling kernel size of 2 pixels, step length of 2 pixels;
layers 10 and 11, a convolution layer, a convolution kernel size of 3 pixels, a step length of 1 pixel and 512 feature images output;
layer 12, pooling layer, pooling kernel size of 2 pixels, step length of 2 pixels;
layers 13 and 14, a convolution layer, a convolution kernel size of 3 pixels, a step length of 1 pixel and 1024 feature images output;
part2: upsampling
Restoring the image size to be the size of the input image so that the deep learning network structure model achieves the semantic segmentation effect of the pixel level;
layer 15, deconvolution layer, convolution kernel size of 2 pixels, step length of 2 pixels, and output combination after layer 14 convolution;
layers 16 and 17, a convolution layer, a convolution kernel size of 3 pixels, a step length of 1 pixel and 512 feature images output;
layer 18, deconvolution layer, convolution kernel size of 2 pixels, step length of 2 pixels, and output combination after convolution of layer 17;
the 19 th and 20 th layers, the convolution kernel size is 3 pixels, the step length is 1 pixel, and 256 feature images are output;
layer 21, deconvolution layer, convolution kernel size of 2 pixels, step length of 2 pixels, and output combination after convolution with layer 20;
the 22 nd layer, the 23 rd layer, the convolution kernel size is 3 pixels, the step length is 1 pixel, and 128 feature images are output;
layer 24, deconvolution layer, convolution kernel size of 2 pixels, step length of 2 pixels, and output combination after layer 23 convolution;
the 25 th and 26 th layers, the convolution kernel size is 3 pixels, the step length is 1 pixel, and 64 feature images are output;
and the 27 th layer, the convolution kernel size is 1 pixel, the step length is 1 pixel, and 2 feature images are output.
2. The method for detecting defects of a PCB board based on semantic segmentation according to claim 1, wherein the step S1 specifically includes: and collecting a large number of PCB pictures and a corresponding number of template pictures which cannot be judged to be defective by a detection device under the traditional automatic process flow from a factory production site, and defining the template pictures as the standard of the defects to be detected of the PCB.
3. The method for detecting defects of a PCB board based on semantic segmentation according to claim 1, wherein the step S1 further includes: and determining the defect type of the PCB by combining the manufacturer requirements and expert knowledge of the industry, and performing data marking by using semantic segmentation marking software as an image defect marking tool.
4. The method for detecting defects of a PCB board based on semantic segmentation according to claim 1, wherein the step S1 further includes: the defect types include: open circuit, short circuit, kong Po, burrs, gaps, copper slag, wire young, pinholes.
5. The method for detecting defects of a PCB board based on semantic segmentation according to claim 1, wherein the step S1 further includes: the captured pictures are uniformly sized 572 a and a corresponding template map is prepared for each picture.
6. The method for detecting defects of a PCB board based on semantic segmentation according to claim 1, wherein the image data cleaning in the step S2 is to classify the actual image acquired in the step S1 into true defects and false defects according to the definition of defects by factories and industry experts.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911134198.1A CN110992317B (en) | 2019-11-19 | 2019-11-19 | PCB defect detection method based on semantic segmentation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911134198.1A CN110992317B (en) | 2019-11-19 | 2019-11-19 | PCB defect detection method based on semantic segmentation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110992317A CN110992317A (en) | 2020-04-10 |
| CN110992317B true CN110992317B (en) | 2023-09-22 |
Family
ID=70084985
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911134198.1A Active CN110992317B (en) | 2019-11-19 | 2019-11-19 | PCB defect detection method based on semantic segmentation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110992317B (en) |
Families Citing this family (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111583187B (en) * | 2020-04-14 | 2023-11-07 | 佛山市南海区广工大数控装备协同创新研究院 | PCB defect detection method based on CNN visualization |
| CN111709910A (en) * | 2020-05-18 | 2020-09-25 | 扬州小纳熊机器人有限公司 | PCB defect detection algorithm based on convolutional neural network |
| CN111812118A (en) * | 2020-06-24 | 2020-10-23 | 阿丘机器人科技(苏州)有限公司 | PCB detection method, device, equipment and computer readable storage medium |
| CN111753732A (en) * | 2020-06-24 | 2020-10-09 | 佛山市南海区广工大数控装备协同创新研究院 | Vehicle multi-target tracking method based on target center point |
| CN111882547A (en) * | 2020-07-30 | 2020-11-03 | 佛山市南海区广工大数控装备协同创新研究院 | PCB missing part detection method based on neural network |
| CN112686833B (en) * | 2020-08-22 | 2023-06-06 | 安徽大学 | Industrial product surface defect detection and classification device based on convolutional neural network |
| CN112164035B (en) * | 2020-09-15 | 2023-04-28 | 郑州金惠计算机***工程有限公司 | Image-based defect detection method and device, electronic equipment and storage medium |
| CN112365478A (en) * | 2020-11-13 | 2021-02-12 | 上海海事大学 | Motor commutator surface defect detection model based on semantic segmentation |
| CN112288741A (en) * | 2020-11-23 | 2021-01-29 | 四川长虹电器股份有限公司 | Product surface defect detection method and system based on semantic segmentation |
| CN112734703A (en) * | 2020-12-28 | 2021-04-30 | 佛山市南海区广工大数控装备协同创新研究院 | PCB defect optimization method by utilizing AI cloud collaborative detection |
| CN113112482A (en) * | 2021-04-15 | 2021-07-13 | 深圳市玻尔智造科技有限公司 | PCB defect detection method based on attention mechanism network |
| CN113362277A (en) * | 2021-04-26 | 2021-09-07 | 辛米尔视觉科技(上海)有限公司 | Workpiece surface defect detection and segmentation method based on deep learning |
| CN113239930B (en) * | 2021-05-14 | 2024-04-05 | 广州广电运通金融电子股份有限公司 | Glass paper defect identification method, system, device and storage medium |
| CN113379685A (en) * | 2021-05-26 | 2021-09-10 | 广东炬森智能装备有限公司 | PCB defect detection method and device based on dual-channel feature comparison model |
| CN113506243A (en) * | 2021-06-04 | 2021-10-15 | 联合汽车电子有限公司 | PCB welding defect detection method and device and storage medium |
| CN113763358B (en) * | 2021-09-08 | 2024-01-09 | 合肥中科类脑智能技术有限公司 | Method and system for detecting oil leakage and metal corrosion of transformer substation based on semantic segmentation |
| CN114529561A (en) * | 2022-02-22 | 2022-05-24 | 成都数之联科技股份有限公司 | Method, system, device and medium for segmenting line image and background plate image |
| CN114882039B (en) * | 2022-07-12 | 2022-09-16 | 南通透灵信息科技有限公司 | PCB defect identification method applied to automatic PCB sorting process |
| CN115713533B (en) * | 2023-01-10 | 2023-06-06 | 佰聆数据股份有限公司 | Power equipment surface defect detection method and device based on machine vision |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109800736A (en) * | 2019-02-01 | 2019-05-24 | 东北大学 | A kind of method for extracting roads based on remote sensing image and deep learning |
| CN110060238A (en) * | 2019-04-01 | 2019-07-26 | 桂林电子科技大学 | Pcb board based on deep learning marks print quality inspection method |
-
2019
- 2019-11-19 CN CN201911134198.1A patent/CN110992317B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109800736A (en) * | 2019-02-01 | 2019-05-24 | 东北大学 | A kind of method for extracting roads based on remote sensing image and deep learning |
| CN110060238A (en) * | 2019-04-01 | 2019-07-26 | 桂林电子科技大学 | Pcb board based on deep learning marks print quality inspection method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110992317A (en) | 2020-04-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110992317B (en) | PCB defect detection method based on semantic segmentation | |
| CN111402226A (en) | Surface defect detection method based on cascade convolution neural network | |
| CN113239930B (en) | Glass paper defect identification method, system, device and storage medium | |
| CN110021005A (en) | Circuit board flaw screening technique and its device and computer-readable recording medium | |
| CN112766110A (en) | Training method of object defect recognition model, object defect recognition method and device | |
| CN113222913B (en) | Circuit board defect detection positioning method, device and storage medium | |
| CN111103307A (en) | Pcb defect detection method based on deep learning | |
| CN115597494B (en) | Precision detection method and system for prefabricated part preformed hole based on point cloud | |
| WO2024002187A1 (en) | Defect detection method, defect detection device, and storage medium | |
| CN112070712B (en) | Printing defect detection method based on self-encoder network | |
| CN113627435A (en) | Method and system for detecting and identifying flaws of ceramic tiles | |
| CN111882547A (en) | PCB missing part detection method based on neural network | |
| CN111091534A (en) | Target detection-based pcb defect detection and positioning method | |
| CN113205511B (en) | Electronic component batch information detection method and system based on deep neural network | |
| Caliskan et al. | Design and realization of an automatic optical inspection system for PCB solder joints | |
| CN114429445A (en) | PCB defect detection and identification method based on MAIRNet | |
| CN113962929A (en) | Photovoltaic cell assembly defect detection method and system and photovoltaic cell assembly production line | |
| CN113705564A (en) | Pointer type instrument identification reading method | |
| CN112184679A (en) | YOLOv 3-based wine bottle flaw automatic detection method | |
| CN112686843B (en) | Board defect detection method and system based on neural network | |
| CN116245882A (en) | Circuit board electronic element detection method and device and computer equipment | |
| CN115601610A (en) | Fabric flaw detection method based on improved EfficientDet model | |
| CN115587989A (en) | Workpiece CT image defect detection and segmentation method and system | |
| CN113034432A (en) | Product defect detection method, system, device and storage medium | |
| CN115719326A (en) | PCB defect detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |