CN113222901A

CN113222901A - Method for detecting surface defects of steel ball based on single stage

Info

Publication number: CN113222901A
Application number: CN202110418127.5A
Authority: CN
Inventors: 王义文; 王恺娇; 刘立佳; 周丽杰
Original assignee: Harbin University of Science and Technology
Current assignee: Harbin University of Science and Technology
Priority date: 2021-04-19
Filing date: 2021-04-19
Publication date: 2021-08-06

Abstract

The invention discloses a method for detecting surface defects of a steel ball based on a single stage, which has higher detection precision, detection efficiency and robustness compared with the traditional manual detection. The single-stage detection method is based on a YOLOv4 network structure pre-training model. The method mainly comprises the following steps: acquiring steel ball surface image data, image augmentation, data set image annotation, data set division, constructing a pre-training model, training a model and verifying the model. The method model can automatically extract the characteristics of the steel ball surface defects and accurately and quickly detect the positions of various defects on the steel ball surface. The method uses Python programming language, is realized by taking a Keras framework as a front end, takes Tensorflow as data processing of a rear end, and uses GPU (NVIDIA, GTX1080Ti) to train, verify and test the model so as to obtain corresponding evaluation indexes and test results. According to the experimental result, the method can quickly and accurately detect the surface defects of the steel balls.

Description

Method for detecting surface defects of steel ball based on single stage

Technical Field

The invention relates to the technical field of digital image processing, in particular to a method for detecting surface defects of a steel ball based on a single stage.

Background

Due to the influence of technical factors such as different steel ball raw material batches, the precision of a processing machine tool, the process control in the processing process and the like, local fine surface defects can be generated in the processing process of the steel balls. The traditional manual detection has the problems of low detection precision, low detection efficiency and poor anti-interference performance. Therefore, a target detection method based on deep learning is developed and widely applied to industrial detection technologies, such as Fast RCNN, SSD and other target detection models, but the detection method has problems of poor model accuracy, slow detection speed, and the like when defects are of unbalanced types and small target defects are detected.

Disclosure of Invention

The invention aims to solve the problems of poor accuracy, low detection speed and the like of a steel ball surface defect detection method, and provides a method based on single-stage steel ball surface defect detection. According to the deep learning, the characteristics can be automatically extracted from the original data, and the extracted characteristics are correspondingly processed. Because the characteristics do not need to be extracted manually, the algorithm can enable the output result to be more objective. The single-stage target detection method considers that all areas of the image are potential targets, target candidate areas do not need to be generated, and target classification and accurate positioning are directly carried out on each position. Therefore, single-stage target detection has faster detection efficiency. The invention mainly solves the problems of low stability, low detection precision, low efficiency and the like of the prior art for detecting the surface defects of the steel balls, thereby effectively improving the detection efficiency and the detection precision of products and finally improving the quality of the products. In order to verify the accuracy and reliability of the algorithm, the algorithm is subjected to experimental analysis, the average progress (mAP) of defect identification reaches 91.37%, and the intersection ratio (IoU) reaches 93.25%.

In order to achieve the purpose, the invention adopts the technical scheme that: a method for detecting surface defects of steel balls based on a single stage comprises the following specific steps:

firstly, acquiring steel ball surface image data and establishing a corresponding data set;

step two, properly expanding the steel ball surface image data acquired in the step one by adopting a data augmentation method;

step three, marking the steel balls with surface defects one by the data set obtained in the step two;

step four, dividing the data set obtained in the step three into a training set, a verification set and a test set, wherein 80% of the data set is used as the training data set, 10% of the data set is used as the verification data set, and 10% of the data set is used as the test data set;

constructing a pre-training model of a YOLOv4 network structure, wherein the main network part is a CSPDarknet53 network, the feature fusion part is SPP and PAN, and the prediction part is YOLOv 3;

step six, training a model: adopting a transfer learning mode, training a finished YOLOv4 pre-training model by using an MS COCO data set, initializing weights in an original model, finely adjusting parameters in the method, freezing a part of convolution layers (generally the first three layers) in a trunk network, setting parameters such as dynamic attenuation learning rate, and the like, wherein a loss function from the training model to a verification set does not fall, finally realizing the accurate positioning of the surface defects of the various types of steel balls, and simultaneously saving the best weights of the trained models to obtain a qualified YOLOv4 model;

step seven, verifying the model: and (4) carrying out defect detection on the steel ball surface image by using a qualified YOLOv4 model to obtain a detection result.

Compared with the prior art, the invention has the beneficial effects that:

1. the invention can accurately and quickly identify the defects and position the defects in actual production by the detection method, the average progress (mAP) of defect identification reaches 91.37%, and the intersection ratio (IoU) reaches 93.25%.

2. The method can be applied to the surface quality detection of other industrial products, and provides a new method for processing by digital image technology.

3. The method can replace manual detection, unify detection standards, reduce labor force, reduce production cost and improve detection efficiency and accuracy.

Drawings

FIG. 1 is a flow chart of a single stage detection method of the present invention;

FIG. 2 is a network architecture diagram of YOLOv4 of the present invention;

FIG. 3 is a schematic view of the detecting device of the present invention;

FIG. 4 is a schematic view of the position of the camera according to the present invention;

FIG. 5 is an image captured by a camera of the present invention;

FIG. 6 is a standard image generated from a camera capturing a picture according to the present invention;

FIG. 7 is a data set image annotation of the present invention;

FIG. 8 is a diagram illustrating the detection results of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. The specific embodiments described herein are merely illustrative of the invention and are not intended to be limiting.

As shown in fig. 1, image data is acquired through a steel ball surface image acquisition system based on machine vision, the acquired image is preprocessed, namely, an image of a data set is enlarged, then the data set is labeled by using a tool LabelImg, the labeled data set is divided to obtain a training set, a verification set and a test set, wherein 80% of the training set, 10% of the verification set and 10% of the test set are used as training data sets, a pre-training model is constructed and trained and adjusted to obtain a qualified Yolov4 model, and finally, a steel ball surface image to be detected is input into an optimal model for defect detection to obtain a detection result.

The method for detecting the surface defects of the steel ball based on the single-stage comprises the following processing steps:

acquiring steel ball surface image data, establishing a steel ball surface defect image acquisition system based on a machine vision method, acquiring the steel ball surface image data by using a double black and white CCD camera, and establishing a corresponding data set;

step two, properly expanding the steel ball surface image data acquired in the step one by adopting a data augmentation method, mainly aiming at preventing the overfitting of the model and enhancing the robustness and the bloom of the model, wherein the specific image augmentation method comprises the following steps: methods that do not change the data properties, such as horizontal flipping, vertical flipping, etc.;

thirdly, marking the steel balls with the surface defects one by using a tool LabelImg on the data set obtained in the second step, and obtaining an xml format file corresponding to each image in the data set after marking, wherein the xml file comprises marking information of the corresponding image;

constructing a pre-training model of a YOLOv4 network structure, wherein a main network part is a CSPDarknet53 network, a feature fusion part is SPP and PAN, a prediction part is YOLOv3, and a CSPDarknet53 network is formed by adding a CSP module into a Darknet53 network, so that the feature extraction capability of the network can be effectively improved, the accuracy can be kept while the weight is reduced, the calculation bottleneck is reduced, and the memory cost is reduced; the SPP network is called Spatial Pyramid Pooling, and the purpose of the method is mainly to increase the receptive field of the YOLOv4 network, complete the maximum Pooling operation on the learned features in the network, and finally splice all the obtained feature maps together according to dimensions to form a new feature map; the PAN is mainly used for further carrying out down-sampling on the features after the features in the network complete the up-sampling, more specifically embodying the sampled features, and finally splicing the sampled features with other features for target detection;

step six, training a model: the method adopts the idea of deep migration learning to carry out model training on the method because the number of collected samples in a data set is limited and cannot support the original training of the method, and in the training process of the method, firstly, a model which is trained by an MS COCO data set, namely a pre-training model, is used for initializing the weight in the original model; then, fine adjustment is carried out on parameters in the method, partial parameters (generally the first three layers) in the main network are frozen firstly, the parameters of the layers are ensured not to participate in updating, the updating efficiency of the model parameters is improved conveniently, parameters such as dynamic attenuation learning rate and the like are set, the curve from the training model to the loss function is stable and does not fall down, accurate positioning of various defects on the surface of the steel ball is finally realized, meanwhile, the weight of the model is saved, and a qualified Yolov4 model is obtained;

The invention can automatically position the surface defect characteristics of the steel ball and improve the detection precision and the detection efficiency of the surface defect detection of the steel ball.

Claims

1. A method for detecting surface defects of steel balls based on a single stage is characterized by comprising the following specific steps:

2. The method for detecting the surface defects of the steel ball based on the single stage as claimed in claim 1, wherein the steel ball surface image acquisition system based on the machine vision is selected to be divided into a feeding mechanism, an illuminating mechanism, a steel ball surface unfolding mechanism and an image acquisition device; in order to reduce the reflection of the surface of the steel ball, the unfolding system is soaked in a detection liquid with a medium of aviation kerosene; the unfolding disc is provided with 90 detection cavities, two CCD cameras are arranged at the same time, each CCD camera can image seven steel balls, the two CCD cameras can image 14 steel balls at the same time, and seven steel balls are arranged between the two cameras at intervals.

3. The method for detecting the surface defects of the steel balls based on the single-stage as claimed in claim 1, wherein the image augmentation method is selected from the methods of horizontal inversion, vertical inversion and the like without changing the data properties.

4. The method for detecting the steel ball surface defects based on the single stage as claimed in claim 1, wherein the image data labels are of the type of dots, clusters of dots, scratches, pits, etc., which are common steel ball surface defects.

5. The method as claimed in claim 1, wherein the labeling of the image of the data set is performed by a LabelImg tool, and an xml-format file corresponding to each image in the data set is obtained after labeling, wherein the xml file includes the labeling information of the corresponding image.

6. The method for detecting the surface defects of the steel balls based on the single stage as claimed in claim 1, wherein the YOLOv4 model takes CSPDarkne53 as a backbone network, and the CSPDarknet53 network is formed by adding a CSP module into a Darknet53 network, so that the feature extraction capability of the network can be effectively improved; the method comprises the steps of taking SPP (spatial pyramid pooling) and PAN (personal area network) as feature fusion layers of the network, improving the feature fusion capability of the method to all levels, mainly aiming at increasing the receptive field of the YOLOv4 network, completing maximum pooling operation on learned features in the network, finally splicing all obtained feature graphs together according to dimensions to form a new feature graph, mainly aiming at the PAN, after completing up-sampling work on the features in the network, further performing down-sampling work on the features, further embodying the sampled features, finally splicing the sampled features together with other features for target detection, and taking YOLOv3 as a prediction part, wherein loss functions during training and nms screened by a prediction frame are mainly improved to be DIOU _ nms.