CN111145258A - Automatic feeding and discharging method for various automobile glasses of industrial robot - Google Patents

Abstract

The invention discloses an automatic loading and unloading method for multiple kinds of automobile glass of an industrial robot, which is characterized in that the intelligent classification of automobile glass categories is realized through an MLP classifier, template primary positioning is carried out by combining image moment characteristics and PCA principal components, then least square method optimization is utilized, higher-precision pose is obtained, the algorithm complexity is O (C), and the time complexity of the traditional template positioning algorithm is up to O (n)⁴) Compared with the prior art, the method greatly improves the timeliness, realizes the efficient positioning of the feeding and discharging of the multi-class materials, reduces the manual intervention and reduces the production cost.

Description

Automatic feeding and discharging method for various automobile glasses of industrial robot

Technical Field

The invention relates to an automatic control method of a robot, in particular to a method for automatic loading and unloading of various automobile glass by the robot.

Background

With the continuous progress of the robot technology, more and more loading and unloading work is completed by the robot in the industrial production field. The camera collects images, coordinates of the workpiece under a robot coordinate system can be obtained through image preprocessing, recognition, positioning and calibration, the coordinates are sent to the robot through serial port communication, and the robot can complete grabbing and placing so as to achieve automation of feeding and discharging.

In the 3C industry, workpieces at the same station are single in type, and the pose of the workpieces can be obtained only by template matching. However, if the glass of a car is very many, even dozens of, and the car is identified and located by template matching in the same production line, the calculation amount becomes very large and time-consuming.

To solve the above problems, there are two solutions: the method is characterized in that during the process of loading automobile glass, categories are manually classified into a plurality of categories or dozens of categories, the similarity between the incoming material and each template in each search space is sequentially calculated through template matching, and the category and the pose of the incoming material are confirmed according to the highest similarity to inform a robot. The other method is to add a bar code or a two-dimensional code on the glass, acquire the incoming material information by scanning the code, identify the type of the incoming material, and then confirm the pose of the incoming material by template matching. This method of identification and location is relatively fast, however, requires the use of bar codes for cooperation.

Chinese patent application CN103464383A discloses a sorting system and method for industrial robots, wherein a camera collects the image sequence of the geometric workpieces to be sorted on a workpiece placing table into a PC, the image sequence of the workpieces to be sorted is analyzed by visual processing software in the PC according to each frame, and the regular geometric workpieces with regular shapes, such as round, rectangular, triangular and hexagonal, are automatically identified, and the relevant characteristics are calculated at the same time, and then sorting is started. The method can quickly identify regular workpieces with conventional shapes, such as circles, rectangles, triangles and the like, however, the shapes of many automobile glass cannot be described by the regular shapes, so that the method is not suitable for irregular automobile glass sorting.

Disclosure of Invention

The invention aims to overcome the defects in the prior art, and provides an automatic loading and unloading method for various automobile glasses of an industrial robot. And finally, solving the pose with higher precision by minimizing an error function.

The method realizes automatic feeding and discharging of various automobile glass by the industrial robot, and is realized by the following technical scheme:

step 1, training classifier

1.1, carrying out class marking on the Image of each material, wherein the format is < Image, L >, adding the Image into a training sample, and adding a Num group into each class to form a sample set.

1.2, Gaussian filtering is carried out on the sample set Image for noise smoothing. The kernel function of gaussian filtering is:

where x, y represents the offset from the center, σ controls the radial extent of the gaussian kernel, and larger values of σ represent larger local areas of influence.

1.3 because the background of the glass image as the sample is simple, the target area and the background area can be obtained by segmentation after binarization pretreatment, and the target area of the sample is marked as R.

1.4 because glass can take place translation and rotation in the material loading and unloading process, in order to discern the kind of target region, when carrying out feature extraction to target region R of sample set image, extract 7 regional characteristics that have the rototranslation invariance, including regional Area, regional outline girth Length, rectangular degree rectangular, the width W of minimum circumscribed rectangle, the height H of minimum circumscribed rectangle, the central second moment (u) of 2 images₁₀,u₀₁)；

Let the feature vector F ═ Area, Length, Rectangularity, W, H, u₁₀,u₀₁]The calculation mode of each element of the feature vector is as follows:

1)

wherein m is₀₀Is the 0 th moment of the image, I (x, y) is the intensity value of the image point (x, y), M and N represent the height and width of the image, respectively;

2)

wherein point is a set of contour points on the region R, and s is the number of contour points of the region;

3)

wherein MSR (R) represents the minimum circumscribed rectangular area of region R;

4) w is the width of the minimum circumscribed rectangle of the region R;

5) h is the height of the minimum circumscribed rectangle of the region R;

6) central second moment (u) of image₁₀,u₀₁) The calculation method is as follows:

wherein

Representing the centroid of the image, i.e.

m₁₀,m₀₁Representing first moments of the images x, y, respectively, i.e.

1.5 training MLP (Multi layer Perceptron) classifier according to feature and sample label.

MLP (multi-layer perception network), namely a multi-layer perception network, is provided that the number of network layers is n, wherein the first layer of the network is input characteristics, and the number of neural units is l₁The intermediate layer is an implicit layer network with n-2 layers in total and the number of network nodes is l_iAdjusting according to input layer, output layer and sample, the last layer is output layer, and output layer node_nThe number of categories to be classified; MLP adopts back propagation algorithm to learn, selects nonlinear microminiature Sigmoid as activation function of neuron, namely

Wherein z is the value of the activation function input;

the last layer from the hidden layer to the output layer is a multi-class regression problem, and the output of the output layer adopts a softmax function, namely

Wherein

Is the output value of the kth node of the n-1 layer, y_kIs the kth node of the output layer;

relevant parameters obtained by the classifier are saved, and only loading is needed for later identification; if a new variety is added to the sample set, retraining is required.

Step 2, identifying workpiece category

2.1, the camera module is used for collecting images, and the operation of the step 1.2 is carried out on the collected images for noise smoothing.

And 2.2, segmenting the image, and obtaining the target area by adopting the preprocessing method 1.3.

And 2.3, calculating the feature vector of the target area obtained in the step 2.2 by adopting the step 1.4, taking the feature vector as an input layer, and identifying by using the classifier C trained in the step 1.5, so that the class L to which the current material belongs can be output.

Step 3, positioning the workpiece

Loading a standard template for identifying the type L, positioning the current workpiece, namely acquiring the pose of the workpiece, and specifically comprising the following steps of:

3.1 moment features for initial positioning

Selecting a standard image as a positioning template from Num group data in a certain class L, and calculating the mass center of a target area of the standard template

And PCA (principal Component analysis) principal Component α₀. And calculating the centroid of the current workpiece target area

And PCA principal component α₁Calculating a translation parameter (t) according to the change of the mass center and the change of the PCA_x,t_y) And a rotation angle theta, i.e.

θ＝α₁-α₀；

3.2 higher precision positioning optimization

The pose accuracy obtained by the method meets most field requirements, however, some high-precision equipment production field application generally has higher accuracy requirements on the pose, the method is further optimized on the basis of the pose obtained by 3.1 calculation, a matching error function d (a) is designed, the minimum value of the d (a) is solved by a least square method, and the pose a (theta, t) with higher accuracy of the current workpiece relative to the standard template image is obtained_x,t_y) I.e. by

Wherein a represents a pose parameter including a rotation angle theta and a translation (t)_x,t_y) Information, denoted as a (θ, t)_x,t_y)，(x_i,y_i) And (x)_i',y_i') the coordinates of the primary matching point pair of the standard template image and the current workpiece according to the pose of step 3.1, d (a) the pose a (theta, t)_x,t_y) The match error in time.

The matching error is minimized by calculation, namely min { d (a) }, and the pose a with higher precision can be obtained₀(θ,t_x,t_y)。

The method of the invention realizes intelligent classification of automobile glass categories based on an MLP classifier, performs template initial positioning by combining image moment features and PCA, and obtains a pose with higher precision by using least square optimization, wherein the algorithm complexity is O (C), while the time complexity of the traditional template positioning algorithm is up to O (n)⁴) Compared with the prior art, the method greatly improves the timeliness, realizes the efficient positioning of the feeding and discharging of the multi-class materials, reduces the manual intervention and reduces the production cost.

Drawings

FIG. 1 is a classifier training flow diagram.

Fig. 2 is a diagram of an MLP training network architecture.

FIG. 3 is a flow chart of the method for automatic loading and unloading of various automobile glass by the industrial robot.

FIG. 4 is a schematic view of three different types of automotive glass according to an embodiment of the present invention.

Detailed Description

The process of the present invention will be described in further detail below with reference to examples and the accompanying drawings.

Taking 3 different glass loading and unloading materials on a certain line as an example, a specific implementation method of the scheme is illustrated in a figure 4.

Step 1, training classifier

1.1 there are 3 different glasses, and 3 categories L are assigned, and each category selects a training sample of which Num is 100 images Image.

1.2 gaussian-filtered Image1 is obtained by applying gaussian filtering to the Image, with a kernel size of 3 × 3 and σ of 0.670.

1.3 the Image1 obtained in the above step 1.2 is binarized to extract the target region R and the background.

1.4 for the target region R extracted in the above 1.3, 7 region features with rotational and translational invariance, namely region Area, region contour perimeter Length, Rectangularity, width W of minimum bounding rectangle, height H of minimum bounding rectangle, and central second moment (u) of 2 images are extracted₁₀,u₀₁)；

Let the feature vector F ═ Area, Length, Rectangularity, W, H, u₁₀,u₀₁]Then, the area feature vector F of a certain sample of the 3 classes of the sample set_i(i-1, 2,3) the results of the calculation are

F₁＝[240158.0,3217.88,648.725,258.812,0.330108,0.0422017,-1.27691e-005]

F₂＝[108871.0,2680.47,481.111,270.077,0.148523,0.187911,2.30686e-005]

F₃＝[244534.0,2566.42,415.659,334.128,0.415405,0.026618,-2.46742e-006]。

1.5 training MLP classifier according to the characteristics and sample labels, designing network structure, properly reducing the number of hidden layer layers and simplifying the model because the number of output samples is less. The first layer of the network is the input characteristic F and the number l of the neural units₁7, i.e., the dimension of F; the second layer is an implicit layer network, and the number l of the neural units is set according to an empirical value₂(ii) 5; the third layer is an output layer, and the number of the nerve units of the output layer is l₃The output category is 3 in total. And the MLP adopts a back propagation algorithm to learn, selects nonlinear differentiable Sigmoid as an activation function of the neuron and finally obtains the classifier C. 2. Identifying workpiece classes

2.1, acquiring an image to be processed to obtain an image I, and carrying out Gaussian filtering on the image by using the step 1.2, wherein the Gaussian filtering parameters are consistent with the step 1.2 to obtain a filtered image I₁。

2.2 pairs of images I₁Performing binarization by the method 1.3 to obtain I₁Target region R of image₁。

2.3 targeting region R₁And calculating to obtain 7-dimensional feature vectors through step 1.4

And F is [262720.0,4122.65,631.169,303.851,0.345984,0.0334388, -3.59867e-006], the feature vector F is used as an input layer of the classifier C, the classifier C obtained in the step 1.5 is used for identification, the classifier outputs the class L to which the current material belongs, and the class identification success rate reaches 99.9%.

3. Workpiece positioning

And loading a standard template for identifying the type L, and positioning the current workpiece to acquire the pose of the workpiece. The specific steps of workpiece positioning are as follows:

step 3.1 moment feature for initial positioning

Selecting standard images from the category as positioned templates, calculating the center positions of the standard templates and PCA pivot (676.0,391.0, -1.7358), and storing the result in each category in order to reduce repeated calculation; calculating the centroid and PCA principal component (1244.0,1501.0, -1.125) of the current workpiece target area, calculating the offset (t) according to the change of the centroid and the change of PCA_x,t_y) And the angle theta is rotated to obtain the pose (568.0,1110.0,0.610 deg.) of the initial positioning. Wherein

θ＝α1-α0。

Step 3.2 higher precision positioning optimization

According to the pose of the initial positioning obtained in the step 3.1, solving the minimum value of an error function by a least square method, namely

A more accurate pose can be obtained. Wherein a is tableThe posture parameters comprise a rotation angle theta and a translation (t)_x,t_y) Information, denoted as a (θ, t)_x,t_y)，(x_i,y_i) And (x)_i',y_i') respectively represents the coordinates of the initial matching point pair obtained by the edge points of the standard template image and the current workpiece according to the pose in the step 3.1, and d (a) represents the pose a (theta, t)_x,t_y) The match error in time. Solving the pose a with the minimum matching error d (a)₀(θ,t_x,t_y) To a sub-pixel level of accuracy (565.695,1112.732,0.601 °).

Claims (1)

1. A method for automatic loading and unloading of various automobile glass by an industrial robot comprises the following steps:

step 1, training classifier

Step 1.1, carrying out class marking on the Image of each material, wherein the format is < Image, L >, adding the Image into a training sample, and adding a Num group into each class to form a sample set;

step 1.2, Gaussian filtering is carried out on the Image of the sample set, noise smoothing is carried out, and the kernel function of the Gaussian filtering is as follows:

wherein x and y represent offset relative to the center, sigma controls the radial range of the Gaussian kernel function, and the larger the sigma value is, the larger the local influence range is;

step 1.3, because the background of the glass image as the sample is simple, the target area and the background area can be obtained by segmentation after binarization pretreatment, and the target area of the sample is marked as R;

step 1.4, extracting the characteristics of a target region R of a sample set image, wherein the target region R comprises 7 region characteristics with rotational translation invariance: area, perimeter of Area outline, Length, Rectangularity, width W of minimum bounding rectangle, height H of minimum bounding rectangle, central second moment of 2 images (u)₁₀,u₀₁)；

Let feature vector F ═ Area, Length, Rectangularity,W,H,u₁₀,u₀₁]The calculation mode of each element of the feature vector is as follows:

1)

wherein m is₀₀Is the 0 th moment of the image, I (x, y) is the intensity value of the image point (x, y), M and N represent the height and width of the image, respectively;

2)

wherein point is a set of contour points on the region R, and s is the number of contour points of the region;

3)

wherein MSR (R) represents the minimum circumscribed rectangular area of region R;

4) w is the width of the minimum circumscribed rectangle of the region R;

5) h is the height of the minimum circumscribed rectangle of the region R;

6) central second moment (u) of image₁₀,u₀₁) The calculation method is as follows:

wherein

Representing the centroid of the image, i.e.

m₁₀,m₀₁Representing first moments of the images x, y, respectively, i.e.

Step 1.5 training MLP (Multi layer Perceptron) classifier according to features and sample labels

MLP (Multi-layer perception network) is a multi-layer perception network, and the number of network layers is assumed to be n, wherein the first layer of the network is input characteristics, and the number of neural units in the layer is l₁The intermediate layer is an implicit layer network with n-2 layers in total and the number of network nodes is l_iAdjusting according to input layer, output layer and sample, the last layer is output layer, and output layer node_nThe number of categories to be classified; MLP adopts back propagation algorithm to learn, selects nonlinear microminiature Sigmoid as activation function of neuron, namely

Wherein z is the value of the activation function input;

the last layer is from a hidden layer to an output layer, and the output of the output layer adopts a softmax function, namely

Wherein

Is the output value of the kth node of the n-1 th layer, y_kIs the kth node of the output layer;

relevant parameters obtained by the classifier are saved, and only loading is needed for later identification; if a new variety is added into the sample set, retraining is needed;

step 2, identifying workpiece category

Step 2.1, acquiring an image by using a camera module, and performing the operation of the step 1.2 on the acquired image to smooth noise;

step 2.2, segmenting the image, and obtaining a target area by adopting the preprocessing method in the step 1.3;

step 2.3, calculating the feature vector of the target area by adopting the step 1.4, using the feature vector as an input layer, and identifying by using the classifier C trained in the step 1.5, so as to output the class L to which the current material belongs;

step 3, positioning the workpiece

Loading a standard template for identifying the type L, positioning the current workpiece, and acquiring the pose of the workpiece, wherein the method specifically comprises the following steps:

step 3.1 moment feature for initial positioning

Selecting a standard image as a positioning template from Num group data in a certain class L, and calculating the mass center of a target area of the standard template

And PCA (principal Component analysis) principal Component α₀Then calculating the mass center of the current workpiece target area

And PCA principal component α₁The translation parameter (t) may be calculated from the change in the centroid and PCA principal components_x,t_y) And a rotation angle theta, i.e.

θ＝α₁-α₀；

Step 3.2 higher precision positioning optimization

Designing a matching error function d (a), solving the minimum value of d (a) by a least square method, and acquiring a pose with higher precision:

wherein a represents a pose parameter including a rotation angle theta and a translation (t)_x,t_y) Information, denoted as a (θ, t)_x,t_y)，(x_i,y_i) And (x'_i,y′_i) Respectively representing the coordinates of the initial matching point pair of the standard template image and the pose of the current workpiece according to the step 3.1, d (a) representing the pose a (theta, t)_x,t_y) The match error in time.

The matching error is minimized by calculation, namely min { d (a) }, and the pose a with higher precision can be obtained₀(θ,t_x,t_y)。

Images