CN105562361A - Independent sorting method of fabric sorting robot - Google Patents

Abstract

The invention discloses an independent sorting method of a fabric sorting robot. Digital images, acquired by the robot, of fabric on an assembly line are adopted as research objects, a weighting form is applied to a color component H and a color component S through image color spatial switching, and an Otsu self-adaption threshold value method is applied to achieve image segmentation; the contours of the fabric are extracted by applying wavelet edge detection and morphological contour compensation; and the actual positions of the fabric are obtained according to the calibration result of a camera. Meanwhile, the weighting form of the color component H and the color component S is adopted as a feature, and a minimum variance criterion is used for classifying K-mean clustering methods. Different kinds of fabric are arranged at corresponding placing positions, the robot reversely solves kinematical parameters of all joints according to the placing positions and a kinematical equation, and finally grabbing actions are implemented. According to the independent sorting method of the fabric sorting robot, manual intervention is not needed, and the fabric can be sorted automatically, efficiently, accurately, stably and enduringly.

Description

The autonomous method for sorting of a kind of fabric sorting machine people

Technical field

The invention belongs to machine vision, robot autonomous fabric classification Sorting Technique field, be specifically related to the autonomous method for sorting of a kind of fabric sorting machine people.

Background technology

Along with the national economic development marches toward new normality, the development model that traditional dependence resource consumption and labour continue to drop into is hard to carry on, and the inner system dynamics of textile clothing industry must constantly strengthen.Slowly pulling open and the disappearance of demographic dividend, the ripe gradually of Robotics and popularizing in every profession and trade application of " industry 4.0 " curtain, robot replaces people to reduce recruitment, will finally become the realistic choice of textile clothing enterprise.Promoted the informationization technology level of industry by " machine substitution ", application messageization initiatively cracks a poor efficiency high consumption difficult problem, and promoting enterprise is changed to technology-intensive type by labor-intensive, is the only selection of textile clothing enterprise sustainable development.Domestic enterprise achieves some achievements to single stations of industry such as welding, spraying, carrying or the research and development of process integration universal type industrial robot, but can not meet the particular/special requirement of textile clothing enterprise to robot.Labor-intensive enterprises are difficult to the high robot of import price, according to the actual demand of textile clothing enterprise " machine substitution ", research and development manufacturing price is relatively low, reliable in quality, practical even process integration robot of single station robot, is the effective ways alleviating enterprise's " machine substitution " cost burden.2014 start, and the ground such as the layout machine device people of single station, the textile machinery people of process integration type and printing and dyeing robot start in Qingdao, stone lion, Shaoxing are researched and developed, and not yet come into operation.Research and development China there is the high-performance robot control system of independent intellectual property right and application system extremely urgent.

Because operative exists the problem of fatigue strength, use the industrial robot based on machine vision to carry out fabric sorting, not only efficiently and accurately but also stable lasting, there is very large advantage.At present, the existing vision system of company of external robot, the M-liA high speed radio frequency etc. as the TrueView system of ABB, the drill bit sorting system of Sehuster-prazision and FANUC be all based on rigid objects sorting application and design; And the domestic business sorting system that there is no maturation.Existing sorting system is all rigid objects exploitation, and general Shape-based interpolation is classified, and the simple mode of geometric templates coupling that adopts identifies, can not adapt to actual conditions well.And for the sorting of the flexible articles such as fabric, at present, home and abroad there is no ripe method and system and releases.

Summary of the invention

The object of this invention is to provide the autonomous method for sorting of a kind of fabric sorting machine people, solve the problem that independently can not sort flexible fabric existed in prior art.

The technical solution adopted in the present invention is, the autonomous method for sorting of a kind of fabric sorting machine people, specifically according to following steps:

Step 1, the binocular camera be arranged on above robot to be demarcated;

Step 2, gather textile image signal by being arranged on binocular camera above robot, then real-time online sends graphics processing unit to;

The image transmitted in step 3, graphics processing unit Real-time Obtaining step 2, and RGB color image is converted to HSV image, application Otsu automatic threshold method, splits each fabric;

The boundary profile of the textile image obtained in step 4, application Wavelet Edge Detection extraction step 3, and applied morphology method carries out outline compensation, obtains the segmentation image based on color segmentation and Wavelet Edge Detection;

Step 5, according to the demarcation of step 1 and step 4 and image edge information, determine the actual crawl position of fabric;

Step 6, structure fabric classification device;

Step 7, robot, according to the classification results of fabric classification device, capture different colours fabric, are placed on diverse location.

Feature of the present invention is also:

Demarcation in step 1 adopts classical Zhang Zhengyou scaling method.

When in step 3, RGB color image is converted to HSV image, consider the needs distinguishing the close fabric of color, for the image of the H obtained, channel S, extract the histogram of H, S color component, adopt two component weighted type, wherein, the weight of H component is 0.2 ~ 0.4, the weight of S component is 0.6 ~ 0.8, two weight sums is 1.

Step 5 determines the actual crawl position of fabric, be specially: according to the position relationship between binocular calibration result and two video cameras, the depth information of image is obtained by Stereo matching, in conjunction with the marginal information that small echo and morphological method detect, determine the three-dimensional information of edge in real world, consider actual crawl situation, inwardly advance 8 ~ 15 pixels to be concrete crawl position with rightmost edges.

Fabric classification device in step 6 adopts unsupervised K-means clustering method, with minimum variance function minimization extreme value for criterion, asks the maximal possibility estimation of textile image color characteristic average, fabric is divided into i class.

According to the classification results of fabric classification device in step 7, capture different colours fabric, be placed on diverse location, the concrete crawl position of the fabric of each color, is the movement position of robot terminal, this position is brought into Robot kinematics equations, by anti-solution, ask for the kinematics parameters in each joint of four-degree-of-freedom robot, this Parameter transfer is to motion control card, robot completes relevant action, and kinematics parameters computational process is as follows:

Robot forward kinematics equation can represent with formula (1):

$T_4^0=T_1^0\left({\mathrm{\θ}}_1\right)T_2^1\left({\mathrm{\θ}}_2\right)T_3^2\left(d_3\right)T_4^3\left({\mathrm{\θ}}_4\right);---\left(1\right)$

In formula, T is the function about joint variable, θ _i(i=1,2,4) are from X _i-1to X _iaround Z _i-1the angle that axle rotates, d ₃for joint distance, corresponding to X ₂to X ₃along Z ₃the distance that axle is measured;

The position of known machine people paw terminal and attitude, ask for each joint angle that robot is corresponding, and with driving joint motor, thus make the attitude of paw meet crawl requirement, inverse kinematics has multi-solution, and the concrete solution of Inverse Kinematics of the present invention is as follows:

A. joint variable θ is asked ₁:

In order to variables separation, to the both sides premultiplication simultaneously of formula (1) obtain formula (2):

${T_1^0}^{-1}\left({\mathrm{\θ}}_1\right)T_4^0=T_2^1\left({\mathrm{\θ}}_2\right)T_3^2\left(d_3\right)T_4^3\left({\mathrm{\θ}}_4\right);---\left(2\right)$

Substitute into the concrete data of four-degree-of-freedom robot, obtain formula (3):

$\left[\begin{array}{cccc}{\mathrm{cos\θ}}_1& {\mathrm{sin\θ}}_1& 0& -l_1\\ -{\mathrm{sin\θ}}_1& {\mathrm{cos\θ}}_1& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}{n}_x& o_x& a_x& p_x\\ n_y& o_y& a_y& p_y\\ n_z& o_z& a_z& p_z\\ 0& 0& 0& 1\end{array}\right]=\left[\begin{array}{cccc}{\mathrm{cos\θ}}_2& -{\mathrm{sin\θ}}_1& 0& l_2{\mathrm{cos\θ}}_2\\ {\mathrm{sin\θ}}_2& {\mathrm{cos\θ}}_1& 0& l_2{\mathrm{sin\θ}}_2\\ 0& 0& 1& -d_3\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}{\mathrm{cos\θ}}_4& -{\mathrm{sin\θ}}_4& 0& 0\\ {\mathrm{sin\θ}}_4& {\mathrm{cos\θ}}_4& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right];---\left(3\right)$

In formula, l ₁for robot links 1 length, l ₂for robot links 2 length, n, o, a are respectively normal, sensing and close vector, and p is Two coordinate initial point length.

Make that the first row the 4th element in formula (3) in the matrix of left and right is equal and the second row the 4th element is equal, as shown in formula (4):

$\left\{\begin{array}{c}{\mathrm{cos\θ}}_1\·p_x+{\mathrm{sin\θ}}_1{p}_y-l_1={\mathrm{cos\θ}}_2\·l_2\\ -{\mathrm{sin\θ}}_1\·p_x+{\mathrm{cos\θ}}_1{p}_y={\mathrm{sin\θ}}_2\·l_2\end{array}\right.;---\left(4\right)$

θ is obtained by formula (4) ₁, as shown in formula (5):

In formula $A=\frac{{l_1}^2-{l_2}^2+{p_x}^2+{p_y}^2}{2l_1\·\sqrt{{p_x}^2+{p_y}^2}},$

B. joint variable θ is asked ₂:

Formula is obtained by formula (4):

In formula $r=\sqrt{{p_x}^2+{p_y}^2},$

C. joint variable d is asked ₃:

Make the third line the 4th element in the left and right matrix in formula (3) equal, obtain formula (7):

D ₃=-(p _z+ d ₄); (7) in formula, d ₄for joint distance, corresponding to X ₃to X ₄along Z ₄the distance that axle is measured;

D. joint variable θ is asked ₄:

Make the second row first element in formula (3) in the matrix of left and right equal, obtain formula (8):

-sinθ ₁·n _x+cosθ ₁·n _y＝sinθ ₂·cosθ ₄+cosθ ₁·sinθ ₄；(8)

θ is tried to achieve by formula (8) ₄, ask for formula as shown in (9):

θ ₄＝arcsin(-sinθ ₁·n _x+cosθ ₁·n _y)-θ ₂；(9)

The invention has the beneficial effects as follows:

1. the autonomous method for sorting of a kind of fabric sorting machine people of the present invention, the color of robot energy ONLINE RECOGNITION fabric, the marginal information of Fusion of Color and image carries out Iamge Segmentation, extracts target fabric;

2. the autonomous method for sorting of a kind of fabric sorting machine people of the present invention, the design of grader makes robot the fabric of industry spot can be carried out automatic classification;

3. the autonomous method for sorting of a kind of fabric sorting machine people of the present invention, can position by the fabric on pipeline, and according to classification results, captures and be placed on diverse location.

Accompanying drawing explanation

Fig. 1 is the autonomous method for sorting flow chart of a kind of fabric sorting machine people of the present invention;

Fig. 2 is a kind of fabric sorting machine of the present invention people fabric classification device flow chart;

Fig. 3 is the autonomous sort process figure of a kind of fabric sorting machine people of the present invention.

Detailed description of the invention

Below in conjunction with the drawings and specific embodiments, the present invention is described in detail.

The present invention with control system form for the four-degree-of-freedom machine of PC and motion control card is artificially routine.

The autonomous method for sorting of a kind of fabric sorting machine people of the present invention, flow process as shown in Figure 1, specifically according to following steps:

Step 1, first the binocular camera be arranged on above robot to be demarcated; Demarcate and adopt classical Zhang Zhengyou scaling method, for the image that binocular vision gathers, every width image gets 6 points respectively, can calculate camera interior and exterior parameter;

Step 2, gather textile image signal by being arranged on binocular camera above robot, then real-time online sends the graphics processing unit on PC to;

Step 3, be positioned at the graphics processing unit of PC, the digital picture transmitted in Real-time Obtaining step 2, and RGB color image is converted to HSV image, consider the needs distinguishing the close fabric of color, for the image of the H obtained, channel S, extract the histogram of H, S color component, adopt two component weighted type, wherein, the weight of H component is 0.2 ~ 0.4, the weight of S component is 0.6 ~ 0.8, and application Otsu adaptive threshold method distinguishes background and target;

The boundary profile of the textile image obtained in step 4, application Wavelet Edge Detection extraction step 3, and applied morphology method carries out outline compensation;

Step 5, according to the demarcation of step 1 and step 4 and image edge information, determine the actual crawl position of fabric, according to the position relationship between binocular calibration result and two video cameras, the depth information of image is obtained by Stereo matching, in conjunction with the marginal information that small echo and morphological method detect, determine the three-dimensional information of edge in real world, consider actual crawl situation, inwardly advance 8 ~ 15 pixels to be concrete crawl position with rightmost edges;

Step 6, structure fabric classification device: grader adopts unsupervised K-means clustering method, and with minimum variance function minimization extreme value for criterion, ask the maximal possibility estimation of textile image color characteristic average, fabric is divided into i class, its flow chart as shown in Figure 2;

Step 7, robot are according to the classification results of fabric classification device, capture different colours fabric, be placed on diverse location, this sort process as shown in Figure 3, the concrete crawl position of the fabric of each color, be the movement position of robot terminal, this position is brought into Robot kinematics equations, by anti-solution, ask for the kinematics parameters in each joint of four-degree-of-freedom robot, this Parameter transfer is to motion control card, and robot completes relevant action, and kinematics parameters computational process is as follows:

Robot forward kinematics equation can represent with formula (1):

$T_4^0=T_1^0\left({\mathrm{\θ}}_1\right)T_2^1\left({\mathrm{\θ}}_2\right)T_3^2\left(d_3\right)T_4^3\left({\mathrm{\θ}}_4\right);---\left(1\right)$

In formula, T is the function about joint variable, θ _i(i=1,2,4) are from X _i-1to X _iaround Z _i-1the angle that axle rotates, d ₃for joint distance, corresponding to X ₂to X ₃along Z ₃the distance that axle is measured;

The position of known machine people paw terminal and attitude, ask for each joint angle that robot is corresponding, and with driving joint motor, thus make the attitude of paw meet crawl requirement, inverse kinematics has multi-solution, and the concrete solution of Inverse Kinematics of the present invention is as follows:

A. joint variable θ is asked ₁:

In order to variables separation, to the both sides premultiplication simultaneously of formula (1) obtain formula (2):

${T_1^0}^{-1}\left({\mathrm{\θ}}_1\right)T_4^0=T_2^1\left({\mathrm{\θ}}_2\right)T_3^2\left(d_3\right)T_4^3\left({\mathrm{\θ}}_4\right);---\left(2\right)$

Substitute into the concrete data of four-degree-of-freedom robot, obtain formula (3):

$\left[\begin{array}{cccc}{\mathrm{cos\θ}}_1& {\mathrm{sin\θ}}_1& 0& -l_1\\ -{\mathrm{sin\θ}}_1& {\mathrm{cos\θ}}_1& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}{n}_x& o_x& a_x& p_x\\ n_y& o_y& a_y& p_y\\ n_z& o_z& a_z& p_z\\ 0& 0& 0& 1\end{array}\right]=\left[\begin{array}{cccc}{\mathrm{cos\θ}}_2& -{\mathrm{sin\θ}}_1& 0& l_2{\mathrm{cos\θ}}_2\\ {\mathrm{sin\θ}}_2& {\mathrm{cos\θ}}_1& 0& l_2{\mathrm{sin\θ}}_2\\ 0& 0& 1& -d_3\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}{\mathrm{cos\θ}}_4& -{\mathrm{sin\θ}}_4& 0& 0\\ {\mathrm{sin\θ}}_4& {\mathrm{cos\θ}}_4& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right];---\left(3\right)$

In formula, l ₁for robot links 1 length, l ₂for robot links 2 length, n, o, a are respectively normal, sensing and close vector, and p is Two coordinate initial point length;

Make the second row the 4th element in the equal and left and right matrix of the first row the 4th element in formula (3) in the matrix of left and right equal, as shown in formula (4):

$\left\{\begin{array}{c}{\mathrm{cos\θ}}_1\·p_x+{\mathrm{sin\θ}}_1{p}_y-l_1={\mathrm{cos\θ}}_2\·l_2\\ -{\mathrm{sin\θ}}_1\·p_x+{\mathrm{cos\θ}}_1{p}_y={\mathrm{sin\θ}}_2\·l_2\end{array}\right.;---\left(4\right)$

θ is obtained by formula (4) ₁, as shown in formula (5):

In formula $A=\frac{{l_1}^2-{l_2}^2+{p_x}^2+{p_y}^2}{2l_1\·\sqrt{{p_x}^2+{p_y}^2}},$

B. joint variable θ is asked ₂:

Obtained by formula (4):

In formula $r=\sqrt{{p_x}^2+{p_y}^2},$

C. joint variable d is asked ₃:

Make the third line the 4th element in the left and right matrix in formula (3) equal:

D ₃=-(p _z+ d ₄); (7) in formula, d ₄for joint distance, corresponding to X ₃to X ₄along Z ₄the distance that axle is measured;

D. joint variable θ is asked ₄:

Make the second row first element in formula (3) in the matrix of left and right equal:

-sinθ ₁·n _x+cosθ ₁·n _y＝sinθ ₂·cosθ ₄+cosθ ₁·sinθ ₄；(8)

θ is tried to achieve by formula (8) ₄, ask for formula as shown in (9):

θ ₄＝arcsin(-sinθ ₁·n _x+cosθ ₁·n _y)-θ ₂；(9)

Above step is separated parameter obtain counter for robot kinematics.

Claims (6)

1. the autonomous method for sorting of fabric sorting machine people, is characterized in that, specifically according to following steps:

Step 1, the binocular camera be arranged on above robot to be demarcated;

Step 2, gather textile image signal by being arranged on binocular camera above robot, then real-time online sends graphics processing unit to;

The image transmitted in step 3, graphics processing unit Real-time Obtaining step 2, and RGB color image is converted to HSV image, application Otsu automatic threshold method, splits each fabric;

The boundary profile of the textile image obtained in step 4, application Wavelet Edge Detection extraction step 3, and applied morphology method carries out outline compensation, obtains the segmentation image based on color segmentation and Wavelet Edge Detection;

Step 5, according to the demarcation of step 1 and step 4 and image edge information, determine the actual crawl position of fabric;

Step 6, structure fabric classification device;

Step 7, robot, according to the classification results of fabric classification device, capture different colours fabric, are placed on diverse location.

2. the autonomous method for sorting of a kind of fabric sorting machine people according to claim 1, is characterized in that, the demarcation in described step 1 adopts classical Zhang Zhengyou scaling method.

3. the autonomous method for sorting of a kind of fabric sorting machine people according to claim 1, it is characterized in that, when in described step 3, RGB color image is converted to HSV image, for the image of the H obtained, channel S, extract the histogram of H, S color component, adopt two component weighted type, wherein, the weight of H component is the weight of 0.2 ~ 0.4, S component be 0.6 ~ 0.8, two weight sums is 1.

4. the autonomous method for sorting of a kind of fabric sorting machine people according to claim 1, it is characterized in that, described step 5 determines the actual crawl position of fabric, be specially: according to the position relationship between binocular calibration result and two video cameras, the depth information of image is obtained by Stereo matching, in conjunction with the marginal information that small echo and morphological method detect, determine the three-dimensional information of edge in real world, inwardly advance 8 ~ 15 pixels to be concrete crawl position with rightmost edges.

5. the autonomous method for sorting of a kind of fabric sorting machine people according to claim 1, it is characterized in that, fabric classification device in described step 6 is for adopting unsupervised K-means clustering method, with minimum variance function minimization extreme value for criterion, ask the maximal possibility estimation of textile image color characteristic average, fabric is divided into i class.

6. the autonomous method for sorting of a kind of fabric sorting machine people according to claim 1, it is characterized in that, in described step 7, robot is according to the classification results of fabric classification device, capture different colours fabric, be placed on diverse location, the concrete crawl position of the fabric of each color, be the movement position of robot terminal, this position is brought into Robot kinematics equations, by anti-solution, ask for the kinematics parameters in each joint of four-degree-of-freedom robot, this Parameter transfer is to motion control card, robot completes relevant action, is specially:

" kinematics parameters computational process is as follows:

Robot forward kinematics equation can represent with formula (1):

$T_4^0=T_1^0\left({\mathrm{\θ}}_1\right)T_2^1\left({\mathrm{\θ}}_2\right)T_3^2\left(d_3\right)T_4^3\left({\mathrm{\θ}}_4\right);---\left(1\right)$

In formula, T is the function about joint variable, θ _i(i=1,2,4) are from X _i-1to X _iaround Z _i-1the angle that axle rotates, d ₃for joint distance, corresponding to X ₂to X ₃along Z ₃the distance that axle is measured;

The position of known machine people paw terminal and attitude, ask for each joint angle that robot is corresponding, with driving joint motor, thus makes the attitude of paw meet crawl requirement, and the concrete solution of Inverse Kinematics of the present invention is as follows:

A. joint variable θ is asked ₁:

In order to variables separation, to the both sides premultiplication simultaneously of formula (1) obtain formula (2):

${T_1^0}^{-1}{\left({\mathrm{\θ}}_1\right)}_4^{0}T=\frac{1}{2}T{\left({\mathrm{\θ}}_2\right)}_3^{2}T{\left(d_3\right)}_4^{3}T\left({\mathrm{\θ}}_4\right);---\left(2\right)$

Substitute into the concrete data of four-degree-of-freedom robot, obtain formula (3):

$\left[\begin{array}{cccc}{\mathrm{cos\θ}}_1& {\mathrm{sin\θ}}_1& 0& -l_1\\ -{\mathrm{sin\θ}}_1& {\mathrm{cos\θ}}_1& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}{n}_x& o_x& a_x& p_x\\ n_y& o_y& a_y& p_y\\ n_z& o_z& a_z& p_z\\ 0& 0& 0& 1\end{array}\right]=\left[\begin{array}{cccc}{\mathrm{cos\θ}}_2& -{\mathrm{sin\θ}}_1& 0& l_2{\mathrm{cos\θ}}_2\\ {\mathrm{sin\θ}}_2& {\mathrm{cos\θ}}_1& 0& l_2{\mathrm{sin\θ}}_2\\ 0& 0& 1& -d_3\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}{\mathrm{cos\θ}}_4& -{\mathrm{sin\θ}}_4& 0& 0\\ {\mathrm{sin\θ}}_4& {\mathrm{cos\θ}}_4& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right];---\left(3\right)$

In formula, l ₁for robot links 1 length, l ₂for robot links 2 length, n, o, a are respectively normal, sensing and close vector, and p is Two coordinate initial point length;

Make the second row the 4th element in the equal and left and right matrix of the first row the 4th element in formula (3) in the matrix of left and right equal, as shown in formula (4):

$\left\{\begin{array}{c}{\mathrm{cos\θ}}_1\·p_x+{\mathrm{sin\θ}}_1{p}_y-l_1={\mathrm{cos\θ}}_2\·l_2\\ -{\mathrm{sin\θ}}_1\·p_x+{\mathrm{cos\θ}}_1{p}_y={\mathrm{sin\θ}}_2\·l_2\end{array}\right.;---\left(4\right)$

θ is obtained by formula (4) ₁, as shown in formula (5):

In formula $A=\frac{{l_1}^2-{l_2}^2+{p_x}^2+{p_y}^2}{2l_1\·\sqrt{{p_x}^2+{p_y}^2}},$

B. joint variable θ is asked ₂:

Obtained by formula (4):

In formula $r=\sqrt{{p_x}^2+{p_y}^2},$

C. joint variable d is asked ₃:

Make the third line the 4th element in the left and right matrix in formula (3) equal:

d ₃＝-(p _z+d ₄)；(7)

In formula, d ₄for joint distance, corresponding to X ₃to X ₄along Z ₄the distance that axle is measured;

D. joint variable θ is asked ₄:

Make the second row first element in formula (3) in the matrix of left and right equal:

-sinθ ₁·n _x+cosθ ₁·n _y＝sinθ ₂·cosθ ₄+cosθ ₁·sinθ ₄；(8)

θ is tried to achieve by formula (8) ₄, ask for formula as shown in (9):

θ ₄＝arcsin(-sinθ ₁·n _x+cosθ ₁·n _y)-θ ₂；(9)”。