CN116958264A - Bolt hole positioning and pose estimation method based on three-dimensional vision - Google Patents

Abstract

A bolt hole positioning and pose estimation method based on three-dimensional vision belongs to the field of bolt hole positioning and pose estimation. The method solves the problems of high calculation complexity, low calculation speed and poor accuracy of positioning and pose estimation in the existing method. According to the method, firstly, point clouds of an iron tower truss scene containing bolt holes are obtained, statistical filtering and downsampling are carried out on the point clouds, then a sampling consistency segmentation algorithm is adopted to obtain truss plane equations and plane point clouds, then each bolt hole point cloud is roughly positioned based on normal calculation, radius search filtering and segmentation clustering, finally, the positions of each bolt hole are precisely positioned according to the denoised point clouds and the roughly positioned bolt hole point clouds, and the pose of each bolt hole is estimated. The method can be applied to bolt hole positioning and pose estimation.

Description

Bolt hole positioning and pose estimation method based on three-dimensional vision

Technical Field

The invention belongs to the field of bolt hole positioning and pose estimation, and particularly relates to a bolt hole positioning and pose estimation method.

Background

The bolt fastening of the iron tower assembly construction site is mainly performed manually at present, the skill proficiency and practical experience of operators greatly influence the bolt fastening quality, and the fastening quality is easily influenced by subjective factors. Meanwhile, the assembly unit hoisting generally has dynamics and is mostly high-altitude electrified operation, the bolt dense array tightening operation precision is high, the labor intensity is high, the manual cooperation link is long, and the requirement on the physical tolerance of workers is extremely high. In an electric power system, manual live working has great potential safety hazards, and particularly the problem is particularly remarkable when the electric power system is operated at an equipotential. Casualties frequently occur in manual operation, and great challenges are also presented to personnel safety. In order to avoid the influence of manual operation on the fastening quality, reduce the labor intensity of constructors, improve the digital construction level of engineering and accelerate the automation of bolt fastening, an iron tower assembling bolt fastening robot is developed. One important technology in bolt fastening robots is bolt hole positioning and pose estimation, which enables the robot to accurately sense and identify the bolt holes in a working scene, and realizes an automatic positioning and alignment process. The visual positioning technology can also compensate for motion errors of other sensors and robots, ensure correct alignment of bolts, and avoid assembly deviation caused by error accumulation. The introduction of robot vision gives a higher level of accuracy, reliability and automation to the bolt-up robot.

Currently, the mainstream bolt hole positioning and pose estimation techniques are divided into two types, namely deep learning-based and traditional algorithm-based. The bolt hole positioning and pose estimation can be performed by using a deep learning technology, so that the characteristic for learning can be automatically extracted, the method has better generalization capability, and the method is good in adaptability to complex scenes. However, deep learning requires a large amount of data, training and reasoning time is long, computing resources are consumed very much, and interpretation of results is poor. The bolt hole positioning and pose estimation algorithm based on deep learning on the actual iron tower assembly bolt fastening robot is difficult to land, so that the advantages based on the traditional algorithm are obvious. The bolt hole data has two kinds of two-dimensional images and three-dimensional point clouds, and three-dimensional point clouds can provide richer and accurate space information compared with two-dimensional images to the influence of outdoor illumination change is less. Three-dimensional sensors such as depth cameras and laser scanners are now mature in technology and cost has been reduced. Therefore, the three-dimensional vision derived from the three-dimensional point cloud has wider application prospect, for example, the invention patent application CN114792312a adopts a RANSAC space circle fitting algorithm added with threshold judgment to position the central pose of the bolt hole, but the three-dimensional vision positioning technology adopted by the invention still has the following problems:

1) The information of the plane where the bolt holes are positioned is not fully utilized: the information on the plane in which the bolt holes are located is not well utilized and is only used for determining Ping Miandian clouds, and the importance of the plane for bolt hole positioning and pose estimation is not fully considered. And the complexity of the spatial three-dimensional circle fitting algorithm for finally determining the position and the posture of the hole is high, the influence of noise points and shielding degree of the bolt hole is great, and the calculation accuracy is difficult to ensure.

2) Only accurate positioning is performed: the precise positioning is performed for the identified bolt holes without involving a coarse positioning of the bolt hole boundaries. This may result in limited accuracy of positioning in complex environments or in the presence of occlusions. And the algorithm only depends on one-step accurate positioning in the whole course, so that a considerable amount of point clouds are needed, and the calculation speed is difficult to ensure.

3) Directly carrying out bolt hole circular fitting after boundary extraction: and obtaining hole edge point clouds and plate edge point clouds by adopting a point cloud boundary extraction algorithm based on normal calculation, and then directly performing three-dimensional circle fitting on the bolt holes. Even with a multi-well plate, the number of plate edge point clouds is greater than the number of hole edge point clouds. The calculation speed is also slowed down by directly performing spatial three-dimensional circle fitting without further distinguishing and filtering the boundary point cloud.

In summary, the existing positioning method based on three-dimensional vision still has the problems of high computational complexity, low computational speed and poor accuracy of positioning and pose estimation, so it is necessary to propose a new technology to solve the above problems.

Disclosure of Invention

The invention aims to solve the problems of high calculation complexity, low calculation speed and poor accuracy of positioning and pose estimation of the existing method, and provides a bolt hole positioning and pose estimation method based on three-dimensional vision.

The technical scheme adopted by the invention for solving the technical problems is as follows:

a bolt hole positioning and pose estimating method based on three-dimensional vision specifically comprises the following steps:

scanning an iron tower truss scene containing bolt holes by adopting a three-dimensional sensor to acquire point cloud data;

step two, denoising the point cloud obtained in the step one to obtain denoised point cloud data;

step three, carrying out downsampling on the denoised point cloud data to obtain downsampled point cloud data;

determining a truss plane and a plane point cloud by adopting a plane model segmentation method with random sampling consistency based on the down-sampled point cloud data;

Step five, respectively calculating normal vectors of each point in the plane point cloud data, and preliminarily determining boundary points according to the calculated normal vectors;

step six, removing the impurity points and the abnormal values from the boundary points preliminarily determined in the step five to obtain boundary points;

step seven, clustering the boundary points obtained in the step six to obtain boundary point clouds of all the bolt holes;

step eight, respectively carrying out two-dimensional circle fitting on boundary point clouds of all the bolt holes based on truss planes, and roughly positioning the center of each bolt hole;

step nine, respectively starting from the circle centers of the bolt holes roughly positioned in the step eight in the denoised point cloud data obtained in the step two, and extracting all point clouds of the parts of the bolt holes;

performing preliminary boundary point determination, boundary point elimination and boundary point clustering on the extracted point cloud to obtain accurate point cloud data of each bolt hole;

and step ten, obtaining bolt hole positioning and pose estimation results according to the accurate point cloud data of each bolt hole.

Further, the denoising processing for the point cloud obtained in the first step adopts a statistical filtering method, and the specific denoising process comprises the following steps:

step two, the total point cloud number contained in the point cloud data acquired in the step one is recorded as N, and any point P in the point cloud data is recorded _i Will be located at point P _i Points within the K-neighborhood of (a) are denoted as P _j J=1, 2, …, K, P is calculated separately _i After the Euclidean distance of each point in the K neighborhood is calculated, the calculated Euclidean distance is averaged to obtain the average valued _iK Is point P _i Corresponding K neighborhood euclidean distance mean, (x) _i ,y _i ,z _i ) Is point P _i Coordinates of (x) _j ,y _j ,z _j ) Is point P _j Coordinates of (c);

similarly, the K neighborhood Euclidean distance mean value d corresponding to each point is calculated _iK ，i＝1,2,...,N；

Step two, calculating d _iK Mean μ and standard deviation σ of (a):

step two and three, setting a standard deviation coefficient s, and calculating a standard distance threshold range [ T ] of statistical filtering according to the standard deviation coefficient s, the mean mu and the standard deviation sigma _std,1 ，T _std,2 ]；

T _std,1 ＝μ-sσ ²

T _std,2 ＝μ+sσ ²

Step two, four, for point P _i Judgment of d _iK Whether or not it is within a standard distance threshold range [ T ] _std,1 ，T _std,2 ]An inner part; if d _iK Is not within the standard distance threshold range [ T ] _std,1 ，T _std,2 ]In, point P _i Is the noise point, point P _i Reject from the point cloud dataset, otherwise Point P _i Not noisy point, not requiring point P _i Processing;

and similarly, after judging each point, obtaining the point cloud data from which the noise is removed.

Further, the specific process of the third step is as follows:

step three, three-dimensional voxel grid division is carried out on the point cloud space of the denoised point cloud data, and the size of each cube grid obtained after division is V= (V) _x ,V _y ,V _z )，V _x ，V _y And V _z The dimensions of the cube mesh in each dimension, respectively;

step three, selecting a point cloud closest to the center of a cube grid from the cube grid for any cube grid;

and similarly, after each cube grid is processed respectively, all the selected point cloud data are the down-sampled point cloud data.

Further, the specific process of the fourth step is as follows:

step four, setting the maximum iteration number as T, and initializing the current iteration number as 1;

step four, randomly selecting a point set from the down-sampled point cloud data, and fitting a plane equation A according to the randomly selected point set _t x+B _t y+C _t z+D _t ＝0，A _t 、B _t 、C _t And D _t The coefficient of the plane equation fitted by the t iteration is that x, y and z are three-dimensional coordinates of points on a plane;

step four, setting a distance threshold d _max Respectively calculating each point in the down-sampled point cloud data to a plane A _t x+B _t y+C _t z+D _t Distance of =0:

wherein d _i′ The distance from the ith point to the plane in the down-sampled point cloud data, i '=1, 2, …, N' is the total number of point clouds contained in the down-sampled point cloud data;

if d _i′ ＜d _max The i 'th point is an intra-office point, otherwise, the i' th point is an extra-office point;

Setting a point threshold value m, and judging whether the number of the local points is larger than m according to the result of the step four;

if the number of the local points is greater than m, the fourth step is executed after the plane equation of the iterative fitting is saved;

otherwise, if the number of the local points is less than or equal to m, directly executing the fourth step;

judging whether the current iteration times T is smaller than T or not;

if T is smaller than T, let t=t+1, and return to execute step four two;

if T is equal to T, selecting a plane equation with the largest number of local points from all plane equations stored By T iterations, marking the selected plane equation as ax+by+cz+d=0, ax+by+cz+d=0 as a truss plane, and taking all local points corresponding to the plane equation ax+by+cz+d=0 as a plane point cloud.

Further, the specific process of the fifth step is as follows:

step five, for any point P in the plane point cloud data _i″ Distance P _i″ The set of the nearest K' points is denoted as M _i″ Utilizing set M _i″ In (a) point-calculation covariance matrix；

Wherein, for set M _i″ Taking the average value of the x-direction coordinates and the y-direction coordinates of all the points in the array, and taking the average value of the z-direction coordinates, and then taking P _mean Is a three-dimensional coordinate composed of an x-direction average value, a y-direction average value and a z-direction average value, P _j′ -P _mean Representing the set M _i″ Three-dimensional coordinates of j' th point in the series and three-dimensional coordinates P _mean Performing difference, wherein an upper corner mark T represents transposition;

for covariance matrixPerforming eigenvalue decomposition, and taking the eigenvector corresponding to the minimum eigenvalue as point P _i″ Normal vector of (2);

step five, normalizing the normal vector calculated in the step five to obtain a point P _i″ Is a normalized normal vector of (2);

step five, executing the processes of the step five and the step five for each point in the plane point cloud data to obtain normalized normal vector of each point;

step five and four, for any point P in the plane point cloud data _i″ Selecting a point P from planar point cloud data _i″ K' neighborhood points of (2), point P _i″ Is written as normalized normal vector of (2)The normalized normal vector of k' neighborhood points is marked as +.>Calculating the points P respectively _i″ After the included angle between the normalized normal vector of each neighborhood point and the normalized normal vector of each neighborhood point, averaging the calculated included angles;

point P _i″ Normalized normal vector N of (a) _pi″ ＝(x _pi″ ,y _pi″ ,z _pi″ ) The included angle between the normalized normal vector of the j' th neighborhood point is as follows:

point P _i″ The average value of the corresponding included angles is as follows:

step five, judging theta _i″ Whether or not it is greater than the angle threshold value theta ₀ If theta _i″ ＞θ ₀ Point P _i″ Is the boundary point, otherwise point P _i″ Is a non-boundary point;

and step five, and the same principle is adopted, each point in the plane point cloud data is processed respectively by adopting the step five, the step four and the step five, and all obtained boundary points are the preliminarily determined boundary points.

Further, the angle threshold value θ ₀ The value of (2) isOr->

Further, the specific process of the step six is as follows:

setting a search radius r, and regarding any point p in the boundary points preliminarily determined in the step five _i″′ Selecting a point p from the preliminarily determined boundary points _i″′ Points having a Euclidean distance less than the search radius r, and a set N (p) of selected points _i″′ ) The method comprises the following steps:

N(p _i″′ )＝{p _j″′ |dist(p _i″′ ,p _j″′ )＜r}

wherein point p _j″′ Is the boundary point preliminarily determined in the fifth step, dist (p _i″′ ,p _j″ ) Representative point p _i″ And point p _j″′ Is a Euclidean distance of (2);

wherein, (x) _i″″ ,y _i″ ,z _i″′ ) Is point p _i″′ Coordinates of (x) _j″ ,y _j″ ,z _j″′ ) Is point p _j″′ Coordinates of (c);

if set N (p) _i″′ ) The point p is the point contained in the data is greater than the threshold value n _i″′ As boundary point, otherwise point p _i″′ Removing;

and similarly, processing each point in the boundary points preliminarily determined in the step five to obtain the rest boundary points.

Further, the specific process of the step seven is as follows:

seventhly, marking all boundary points obtained in the step six as unclassified states, and creating a list containing all unclassified points;

Step seven, randomly selecting a point from unclassified points in the list as a seed point, removing the seed point from the unclassified list, respectively calculating Euclidean distances between the seed point and each unclassified point, and if the Euclidean distance between a certain unclassified point and the seed point is smaller than a given distance threshold value, adding the unclassified point into a cluster of the seed points, and removing the unclassified point from the unclassified list;

step seven, judging whether unclassified points exist in the list, if so, returning to the step seven, otherwise, ending the clustering, and respectively obtaining the cloud data of each clustered point;

setting the minimum clustering point number and the maximum clustering point number, if the point number contained in a certain cluster is between the minimum clustering point number and the maximum clustering point number, the cluster belongs to the boundary point cloud of the bolt hole, each cluster corresponds to the boundary point cloud of one bolt hole, and otherwise, the cluster does not belong to the boundary point cloud of the bolt hole.

In the eighth step, two-dimensional circle fitting is performed on the boundary point clouds of each bolt hole respectively to obtain two-dimensional circle fitting results of each bolt hole; the specific process is as follows:

Step eight, projecting boundary point clouds of any bolt hole onto a truss plane to obtain a projected point set;

step eight, randomly selecting a plurality of data points from the projected point set, and performing two-dimensional circle fitting by using the x-direction coordinates and the y-direction coordinates of the selected data points;

the fitted two-dimensional round equation is: (x-x) _center ) ² +(y-y _center ) ² ＝r′ ² Wherein (x) _center ,y _center ) Is the center coordinates of a two-dimensional circle, and r' is the radius of the two-dimensional circle;

step eight, respectively calculating the distance from each point in the boundary point cloud of the bolt hole to the circle center after projection, and for any point in the boundary point cloud of the bolt hole after projection, making a difference between the distance from the point to the circle center and the radius r', if the absolute value of the difference result is smaller than a threshold epsilon, considering the point to be on a circle, otherwise, considering the point not to be on a circle;

after each point in the projected boundary point cloud of the bolt hole is judged, counting the total point number on the circle, and if the total point number on the circle is larger than a set threshold value, the two-dimensional circle equation obtained by fitting meets the requirement; otherwise, the two-dimensional round equation obtained by the fitting is not in accordance with the requirement;

step eight four, repeating the process from step eight two to step eight three until the set maximum iteration number is reached;

Selecting a two-dimensional round equation with the largest total number of points on a circle from two-dimensional round equations meeting requirements, taking the selected two-dimensional round equation as a round equation corresponding to the bolt hole, wherein the center of the selected two-dimensional round equation is the center of the roughly positioned bolt hole, the center coordinates of the two-dimensional round equation are the coordinates of the center of the bolt hole in the x direction and the y direction, and substituting the coordinates of the center of the bolt hole in the x direction and the y direction into a truss plane ax+by+cz+D=0 to obtain the z-direction coordinates of the center of the bolt hole;

and step eight, five, and the same way, executing the processes from step eight to step eight four on the boundary point clouds of each bolt hole.

Further, the specific process of the step ten is as follows:

step eleven, for the accurate point cloud data of any one bolt hole, projecting the accurate point cloud data of the bolt hole to a truss plane ax+by+cz+d=0 to obtain projected point cloud data;

performing two-dimensional circle fitting on the accurate point cloud data projected by the bolt hole by adopting the method of the step eight to obtain the circle center and the radius of the two-dimensional circle, and obtaining the center of the bolt hole according to the circle center of the two-dimensional circle;

the axial attitude (Nx, ny, nz) of the bolt hole _{Bolt hole} Determined by the normal vector of the truss plane, i.e. (Nx, ny, nz) _{Bolt hole} = (a ', B ', C '), wherein,

and step twelve, executing the process of step eleven for the accurate point cloud data of any one bolt hole, and completing the positioning and pose estimation of each bolt hole.

The beneficial effects of the invention are as follows:

(1) The invention reduces the total number of clouds of the scene points of the bolt hole through downsampling so as to ensure the running speed of the algorithm. After roughly positioning the bolt hole center, returning to the scene before downsampling to acquire local all point clouds of each bolt hole to precisely position the identified bolt hole. Under a complex environment or in the presence of shielding, the coarse-before-fine positioning mode of the invention ensures the calculation speed and improves the accuracy of positioning and pose estimation.

(2) According to the invention, a point cloud boundary extraction algorithm based on normal calculation is adopted to obtain hole boundary point cloud and plate boundary point cloud, then a radius search filtering algorithm is adopted to remove miscellaneous points, and then a Euclidean clustering algorithm is adopted to obtain each bolt hole boundary point cloud, so that the calculation speed of subsequent bolt hole positioning is improved through further distinguishing and filtering treatment of the boundary point cloud.

(3) And taking the normal vector of the truss fitting plane where the bolt hole is positioned as the normal vector of the bolt hole, projecting the bolt hole boundary point cloud to the truss plane in the process of accurately positioning the bolt hole, and performing two-dimensional circle fitting on the projected points to obtain the circle center and the radius of the two-dimensional circle. Compared with three-dimensional circle fitting, the two-dimensional circle fitting is simpler and more efficient, the convergence is better, the calculation amount and calculation time are reduced, the calculation complexity is reduced, and the influence of noise and uncertainty from a non-truss plane and the surrounding environment is reduced. The number of point clouds used to determine the truss plane is high and less disturbed by noise points, which can ensure the accuracy and integrity of the slab plane.

Drawings

Fig. 1 is a schematic structural view of an iron tower assembly bolt screwing robot;

in the figure, an industrial robot for screwing bolts, a binocular grating structure optical camera, 3, a tail end clamp, 4 and bolts are adopted;

FIG. 2 is a flow chart of a three-dimensional vision based bolt hole positioning and pose estimation method of the invention;

FIG. 3 is a schematic diagram of a point-to-plane projection process;

FIG. 4 is a diagram of the result of coarse positioning of bolt holes;

FIG. 5 is a schematic illustration of a preliminary determination of boundary points;

fig. 6 is a graph of the result of the precise positioning of the bolt holes.

Detailed Description

Detailed description of the inventionin the first embodiment, this embodiment will be described with reference to fig. 2. The bolt hole positioning and pose estimating method based on three-dimensional vision specifically comprises the following steps:

step one, scanning an iron tower truss scene containing bolt holes by adopting a three-dimensional sensor (which can be a binocular grating structure light camera) to obtain point cloud data;

step two, denoising the point cloud obtained in the step one to obtain denoised point cloud data;

step three, carrying out downsampling on the denoised point cloud data to obtain downsampled point cloud data;

determining a truss plane and a plane point cloud by adopting a random sampling consistency planar model segmentation method (SACSeg) based on the down-sampled point cloud data;

Step five, respectively calculating normal vectors of each point in the plane point cloud data, and preliminarily determining boundary points according to the calculated normal vectors;

step six, removing the impurity points and the abnormal values from the boundary points preliminarily determined in the step five to obtain boundary points;

step seven, clustering the boundary points obtained in the step six to obtain boundary point clouds of all the bolt holes;

step eight, respectively carrying out two-dimensional circle fitting on boundary point clouds of all the bolt holes based on truss planes, and roughly positioning the center of each bolt hole;

step nine, respectively starting from the circle centers of the bolt holes roughly positioned in the step eight in the denoised point cloud data obtained in the step two, and extracting all point clouds of the parts of the bolt holes;

then, carrying out preliminary boundary point determination, boundary point elimination (namely, the method of the step six) and boundary point clustering on the extracted point cloud to obtain accurate point cloud data of each bolt hole;

and step ten, obtaining bolt hole positioning and pose estimation results according to the accurate point cloud data of each bolt hole.

According to the invention, a three-dimensional sensor is used for shooting a scene of the iron tower truss comprising the bolt holes, and then an input detection algorithm is carried out on point clouds. The algorithm firstly performs statistical filtering to remove noise points, and reduces the number of point clouds through voxel grid downsampling, so that the calculation speed is improved. Next, a sampling consistency segmentation algorithm is employed to obtain truss plane equations, plane normal vectors, and plane point clouds. The normal vector of the plane in which the bolt hole lies is considered to be the normal vector of the bolt hole. And then, obtaining the boundary point clouds of the rectangular outside the truss and the boundary point clouds inside the bolt holes through a scattered point cloud boundary extraction algorithm based on normal calculation. And removing the miscellaneous points through a radius search filtering algorithm, and then performing segmentation clustering again to obtain an inner boundary, namely a plurality of bolt hole point clouds. Finally, the rough positioning is further improved by calculating the circle center of the point cloud inside the bolt hole. Starting from the three-dimensional circle centers of all the bolt holes calculated on the truss plane, acquiring all the local point clouds around the bolt holes after denoising of the original input scene in a radius searching mode, then accurately positioning again, extracting inner and outer boundaries based on normal calculation, and analyzing the inner boundaries to acquire the geometric parameters of the target bolt holes. The coarse-before-fine positioning mode ensures the calculation speed, and increases the number of local point clouds for calculating the bolt holes, thereby improving the positioning accuracy.

And accurately positioning, namely projecting the acquired bolt hole boundary point cloud onto a truss plane, and performing two-dimensional circle fitting on the projected points to obtain the circle center and the radius of the two-dimensional circle. The purpose of this is to take advantage of the simplicity and efficiency of two-dimensional circle fitting to reduce computational effort and computational time. By projecting the point cloud onto the truss plane, the effects of noise and uncertainty from the non-truss plane and the surrounding environment can be reduced. The number of point clouds previously used to determine the truss plane is high and less disturbed by noise points, which can ensure the accuracy and integrity of the slab plane. Thus, the bolt hole boundary point cloud is projected onto the truss plane and the disturbance in the data is reduced by two-dimensional circle fitting.

The detection effect of the algorithm of the invention is shown in fig. 4, 5 and 6. The left side of fig. 5 is the space annular point cloud obtained in the step nine, the right side of fig. 5 is the inner boundary extraction point cloud calculated based on normal, the right side of fig. 6 is the result of accurate positioning of the bolt hole, and the white point at the center of the circle is the positioning center generated after two-dimensional circle fitting.

The work is performed by adopting the iron tower assembling bolt screwing robot shown in fig. 1. The general workflow of the bolt screwing robot is as follows: the binocular grating structure light camera 2 scans scene point clouds, and gives out position and pose information of target bolts after bolt hole positioning and pose estimation algorithm processing and guides the bolt screwing robot 1 to a working space. The tail end clamp 3 of the bolt screwing robot automatically loads and clamps the bolts 4, and the subsequent screwing action is completed aiming at the target bolt holes.

The second embodiment, which is different from the first embodiment, is that the denoising processing for the point cloud obtained in the first step adopts a statistical filtering method, and the specific denoising process is as follows:

step two, the total point cloud number contained in the point cloud data acquired in the step one is recorded as N, and any point P in the point cloud data is recorded _i Will be located at point P _i Points within the K-neighborhood of (a) are denoted as P _j J=1, 2, …, K, P is calculated separately _i After the Euclidean distance of each point in the K neighborhood is calculated, the calculated Euclidean distance is averaged to obtain the average valued _iK Is point P _i Corresponding K neighborhood euclidean distance mean, (x) _i ,y _i ,z _i ) Is point P _i Coordinates of (x) _j ,y _j ,z _j ) Is point P _j Coordinates of (c);

similarly, the K neighborhood Euclidean distance mean value d corresponding to each point is calculated _iK ，i＝1,2,...,N；

Step two, assume d _iK Obeying normal distribution, calculating d _iK Mean μ and standard deviation σ of (a):

step two and three, setting a standard deviation coefficient s, and calculating a standard distance threshold range [ T ] of statistical filtering according to the standard deviation coefficient s, the mean mu and the standard deviation sigma _std,1 ，T _std,2 ]；

T _std,1 ＝μ-sσ ²

T _std,2 ＝μ+sσ ²

Step two, four, for point P _i Judgment of d _iK Whether or not it is within a standard distance threshold range [ T ] _std,1 ，T _std,2 ]An inner part; if d _iK Is not within the standard distance threshold range [ T ] _std,1 ，T _std,2 ]In, point P _i Is the noise point, point P _i Reject from the point cloud dataset, otherwise Point P _i Not noisy point, not requiring point P _i Processing;

and similarly, after judging each point, obtaining the point cloud data from which the noise is removed.

Other steps and parameters are the same as in the first embodiment.

At point P _i Points within the K-neighborhood of (c) refer to: respectively calculating the points P _i After the Euclidean distance from any other point, the calculated Euclidean distances are ordered from small to large, and the point corresponding to the distance of the K position at the front row is taken as a point P _i Points within K-neighborhood of (c).

In the embodiment, the abnormal points are judged based on the distance information, and the noise points far away from the average value of the point cloud are removed.

The third embodiment is different from the first embodiment or the second embodiment in that the specific process of the third step is:

step three, three-dimensional voxel grid division is carried out on the point cloud space of the denoised point cloud data, each voxel grid is actually a cube with a settable size, and the size of each cube grid obtained after division is V= (V) _x ,V _y ,V _z )，V _x ，V _y And V _z The dimensions of the cube mesh in each dimension, respectively;

step three, selecting a point cloud closest to the center of a cube grid from the cube grid for any cube grid;

And similarly, after each cube grid is processed respectively, all the selected point cloud data are the down-sampled point cloud data.

Other steps and parameters are the same as in the first or second embodiment.

The method of the embodiment can keep the shape characteristics and the space information of the original point cloud while reducing the number of the point clouds in the scene, and reduces the time consumption in the subsequent point cloud normal vector calculation process by reducing the number of the point clouds in the scene.

The fourth embodiment is different from one of the first to third embodiments, and the specific process of the fourth step is as follows:

step four, setting the maximum iteration number as T, and initializing the current iteration number as 1;

step four, randomly selecting a point set from the down-sampled point cloud data, and fitting a plane equation A according to the randomly selected point set _t x+B _t y+C _t z+D _t ＝0，A _t 、B _t 、C _t And D _t The coefficient of the plane equation fitted by the t iteration is that x, y and z are three-dimensional coordinates of points on a plane;

step four, setting a distance threshold d _max Respectively calculating each point in the down-sampled point cloud data to a plane A _t x+B _t y+C _t z+D _t Distance of =0:

wherein d _i′ The distance from the ith point to the plane in the down-sampled point cloud data, i '=1, 2, …, N' is the total number of point clouds contained in the down-sampled point cloud data;

If d _i′ ＜d _max The i 'th point is an intra-office point, otherwise, the i' th point is an extra-office point;

setting a point threshold value m, and judging whether the number of the local points is larger than m according to the result of the step four;

if the number of the local points is greater than m, the fourth step is executed after the plane equation of the iterative fitting is saved;

otherwise, if the number of the local points is less than or equal to m, directly executing the fourth step;

judging whether the current iteration times T is smaller than T or not;

if T is smaller than T, let t=t+1, and return to execute step four two;

if T is equal to T, selecting a plane equation with the largest number of local points from all plane equations stored By T iterations, marking the selected plane equation as ax+by+cz+d=0, ax+by+cz+d=0 as a truss plane, and taking all local points corresponding to the plane equation ax+by+cz+d=0 as a plane point cloud.

It should be noted that: because the bolt holes are uniformly distributed on the truss plane, the truss plane needs to be extracted and calculated before final positioning and pose estimation of the bolt holes are realized.

Other steps and parameters are the same as in one to three embodiments.

The fifth embodiment is different from the first to fourth embodiments in that the specific process of the fifth step is:

Step five, for any point P in the plane point cloud data _i″ Distance P _i″ The set of the nearest K' points is denoted as M _i″ (calculate P separately) _i″ The Euclidean distance with any other point, the calculated Euclidean distance is ordered from small to large, and K' points arranged in front are utilized to form a set M _i″ ) Utilizing set M _i″ In (a) point-calculation covariance matrix

Wherein, for set M _i″ Taking the average value of the x-direction coordinates and the y-direction coordinates of all the points in the array, and taking the average value of the z-direction coordinates, and then taking P _mean Is composed of x-direction average value, y-direction average value and z-direction average valueThree-dimensional coordinates, P _j′ -P _mean Representing the set M _i″ Three-dimensional coordinates of j' th point in the series and three-dimensional coordinates P _mean Make difference (i.e. will P _j′ X-direction coordinate value of (2) and P _mean Is differenced by the coordinate value in the x direction of (2), P _j′ Coordinate value of y direction of (2) and P _mean Is differenced by the y-direction coordinate value of P _j′ Is the z-direction coordinate value of (2) and P _mean Difference in z-direction coordinate values), upper corner mark T represents transpose;

for covariance matrixPerforming eigenvalue decomposition, and taking the eigenvector corresponding to the minimum eigenvalue as point P _i″ Normal vector of (2);

it should be noted that: the result of the calculation vector is affected by the parameter K'; smaller values of K 'can result in finer normal vectors, while larger values of K' can result in smoother normal vectors. Selecting a proper K' value according to the density and the geometric structure of the point cloud under the background of bolt hole positioning and pose estimation;

Step five, normalizing the normal vector calculated in the step five to obtain a point P _i″ Is a normalized normal vector of (2);

step five, executing the processes of the step five and the step five for each point in the plane point cloud data to obtain normalized normal vector of each point;

step five and four, for any point P in the plane point cloud data _i″ Selecting a point P from planar point cloud data _i″ K' neighborhood points of (2), point P _i″ Is written as normalized normal vector of (2)The normalized normal vector of k' neighborhood points is marked as +.>Calculating the points P respectively _i″ Normalized normal vector of (c) and normalized for each neighborhood pointAfter the included angle of the normal vector, averaging the calculated included angle;

point P _i″ Normalized normal vector of (a)The included angle between the normalized normal vector of the j' th neighborhood point is as follows:

point P _i″ The average value of the corresponding included angles is as follows:

step five, judging theta _i″ Whether or not it is greater than the angle threshold value theta ₀ If theta _i″ ＞θ ₀ Point P _i″ Is the boundary point, otherwise point P _i″ Is a non-boundary point;

and fifthly, processing each point in the plane point cloud data by adopting the fifth fourth step and the fifth step, wherein all obtained boundary points are preliminarily determined boundary points, the boundary points are marked as 1, and the non-boundary points are marked as 0, so that the boundary point cloud is extracted from the original point cloud, and the preliminarily determined boundary points are truss plane boundary point cloud containing miscellaneous points and boundary point cloud of internal bolt holes.

It should be noted that: the nearest neighbor search algorithm based on Euclidean distance is used for normal vector calculation and boundary point evaluation, but the normal vector calculation and the boundary point evaluation are independent of each other, and the nearest neighbor numbers of the normal vector calculation and the boundary point evaluation are different. In normal vector calculation, the K' value is generally smaller than 10, and the calculation speed and the accuracy can be simultaneously considered; however, in the evaluation of boundary points, in order to prevent overcoupling with a neighboring point used for determining the normal vector of the point, and for the purpose of improving the accuracy of boundary recognition, the value of the neighboring point k 'can be taken to be 50 to 100, and must be greater than 20, otherwise, the extracted boundary point is excessively associated with k' nearest neighboring points of the calculation method vector, resulting in problems of boundary unevenness, missing and the like.

Other steps and parameters are the same as in one to four embodiments.

In a sixth embodiment, the difference between the present embodiment and one to fifth embodiments is that the angle threshold value θ ₀ The value of (2) isOr->

Other steps and parameters are the same as in one of the first to fifth embodiments.

The seventh embodiment is different from one of the first to sixth embodiments, and the specific process of the sixth step is:

setting a search radius r, and regarding any point p in the boundary points preliminarily determined in the step five _i″′ Selecting a point p from the preliminarily determined boundary points _i″′ Points having a Euclidean distance less than the search radius r, and a set N (p) of selected points _i″′ ) The method comprises the following steps:

N(p _i″′ )＝{p _j″′ |dist(p _i″′ ,p _j″′ )＜r}

wherein point p _j″′ Is the boundary point preliminarily determined in the fifth step, dist (p _i″′ ,p _j″′ ) Representative point p _i″′ And point p _j″′ Is a Euclidean distance of (2);

wherein, (x) _i″′ ,y _i″′ ,z _i″′ ) Is point p _i″′ Coordinates of (x) _j″′ ,y _j″′ ,z _j″′ ) Is point p _j″′ Coordinates of (c);

if set N (p) _i″′ ) The point p is the point contained in the data is greater than the threshold value n _i″′ As boundary point, otherwise point p _i″′ Removing;

and similarly, processing each point in the boundary points preliminarily determined in the step five to obtain the rest boundary points.

Other steps and parameters are the same as in one of the first to sixth embodiments.

According to the method, radius search filtering is conducted on the boundary point cloud which is initially extracted, outliers or noise which does not belong to the boundary are removed, accuracy of boundary extraction is improved, and noise interference is reduced.

The eighth embodiment is different from one of the first to seventh embodiments in that the specific process of the step seven is:

seventhly, marking all boundary points obtained in the step six as unclassified states, and creating a list containing all unclassified points;

step seven, randomly selecting a point from unclassified points in the list as a seed point, removing the seed point from the unclassified list, respectively calculating Euclidean distance between the seed point and each unclassified point, if the Euclidean distance between a certain unclassified point and the seed point is smaller than a given distance threshold value, adding the unclassified point into a cluster of the seed points (the unclassified point and the seed point are considered to belong to the same class), and removing the unclassified point from the unclassified list;

Step seven, judging whether unclassified points exist in the list, if so, returning to the step seven, otherwise, ending the clustering, and respectively obtaining the cloud data of each clustered point;

setting the minimum clustering point number and the maximum clustering point number, if the point number contained in a certain cluster is between the minimum clustering point number and the maximum clustering point number, the cluster belongs to the boundary point cloud of the bolt hole, each cluster corresponds to the boundary point cloud of one bolt hole, and otherwise, the cluster does not belong to the boundary point cloud of the bolt hole.

Other steps and parameters are the same as those of one of the first to seventh embodiments.

And clustering according to the seventh and fourth steps to obtain the boundary point cloud of each bolt hole and the boundary point cloud of the truss edge, and further distinguishing the boundary point cloud of the truss edge from the boundary point cloud of a plurality of bolt holes on the truss to obtain the boundary point cloud only containing a plurality of bolt holes.

In the eighth step, two-dimensional circle fitting is performed on the boundary point clouds of each bolt hole to obtain two-dimensional circle fitting results of each bolt hole; the specific process is as follows:

Step eight, projecting boundary point clouds of any bolt hole onto a truss plane to obtain a projected point set;

step eight, randomly selecting a plurality of data points from the projected point set, and performing two-dimensional circle fitting by using the x-direction coordinates and the y-direction coordinates of the selected data points;

the fitted two-dimensional round equation is: (x-x) _center ) ² +(y-y _center ) ² ＝r′ ² Wherein (x) _center ,y _center ) Is the center coordinates of a two-dimensional circle, and r' is the radius of the two-dimensional circle;

step eight, respectively calculating the distance from each point in the boundary point cloud of the bolt hole to the circle center after projection, and for any point in the boundary point cloud of the bolt hole after projection, making a difference between the distance from the point to the circle center and the radius r', if the absolute value of the difference result is smaller than a threshold epsilon, considering the point to be on a circle, otherwise, considering the point not to be on a circle;

after each point in the projected boundary point cloud of the bolt hole is judged, counting the total point number on the circle, and if the total point number on the circle is larger than a set threshold value, the two-dimensional circle equation obtained by fitting meets the requirement; otherwise, the two-dimensional round equation obtained by the fitting is not in accordance with the requirement;

step eight four, repeating the process from step eight two to step eight three until the set maximum iteration number is reached;

Selecting a two-dimensional round equation with the largest total number of points on a circle from two-dimensional round equations meeting requirements, taking the selected two-dimensional round equation as a round equation corresponding to the bolt hole, wherein the center of the selected two-dimensional round equation is the center of the roughly positioned bolt hole, the center coordinates of the two-dimensional round equation are the coordinates of the center of the bolt hole in the x direction and the y direction, and substituting the coordinates of the center of the bolt hole in the x direction and the y direction into a truss plane ax+by+cz+D=0 to obtain the z-direction coordinates of the center of the bolt hole;

and step eight, five, and the same way, executing the processes from step eight to step eight four on the boundary point clouds of each bolt hole.

Other steps and parameters are the same as in one to eight of the embodiments.

Detailed description of the inventionthe present embodiment will be described with reference to fig. 3. This embodiment is different from one of the first to ninth embodiments in that the specific process of the step ten is:

step eleven, for the accurate point cloud data of any one bolt hole, projecting the accurate point cloud data of the bolt hole to a truss plane ax+by+cz+d=0 to obtain projected point cloud data;

performing two-dimensional circle fitting on the accurate point cloud data projected by the bolt hole by adopting the method of the step eight to obtain the circle center and the radius of the two-dimensional circle, and obtaining the center of the bolt hole according to the circle center of the two-dimensional circle;

The axial attitude (Nx, ny, nz) of the bolt hole _{Bolt hole} Determined by the normal vector of the truss plane, i.e. (Nx, ny, nz) _{Bolt hole} = (a ', B ', C '), wherein,

and step twelve, executing the process of step eleven on the accurate point cloud data of any bolt hole, wherein the positioning and the pose estimation of each bolt hole are all completed (the circle center is the positioning result of the bolt hole, and the axial pose is the pose estimation result).

Other steps and parameters are the same as in one of the first to ninth embodiments.

Projecting accurate point cloud data of the bolt holes to a truss plane ax+by+cz+d=0, wherein the specific process of projection is as follows:

for any point p in accurate point cloud data of bolt holes _i = (x, y, z), point p _i The projected point of = (x, y, z) to the truss plane is denoted q _i = (x ', y ', z ') vectorParallel to the normal vector (a, B, C) of truss plane ax+by+cz+d=0, then:

x＝x'-At

y＝y'-Bt

z＝z'-Ct

wherein t is an intermediate parameter;

since the point q= (x ', y', z ') is on the truss plane ax+by+cz+d=0, ax' +by '+cz' +d=0 is foundSubstituting t to obtain the projection point q= (x ', y ', z ').

The eleventh embodiment, which is different from the first to tenth embodiments, is that, starting from the center of each bolt hole roughly located in the eighth step, extracting all local point clouds of each bolt hole; the specific process is as follows:

Setting a searching radius r, selecting points meeting the condition (1) from the denoised point cloud data for any bolt hole center O roughly positioned in the step eight, and taking the selected points as all point clouds of the bolt hole part;

the condition (1) is: if the Euclidean distance between a certain point and the center O is smaller than the searching radius r, the certain point is a point meeting the condition (1);

and similarly, starting from the center of each bolt hole roughly positioned in the step eight, and respectively obtaining all local point clouds of each bolt hole.

Other steps and parameters are the same as in one to one tenth of the embodiments.

The above examples of the present invention are only for describing the calculation model and calculation flow of the present invention in detail, and are not limiting of the embodiments of the present invention. Other variations and modifications of the above description will be apparent to those of ordinary skill in the art, and it is not intended to be exhaustive of all embodiments, all of which are within the scope of the invention.

Claims (10)

1. The bolt hole positioning and pose estimating method based on three-dimensional vision is characterized by comprising the following steps of:

Scanning an iron tower truss scene containing bolt holes by adopting a three-dimensional sensor to acquire point cloud data;

step two, denoising the point cloud obtained in the step one to obtain denoised point cloud data;

step three, carrying out downsampling on the denoised point cloud data to obtain downsampled point cloud data;

determining a truss plane and a plane point cloud by adopting a plane model segmentation method with random sampling consistency based on the down-sampled point cloud data;

step five, respectively calculating normal vectors of each point in the plane point cloud data, and preliminarily determining boundary points according to the calculated normal vectors;

step six, removing the impurity points and the abnormal values from the boundary points preliminarily determined in the step five to obtain boundary points;

step seven, clustering the boundary points obtained in the step six to obtain boundary point clouds of all the bolt holes;

step eight, respectively carrying out two-dimensional circle fitting on boundary point clouds of all the bolt holes based on truss planes, and roughly positioning the center of each bolt hole;

step nine, respectively starting from the circle centers of the bolt holes roughly positioned in the step eight in the denoised point cloud data obtained in the step two, and extracting all point clouds of the parts of the bolt holes;

Performing preliminary boundary point determination, boundary point elimination and boundary point clustering on the extracted point cloud to obtain accurate point cloud data of each bolt hole;

and step ten, obtaining bolt hole positioning and pose estimation results according to the accurate point cloud data of each bolt hole.

2. The method for positioning and estimating the pose of the bolt hole based on the three-dimensional vision according to claim 1, wherein the denoising processing of the point cloud obtained in the first step adopts a statistical filtering method, and the specific denoising process comprises the following steps:

step two, the total point cloud number contained in the point cloud data acquired in the step one is recorded as N, and any point P in the point cloud data is recorded _i Will be located at point P _i Points within the K-neighborhood of (a) are denoted as P _j J=1, 2, …, K, P is calculated separately _i After the Euclidean distance of each point in the K neighborhood is calculated, the calculated Euclidean distance is averaged to obtain the average valued _iK Is point P _i Corresponding K neighborhood euclidean distance mean, (x) _i ,y _i ,z _i ) Is point P _i Coordinates of (x) _j ,y _j ,z _j ) Is point P _j Coordinates of (c);

similarly, the K neighborhood Euclidean distance mean value d corresponding to each point is calculated _iK ，i＝1,2,...,N；

Step two, calculating d _iK Mean μ and standard deviation σ of (a):

step two and three, setting a standard deviation coefficient s, and calculating a standard distance threshold range [ T ] of statistical filtering according to the standard deviation coefficient s, the mean mu and the standard deviation sigma _std,1 ，T _std,2 ]；

T _std,1 ＝μ-sσ ²

T _std,2 ＝μ+sσ ²

Step two, four, for point P _i Judgment of d _iK Whether or not it is within a standard distance threshold range [ T ] _std,1 ，T _std,2 ]An inner part; if d _iK Is not within the standard distance threshold range [ T ] _std,1 ，T _std,2 ]In, point P _i Is the noise point, point P _i Reject from the point cloud dataset, otherwise Point P _i Not noisy point, not requiring point P _i Processing;

and similarly, after judging each point, obtaining the point cloud data from which the noise is removed.

3. The three-dimensional vision-based bolt hole positioning and pose estimation method according to claim 2, wherein the specific process of the third step is as follows:

step three, three-dimensional voxel grid division is carried out on the point cloud space of the denoised point cloud data, and the size of each cube grid obtained after division is V= (V) _x ,V _y ,V _z )，V _x ，V _y And V _z The dimensions of the cube mesh in each dimension, respectively;

step three, selecting a point cloud closest to the center of a cube grid from the cube grid for any cube grid;

and similarly, after each cube grid is processed respectively, all the selected point cloud data are the down-sampled point cloud data.

4. The three-dimensional vision-based bolt hole positioning and pose estimation method according to claim 3, wherein the specific process of the fourth step is as follows:

Step four, setting the maximum iteration number as T, and initializing the current iteration number as 1;

step four, randomly selecting a point set from the down-sampled point cloud data, and fitting a plane equation A according to the randomly selected point set _t x+B _t y+C _t z+D _t ＝0，A _t 、B _t 、C _t And D _t The coefficient of the plane equation fitted by the t iteration is that x, y and z are three-dimensional coordinates of points on a plane;

step four, setting a distance threshold d _max Respectively calculating each point in the down-sampled point cloud data to a plane A _t x+B _t y+C _t z+D _t Distance of =0:

wherein d _i′ The distance from the ith point to the plane in the down-sampled point cloud data, i '=1, 2, …, N' is the total number of point clouds contained in the down-sampled point cloud data;

if d _i′ ＜d _max The i 'th point is an intra-office point, otherwise, the i' th point is an extra-office point;

setting a point threshold value m, and judging whether the number of the local points is larger than m according to the result of the step four;

if the number of the local points is greater than m, the fourth step is executed after the plane equation of the iterative fitting is saved;

otherwise, if the number of the local points is less than or equal to m, directly executing the fourth step;

judging whether the current iteration times T is smaller than T or not;

if T is smaller than T, let t=t+1, and return to execute step four two;

If T is equal to T, selecting a plane equation with the largest number of local points from all plane equations stored By T iterations, marking the selected plane equation as ax+by+cz+d=0, ax+by+cz+d=0 as a truss plane, and taking all local points corresponding to the plane equation ax+by+cz+d=0 as a plane point cloud.

5. The three-dimensional vision-based bolt hole positioning and pose estimation method according to claim 4, wherein the specific process of the fifth step is as follows:

step five, for any point P in the plane point cloud data _i″ Distance P _i″ The set of the nearest K' points is denoted as M _i″ Utilizing set M _i″ In (a) point-calculation covariance matrix

Wherein, for set M _i″ Taking the average value of the x-direction coordinates and the y-direction coordinates of all the points in the array, and taking the average value of the z-direction coordinates, and then taking P _mean Is a three-dimensional coordinate composed of an x-direction average value, a y-direction average value and a z-direction average value, P _j′ -P _mean Representing the set M _i″ Three-dimensional coordinates of j' th point in the series and three-dimensional coordinates P _mean Performing difference, wherein an upper corner mark T represents transposition;

for covariance matrixPerforming eigenvalue decomposition, and taking the eigenvector corresponding to the minimum eigenvalue as point P _i″ Normal vector of (2);

step five, normalizing the normal vector calculated in the step five to obtain a point P _i″ Is a normalized normal vector of (2);

step five, executing the processes of the step five and the step five for each point in the plane point cloud data to obtain normalized normal vector of each point;

step five and four, for any point P in the plane point cloud data _i″ Selecting a point P from planar point cloud data _i″ K' neighborhood points of (2), point P _i″ Is written as normalized normal vector of (2)The normalized normal vector of k' neighborhood points is marked as +.>Calculating the points P respectively _i″ After the included angle between the normalized normal vector of each neighborhood point and the normalized normal vector of each neighborhood point, averaging the calculated included angles;

point P _i″ Normalized normal vector of (a)The included angle between the normalized normal vector of the j' th neighborhood point is as follows:

point P _i″ The average value of the corresponding included angles is as follows:

step five, judging theta _i″ Whether or not it is greater than the angle threshold value theta ₀ If theta _i″ ＞θ ₀ Point P _i″ Is the boundary point, otherwise point P _i″ Is a non-boundary point;

and step five, and the same principle is adopted, each point in the plane point cloud data is processed respectively by adopting the step five, the step four and the step five, and all obtained boundary points are the preliminarily determined boundary points.

6. The three-dimensional vision-based bolt hole positioning and pose estimation method according to claim 5, wherein the angle threshold value θ ₀ The value of (2) isOr->

7. The three-dimensional vision-based bolt hole positioning and pose estimation method according to claim 6, wherein the specific process of the sixth step is as follows:

setting a search radius r, and regarding any point p in the boundary points preliminarily determined in the step five _i″′ Selecting a point p from the preliminarily determined boundary points _i″′ Points having a Euclidean distance less than the search radius r, and a set N (p) of selected points _i″′ ) The method comprises the following steps:

N(p _i″′ )＝{p _j″′ |dist(p _i″′ ,p _j″″ )＜r}

wherein point p _j″′ Is the boundary point preliminarily determined in the fifth step, dist (p _i″′ ,p _j″′ ) Representative point p _i″′ And point p _j″′ Is a Euclidean distance of (2);

wherein, (x) _i″′ ,y _i″′ ,z _i″′ ) Is point p _i″′ Coordinates of (x) _j″′ ,y _j″′ ,z _j″′ ) Is point p _j″′ Coordinates of (c);

if set N (p) _i″′ ) The point p is the point contained in the data is greater than the threshold value n _i″′ As boundary point, otherwise point p _i″′ Removing;

and similarly, processing each point in the boundary points preliminarily determined in the step five to obtain the rest boundary points.

8. The three-dimensional vision-based bolt hole positioning and pose estimation method according to claim 7, wherein the specific process of the step seven is as follows:

seventhly, marking all boundary points obtained in the step six as unclassified states, and creating a list containing all unclassified points;

Step seven, randomly selecting a point from unclassified points in the list as a seed point, removing the seed point from the unclassified list, respectively calculating Euclidean distances between the seed point and each unclassified point, and if the Euclidean distance between a certain unclassified point and the seed point is smaller than a given distance threshold value, adding the unclassified point into a cluster of the seed points, and removing the unclassified point from the unclassified list;

step seven, judging whether unclassified points exist in the list, if so, returning to the step seven, otherwise, ending the clustering, and respectively obtaining the cloud data of each clustered point;

setting the minimum clustering point number and the maximum clustering point number, if the point number contained in a certain cluster is between the minimum clustering point number and the maximum clustering point number, the cluster belongs to the boundary point cloud of the bolt hole, each cluster corresponds to the boundary point cloud of one bolt hole, and otherwise, the cluster does not belong to the boundary point cloud of the bolt hole.

9. The method for positioning and estimating the pose of the bolt hole based on three-dimensional vision according to claim 8, wherein in the eighth step, two-dimensional circle fitting is performed on the boundary point clouds of each bolt hole respectively to obtain two-dimensional circle fitting results of each bolt hole; the specific process is as follows:

Step eight, projecting boundary point clouds of any bolt hole onto a truss plane to obtain a projected point set;

step eight, randomly selecting a plurality of data points from the projected point set, and performing two-dimensional circle fitting by using the x-direction coordinates and the y-direction coordinates of the selected data points;

the fitted two-dimensional round equation is: (x-x) _center ) ² +(y-y _center ) ² ＝r′ ² Wherein (x) _center ,y _center ) Is the center coordinates of a two-dimensional circle, and r' is the radius of the two-dimensional circle;

step eight, respectively calculating the distance from each point in the boundary point cloud of the bolt hole to the circle center after projection, and for any point in the boundary point cloud of the bolt hole after projection, making a difference between the distance from the point to the circle center and the radius r', if the absolute value of the difference result is smaller than a threshold epsilon, considering the point to be on a circle, otherwise, considering the point not to be on a circle;

after each point in the projected boundary point cloud of the bolt hole is judged, counting the total point number on the circle, and if the total point number on the circle is larger than a set threshold value, the two-dimensional circle equation obtained by fitting meets the requirement; otherwise, the two-dimensional round equation obtained by the fitting is not in accordance with the requirement;

step eight four, repeating the process from step eight two to step eight three until the set maximum iteration number is reached;

Selecting a two-dimensional round equation with the largest total number of points on a circle from two-dimensional round equations meeting requirements, taking the selected two-dimensional round equation as a round equation corresponding to the bolt hole, wherein the center of the selected two-dimensional round equation is the center of the roughly positioned bolt hole, the center coordinates of the two-dimensional round equation are the coordinates of the center of the bolt hole in the x direction and the y direction, and substituting the coordinates of the center of the bolt hole in the x direction and the y direction into a truss plane ax+by+cz+D=0 to obtain the z-direction coordinates of the center of the bolt hole;

and step eight, five, and the same way, executing the processes from step eight to step eight four on the boundary point clouds of each bolt hole.

10. The three-dimensional vision-based bolt hole positioning and pose estimation method according to claim 9, wherein the specific process of the step ten is as follows:

step eleven, for the accurate point cloud data of any one bolt hole, projecting the accurate point cloud data of the bolt hole to a truss plane ax+by+cz+d=0 to obtain projected point cloud data;

performing two-dimensional circle fitting on the accurate point cloud data projected by the bolt hole by adopting the method of the step eight to obtain the circle center and the radius of the two-dimensional circle, and obtaining the center of the bolt hole according to the circle center of the two-dimensional circle;

The axial attitude (Nx, ny, nz) of the bolt hole _{Bolt hole} Determined by the normal vector of the truss plane, i.e. (Nx, ny, nz) _{Bolt hole} = (a ', B ', C '), wherein,

and step twelve, executing the process of step eleven for the accurate point cloud data of any one bolt hole, and completing the positioning and pose estimation of each bolt hole.