CN107564059A - Object positioning method, device and NI Vision Builder for Automated Inspection based on RGB D information - Google Patents
Object positioning method, device and NI Vision Builder for Automated Inspection based on RGB D information Download PDFInfo
- Publication number
- CN107564059A CN107564059A CN201710562324.8A CN201710562324A CN107564059A CN 107564059 A CN107564059 A CN 107564059A CN 201710562324 A CN201710562324 A CN 201710562324A CN 107564059 A CN107564059 A CN 107564059A
- Authority
- CN
- China
- Prior art keywords
- point
- information
- plane
- rgb
- point cloud
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Image Analysis (AREA)
Abstract
The invention provides a kind of object positioning method based on RGB D information, device and NI Vision Builder for Automated Inspection, it is related to digital image processing techniques field.Wherein, the object positioning method based on RGB D information includes：Obtain the ambient image under current view point；Based on RANSAC algorithms, in the criterion of geometric similarity measurement, the similitude that color similarity module comes metric point and plane is introduced, carries out piecemeal to point cloud chart, and successively from selected seed point in each piece to obtain plane parameter；Intra-office point judgement is carried out according to the colouring information of ambient image and geological information, plane parameter is reevaluated, extracts plane characteristic equation；The point cloud cluster that each object under current view point is obtained using the region growing algorithm based on RGB D information realizes that object positions.The present invention is made better use of RGB D information and can positioned with the multiple objects necessarily mutually blocked with reference to the object location algorithm that a kind of efficient stable is realized based on region growing algorithm, while has very high real-time and robustness.
Description
Technical field
The present invention relates to digital image processing techniques field, determines in particular to a kind of object based on RGB-D information
Position method, apparatus and a kind of NI Vision Builder for Automated Inspection.
Background technology
Object positioning is important component that intellect service robot realizes environment sensing, and one of machine vision
Important branch.The object positioning requirements robot of intelligent robot can be in the complicated family's ring of such as desktop, refrigerator or ground etc.
Certain objects are accurately positioned in border, subsequently object identification and operation are submitted necessary information for it, size including object,
The information such as center of gravity, position, posture.Traditional intellect service robot is based primarily upon image and realizes that object positions.But this kind of method
Just for certain objects, and need the offline model for establishing object or gather the image of object, pass through distinguished point based online
The method matched somebody with somebody obtains the pose of object.With the development of 3-D scanning technology, cloud acquisition device such as Kinect etc. three is put in real time
Dimension sensor is popularized, and the object positioning based on a cloud has also obtained extensive research.Blodow etc. exists《Partial view
modeling and validation in 3D laser scans for grasping》Pass through plane extraction algorithm in one text
Desktop is obtained, is positioned followed by the segmentation for realizing object on table top under single view based on an object segmentation algorithm of cluster.
In order to reduce the time complexity of object on table top positioning, Deepak and Quo etc. exist《Grasping novel objects with
depth segmentation》RGB-D data are directly based upon in one text depth map split using region growing algorithm
To block of pixels, the quantity for being next based on block of pixels pixel realizes object on table top segmentation positioning.But it is sentenced based on pixel quantity
The method of earnest body causes algorithm application to be limited, while is easily influenceed by data noise.
Therefore a kind of object location algorithm of efficient stable is needed, can be positioned with the multiple objects necessarily mutually blocked,
There is very high real-time and robustness simultaneously.
The content of the invention
In order to overcome the shortcomings of prior art, the present invention provide a kind of object positioning method based on RGB-D information,
Device and NI Vision Builder for Automated Inspection.The present invention makes better use of RGB-D information realizes one with reference to based on region growing algorithm
The object location algorithm of kind efficient stable can be positioned with the multiple objects necessarily mutually blocked, while has very high real-time
And robustness.
The first aspect of the present invention provides a kind of object positioning method based on RGB-D information, including：
The ambient image under current view point is obtained, wherein, the ambient image includes RGB figures, depth map, point cloud chart and covered
Film；
Based on RANSAC algorithms, in the criterion of geometric similarity measurement, color similarity module is introduced to measure
The similitude of point and plane, piecemeal is carried out to the point cloud chart, and joined successively from selected seed point in each piece to obtain plane
Number；
Intra-office point judgement is carried out according to the colouring information of the ambient image and geological information, to the plane parameter again
Estimation, extracts plane characteristic equation；
The point cloud cluster of each object under current view point is obtained using the region growing algorithm based on RGB-D information, utilizes point
The pose of each object described in cloud acquisition of information realizes that object positions.
Preferably, using Kinect as IMAQ instrument to obtain the ambient image.
In any of the above-described scheme preferably, piecemeal is carried out to the point cloud chart, specifically included：
Its neighborhood point set is determined according to the position of the pixel of each point, piecemeal is carried out to the point cloud chart, to ensure space
In as far as possible more plane be detected.
In any of the above-described scheme preferably, the distance conception of three dimensions is expanded to bag by the selection process of seed point
The distance of the sextuple space of information containing space geometry (x, y, z) and colouring information (R, G, B), according to the similar of this Distance Judgment point
Degree.
In any of the above-described scheme preferably, it is described to be carried out according to the colouring information and geological information of the ambient image
Intra-office point judges, the plane parameter is reevaluated, plane characteristic equation is extracted, specifically includes：
By the color similarity of point and the point to the geometric distance Weighted Fusion of plane, the face is adjusted according to environmental quality
The weight of color similitude and the described point are to the weight of the geometric distance of plane, the accuracy split with guarantee.
In any of the above-described scheme preferably, if under rejecting above support plane if object is just placed on a support
The cloud data of side.
In any of the above-described scheme preferably, value of the mask at image cavity is arranged to 0.
The Part II of the present invention is related to a kind of positioning device for body based on RGB-D information, for NI Vision Builder for Automated Inspection,
Including：
Data cell, obtain current view point under ambient image, wherein, the ambient image include RGB figures, depth map,
Point cloud chart and mask；
Blocking unit, based on RANSAC algorithms, in the criterion of geometric similarity measurement, introduce color similarity measurement mark
Standard carrys out the similitude of metric point and plane, and piecemeal is carried out to the point cloud chart, and successively from selected seed point in each piece to obtain
Obtain plane parameter；
Optimize unit, intra-office point judgement is carried out according to the colouring information of the ambient image and geological information, to described flat
Face parameter reevaluates, and extracts plane characteristic equation；
Positioning unit, the point cloud of each object under current view point is obtained using the region growing algorithm based on RGB-D information
Cluster, realize that object positions using the pose of each object described in a cloud acquisition of information.
Preferably, image unit, using Kinect as IMAQ instrument to obtain the ambient image.
In any of the above-described scheme preferably, the blocking unit is specifically used for：
Its neighborhood point set is determined according to the position of the pixel of each point, piecemeal is carried out to the point cloud chart, to ensure space
In as far as possible more plane be detected.
In any of the above-described scheme preferably, the distance conception of three dimensions is expanded to bag by the selection process of seed point
The distance of the sextuple space of information containing space geometry (x, y, z) and colouring information (R, G, B), according to the similar of this Distance Judgment point
Degree.
In any of the above-described scheme preferably, the geometric distance weighting of the color similarity of point and the point to plane is melted
Close, the weight of the color similarity is adjusted according to environmental quality and the described point arrives the weight of the geometric distance of plane, with guarantor
Demonstrate,prove the accuracy of segmentation.
In any of the above-described scheme preferably, culling unit, the branch is rejected if object is just placed on a support
Support the cloud data below object plane.
In any of the above-described scheme preferably, mask cell, value of the mask at image cavity is arranged to 0.
The third aspect of the present invention is related to a kind of NI Vision Builder for Automated Inspection, including the positioner based on RGB-D information.
The beneficial effects of the invention are as follows：Object orientation problem is converted into point cloud segmentation problem, passes through improved RANSAC
Plane in algorithm extraction environment, added on the basis of traditional RANSAC algorithms：The piecemeal of seed point chooses, plane of similarity
The improved methods such as cluster, plane parameter re-optimization, an iteration can is only needed accurately to extract multiple planes in space, while profit
The segmentation of object point cloud is realized with the region growing algorithm based on RGB-D data, finally utilizes its position of object point cloud acquisition of information
Appearance realizes that object positions, and can position with the multiple objects necessarily mutually blocked, while have very high real-time and robustness.
The additional aspect and advantage of the present invention will be set forth in part in the description, and will partly become from the following description
Obtain substantially, or recognized by the practice of the present invention.
Brief description of the drawings
The above-mentioned and/or additional aspect and advantage of the present invention will become in the description from combination accompanying drawings below to embodiment
Substantially and it is readily appreciated that, wherein：
Fig. 1 is the exemplary flow according to a preferred embodiment of the object positioning method based on RGB-D information of the present invention
Figure；
Fig. 2 is the schematic block diagram according to a preferred embodiment of the positioning device for body based on RGB-D information of the present invention；
Fig. 3 is the schematic block diagram according to a preferred embodiment of the NI Vision Builder for Automated Inspection of the present invention；
Fig. 4 is the space plane equation according to an embodiment of the object positioning method based on RGB-D information of the present invention
Normal vector form；
Fig. 5 is to split according to the point cloud plane of an embodiment of the object positioning method based on RGB-D information of the present invention
Flow chart；
Fig. 6 is to determine according to the robot object of an embodiment of the object positioning method based on RGB-D information of the present invention
Position schematic diagram.
Embodiment
It is below in conjunction with the accompanying drawings and specific real in order to be more clearly understood that the above objects, features and advantages of the present invention
Mode is applied the present invention is further described in detail.It should be noted that in the case where not conflicting, the implementation of the application
Feature in example and embodiment can be mutually combined.
Many details are elaborated in the following description to facilitate a thorough understanding of the present invention, still, the present invention may be used also
To be different from other modes described here using other to implement, therefore, protection scope of the present invention is not by described below
Specific embodiment limitation.
As shown in figure 1, a kind of object positioning method based on RGB-D information according to embodiments of the present invention, for machine
Vision system, including：Step 102, obtain current view point under ambient image, wherein, ambient image include RGB figures, depth map,
Point cloud chart and mask；Step 104, based on RANSAC algorithms, in the criterion of geometric similarity measurement, color similarity degree is introduced
Amount standard carrys out the similitude of metric point and plane, and piecemeal is carried out to point cloud chart, and successively from selected seed point in each piece to obtain
Obtain plane parameter；Step 106, intra-office point judgement is carried out according to the colouring information of ambient image and geological information, to plane parameter
Reevaluate, extract plane characteristic equation；Step 108, obtained using the region growing algorithm based on RGB-D information and work as forward sight
The point cloud cluster of each object under point, realize that object positions using a pose for each object of cloud acquisition of information.
Object positioning side according to an embodiment of the invention based on RGB-D information (color and depth information of image)
Method, by being added in traditional RANSAC algorithms：The piecemeal selection of seed point, the cluster of plane of similarity, plane parameter are excellent again
The improved methods such as change, only need an iteration can accurately to extract multiple planes in space.Specifically, due to RGB-D images
Cloud data has the property of structuring, i.e., each point can determine its neighborhood point set, therefore algorithm according to the position of its pixel
Piecemeal is carried out to point cloud chart first, can so ensure that plane as far as possible more in space is detected.Then successively from each piece
Middle selection a quantity of seeds point is based on RANSAC algorithms and obtains plane parameter and preserve effective plane ginseng according to threshold condition
Number p, wherein each p is includedThree data.It can thus be concluded that to a parameter sets P={ p1...pn}.P is analyzed,
It can find to be distributed in several clusters in the element set in set, and each cluster is precisely the parameter place corresponding to the midplane of space
Region.Therefore first by the selection of seed point, plane parameter initial value is determined, colouring information is then based on and geological information enters
Row intra-office point judges, and plane parameter is reevaluated, and extracts plane characteristic equation.
The object positioning method based on RGB-D information according to the abovementioned embodiments of the present invention, there can also be following skill
Art feature：Using Kinect as IMAQ instrument to obtain ambient image.
In this embodiment, more ripe Kinect is used easily to obtain object as IMAQ instrument
RGB-D information and corresponding cloud data.
In this embodiment, the step of carrying out piecemeal to point cloud chart specifically includes：It is true according to the position of the pixel of each point
Its fixed neighborhood point set, piecemeal is carried out to point cloud chart, is detected with the plane for ensureing as far as possible more in space.
In this embodiment, because cloud data has the property of structuring, i.e., each point can be according to the position of its pixel
Put and determine its neighborhood point set, therefore algorithm carries out piecemeal to point cloud chart first, can so ensure plane quilt as far as possible more in space
Detect.
The selection process of seed point expands to the distance conception of three dimensions comprising space geometry information (x, y, z) and face
The distance of the sextuple space of color information (R, G, B), according to the similarity degree of this Distance Judgment point.
In this embodiment, the distance of two points is nearer in point cloud, and the probability that they belong to same model is bigger.Ring
The three dimensional point cloud set P={ p in borderi|pi=(xi,yi,zi,Ri,Gi,Bi), (x, y, z) represents some point relative to taking
As the three-dimensional coordinate of device, (Ri,Gi,Bi) represent the colouring information of the point.In traditional RANSAC algorithms, seed point be from
3 points randomly generated in initial data, and thus calculate initial plane model.But the seed obtained by this method
Probability of the point not in approximately the same plane is bigger, therefore can reduce the efficiency of segmentation.It is of the invention then by the distance of three dimensions
Concept expands to the distance of the sextuple space comprising space geometry information (x, y, z) and colouring information (R, G, B), effective to reduce
Seed point chooses the risk of failure brought, and largely improves the overall efficiency of data processing in algorithm.
Intra-office point judgement is carried out according to the colouring information of ambient image and geological information, plane parameter is reevaluated, carried
The step of taking out plane characteristic equation specifically includes：The geometric distance weighting of the color similarity of point and the point to plane is melted
Close, the weight and the weight of the point to the geometric distance of plane of color similarity are adjusted according to environmental quality, to ensure segmentation
Accuracy.
In this embodiment, for most of objects in environment, the difference on same object is in color, material
Etc. often there is uniformity.According to this characteristic, compared to simple geometric properties, the involvement of colouring information can be point
Cut and more accurate foundation is provided, it is therefore, of the invention in the criterion of the only geometric similarity measurement of traditional RANSAC algorithms,
The similitude that color similarity module comes metric point and plane is introduced, and by adjusting geometry and color characteristic in formula
Weight make it that segmentation result is more accurate.
The cloud data below support object plane is rejected if object is just placed on a support.
In this embodiment, NI Vision Builder for Automated Inspection object to be processed is general all on the supporting surfaces such as desktop, i.e. object
Point cloud is square in the plane.Thus, if point rejecting by plane and below can reduce computing pressure in point cloud chart
Improve algorithm accuracy.
Value of the mask at image cavity is arranged to 0.
In this embodiment, because each pixel of depth map represents that certain point in environment arrives the distance of camera, point
Each pixel of cloud then includes (x, y, the z) coordinate information of relevant position under camera coordinates system in space, and mask is then one
The width only image comprising 0 and 1 two kind of data, the depth map of collection is because environmental factor there may be cavity, so mask is existed
Value at cavity is arranged to 0.
As shown in Fig. 2 a kind of positioning device for body 200 based on RGB-D information is additionally provided, for machine vision system
System, including：Data cell 202, the ambient image under current view point is obtained, wherein, ambient image includes RGB figures, depth map, point
Cloud atlas and mask；Blocking unit 204, based on RANSAC algorithms, in the criterion of geometric similarity measurement, introduce color similarity
Module carrys out the similitude of metric point and plane, to point cloud chart carry out piecemeal, and successively from selected seed point in each piece with
Obtain plane parameter；Optimize unit 206, intra-office point judgement is carried out according to the colouring information of ambient image and geological information, to flat
Face parameter reevaluates, and extracts plane characteristic equation；Positioning unit 208, utilize the region growing algorithm based on RGB-D information
The point cloud cluster of each object under current view point is obtained, realizes that object positions using a pose for each object of cloud acquisition of information.
Positioning device for body 200 according to an embodiment of the invention based on RGB-D information, RGB-D information are image
Color and depth information, by being added in traditional RANSAC algorithms：The piecemeal of seed point is chosen, the cluster of plane of similarity,
The improved methods such as plane parameter re-optimization, an iteration can is only needed accurately to extract multiple planes in space.Specifically, due to
The cloud data of RGB-D images has the property of structuring, i.e., each point can determine its neighborhood point according to the position of its pixel
Collection, therefore algorithm carries out piecemeal to point cloud chart first, can so ensure that plane as far as possible more in space is detected.Then according to
Secondary a quantity of seeds point of being chosen from each piece obtains plane parameter based on RANSAC algorithms and preserved according to threshold condition
Effective plane parameter p, wherein each p is includedThree data.It can thus be concluded that to a parameter sets P=
{p1...pn}.P is analyzed, it is possible to find it is distributed in the element set in set in several clusters, and each cluster is precisely to be put down in space
The region where parameter corresponding to face.Therefore first by the selection of seed point, plane parameter initial value is determined, is then based on face
Color information and geological information carry out intra-office point judgement, and plane parameter is reevaluated, and extract plane characteristic equation.
The positioning device for body 200 based on RGB-D information according to the abovementioned embodiments of the present invention, can also have following
Technical characteristic：
Positioning device for body 200 based on RGB-D information, in addition to：Image unit, IMAQ is used as using Kinect
Instrument is to obtain ambient image.
In this embodiment, more ripe Kinect is used easily to obtain object as IMAQ instrument
RGB-D information and corresponding cloud data.
The blocking unit 204 of positioning device for body 200 based on RGB-D information is specifically used for：According to the pixel of each point
Position determine its neighborhood point set, piecemeal is carried out to point cloud chart, is detected with the plane for ensureing in space as far as possible more.
In this embodiment, because cloud data has the property of structuring, i.e., each point can be according to the position of its pixel
Put and determine its neighborhood point set, therefore algorithm carries out piecemeal to point cloud chart first, can so ensure plane quilt as far as possible more in space
Detect.
In positioning device for body 200 based on RGB-D information, the selection process of seed point is by the distance conception of three dimensions
The distance of the sextuple space comprising space geometry information (x, y, z) and colouring information (R, G, B) is expanded to, according to this Distance Judgment
The similarity degree of point.
In this embodiment, the distance of two points is nearer in point cloud, and the probability that they belong to same model is bigger.Ring
The three dimensional point cloud set P={ p in borderi|pi=(xi,yi,zi,Ri,Gi,Bi), (x, y, z) represents some point relative to taking
As the three-dimensional coordinate of device, (Ri,Gi,Bi) represent the colouring information of the point.In traditional RANSAC algorithms, seed point be from
3 points randomly generated in initial data, and thus calculate initial plane model.But the seed obtained by this method
Probability of the point not in approximately the same plane is bigger, therefore can reduce the efficiency of segmentation.It is of the invention then by the distance of three dimensions
Concept expands to the distance of the sextuple space comprising space geometry information (x, y, z) and colouring information (R, G, B), effective to reduce
Seed point chooses the risk of failure brought, and largely improves the overall efficiency of data processing in algorithm.
Optimize unit 206 in the positioning device for body 200 based on RGB-D information to be specifically used for：The color of point is similar
Property the geometric distance Weighted Fusion of plane is arrived with the point, according to environmental quality adjust color similarity weight and the point to plane
Geometric distance weight, with ensure segmentation accuracy.
In this embodiment, for most of objects in environment, the difference on same object is in color, material
Etc. often there is uniformity.According to this characteristic, compared to simple geometric properties, the involvement of colouring information can be point
Cut and more accurate foundation is provided, it is therefore, of the invention in the criterion of the only geometric similarity measurement of traditional RANSAC algorithms,
The similitude that color similarity module comes metric point and plane is introduced, and by adjusting geometry and color characteristic in formula
Weight make it that segmentation result is more accurate.
Positioning device for body 200 based on RGB-D information also includes：Culling unit, if object is just placed on a support
Then reject the cloud data below support object plane.
In this embodiment, NI Vision Builder for Automated Inspection object to be processed is general all on the supporting surfaces such as desktop, i.e. object
Point cloud is square in the plane.Thus, if point rejecting by plane and below can reduce computing pressure in point cloud chart
Improve algorithm accuracy.
Positioning device for body 200 based on RGB-D information also includes：
Mask cell, value of the mask at image cavity is arranged to 0.
In this embodiment, because each pixel of depth map represents that certain point in environment arrives the distance of camera, point
Each pixel of cloud then includes (x, y, the z) coordinate information of relevant position under camera coordinates system in space, and mask is then one
The width only image comprising 0 and 1 two kind of data, the depth map of collection is because environmental factor there may be cavity, so mask is existed
Value at cavity is arranged to 0.
As shown in figure 3, according to still another embodiment of the invention, it is also proposed that a kind of NI Vision Builder for Automated Inspection 300, including：Such as
The positioning device for body 200 based on RGB-D information any one of above-described embodiment.
(it is applied to intelligent Service below in conjunction with a kind of way of realization of the object positioning method of Fig. 4 to Fig. 6 description present invention
Robot object positioning)：
Overall framework is as shown in fig. 6, wherein, vision system acquisition module 602 is responsible for the figure of environment under collection current view point
Picture, including RGB figures, depth map, point cloud chart and mask.Wherein RGB is the coloured image of current scene, each pixel of depth map
Point represents certain point in environment to the distance of camera, and each pixel for putting cloud is then sat comprising relevant position in space in camera
Lower (x, y, the z) coordinate information of mark system, mask are then a width only images comprising 0 and 1 two kind of data, the depth that Kinect is gathered
Figure is because environmental factor there may be cavity, so value of the mask at cavity is 0.Environment plane extraction module 604 then utilizes
Point cloud chart extracts to the plane in space.Object segmentation locating module 606 is then done into one to the point cloud after plane is extracted
Step processing, based on scene RGB and depth information calmodulin binding domain CaM growth algorithm can fast and stable realize the segmentations of more objects in space
Positioning.
Specifically, in 3d space (OXYZ), space plane can be represented by the normal vector of plane：
WhereinTo lead to zeroaxial planar process vector parameter, as shown in Figure 4.
Plane extraction algorithm based on RANSAC is in the nature to find optimal dough sheet in three-dimensional point set.Traditional RANSAC is calculated
Method calculates its corresponding plane parameter by randomly choosing three points.Then plane parameter is reversely verified using the point in plane,
This process n times of iteration, in iterating to calculate each time, compare this result of calculation and the preceding optimal knot once preserved
Fruit, if this result is more preferable, optimum detection result is saved as with this result of calculation.
The present invention is improved RANSAC plane extraction algorithms.Because the cloud data obtained from Kinect has structure
The property of change, i.e., each point can determine its neighborhood point set according to the position of its pixel, therefore algorithm is carried out to point cloud chart first
Piecemeal, it can so ensure that plane as far as possible more in space is detected.Then choose successively from each piece a number of
Seed point is based on RANSAC algorithms and obtains plane parameter and preserve effective plane parameter p according to threshold condition, wherein each p is includedThree data.It can thus be concluded that to a parameter sets P={ p1...pn}.P is analyzed, it is possible to find the element in set
Integrated distribution is in several clusters, and each cluster is precisely the region where the parameter corresponding to the midplane of space.Therefore it is logical first
The selection of seed point is crossed, determines plane parameter initial value, colouring information is then based on and geological information carries out intra-office point judgement, and it is right
Plane parameter reevaluates, and extracts plane characteristic equation.
Selected seed point：
In theory, the distance of two points is nearer in point cloud, and the probability that they belong to same model is bigger.The three of environment
Tie up cloud data set P={ pi|pi=(xi,yi,zi,Ri,Gi,Bi), (x, y, z) represents some point relative to Kinect's
Three-dimensional coordinate, (Ri,Gi,Bi) represent the colouring information of the point.In traditional RANSAC algorithms, seed point is from initial data
In 3 points randomly generating, and thus calculate initial plane model.But the seed point obtained by this method is not same
Probability in one plane is bigger, therefore can reduce the efficiency of segmentation.The present invention on this basis, to the selection side of seed point
Formula is improved, by the distance conception of three dimensions expand to comprising space geometry information (x, y, z) and colouring information (R, G,
B the distance of sextuple space), according to the similarity degree of this Distance Judgment point.Specific method is:One is randomly generated in point set P
Point pm, using this point as starting point, find other seed points p for meeting formula (2) in sextuple space distance with itn。
After seed point determines, using plane where least square fitting seed point, if plane equation is in space
Expression-form is:
Ax+by+cz+d=0 (3)
Estimate whether plane meets to require according to evaluation function (formula (4))：
Wherein, diFor the distance (formula (5)) of seed point to plane, d is apart from average (formula (6)).
If standard deviation E be more than setting threshold value, illustrate these point fluctuation it is larger, belong to conplane probability compared with
Small, the plane is ineligible, it is necessary to re-start seed point selection.This method effectively reduces seed point selection and brought
Risk of failure, largely improve the overall efficiency of data processing in algorithm.
Intra-office point judges：
After preliminary acquisition plane parameter model [a, b, c, d], other intra-offices of the model in data acquisition system are found out
Point, i.e., whether plane is belonged to left point and judged.In theory, if point arrives the distance d of planei=0, then point piIn the plane.
But because error be present in the Kinect cloud datas collected, therefore, it is difficult to obtain proper plane, i.e., point and its
Distance to plane always fluctuates within the specific limits.
The cloud data that Kinect is obtained, not only comprising three-dimensional space data, also with corresponding RGB color information.It is right
For most of objects in environment, the difference on same object often has uniformity in color, material etc..Root
According to this characteristic, compared to simple geometric properties, the involvement of colouring information can provide more accurate foundation for segmentation, because
This, the present invention introduces color similarity module in the criterion of the only geometric similarity measurement of traditional RANSAC algorithms
Carry out the similitude of metric point and plane, and by adjusting the weight segmentation result of geometry and color characteristic in formula more
Accurately.
Color similarity c is defined to measure a point p in cloud dataiWith the color similarity of plane.Color similarity c's
It is worth smaller, illustrates point piHigher with the similarity of plane, the probability that point belongs to plane is also bigger, otherwise smaller.Color is similar
Property is defined as:
Wherein cikFor point piThe value of three Color Channels,For plane (being made up of n points) interior each point color
The average of passage.
I.e.：
c(pi)=[(Ri-R)2+(Gi-G)2+(BI-B)2]1/2 (8)
Will point piColor similarity c (pi) with its arrive plane geometric distance d (pi) fusion, defining point piWith plane
Relational expression
H=α c (pi)+βd(pi) (9)
Wherein, α, β are respectively c (pi) and d (pi) weight.Set suitable threshold value Hth, when meeting H ＜ HthWhen, then recognize
Belong to plane for point, be the intra-office point of areal model.H is smaller, illustrates that the probability for a little belonging to plane is bigger, otherwise smaller.Calculate
The number of intra-office point, if greater than threshold value Snum, then split successfully.
Weight α, β proportionate relationship according to environmental quality, it is necessary to be adjusted.For the indoor environments such as office, object
Color uniformity it is higher, then α is larger；For outdoor elements, object color homogeneity is relatively low, and it is 0 that can take α,
Then algorithm deteriorates to traditional plane space diagnostic method.Only in this way, the accuracy of segmentation is just can guarantee that, segmentation is met user
Demand.
Progress successive ignition is needed just to be partitioned into all planes when carrying out plane segmentation to a cloud using RANSAC algorithms,
Each run is complete once, and remaining point is split next time as original point cloud, and constantly circulation is all until extracting
Plane point set.Analyzed more than, point cloud plane segmentation flow of the present invention based on RANSAC methods is as shown in Figure 5.Object is determined
The input data that position algorithm needs, it is respectively:Depth map depthMap, it is directly obtained by Kinect, each pixel of figure
Value be distance of the corresponding points in scene to depth camera photocentre.RGB schemes, and it is the true figure of the current scene of camera collection
Picture.Mask Mask, its essence are a bianry images, and each pixel is made up of 0 or 1 two numeral, and 0 represents the picture of current location
Element is invalid, and 1 represents that current pixel is effective.Depth increases threshold valueRepresent the minimum value of adjacent pixel depth difference when region increases.
The minimum M inPoint of object pixel quantity.In the data of input, depthMap, RGB and Mask pixel are as mutually
It is corresponding.
Usual robot object to be processed is general all on the supporting surfaces such as desktop, i.e. the point cloud of object is square in the plane
's.Thus, if point by plane and below is rejected and can be obtained by point cloud above desktop in point cloud chart, i.e. object
Point cloud.Based on this, on the basis of plane extraction, we obtain each table using the region growing algorithm based on RGB-D data
The point cloud cluster of face object come realize object position.
Understand that object orientation problem is converted to point cloud segmentation problem by the present invention according to above-described embodiment, by improved
Plane in RANSAC algorithm extraction environments, added on the basis of traditional RANSAC algorithms：The piecemeal of seed point is chosen, is similar
The improved methods such as the cluster of plane, plane parameter re-optimization, only need an iteration can accurately to extract multiple planes in space,
The segmentation of object point cloud is realized using the region growing algorithm based on RGB-D data simultaneously, is finally obtained using object point cloud information
Take its pose to realize that object positions, can position with the multiple objects necessarily mutually blocked, at the same have very high real-time and
Robustness.
The preferred embodiments of the present invention are the foregoing is only, are not intended to limit the invention, for the skill of this area
For art personnel, the present invention can have various modifications and variations.Within the spirit and principles of the invention, that is made any repaiies
Change, equivalent substitution, improvement etc., should be included in the scope of the protection.
Claims (10)
- A kind of 1. object positioning method based on RGB-D information, for NI Vision Builder for Automated Inspection, it is characterised in that including：The ambient image under current view point is obtained, wherein, the ambient image includes RGB figures, depth map, point cloud chart and mask；Based on RANSAC algorithms, in the criterion of geometric similarity measurement, introduce color similarity module come metric point with The similitude of plane, piecemeal is carried out to the point cloud chart, and successively from selected seed point in each piece to obtain plane parameter；Intra-office point judgement is carried out according to the colouring information of the ambient image and geological information, the plane parameter is estimated again Meter, extracts plane characteristic equation；The point cloud cluster of each object under current view point is obtained using the region growing algorithm based on RGB-D information, is believed using a cloud The pose that breath obtains each object realizes that object positions.
- 2. the localization method according to claim 1 based on RGB-D information, it is characterised in that also include：Using Kinect as IMAQ instrument to obtain the ambient image.
- 3. the localization method according to claim 1 based on RGB-D information, it is characterised in that carried out to the point cloud chart Piecemeal, specifically include：Its neighborhood point set is determined according to the position of the pixel of each point, piecemeal is carried out to the point cloud chart, to ensure in space to the greatest extent The more plane of amount is detected.
- 4. the localization method according to claim 1 based on RGB-D information, it is characterised in that the selection process of seed point The distance conception of three dimensions is expanded to comprising space geometry information（X, y, z）And colouring information（R, G, B）Sextuple space Distance, according to the similarity degree of this Distance Judgment point.
- 5. the localization method according to claim 1 based on RGB-D information, it is characterised in that described according to the environment The colouring information and geological information of image carry out intra-office point judgement, and the plane parameter is reevaluated, extracts plane characteristic Equation, specifically include：By the color similarity of point and the point to the geometric distance Weighted Fusion of plane, the color phase is adjusted according to environmental quality Like the weight of property and the described point to the weight of the geometric distance of plane, the accuracy split with guarantee.
- 6. the localization method according to claim 1 based on RGB-D information, it is characterised in that also include：The cloud data below above support plane is rejected if object is just placed on a support.
- 7. the localization method based on RGB-D information according to any one of claim 1-6, it is characterised in that also include：Value of the mask at image cavity is arranged to 0.
- A kind of 8. positioning device for body based on RGB-D information, for NI Vision Builder for Automated Inspection, it is characterised in that including：Data cell, the ambient image under current view point is obtained, wherein, the ambient image includes RGB figures, depth map, point cloud Figure and mask；Blocking unit, based on RANSAC algorithms, in the criterion of geometric similarity measurement, introduce color similarity module and come The similitude of metric point and plane, piecemeal is carried out to the point cloud chart, and it is flat to obtain from selected seed point in each piece successively Face parameter；Optimize unit, intra-office point judgement is carried out according to the colouring information of the ambient image and geological information, the plane is joined Number reevaluates, and extracts plane characteristic equation；Positioning unit, the point cloud cluster of each object under current view point, profit are obtained using the region growing algorithm based on RGB-D information The pose of each object described in a cloud acquisition of information realizes that object positions.
- 9. the positioner according to claim 8 based on RGB-D information, it is characterised in that also include：Image unit, using Kinect as IMAQ instrument to obtain the ambient image.
- 10. a kind of NI Vision Builder for Automated Inspection, it is characterised in that including being believed any one of claim 8 to 9 based on RGB-D The positioner of breath.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710562324.8A CN107564059A (en) | 2017-07-11 | 2017-07-11 | Object positioning method, device and NI Vision Builder for Automated Inspection based on RGB D information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710562324.8A CN107564059A (en) | 2017-07-11 | 2017-07-11 | Object positioning method, device and NI Vision Builder for Automated Inspection based on RGB D information |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107564059A true CN107564059A (en) | 2018-01-09 |
Family
ID=60973499
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710562324.8A Pending CN107564059A (en) | 2017-07-11 | 2017-07-11 | Object positioning method, device and NI Vision Builder for Automated Inspection based on RGB D information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107564059A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108460779A (en) * | 2018-02-12 | 2018-08-28 | 浙江大学 | A kind of mobile robot image vision localization method under dynamic environment |
| CN109146894A (en) * | 2018-08-07 | 2019-01-04 | 庄朝尹 | A kind of model area dividing method of three-dimensional modeling |
| CN109176507A (en) * | 2018-08-13 | 2019-01-11 | 国网陕西省电力公司电力科学研究院 | The intelligent mode of connection and device of a kind of robot to transformer |
| CN109448034A (en) * | 2018-10-24 | 2019-03-08 | 华侨大学 | A kind of part pose acquisition methods based on geometric primitive |
| CN109493405A (en) * | 2018-10-31 | 2019-03-19 | 南京和瑞供应链管理有限公司 | Machine vision point cloud chart generation method suitable for large scene storage |
| CN110530376A (en) * | 2019-10-10 | 2019-12-03 | 上海钛米机器人科技有限公司 | Robot localization method, apparatus, robot and storage medium |
| CN110542421A (en) * | 2019-10-10 | 2019-12-06 | 上海钛米机器人科技有限公司 | Robot positioning method, positioning device, robot, and storage medium |
| CN112634366A (en) * | 2020-12-23 | 2021-04-09 | 北京百度网讯科技有限公司 | Position information generation method, related device and computer program product |
| CN112650790A (en) * | 2020-12-28 | 2021-04-13 | 北京字节跳动网络技术有限公司 | Target point cloud plane determining method and device, electronic equipment and storage medium |
| CN113343840A (en) * | 2021-06-02 | 2021-09-03 | 合肥泰瑞数创科技有限公司 | Object identification method and device based on three-dimensional point cloud |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106570903A (en) * | 2016-10-13 | 2017-04-19 | 华南理工大学 | Visual identification and positioning method based on RGB-D camera |
-
2017
- 2017-07-11 CN CN201710562324.8A patent/CN107564059A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106570903A (en) * | 2016-10-13 | 2017-04-19 | 华南理工大学 | Visual identification and positioning method based on RGB-D camera |
Non-Patent Citations (2)
| Title |
|---|
| 孙小凯: "基于RGB-D信息的物体定位与识别", 《中国优秀硕士学位论文全文数据库 信息科技辑（月刊） 无线电电子学》 * |
| 王梅,于远芳,屠大维,周华: "基于Kinect的环境平面特征提取与重构", 《计算机应用》 * |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108460779B (en) * | 2018-02-12 | 2021-09-24 | 浙江大学 | Mobile robot image visual positioning method in dynamic environment |
| CN108460779A (en) * | 2018-02-12 | 2018-08-28 | 浙江大学 | A kind of mobile robot image vision localization method under dynamic environment |
| CN109146894A (en) * | 2018-08-07 | 2019-01-04 | 庄朝尹 | A kind of model area dividing method of three-dimensional modeling |
| CN109176507A (en) * | 2018-08-13 | 2019-01-11 | 国网陕西省电力公司电力科学研究院 | The intelligent mode of connection and device of a kind of robot to transformer |
| CN109448034A (en) * | 2018-10-24 | 2019-03-08 | 华侨大学 | A kind of part pose acquisition methods based on geometric primitive |
| CN109448034B (en) * | 2018-10-24 | 2021-10-01 | 华侨大学 | Part pose acquisition method based on geometric elements |
| CN109493405A (en) * | 2018-10-31 | 2019-03-19 | 南京和瑞供应链管理有限公司 | Machine vision point cloud chart generation method suitable for large scene storage |
| CN110542421A (en) * | 2019-10-10 | 2019-12-06 | 上海钛米机器人科技有限公司 | Robot positioning method, positioning device, robot, and storage medium |
| CN110530376A (en) * | 2019-10-10 | 2019-12-03 | 上海钛米机器人科技有限公司 | Robot localization method, apparatus, robot and storage medium |
| CN112634366A (en) * | 2020-12-23 | 2021-04-09 | 北京百度网讯科技有限公司 | Position information generation method, related device and computer program product |
| CN112634366B (en) * | 2020-12-23 | 2023-10-17 | 北京百度网讯科技有限公司 | Method for generating position information, related device and computer program product |
| CN112650790A (en) * | 2020-12-28 | 2021-04-13 | 北京字节跳动网络技术有限公司 | Target point cloud plane determining method and device, electronic equipment and storage medium |
| CN112650790B (en) * | 2020-12-28 | 2023-10-13 | 北京字节跳动网络技术有限公司 | Target point cloud plane determining method and device, electronic equipment and storage medium |
| CN113343840A (en) * | 2021-06-02 | 2021-09-03 | 合肥泰瑞数创科技有限公司 | Object identification method and device based on three-dimensional point cloud |
| CN113343840B (en) * | 2021-06-02 | 2022-03-08 | 合肥泰瑞数创科技有限公司 | Object identification method and device based on three-dimensional point cloud |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107564059A (en) | Object positioning method, device and NI Vision Builder for Automated Inspection based on RGB D information | |
| CN106327532B (en) | A kind of three-dimensional registration method of single image | |
| CN108334816B (en) | Multi-pose face recognition method based on contour symmetric constraint generation type countermeasure network | |
| CN109102547A (en) | Robot based on object identification deep learning model grabs position and orientation estimation method | |
| CN104933755B (en) | A kind of stationary body method for reconstructing and system | |
| CN106023298B (en) | Point cloud Rigid Registration method based on local Poisson curve reestablishing | |
| CN104820991B (en) | A kind of multiple soft-constraint solid matching method based on cost matrix | |
| CN104596519B (en) | Vision positioning method based on RANSAC algorithms | |
| CN106910242A (en) | The method and system of indoor full scene three-dimensional reconstruction are carried out based on depth camera | |
| CN107248159A (en) | A kind of metal works defect inspection method based on binocular vision | |
| CN107679503A (en) | A kind of crowd's counting algorithm based on deep learning | |
| CN104240264B (en) | The height detection method and device of a kind of moving object | |
| CN105160686B (en) | A kind of low latitude various visual angles Remote Sensing Images Matching Method based on improvement SIFT operators | |
| CN108629843A (en) | A kind of method and apparatus for realizing augmented reality | |
| CN110378997A (en) | A kind of dynamic scene based on ORB-SLAM2 builds figure and localization method | |
| CN108955718A (en) | A kind of visual odometry and its localization method, robot and storage medium | |
| CN105678241B (en) | A kind of cascade two dimensional image face pose estimation | |
| CN106091984A (en) | A kind of three dimensional point cloud acquisition methods based on line laser | |
| CN104615996B (en) | A kind of various visual angles two-dimension human face automatic positioning method for characteristic point | |
| CN107953329A (en) | Object identification and Attitude estimation method, apparatus and mechanical arm grasping system | |
| CN111998862B (en) | BNN-based dense binocular SLAM method | |
| CN109086795A (en) | A kind of accurate elimination method of image mismatch | |
| CN110006444B (en) | Anti-interference visual odometer construction method based on optimized Gaussian mixture model | |
| CN109493384A (en) | Camera position and orientation estimation method, system, equipment and storage medium | |
| CN110059602A (en) | A kind of vertical view face antidote based on orthographic projection eigentransformation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180109 |
|
| RJ01 | Rejection of invention patent application after publication |