CN115272465A - Object positioning method, device, autonomous mobile device and storage medium - Google Patents
Object positioning method, device, autonomous mobile device and storage medium Download PDFInfo
- Publication number
- CN115272465A CN115272465A CN202210861930.0A CN202210861930A CN115272465A CN 115272465 A CN115272465 A CN 115272465A CN 202210861930 A CN202210861930 A CN 202210861930A CN 115272465 A CN115272465 A CN 115272465A
- Authority
- CN
- China
- Prior art keywords
- voxels
- front surface
- grid
- target object
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 66
- 238000003860 storage Methods 0.000 title claims abstract description 21
- 230000033001 locomotion Effects 0.000 claims abstract description 15
- 230000005484 gravity Effects 0.000 claims description 48
- 230000011218 segmentation Effects 0.000 claims description 45
- 238000004590 computer program Methods 0.000 claims description 16
- 238000013507 mapping Methods 0.000 claims description 5
- 230000004807 localization Effects 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 2
- 239000000126 substance Substances 0.000 claims description 2
- 238000012545 processing Methods 0.000 description 12
- 238000004364 calculation method Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 8
- 238000004422 calculation algorithm Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 239000007787 solid Substances 0.000 description 4
- 238000009826 distribution Methods 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000000877 morphologic effect Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/155—Segmentation; Edge detection involving morphological operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20036—Morphological image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The application discloses an object positioning method, an object positioning device, autonomous mobile equipment and a storage medium. The method specifically comprises the following steps: acquiring three-dimensional voxels of the autonomous mobile device in a motion direction in a target space, and generating a grid projection model comprising the three-dimensional voxels; aiming at each grid, determining non-ground voxels in a target space according to the barycentric coordinates of the three-dimensional voxels in the grid; determining a front surface image of the target object according to the non-ground voxels; and positioning the target object according to the pixel boundary in the front surface image. According to the technical scheme, the point cloud influencing target identification can be screened out, the interference of influencing factors in a target environment is reduced, the identification accuracy of the target object is improved, and the universality of target object positioning can be improved due to the identification requirements of various different target objects.
Description
Technical Field
The present application relates to the field of robotics, and in particular, to an object positioning method and apparatus, an autonomous mobile device, and a storage medium.
Background
With the continuous improvement of the demand of social productivity, the robot technology is valued and vigorously developed. The robot with the space sensing capability is widely applied to the industrial requirements of various services, storage, logistics and the like, and the working efficiency of various industries is greatly improved.
In order to meet the demand of automation logistics, a robot with vision or scanning capability senses objects in a space for transportation. In the prior art, in the process of space perception, an electronic tag method is usually adopted to help a robot identify an object, but the identification precision is low; or the laser radar is adopted to scan the object for identification, but a three-dimensional model of the corresponding object needs to be established in advance for comparison, so that the adaptability is poor.
Disclosure of Invention
The application provides an object positioning method, an object positioning device, autonomous mobile equipment and a storage medium, so as to improve the identification precision and adaptability of a target object.
According to an aspect of the present application, there is provided an object positioning method, the method including:
acquiring three-dimensional voxels of the autonomous mobile device in a motion direction in a target space, and generating a grid projection model comprising the three-dimensional voxels;
aiming at each grid, determining non-ground voxels in a target space according to the barycentric coordinates of the three-dimensional voxels in the grid;
determining a front surface image of the target object according to the non-ground voxels;
and positioning the target object according to the pixel boundary in the front surface image.
According to another aspect of the present application, there is provided an object positioning device including:
the grid projection generation module is used for acquiring three-dimensional voxels of the autonomous mobile equipment in the motion direction in the target space and generating a grid projection model comprising the three-dimensional voxels;
the non-ground voxel determining module is used for determining a non-ground voxel in the target space according to the barycentric coordinates of the three-dimensional voxels in the grids aiming at each grid;
a front surface image determination module for determining a front surface image of the target object based on the non-ground voxels;
and the target object positioning module is used for positioning the target object according to the pixel boundary in the front surface image.
According to another aspect of the present application, there is provided an autonomous mobile apparatus including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the object localization method of any of the embodiments of the present application.
According to another aspect of the present application, there is provided a computer-readable storage medium storing computer instructions for causing a processor to implement the object positioning method according to any of the embodiments of the present application when the computer instructions are executed.
According to the technical scheme of the embodiment of the application, the three-dimensional voxels are processed from the grid projection model, the target object in the non-ground voxels is determined, and then the positioning of the target object is determined according to the front surface image of the target object. By the method, point clouds influencing target identification can be screened out, interference of influencing factors in a target environment is reduced, the identification accuracy of the target object is improved, and the universality of target object positioning can be improved due to the identification requirements of various different target objects.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present application, nor do they limit the scope of the present application. Other features of the present application will become apparent from the following description.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1A is a flowchart of an object positioning method according to an embodiment of the present application;
fig. 1B is a schematic diagram of a grid projection model according to an embodiment of the present application;
fig. 2A is a flowchart of an object positioning method according to a second embodiment of the present application;
fig. 2B is a binary image of a front surface of a logistics tray according to the second embodiment of the present application;
fig. 2C is a schematic diagram of a logistics tray lateral pixel boundary provided in accordance with an embodiment two of the present application;
fig. 2D is a schematic diagram of an identification logistics tray provided in accordance with the second embodiment of the present application;
fig. 3 is a schematic structural diagram of an object positioning device according to a third embodiment of the present application;
fig. 4 is a schematic structural diagram of an autonomous mobile apparatus implementing an object positioning method according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions of the present application better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example one
Fig. 1A is a flowchart of an embodiment of the present application, which provides an object positioning method, and this embodiment is applicable to a case where a robot identifies a position of a logistics tray, and the method may be performed by an object positioning device, which may be implemented in a form of hardware and/or software, and the object positioning device may be configured in an autonomous mobile apparatus. As shown in fig. 1A, the method includes:
s110, three-dimensional voxels in the movement direction of the autonomous mobile device in the target space are obtained, and a grid projection model comprising the three-dimensional voxels is generated.
The autonomous moving device may be any device capable of moving in a space according to a preset program or according to a space sensing capability, such as a planar mobile robot (which may be a service robot, a logistics robot, or the like according to different application scenarios), an AGV (Automated Guided Vehicle), or a multi-degree-of-freedom robot that needs to avoid an obstacle while performing work in the space. The target space may be a work space of the autonomous mobile device, such as a three-dimensional space of a warehouse or a production line where the logistics robot works, or the like.
The three-dimensional voxel can be obtained by converting point cloud data in a space acquired by the autonomous mobile equipment, and is used for reducing the number of point clouds and saving computing resources while ensuring that the shape distribution of the point clouds can be maintained, thereby improving the computing efficiency. The grid projection model may be a projection (vertical projection) of a three-dimensional voxel acquired by the autonomous mobile device perpendicular to a horizontal plane, and a preset grid size is set, and each three-dimensional voxel is projected onto a grid of the horizontal plane.
For convenience of understanding and calculation, the center of gravity of a three-dimensional voxel is generally used instead of the three-dimensional voxel for projection and calculation. When the autonomous mobile device moves in the target space, the three-dimensional voxels in the moving direction are acquired in real time, so that whether obstacles exist in the moving direction or not is judged in an auxiliary manner (the obstacles in the moving direction include some obstacles influencing the movement of the autonomous mobile device on the ground, such as a solid obstacle and a ground depression).
For example, as shown in FIG. 1B, which is a planar model of a grid process, the autonomous mobile device's center of motion is the origin C, RmaxThe length and width of each grid cell are d for the maximum shooting distance of the depth cameraxAnd dyThe center of gravity P (x) of each three-dimensional voxelp，yp，zp) Assigned to each grid (row, col) according to the following formula:
wherein Rdown () is a rounding-down function, the gravity center of each input three-dimensional voxel is subjected to rasterization processing and then is distributed into a unique corresponding grid, and finally, the projection points of the gravity centers of all three-dimensional voxels are summarized into the grid.
In an alternative embodiment, the acquiring three-dimensional voxels in a motion direction of the autonomous mobile device in the target space and generating a grid projection model including the three-dimensional voxels may include: acquiring a colored point cloud of the autonomous mobile equipment in the motion direction of a target space; carrying out downsampling on the colored point cloud to obtain a three-dimensional voxel; and generating a grid projection model according to the barycentric coordinates of the three-dimensional voxels.
The colored point cloud may be point cloud data with color information, and the downsampling may be any downsampling method in the prior art, which is not limited in the embodiment of the present application. And dividing all colored point clouds into a plurality of three-dimensional voxels through downsampling processing, and representing all colored point clouds in each three-dimensional voxel by using the gravity center of the three-dimensional voxel. When in down-sampling, all three-dimensional voxels need to be checked, and the voxel gravity center replaces a point set in the three-dimensional voxels, so that the shape of the point cloud is ensured, the data volume of point cloud data needing to be processed is reduced, and the complexity of calculation is reduced. And then, by establishing a grid projection model, the gravity center of each three-dimensional voxel with color information is placed in the grid projection model.
And S120, aiming at each grid, determining non-ground voxels in the target space according to the barycentric coordinates of the three-dimensional voxels in the grid.
The non-ground voxels may be voxels in the retargeted space that represent objects above ground, for example, when the logistics/warehousing robot identifies goods in a warehouse, three-dimensional voxels corresponding to objects above ground such as goods or logistics pallets all belong to the non-ground voxels.
Since the height information (i.e., the coordinate value in the vertical direction) is included in the barycentric coordinates of the three-dimensional voxels, voxels higher than the ground height can be determined from the barycentric coordinates of each three-dimensional voxel.
And S130, determining the front surface image of the target object according to the non-ground voxels.
Wherein the front surface of the target object may be a side facing the autonomous mobile device. And generating a front surface image of the target object according to the coordinate data of the gravity centers of the non-ground voxels. For example, the center of gravity of the non-ground voxels may be filtered or filtered to select the center of gravity of the voxels on the front surface of the target object, and then the corresponding front surface image may be generated. The filtering or screening operation is only for illustration and should not be construed as a limitation on the technical solutions of the embodiments of the present application.
In an alternative embodiment, the determining the front surface image of the target object from the non-ground voxels may include: performing point cloud segmentation on non-ground voxels to determine front surface voxels of the target object; from the anterior surface voxels, an anterior surface image is determined.
The point cloud segmentation may be a segmentation screening of all non-ground voxels, from which front surface voxels of the target object are determined, and then a front surface image of the target object is generated through the center of gravity of the front surface voxels. Certainly, the front surface image may be a two-dimensional image based on a point cloud (voxel gravity center), or may be a two-dimensional image obtained by fusing a point cloud and an actual image, which is not limited in this embodiment of the present application.
In an alternative embodiment, the point cloud segmentation of the non-ground voxels to determine front surface voxels of the target object may include: performing preset point cloud segmentation on non-ground voxels in a grid projection model; the preset point cloud segmentation comprises at least one of color segmentation, cluster segmentation and plane segmentation; and mapping the point cloud segmentation result to a target space to obtain a front surface voxel.
The point cloud segmentation process for the barycenter of the non-ground voxels may be performed in a grid projection model, and it can be understood that the barycenter of each voxel is projected in a two-dimensional plane. When the colored point cloud obtained initially is down-sampled and converted into voxels, color information is given to each voxel at a time, and thus each voxel has a center of gravity with color information. Non-ground voxels may be screened by a pre-defined color segmentation algorithm. For example, in a warehouse, a logistics tray needs to be identified, the color of the logistics tray (corresponding to the target item in the embodiment of the present application) may be determined in advance, and a technician performs color space conversion on the obtained point cloud in a test, and converts an RGB (Red Green Blue) space into an HSV (Hue failure Value) space, so as to better represent the color of the logistics tray. It is assumed that after a large number of experiments, the threshold range of the color of the logistics tray is: 90-H-Ap-110, S-153, 51-V-Ap-153 (a blue color). And segmenting the gravity centers of all non-ground voxels through the color threshold value under the preset HSV color space so as to obtain the voxel gravity center of the logistics tray.
The method may also be a cluster segmentation method, and the cluster segmentation may be performed by finding a description object (i.e., an object in the target space) and information related to the description object in the data, and segmenting the point cloud data into different classes or clusters according to a preset standard, where the preset standard may be determined by a relevant technician according to manual experience or a large number of tests. For example, clustering segmentation is performed using euclidean distance as a distance index. The three-dimensional point cloud (voxel gravity center) acquired by the autonomous mobile device is composed of a mass of scattered and disordered discrete points, and a topological relation between the discrete points, such as an octree or a k-dimensional tree, can be created by using any topological relation establishment method in the prior art.
A planar segmentation method may also be used, which also needs to be implemented in the grid projection model. After the gravity center point of the three-dimensional voxel is projected to a two-dimensional plane, plane segmentation is carried out by adopting a straight line fitting mode. A corresponding distance threshold value can be set in the algorithm parameters according to the point cloud resolution (voxel gravity center resolution), and when the distance between a certain point and the fitted straight line is less than or equal to the distance threshold value, the point is regarded as an inner point of the fitted straight line; conversely, when the distance between a certain point and the fitted straight line is greater than the distance threshold value, the point is regarded as the outer point of the fitted straight line. Thereby determining the point cloud straight line of the target object after being projected to the two-dimensional plane. The straight line fitting method may use, for example, a RANSAC (Random Sample Consensus) algorithm.
The point cloud segmentation result may be a calculation result of the aforementioned point cloud segmentation method. It should be noted that, since some segmentation methods need to perform calculation based on the two-dimensional plane in the grid projection model, after the segmentation result of the voxel gravity center on the two-dimensional plane is obtained through calculation, the point cloud segmentation result may be mapped back to the target space (three-dimensional space), thereby helping to determine the front surface voxel. For example, the front surface of the target object on the two-dimensional plane is obtained as a line segment during plane segmentation. And remapping all points on the line segment back to a three-dimensional space, so that the gravity center distribution condition of the front surface voxel of the target object can be obtained, and the front surface picture can be determined. The method for dividing and mapping the point cloud back to the three-dimensional space has the advantages that the identification precision of the target object can be improved, the method is universal, and the method can be effective in different target objects.
And S140, positioning the target object according to the pixel boundary in the front surface image.
The pixel boundary may be a contour boundary of an object point cloud (three-dimensional voxel gravity center) in the front surface image obtained in the foregoing step. The shape of the target object, the relative distance from the autonomous mobile apparatus, the relative position, and the like may be determined according to the pixel boundary, thereby locating the target object.
According to the technical scheme of the embodiment of the application, the three-dimensional voxels are processed from the grid projection model, the target object in the non-ground voxels is determined, and then the positioning of the target object is determined according to the front surface image of the target object. By the method, point clouds influencing target identification can be screened out, interference of influencing factors in a target environment is reduced, the identification accuracy of the target object is improved, and the universality of target object positioning can be improved due to the identification requirements of various different target objects.
Example two
Fig. 2A is a flowchart of an object positioning method according to a second embodiment of the present application, where the present embodiment further details the positioning operation of the target object based on the foregoing embodiment. As shown in fig. 2A, the method includes:
s210, three-dimensional voxels in the movement direction of the autonomous mobile device in the target space are obtained, and a grid projection model comprising the three-dimensional voxels is generated.
And S220, aiming at each grid, determining non-ground voxels in the target space according to the barycentric coordinates of the three-dimensional voxels in the grid.
In an alternative embodiment, the determining, for each grid, a non-ground voxel in the target space according to the barycentric coordinates of a three-dimensional voxel in the grid may include: for each grid, determining basic attribute information of the three-dimensional voxels in the grid according to the barycentric coordinates of the three-dimensional voxels in the grid; wherein the basic attribute information comprises at least one of the number of barycenters, the average height, the height variance and the maximum height difference; and determining non-ground voxels in the target space according to the basic attribute information of the three-dimensional voxels.
Wherein the basic attribute information includes at least one of the number of barycenters, the average height, the height variance, and the maximum height difference. The number of centers of gravity is the number of projected three-dimensional voxel centers of gravity within a grid. The mean height is then the average of the heights of the centers of gravity of all three-dimensional voxels in a grid. The height variance is also the variance of the height of the center of gravity of the associated three-dimensional voxel in a grid. The maximum height difference refers to the height difference of two barycenters with the highest value and the lowest value in the barycenters of all three-dimensional voxels in one grid. According to at least one of the four indexes, whether the gravity center of the three-dimensional voxel included in the grid is a non-ground voxel can be judged. Since the three-dimensional voxels of the object in the target space are all projected onto the two-dimensional plane, it can be understood that the number of points is approximately constant when points (three-dimensional voxel gravity centers) on a plane of the object parallel to the horizontal plane are projected onto the two-dimensional plane; however, when points on a plane of the object perpendicular to the horizontal plane are projected onto the two-dimensional plane, the number of points in the grid is greater than the number of points parallel to the horizontal plane. Therefore, the three-dimensional voxel gravity center of the three-dimensional surface of the object can be determined by the number of gravity centers in the grid.
Similarly, if the average value of the height of the center of gravity of each three-dimensional voxel in one grid is greater than the ground height (which may be set to 0 when the grid projection model is built) and exceeds a preset average height threshold, it may be determined that a three-dimensional surface of an object exists in the grid. The preset average height threshold is to prevent errors and can be determined by a person skilled in the art based on manual experience or a lot of experiments. The height variance reflects the height change of the gravity centers of all three-dimensional voxels in a grid, if the height variance is large, the fluctuation of the gravity center height of each three-dimensional voxel in the grid can be considered to be obvious, and then the existence of a three-dimensional surface of an object in the grid can be judged; if the height variance is small, it is considered that the height of the center of gravity of each three-dimensional voxel in the grid does not fluctuate significantly, and it is determined that there is no solid surface of the object in the grid. The maximum height difference can represent the fluctuation of the gravity center height of the three-dimensional voxels in the grid, if the maximum height difference exceeds a height difference threshold value, the gravity center height fluctuation of each three-dimensional voxel in the grid can be considered to be obvious, and the three-dimensional surface of the object in the grid can be judged; if the maximum height difference is small (for example, within a height difference threshold range), it is considered that the height of the center of gravity of each three-dimensional voxel in the grid does not significantly fluctuate, and it is determined that there is no solid surface of the object in the grid.
For example, the basic attribute information of the three-dimensional voxel gravity center in each grid cell can be calculated by: the gravity center point number, the average height, the (maximum) height difference and the height variance are selected, and a proper parameter setting threshold value is selected from the gravity center number, the average height, the (maximum) height difference and the height variance to be used for filtering out ground points in the voxel gravity center.
Taking i as the row number of the grid cell, j as the column number of the grid cell, each grid cell can be labeled ceil (i, j), and the number of points stored in ceil (i, j) is recorded as sizeceil(i,j)The average height of ceil (i, j) is calculated as follows:
where k is a certain point in ceil (i, j), ZkExpressed as a height value (i.e., Z-axis directional coordinate) for the store. The operating environment of an autonomous mobile device is generally a flat ground, and therefore the average height of the grid storing the point cloud (voxel center of gravity) of the object is generally large.
The following formula can be used to calculate the (maximum) height difference of the midpoint of ceil (i, j):
Zdiff(i,j)＝zmax-zmin；
wherein, ZmaxAnd ZminRespectively representing the maximum value and the minimum value of the Z-direction coordinate value of the gravity center of each voxel in the grid ceil (i, j). Similar to the above criteria, since autonomous mobile devices primarily move on flat ground, it is common to have a grid storing non-ground pointsA large height difference.
In order to better represent the height distribution of the point cloud in each grid, the height variance, sigma, of the grid can be introduced2(i, j) is the variance of the height of the center of gravity of each voxel in grid ceil (i, j):
in general, the intra-grid height variance information of the voxel barycenter of the storage object solid plane is large, and the intra-grid variance information of the storage ground point is small. Size in combination with the aboveceil(i,j)And the above three grid unit attributes are integrated, the four thresholds can be set respectively, and some thresholds can be selected for ground point filtering. Because the segmentation indexes are comprehensive, a good segmentation effect can be realized without setting threshold parameters according to strict indexes, and the algorithm has low dependence on the parameters.
Therefore, according to at least one of the four ways or the free combination, whether the object exists in the grid can be accurately and effectively judged, the accuracy and the efficiency of object identification are improved, and a basis is provided for the identification of the front surface of the subsequent target object.
And S230, determining the front surface image of the target object according to the non-ground voxels.
And S240, determining the pixel boundary of the target object in the front surface image according to the front surface image.
The voxel gravity center of the front surface of the object obtained after the processing in the previous step can be converted into a gray-scale image, and the gray-scale image is subjected to binarization processing to convert the gray-scale image of the front surface of the object into a black-and-white image. For example, a segmentation threshold is set, and for pixels with a gray value less than or equal to the segmentation threshold, the pixels can be determined as background pixels in the image, and the background pixels are converted into black pixels with a gray value of 0; for the pixels with the gray value larger than the segmentation threshold, the pixels can be determined as the pixels of the target object, and the pixels of the target object are converted into white pixels with the gray value of 255.
Optionally, morphological processing may be performed on the black-and-white image (binary image) to determine a set of pixel points that conform to the shape of the target object. As shown in fig. 2B, taking the identification of the logistics tray as an example, fig. 2B shows the identification result of the binary image of the front surface of the logistics tray. The morphological processing may be any morphological calculation method in the prior art, for example, a corrosion-expansion method, which is not limited in the embodiments of the present application.
Further, contour calculation can be performed on pixel points in the binary image. Traversing the pixels in each row from left to right in a row unit from top to bottom in the pixel in the whole image, performing preset operation (for example, first-order difference operation), extracting the pixels with the pixel gray value difference of 255 between adjacent points, and putting the pixels into a set, namely contour points, thereby determining the pixel boundary between the target object and the background, as shown in fig. 2C.
And S250, determining the center of the front surface of the target object according to the pixel boundary.
The front surface center may be a center of a width of the front surface of the target object, the center of the lateral width of the target object being determined by pixel boundaries in the binary image to assist the autonomous mobile device in determining the location of the target object.
In an alternative embodiment, if the target object is a logistics tray, before determining the center of the front surface of the target object according to the pixel boundary, the method may further include: determining the tray type of the logistics tray according to the number of the pixel boundaries; determining the size of the logistics tray according to the type of the tray; accordingly, the determining the center of the front surface of the target object according to the pixel boundary may include: and determining the center of the front surface of the target object according to the size of the logistics tray.
Taking the target object as the logistics tray as an example, the pixel boundary in the image of the front surface of the logistics tray is determined in the foregoing step, and the pixel boundary is emphasized on the transverse profile. It can be understood that there is a certain noise in the contour point set due to the influence of interference factors such as sundries, light or goods, as shown in fig. 2D. For the logistics tray, the stable and quantifiable features are the columns and the holes of the logistics tray, so that the number of the columns and the number of the holes of the logistics tray can be determined through the number of the pixel boundaries. The tray type of the logistics tray is thus determined according to the number of pixel boundaries, and may include, but is not limited to, single-well trays, which have 4 pixel boundaries, and double-well trays, which have 6 pixel boundaries, for example. The specific type of the single-hole tray or the double-hole tray, such as a square-hole tray or a trapezoidal-hole tray, may also be determined by a preset image processing algorithm according to the shape of the pixel boundary. In addition, when the pixel points of the adjacent pixel boundaries are judged, a certain threshold value can be set to ensure that the distance between the pixel points of the adjacent pixel boundaries in the same row cannot be too close or too far.
After the tray type is identified, the size of the currently identified tray can be determined according to the preset incidence relation between the tray type and the tray size, and the center of the front surface of the logistics tray is determined according to the tray size, so that a foundation is provided for positioning the logistics tray.
And S260, determining the positioning of the target object in the target space according to the center of the front surface of the target object.
After the parameters of the front surface center are obtained in the previous step, the positioning of the target object is determined according to the coordinates of the voxel gravity center (or point cloud data) corresponding to the front surface center in the target space.
According to the technical scheme, the center of the front surface is determined through the pixel boundary of the front surface image of the target object, so that the autonomous mobile equipment is assisted in object positioning. The method has the advantages that the type, the size and the like of the logistics tray can be simply and efficiently determined through the pixel boundary, and the logistics tray identification efficiency is improved.
EXAMPLE III
Fig. 3 is a schematic structural diagram of an object positioning device according to a third embodiment of the present application. As shown in fig. 3, the object positioning device 300 includes:
a grid projection generating module 310, configured to obtain three-dimensional voxels of the autonomous mobile device in a motion direction in the target space, and generate a grid projection model including the three-dimensional voxels;
a non-ground voxel determining module 320, configured to determine, for each grid, a non-ground voxel in the target space according to barycentric coordinates of three-dimensional voxels in the grid;
a front surface image determination module 330 for determining a front surface image of the target object based on the non-ground voxels;
and the target object positioning module 340 is configured to position the target object according to the pixel boundary in the front surface image.
According to the technical scheme of the embodiment of the application, the three-dimensional voxels are processed from the grid projection model, the target object in the non-ground voxels is determined, and then the positioning of the target object is determined according to the front surface image of the target object. Doing so can sieve out the some cloud that influences target identification, reduce the interference of influence factor among the target environment, improve target object's the identification degree of accuracy, but also can improve target object location's commonality because of the target object identification demand to various differences.
In an alternative embodiment, the target object locating module 340 may include:
a pixel boundary determining unit for determining a pixel boundary of the target object in the front surface image based on the front surface image;
a front surface center determining unit for determining a front surface center of the target object based on the pixel boundary;
and the positioning determining unit is used for determining the positioning of the target object in the target space according to the center of the front surface of the target object.
In an alternative embodiment, the object positioning device 300 may further include:
the tray type determining module is used for determining the tray type of the logistics tray according to the number of the pixel boundaries;
the tray size determining module is used for determining the size of the logistics tray according to the type of the tray;
accordingly, the front surface center determining unit may be specifically configured to: and determining the center of the front surface of the target object according to the size of the logistics tray.
In an alternative embodiment, the non-ground voxel determination module 320 may include:
a voxel information determining unit, which is used for determining the basic attribute information of the three-dimensional voxels in the grids according to the barycentric coordinates of the three-dimensional voxels in the grids aiming at each grid; wherein the basic attribute information includes at least one of the number of centers of gravity, an average height, a height variance, and a maximum height difference;
and the non-ground voxel determining unit is used for determining the non-ground voxels in the target space according to the basic attribute information of the three-dimensional voxels.
In an alternative embodiment, the front surface image determining module 330 may include:
the front surface voxel determining unit is used for performing point cloud segmentation on the non-ground voxels to determine the front surface voxels of the target object;
a front surface image determining unit for determining a front surface image based on the front surface voxels.
In an alternative embodiment, the anterior surface voxel determination unit may include:
the point cloud segmentation subunit is used for carrying out preset point cloud segmentation on the non-ground voxels in the grid projection model; the preset point cloud segmentation comprises at least one of color segmentation, cluster segmentation and plane segmentation;
and the segmentation result mapping subunit is used for mapping the point cloud segmentation result to a target space to obtain a front surface voxel.
In an alternative embodiment, the grid projection generating module 310 may include:
the colored point cloud acquisition unit is used for acquiring a colored point cloud in the motion direction of the autonomous mobile equipment in a target space;
the three-dimensional voxel determining unit is used for performing downsampling on the colored point cloud to obtain a three-dimensional voxel;
and the grid projection generating unit is used for generating a grid projection model according to the barycentric coordinates of the three-dimensional voxels.
The object positioning device provided by the embodiment of the application can execute the object positioning method provided by any embodiment of the application, and has the corresponding functional modules and beneficial effects of executing each object positioning method.
Example four
Fig. 4 shows a schematic structural diagram of an autonomous mobile device 10 that can be used to implement embodiments of the present application. An autonomous mobile device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the applications described and/or claimed herein.
As shown in fig. 4, the autonomous mobile apparatus 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, and the like, wherein the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various suitable actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from a storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data necessary for the operation of the autonomous mobile apparatus 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
A number of components in the autonomous mobile device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, or the like; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the autonomous mobile device 10 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
In some embodiments, the object-locating method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the autonomous mobile device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the object positioning method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the object localization method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on a machine, as a stand-alone software package partly on a machine and partly on a remote machine or entirely on a remote machine or server.
In the context of this application, a computer readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here may be implemented on an autonomous mobile device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user may provide input to the autonomous mobile device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.
It should be understood that various forms of the flows shown above, reordering, adding or deleting steps, may be used. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solution of the present application can be achieved, and the present invention is not limited thereto.
The above-described embodiments are not intended to limit the scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (10)
1. A method of locating an object, the method comprising:
acquiring three-dimensional voxels of the autonomous mobile device in a motion direction in a target space, and generating a grid projection model comprising the three-dimensional voxels;
for each grid, determining non-ground voxels in the target space according to the barycentric coordinates of the three-dimensional voxels in the grid;
determining a front surface image of the target object according to the non-ground voxels;
and positioning the target object according to the pixel boundary in the front surface image.
2. The method of claim 1, wherein locating the target object according to pixel boundaries in the front surface image comprises:
determining a pixel boundary of the target object in the front surface image according to the front surface image;
determining the center of the front surface of the target object according to the pixel boundary;
and determining the position of the target object in the target space according to the center of the front surface of the target object.
3. The method of claim 2, wherein if the target object is a logistics tray, before the determining the center of the front surface of the target object according to the pixel boundaries, the method further comprises:
determining the tray type of the logistics tray according to the number of the pixel boundaries;
determining the size of the logistics tray according to the tray type;
correspondingly, the determining the front surface center of the target object according to the pixel boundary includes:
and determining the center of the front surface of the target object according to the size of the logistics tray.
4. The method according to any one of claims 1-3, wherein the determining, for each grid, non-ground voxels in the target space based on coordinates of a center of gravity of the three-dimensional voxels within that grid comprises:
for each grid, determining basic attribute information of the three-dimensional voxels in the grid according to the barycentric coordinates of the three-dimensional voxels in the grid; wherein the basic attribute information includes at least one of a number of centers of gravity, an average height, a height variance, and a maximum height difference;
and determining non-ground voxels in the target space according to the basic attribute information of the three-dimensional voxels.
5. The method according to any one of claims 1-3, wherein determining a front surface image of the target object from the non-ground voxels comprises:
performing point cloud segmentation on the non-ground voxels to determine front surface voxels of the target object;
determining the anterior surface image from the anterior surface voxels.
6. The method of claim 5, wherein the point cloud segmentation of the non-ground voxels, determining front surface voxels of the target object, comprises:
performing preset point cloud segmentation on the non-ground voxels in the grid projection model; the preset point cloud segmentation comprises at least one of color segmentation, cluster segmentation and plane segmentation;
and mapping the point cloud segmentation result to the target space to obtain the front surface voxel.
7. The method of any one of claims 6, wherein obtaining three-dimensional voxels in a direction of motion of the autonomous mobile device in the target space and generating a grid projection model including the three-dimensional voxels comprises:
acquiring a colored point cloud of the autonomous mobile equipment in a motion direction in a target space;
down-sampling the colored point cloud to obtain the three-dimensional voxel;
and generating the grid projection model according to the barycentric coordinates of the three-dimensional voxels.
8. An object positioning device, comprising:
the grid projection generation module is used for acquiring three-dimensional voxels of the autonomous mobile equipment in the motion direction in the target space and generating a grid projection model comprising the three-dimensional voxels;
a non-ground voxel determining module, configured to determine, for each grid, a non-ground voxel in the target space according to a barycentric coordinate of the three-dimensional voxel in the grid;
a front surface image determination module for determining a front surface image of the target object based on the non-ground voxels;
and the target object positioning module is used for positioning the target object according to the pixel boundary in the front surface image.
9. An autonomous mobile device, characterized in that the autonomous mobile device comprises:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the object positioning method of any one of claims 1-7.
10. A computer-readable storage medium storing computer instructions for causing a processor to perform the object localization method according to any one of claims 1-7 when executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210861930.0A CN115272465A (en) | 2022-07-21 | 2022-07-21 | Object positioning method, device, autonomous mobile device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210861930.0A CN115272465A (en) | 2022-07-21 | 2022-07-21 | Object positioning method, device, autonomous mobile device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115272465A true CN115272465A (en) | 2022-11-01 |
Family
ID=83768056
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210861930.0A Pending CN115272465A (en) | 2022-07-21 | 2022-07-21 | Object positioning method, device, autonomous mobile device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115272465A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116383229A (en) * | 2023-06-02 | 2023-07-04 | 未来机器人（深圳）有限公司 | Tray identification method, unmanned forklift and storage medium |
-
2022
- 2022-07-21 CN CN202210861930.0A patent/CN115272465A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116383229A (en) * | 2023-06-02 | 2023-07-04 | 未来机器人（深圳）有限公司 | Tray identification method, unmanned forklift and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107610176B (en) | Pallet dynamic identification and positioning method, system and medium based on Kinect | |
| EP3168812B1 (en) | System and method for scoring clutter for use in 3d point cloud matching in a vision system | |
| US8199977B2 (en) | System and method for extraction of features from a 3-D point cloud | |
| CN113012210B (en) | Method and device for generating depth map, electronic equipment and storage medium | |
| CN112927353B (en) | Three-dimensional scene reconstruction method, storage medium and terminal based on two-dimensional target detection and model alignment | |
| CN110889828A (en) | Pallet identification method in predetermined scene, terminal device and computer storage medium | |
| CN112097732A (en) | Binocular camera-based three-dimensional distance measurement method, system, equipment and readable storage medium | |
| Yogeswaran et al. | 3d surface analysis for automated detection of deformations on automotive body panels | |
| CN115546202B (en) | Tray detection and positioning method for unmanned forklift | |
| CN115719436A (en) | Model training method, target detection method, device, equipment and storage medium | |
| CN116503803A (en) | Obstacle detection method, obstacle detection device, electronic device and storage medium | |
| CN115272465A (en) | Object positioning method, device, autonomous mobile device and storage medium | |
| Tombari et al. | Stereo for robots: quantitative evaluation of efficient and low-memory dense stereo algorithms | |
| CN111369611B (en) | Image pixel depth value optimization method, device, equipment and storage medium thereof | |
| CN114170596A (en) | Posture recognition method and device, electronic equipment, engineering machinery and storage medium | |
| US11468609B2 (en) | Methods and apparatus for generating point cloud histograms | |
| Sert | A new modified neutrosophic set segmentation approach | |
| CN116309882A (en) | Tray detection and positioning method and system for unmanned forklift application | |
| EP4202858A1 (en) | Method and device for cargo counting, computer equipment, and storage medium | |
| CN116309817A (en) | Tray detection and positioning method based on RGB-D camera | |
| CN114295053B (en) | Method and device for determining volume of material, equipment, medium and product | |
| US11900652B2 (en) | Method and computing system for generating a safety volume list for object detection | |
| CN115035492A (en) | Vehicle identification method, device, equipment and storage medium | |
| CN114972495A (en) | Grabbing method and device for object with pure plane structure and computing equipment | |
| Kovacs et al. | Edge detection in discretized range images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |