WO2022099620A1 - Procédé et appareil de segmentation de nuage de points tridimensionnel et plate-forme mobile - Google Patents

Procédé et appareil de segmentation de nuage de points tridimensionnel et plate-forme mobile Download PDF

Info

Publication number
WO2022099620A1
WO2022099620A1 PCT/CN2020/128711 CN2020128711W WO2022099620A1 WO 2022099620 A1 WO2022099620 A1 WO 2022099620A1 CN 2020128711 W CN2020128711 W CN 2020128711W WO 2022099620 A1 WO2022099620 A1 WO 2022099620A1
Authority
WO
WIPO (PCT)
Prior art keywords
point cloud
point
dimensional point
points
dimensional
Prior art date
Application number
PCT/CN2020/128711
Other languages
English (en)
Chinese (zh)
Inventor
李星河
韩路新
于亦奇
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to CN202080071116.8A priority Critical patent/CN114631124A/zh
Priority to PCT/CN2020/128711 priority patent/WO2022099620A1/fr
Publication of WO2022099620A1 publication Critical patent/WO2022099620A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition

Definitions

  • the present disclosure relates to the technical field of computer vision, and in particular, to a three-dimensional point cloud segmentation method and device, and a movable platform.
  • a path planning module on the movable platform can perform decision planning on the traveling state (eg, pose and speed) of the movable platform.
  • the point cloud acquisition device on the movable platform needs to collect the 3D point cloud of the surrounding environment, and perform point cloud segmentation to distinguish the ground and obstacles in the 3D point cloud, and further Distinguish dynamic and static objects from obstacles. Therefore, point cloud segmentation is an important part of decision planning for the driving state of the mobile platform.
  • the embodiments of the present disclosure propose a three-dimensional point cloud segmentation method and device, and a movable platform, so as to accurately perform point cloud segmentation on the three-dimensional point cloud collected by the movable platform.
  • a method for segmenting a 3D point cloud which is used to segment a 3D point cloud collected by a movable platform, the method comprising: acquiring multiple points in the 3D point cloud. search for the multiple candidate points on the v-disparity plane, and determine the target candidate point located on the road surface of the movable platform among the multiple candidate points; fit the target candidate point based on the The model of the driving road surface is obtained, and a second point cloud segmentation is performed on the three-dimensional point cloud based on the model of the driving road surface on the u-disparity plane to obtain a point cloud segmentation result.
  • a three-dimensional point cloud segmentation device including a processor, the three-dimensional point cloud segmentation device is configured to perform point cloud segmentation on a three-dimensional point cloud collected by a movable platform, and the processing The device is configured to perform the following steps: acquiring multiple candidate points in the three-dimensional point cloud; searching the multiple candidate points on the v-disparity plane, and determining that the multiple candidate points are located on the movable platform for driving target candidate points on the road surface; fit a model of the driving road surface based on the target candidate points, and perform a second point cloud segmentation on the three-dimensional point cloud based on the model of the driving road surface on the u-disparity plane.
  • a movable platform which is characterized by comprising: a casing; a point cloud collecting device, disposed on the casing, for collecting a three-dimensional point cloud; and a three-dimensional point cloud A dividing device, which is arranged in the casing, is used for executing the method described in any embodiment of the present disclosure.
  • a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the method described in any of the embodiments of the present disclosure.
  • the candidate points are searched on the v-disparity plane, the target candidate points located on the road surface of the movable platform among the plurality of candidate points are determined, and then based on the target candidate points
  • the model of the driving road surface is combined, and the model is used as the benchmark for point cloud segmentation, and the second point cloud segmentation is performed on the three-dimensional point cloud based on the model of the driving road surface on the u-disparity plane, which improves the accuracy of point cloud segmentation.
  • the accuracy makes it possible to accurately segment the areas that are difficult to segment, such as slopes and distances in the 3D point cloud.
  • Figure 1 is a schematic diagram of a point cloud segmentation process of some embodiments.
  • FIG. 2 is a schematic diagram of a decision-making planning process during travel of a mobile platform according to some embodiments.
  • FIG. 3 is a flowchart of a point cloud segmentation method according to an embodiment of the present disclosure.
  • 4A and 4B are schematic diagrams of the uvd coordinate system according to an embodiment of the present disclosure, respectively.
  • FIG. 5 is a schematic diagram of a projection process of a u-disparity plane according to an embodiment of the present disclosure.
  • FIG. 6 is a schematic diagram of the relationship between parallax and depth according to an embodiment of the present disclosure.
  • FIG. 7 is a schematic diagram of a point cloud segmentation apparatus according to an embodiment of the present disclosure.
  • FIG. 8 is a schematic diagram of a movable platform according to an embodiment of the present disclosure.
  • first, second, third, etc. may be used in this disclosure to describe various pieces of information, such information should not be limited by these terms. These terms are only used to distinguish the same type of information from each other.
  • first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information, without departing from the scope of the present disclosure.
  • word "if” as used herein can be interpreted as "at the time of” or "when” or "in response to determining.”
  • a path planning module on the movable platform can be used to make decision planning on the traveling state of the movable platform.
  • point cloud segmentation is an important part of decision-making planning for the driving state of the mobile platform.
  • FIG. 1 it is a schematic diagram of a point cloud segmentation process in some embodiments.
  • a 3D point cloud can be collected by a point cloud collection device on the movable platform, and then, in step 102, for a movable platform (such as an unmanned vehicle) running on the ground, the collected 3D point cloud can be collected.
  • the point cloud is divided into ground points and non-ground points.
  • the collected 3D point cloud can be segmented to segment the 3D points in the 3D point cloud into points on the road that the mobile platform is driving on and those not driving on the mobile platform. point on the road.
  • the following description will be made by taking the driving road as the ground.
  • step 103 if a 3D point is a ground point, step 104 is performed to add a ground point label to the 3D point; otherwise, step 105 is performed to perform dynamic and static segmentation on the 3D point, that is, segment the 3D point into stationary static point and dynamic point where motion occurs.
  • step 106 if a 3D point is a static point, step 107 is performed to add a static point label to the 3D point; otherwise, step 108 is performed to add a dynamic point label to the 3D point, and in step 109 the labelled output is output.
  • 3D point cloud to downstream modules all or part of the three-dimensional points in the three-dimensional point cloud can be marked.
  • the label may include at least one of a first label used to characterize whether the 3D point is a ground point and a second label used to characterize whether the 3D point is a static point, and may also include a label used to characterize other information of the 3D point. Label.
  • the downstream module may be a planning module on a movable platform, such as an electronic control unit (Electronic Control Unit, ECU), a central processing unit (Central Processing Unit, CPU) and the like.
  • ECU Electronic Control Unit
  • CPU Central Processing Unit
  • the Planning module can make decision planning on the driving state of the movable platform based on the label of the 3D point.
  • the driving state may include at least one of a pose and a speed of the movable platform.
  • FIG. 2 it is a schematic diagram of the decision planning process of some embodiments.
  • the planning module can receive the 3D point cloud and read the tags carried in the 3D point cloud.
  • step 203 it may be determined whether the three-dimensional point in the three-dimensional point cloud is a point on the road (eg, ground) on which the movable platform travels based on the label.
  • the road eg, ground
  • step 204 identify the three-dimensional point belonging to the lane line from the ground point, and determine the posture of the movable platform according to the direction of the lane line, so that the movable platform can follow the direction of the lane line. drive in the direction.
  • step 205 is executed to determine whether the non-ground point is a static point. If yes, step 206 is executed to determine the pose of the movable platform according to the orientation of the static point.
  • step 207 is executed to determine at least one of the attitude and speed of the movable platform according to the orientation and speed of the static point. For example, if the dynamic point is on the pre-planned travel path of the movable platform, and the moving speed of the dynamic point is less than or equal to the moving speed of the movable platform, control the movable platform to slow down, or adjust the posture of the movable platform, so that the movable platform bypasses the dynamic point.
  • the movable platform can be controlled to travel at the same speed as the dynamic point.
  • point cloud segmentation is an important part of decision-making and planning for the driving state of the mobile platform, and accurate point cloud segmentation is helpful for accurate decision-making and planning of the driving state of the mobile platform.
  • the current point cloud segmentation methods are mainly based on local features. Specifically, the three-dimensional point cloud is transformed into the xyz space, and rasterization or proximity search is performed in the xyz space. Find the adjacent points around the candidate point, and determine the probability that the candidate point belongs to the ground point according to the thickness, height and normal vector of the adjacent point cloud. This method will have obvious degradation when dealing with 3D point clouds with a long distance, the segmentation accuracy is low, and it is difficult to establish a global ground model, and it is impossible to make correct judgments for non-ground planes.
  • the present disclosure provides a three-dimensional point cloud segmentation method, which is used to perform point cloud segmentation on a three-dimensional point cloud collected by a movable platform. As shown in FIG. 3 , the method includes:
  • Step 301 Obtain multiple candidate points in the 3D point cloud
  • Step 302 Search the multiple candidate points on the v-disparity plane, and determine a target candidate point located on the road surface of the movable platform among the multiple candidate points;
  • Step 303 Fit the model of the driving road surface based on the target candidate points, and perform a second point cloud segmentation on the three-dimensional point cloud based on the model of the driving road surface on the u-disparity plane to obtain a point cloud segmentation result.
  • a three-dimensional point cloud may be collected by a point cloud collection device (eg, lidar, vision sensor, etc.) on the movable platform.
  • the movable platform may be an unmanned vehicle, an unmanned aerial vehicle, an unmanned ship, a movable robot, and the like.
  • the candidate points may be some or all of the points in the three-dimensional point cloud.
  • semantic segmentation may be performed on the three-dimensional point cloud, and multiple candidate points in the three-dimensional point cloud may be acquired based on the semantic segmentation result.
  • the categories of multiple 3D points in the 3D point cloud can be obtained, such as vehicle category, traffic light category, pedestrian category, lane line category, etc.
  • candidate points can be determined based on the categories of the three-dimensional points. For example, three-dimensional points of the lane line category are determined as candidate points.
  • the 3D point cloud may also be pre-segmented on the u-disparity plane, and candidate points are determined based on the pre-segmentation result.
  • the 3D point cloud can be obtained based on the projection density of the 3D point cloud on the u-disparity plane and the first reference projection density of the plane model of the driving road on the u-disparity plane. multiple candidate points.
  • the traveling road surface of the movable platform is assumed to be a plane, a first reference projection density is determined based on the plane, and then pre-segmentation is performed based on the first reference projection density to determine candidate points on the traveling surface of the movable platform.
  • the pre-segmentation on the u-disparity plane can improve the signal-to-noise ratio of the candidate points, so that the candidate points can be selected in long-distance regions (with less signal amount), and the distance and accuracy of the search on the v-disparity plane can be improved.
  • each coordinate axis in the uvd space can be determined based on the direction of the driving road surface of the movable platform.
  • the movable platform 401 is driving on the horizontal ground 402
  • the u-axis, v-axis and d-axis ie, the disparity axis
  • the u-axis, v-axis and d-axis can be the coordinate axes on the ground that are perpendicular to the traveling direction of the movable platform, and the A coordinate axis parallel to the traveling direction of the movable platform, and a coordinate axis in the height direction of the movable platform (ie, vertically upward).
  • the movable platform 404 is a glass-cleaning robot traveling on the vertical glass plane 403
  • the u-axis, v-axis and d-axis may be the coordinate axes on the glass plane that are perpendicular to the traveling direction of the movable platform,
  • each coordinate axis of the uvd space may also point in other directions, and the specific direction may be set according to actual needs, which is not limited in the present disclosure.
  • the u-disparity plane may be pre-divided into multiple grids.
  • a first grid of the plurality of grids if the ratio of the projection density to the first reference projection density is greater than or equal to a first preset ratio, project the three-dimensional point cloud to the Points in the first pixel grid are determined as candidate points.
  • the first preset ratio is greater than 1.
  • the u-disparity plane can be pre-divided into multiple grids, and each grid can be of the same size in order to compare projection densities.
  • Each black point represents a projection point of a 3D point in the 3D point cloud on the u-disparity plane, and the number of projected points in a grid is equal to the number of 3D points in the 3D point cloud projected to the grid.
  • the projected density within a grid can be determined as the ratio of the number of projected points within the grid to the area of the grid.
  • the three-dimensional points of whose plane is parallel to the driving direction of the vehicle (that is, the direction of the disparity coordinate axis) or have a small included angle, these three-dimensional points extend along the disparity coordinate axis, and the parallax change range is large, that is, the first
  • the density of three-dimensional points within a region is low.
  • there is an obstacle in the area d3 to d4 away from the vehicle (referred to as the second area), and the plane where the obstacle is located is generally perpendicular to the driving direction of the vehicle or has a large included angle, which will hinder the vehicle from moving forward.
  • the parallax variation range of the three-dimensional points in the second area is small, and the density of the three-dimensional points in the second area is large. Therefore, as long as the first reference density of the travel plane of the movable platform is known, it can be roughly inferred whether a three-dimensional point is a point on the travel plane or a point outside the travel plane (eg, an obstacle).
  • the preset ratio ⁇ (that is, the redundancy of the segmentation) is set to a value greater than 1 here, so as to provide a certain amount of redundancy and reduce the selection error of candidate points.
  • ⁇ can be fixedly set according to the model of the vision sensor, or can also be dynamically set according to the actual application scenario. When the reliability of the vision sensor is low, ⁇ can be set to a larger value, otherwise, ⁇ can be set to a smaller value. For example, when the focal length of the vision sensor is long, or the surrounding light is dim, etc., ⁇ can be set to a larger value.
  • the first reference projection density of the plane model on the u-disparity plane is proportional to disparity values of points on the plane model.
  • the first coordinate axis may be determined according to the baseline length of the visual sensor, the ending of the first coordinate axis of the plane model in the coordinate system of the visual sensor, and the disparity value of points on the plane model. Baseline projected density. Assume that the plane model of the driving road is:
  • ⁇ is the slope of the driving road surface.
  • a ratio of the intercept to a baseline length of the vision sensor may be calculated, and a product of the ratio and a disparity value of a point on the planar model may be determined as the first reference projection density. Then the first reference projection density of the points on the plane model on the u-disparity plane can be recorded as:
  • the first coordinate axis is the coordinate axis in the height direction of the movable platform.
  • the first coordinate axis may be a vertically upward coordinate axis.
  • the first coordinate axis may be a coordinate axis in the horizontal direction.
  • the first reference projection density is only related to the intercept, baseline length and disparity value, but has nothing to do with the distance (z value) of the ground. Therefore, here, the driving road of the movable platform is assumed to be a plane, and then the first reference projection density is determined based on the plane model to segment on the u-disparity plane to determine candidate points.
  • the amount of calculation is reduced, and on the other hand, the The signal-to-noise ratio of the candidate points is improved, so that the candidate points can be selected in the case of a long distance (less signal amount), which improves the subsequent search distance and accuracy on the v-disparity plane. Areas that are difficult to divide equally can also be accurately cut out.
  • the point cloud acquisition device does not acquire a complete point cloud frame due to a clock reset or other reasons. Therefore, the acquired 3D point cloud may include both valid points and invalid points.
  • multiple candidate points may be acquired only from valid points in the three-dimensional point cloud. Among them, invalid points can be set as invalid to avoid invalid points being selected as candidate points.
  • outlier points may also be filtered out of the three-dimensional points, and multiple candidate points in the filtered three-dimensional point cloud may be obtained.
  • Outliers are points whose value range is outside the valid range. Outliers can be filtered out of 3D points by filtering.
  • preset scale transformation parameters can be obtained, and based on the scale transformation parameters, scale transformation is performed on the u-coordinate values of each 3D point in the 3D point cloud, and the scaled transformation parameters are scaled.
  • the 3D point cloud is projected onto the u-disparity plane.
  • the scale transformation parameter scale of a three-dimensional point is used to enlarge or reduce the u-coordinate value of the three-dimensional point.
  • the scale transformation parameter scale is greater than 1, it means that the u-coordinate value of the three-dimensional point is enlarged, that is, a row of projection points on the u-disparity plane is mapped to the transformed scale row.
  • the scale transformation parameter scale is less than 1, it means that the u-coordinate value of the three-dimensional point is reduced, that is, the scale row projection point on the u-disparity plane is mapped to the transformed first row.
  • the value of the scale transformation parameter of a three-dimensional point corresponds to the u-coordinate value of the three-dimensional point. For example, if the u-coordinate value of the first three-dimensional point in the three-dimensional point cloud is smaller than the first preset coordinate value, the scaling parameter of the first three-dimensional point is greater than 1. For another example, if the u-coordinate value of the first three-dimensional point in the three-dimensional point cloud is greater than or equal to the second preset coordinate value, the scaling parameter of the first three-dimensional point is less than 1. Wherein, the first preset coordinate value may be less than or equal to the second preset coordinate value.
  • the first preset coordinate value is less than the second preset coordinate value, then when the u coordinate value of the first three-dimensional point in the three-dimensional point cloud is greater than or equal to the first preset coordinate value, and When it is less than the second preset coordinate value, the scale transformation parameter of the first three-dimensional point is equal to 1.
  • multiple scaling parameters greater than 1 and/or multiple scaling parameters less than 1 may be set. For example, if the u-coordinate value of the first three-dimensional point is less than the third preset coordinate value, set the scale transformation parameter of the first three-dimensional point to the first parameter value; if the u-coordinate value of the first three-dimensional point is greater than or equal to the The third preset coordinate value is smaller than the first preset coordinate value, and the scale transformation parameter of the first three-dimensional point is set as the second parameter value. Wherein, both the first parameter value and the second parameter value are greater than 1, and the first parameter value is greater than the second parameter value, and the third preset coordinate value is less than the first preset coordinate value.
  • the scale transformation parameter of the first three-dimensional point is set to the third parameter value, and if the u-coordinate value of the first three-dimensional point is less than
  • the fourth preset coordinate value is greater than or equal to the second preset coordinate value, and the scale transformation parameter of the first three-dimensional point is set as the fourth parameter value.
  • the fourth preset coordinate value is greater than the second preset coordinate value, and the fourth parameter value is less than the third parameter value.
  • Table 1 shows the corresponding relationship between the scale parameter in some embodiments and the number of rows before and after transformation. Those skilled in the art can understand that Table 1 is only an exemplary description and is not used to limit the present disclosure.
  • the scale transformation parameter scale is 4.0
  • the projection points of 1 row before transformation are mapped to the projection points of 4 rows after transformation.
  • the projection points of the 3rd row before transformation are mapped to the projection points of the 0th row to the 3rd row after transformation
  • the projection points of the 4th row before transformation are mapped to the projection points of the 4th row to the 7th row after transformation, And so on.
  • the reason why the starting line number of the projection point before transformation is 3 is because the error corresponding to the point with too small parallax value is relatively large. Therefore, only the points with a disparity value greater than or equal to 3 are taken for processing here. Those skilled in the art can understand that the point before the third row can also be used without considering the error; in other cases, the starting row number of the projection point before transformation can also be set to a value greater than 3.
  • f is the focal length of the vision sensor used to collect the three-dimensional point cloud
  • b is the focal length of the vision sensor
  • d is the parallax value
  • z is the depth. It can be seen that the relationship between z and d is inversely proportional, as shown in Figure 6.
  • the u-coordinate value is scaled by using the scale transformation parameter. The purpose is to counteract the characteristic of nonlinear variation of depth corresponding to disparity, compress the near high-resolution area, and give full play to the sub-pixel accuracy in the distance, thereby improving the point cloud. Segmentation accuracy.
  • the number of rows of three-dimensional points after transformation is close to the number of rows before transformation, which avoids the situation that the number of rows is too large due to scale transformation, thereby greatly increasing the computing power, and realizes the computing power and the number of points.
  • the balance between cloud segmentation accuracy is very important.
  • the candidate points selected in step 301 can be projected onto the v-disparity plane.
  • the v-disparity plane is of equal scale, that is, the number of rows of projection points on the v-disparity plane.
  • the plurality of candidate points may be searched on the v-disparity plane to determine a target candidate point located on the road surface of the movable platform among the plurality of candidate points.
  • the following takes the dynamic programming method as an example to describe the process of determining the target candidate point. In practical applications, other methods may also be adopted to determine the target candidate point, which will not be described here.
  • a search cost of the candidate point may be determined, and a target candidate point may be determined from the candidate points based on the search cost of the candidate point. If the search cost of the candidate point is less than the preset cost, the candidate point is determined as the target candidate point.
  • the search cost includes a first search cost and a second search cost; wherein the first search cost is used to characterize whether a target candidate point is observed on the candidate point, and the second search cost The cost is used to characterize whether the candidate point and the neighbor target candidate point of the candidate point are smooth.
  • the density cost can be calculated as follows:
  • p is a certain point on the v-disparity image
  • cost is the density cost of the point
  • th is related to the parameters of the visual sensor.
  • the driving surface of the movable platform is the ground
  • the width of the lane line on the ground is about 3 meters
  • the 3-meter wide area captured by the vision sensor is generally one frame of 3D points.
  • the cloud image includes 5 pixels, so the value of th can be 5. In other cases, th can also be set to other values according to the actual situation.
  • a model of the driving road surface may be fitted based on the target candidate points.
  • polynomial fitting may be performed on the target candidate points based on the least squares method to obtain a polynomial model of the driving road surface.
  • the resulting model can be expressed as:
  • A, B, C, D, and E are all constants, and z is the depth.
  • the above model is only an exemplary description, and the above model can be adjusted to a cubic polynomial model or a quintic polynomial model, etc. according to actual application scenarios. Then, the slope of the model of the driving road surface may be obtained; the second reference projection density of the driving road surface on the u-disparity plane may be determined based on the slope; Second point cloud segmentation.
  • the manner of performing the second point cloud segmentation on the 3D point cloud based on the second reference projection density is similar to the foregoing manner of performing the point cloud segmentation based on the first reference projection density, that is, the second pixel grid on the u-disparity plane , if the ratio of the projection density of the 3D point cloud on the u-disparity plane to the second reference projection density is greater than or equal to a second preset ratio, project the 3D point cloud to the second pixel
  • the points in the grid are divided into target points on the road surface.
  • the second preset ratio may be set to a value greater than or equal to 1, and the second preset value may be the same as or different from the first preset value.
  • the second reference projection density may be determined based on the model of the driving road surface, the slope of the model and the depth of the driving road surface. For example, a product of the slope of the model and the depth of the travel surface may be calculated, a difference between the model of the travel surface and the product may be calculated, and a determination may be made based on the ratio of the difference to the baseline length of the vision sensor is the second reference projection density, specifically as follows:
  • the above process performs search and model fitting on the v-disparity plane, and performs point cloud segmentation on the u-disparity plane.
  • the two steps are iteratively performed.
  • the segmentation of the u-disparity plane can improve the signal-to-noise ratio of the ground candidate points, so that the The candidate points can be selected in long-distance regions (with less semaphore) to improve the search distance and accuracy on the v-disparity plane, and the resulting model can provide important information about the benchmark density for u-disparity segmentation, so that Areas that are difficult to segment, such as slopes and distances, can also be accurately cut out.
  • each 3D point in the 3D point cloud may be labeled based on the point cloud segmentation result, and one 3D point label is used to represent the category of the 3D point.
  • the categories may include a first category and a second category, wherein the first category is used to represent that the three-dimensional point belongs to a point on the road where the movable platform travels, and the second category is used to represent that the three-dimensional point belongs to a point on an obstacle.
  • the category may further include a third category, which is used to represent that the three-dimensional point does not belong to the point on the driving road, nor the point on the obstacle.
  • the points of the third category may be reflection points, or points whose category cannot be determined, and the like.
  • each 3D point in the 3D point cloud may be tagged based on the point cloud segmentation result and the height of each 3D point in the 3D point cloud.
  • the 3D points are labeled based on the height of the 3D point and the point cloud segmentation result, which improves the accuracy of the label. Specifically, if the height of a three-dimensional point is lower than the height of the driving road surface, the label of the three-dimensional point may be determined as a first label, and the first label is used to indicate that the three-dimensional point is a reflection point. If the height of a 3D point is not lower than the height of the driving road, it can be further combined with the point cloud segmentation result for labeling.
  • a first confidence level that the 3D point is a point on the driving road may be determined based on the height of the 3D point;
  • the three-dimensional point is the second confidence level of the point on the driving road surface; the three-dimensional point is labeled based on the first confidence level and the second confidence level of the three-dimensional point.
  • the height of a three-dimensional point is not lower than the height of the driving road
  • the first confidence that the three-dimensional point is a point on the driving road is lower, otherwise the first confidence is lower. high.
  • the ratio of the projected density of a three-dimensional point to the second reference projected density is larger, the second confidence level that the three-dimensional point is a point on the driving road is lower, otherwise the second confidence level is higher.
  • the label of the 3D point may be determined as the first confidence level.
  • a label the first label is used to indicate that the three-dimensional point is a point on the driving road surface.
  • the label of the 3D point can be determined to be first label.
  • the labels of the three-dimensional points may also be determined based on other methods, which will not be listed one by one here.
  • the point cloud segmentation result can be used by the planning unit on the movable platform to plan the driving state of the movable platform.
  • the planning unit can determine whether there are obstacles on the driving path based on the labels obtained from the segmentation results of the point cloud, so as to decide whether to control the speed and attitude of the movable platform to avoid obstacles.
  • the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the specific execution order of each step should be based on its function and possible Internal logic is determined.
  • An embodiment of the present disclosure further provides a point cloud segmentation device, including a processor, where the processor is configured to perform the following steps:
  • a model of the driving road surface is fitted based on the target candidate points, and a second point cloud segmentation is performed on the three-dimensional point cloud based on the model of the driving road surface on the u-disparity plane.
  • the processor is configured to: determine all 3D points in the 3D point cloud as candidate points; or perform semantic segmentation on the 3D point cloud, and obtain the 3D point cloud based on the semantic segmentation result. or based on the projection density of the 3D point cloud on the u-disparity plane and the first reference projection density of the plane model of the driving road on the u-disparity plane, obtain the 3D point Multiple candidate points in the cloud.
  • the processor is configured to: in the first pixel grid on the u-disparity plane, if the ratio of the projected density to the first reference projected density is greater than or equal to a first preset A ratio, and a point projected into the first pixel grid in the three-dimensional point cloud is determined as a candidate point, and the first preset ratio is greater than 1.
  • the first reference projection density of the plane model on the u-disparity plane is proportional to disparity values of points on the plane model.
  • the three-dimensional point cloud is acquired by a vision sensor on the movable platform; the processor is configured to: acquire the first coordinate axis of the plane model in the coordinate system of the vision sensor intercept, the first coordinate axis is the coordinate axis in the height direction of the movable platform; the intercept is determined based on the intercept, the baseline length of the vision sensor and the parallax value of the point on the plane The first reference projected density.
  • the processor is configured to: calculate a ratio of the intercept to a baseline length of the vision sensor; and determine a product of the ratio and a disparity value of a point on the plane model as the The first reference projected density.
  • the three-dimensional point cloud includes valid points and invalid points; the processor is configured to: obtain a plurality of candidate points from the valid points in the three-dimensional point cloud.
  • the processor is configured to: filter out outlier points from the three-dimensional points; and obtain a plurality of candidate points in the filtered three-dimensional point cloud.
  • the processor is further configured to: obtain preset scaling parameters; perform scaling on the u-coordinate values of each 3D point in the 3D point cloud based on the scaling parameters; The transformed three-dimensional point cloud is projected onto the u-disparity plane.
  • the scaling parameter of a 3D point corresponds to the u-coordinate value of the 3D point.
  • the scaling parameter of the first three-dimensional point is greater than 1; and/or if the three-dimensional point The u-coordinate value of the first three-dimensional point in the point cloud is greater than the second preset coordinate value, and the scale transformation parameter of the first three-dimensional point is less than 1.
  • the processor is configured to: for each candidate point in the plurality of candidate points, determine a search cost of the candidate point; determine from the candidate points based on the search cost of the candidate point target candidate point.
  • the processor is configured to: if the search cost of the candidate point is less than a preset cost, determine the candidate point as the target candidate point.
  • the search cost includes a first search cost and a second search cost; wherein the first search cost is used to characterize whether a target candidate point is observed on the candidate point, and the second search cost The cost is used to characterize whether the candidate point and the neighbor target candidate point of the candidate point are smooth.
  • the processor is configured to: perform polynomial fitting on the target candidate points based on a least squares method to obtain a polynomial model of the driving road surface.
  • the processor is configured to: obtain a slope of the model of the driving road surface; determine a second reference projection density of the driving road surface on the u-disparity plane based on the slope; The second point cloud segmentation is performed on the 3D point cloud by the two reference projection densities.
  • the processor is configured to: in the second pixel grid on the u-disparity plane, if the projection density of the three-dimensional point cloud on the u-disparity plane is the same as the second reference projection
  • the density ratio is greater than or equal to a second preset ratio, and the points projected into the second pixel grid in the three-dimensional point cloud are divided into target points on the driving road, and the second preset ratio is greater than or equal to 1.
  • the processor is configured to: determine the depth of the driving surface; and determine the second reference projected density based on a model of the driving surface, a slope of the model, and the depth of the driving surface.
  • the three-dimensional point cloud is collected by a vision sensor on the movable platform; the processor is configured to: calculate the product of the slope of the model and the depth of the driving surface; calculate the driving The difference between the model of the road surface and the product; the second reference projection density is determined based on the ratio of the difference to the baseline length of the vision sensor.
  • the processor is further configured to: label each 3D point in the 3D point cloud based on the point cloud segmentation result, and a label of a 3D point is used to represent the category of the 3D point .
  • the processor is configured to: label each 3D point in the 3D point cloud based on the point cloud segmentation result and the height of each 3D point in the 3D point cloud.
  • the processor is configured to: if the height of the three-dimensional point is lower than the height of the driving road surface, determine the label of the three-dimensional point as a first label, and the first label is used to represent the The three-dimensional point is referred to as the reflection point.
  • the processor is configured to: for each 3D point in the 3D point cloud, determine a first confidence level that the 3D point is a point on the driving road based on the height of the 3D point; A second confidence level of the three-dimensional point as a point on the driving road is determined based on the point cloud segmentation result; the three-dimensional point is labeled based on the first confidence level and the second confidence level of the three-dimensional point.
  • the processor is configured to: if at least one of the first confidence level and the second confidence level of the 3D point is greater than a preset confidence level, determine that the label of the 3D point is the first A label, the first label is used to indicate that the three-dimensional point is a point on the driving road surface.
  • the three-dimensional point cloud is acquired based on a vision sensor or lidar installed on the movable platform; and/or the point cloud segmentation result is used for the planning unit on the movable platform to pair The traveling state of the movable platform is planned.
  • FIG. 7 shows a schematic diagram of a more specific hardware structure of a data processing apparatus provided by an embodiment of this specification.
  • the apparatus may include: a processor 701 , a memory 702 , an input/output interface 703 , a communication interface 704 and a bus 705 .
  • the processor 701 , the memory 702 , the input/output interface 703 and the communication interface 704 realize the communication connection among each other within the device through the bus 705 .
  • the processor 701 can be implemented by a general-purpose CPU (Central Processing Unit, central processing unit), a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc. program to implement the technical solutions provided by the embodiments of this specification.
  • a general-purpose CPU Central Processing Unit, central processing unit
  • a microprocessor central processing unit
  • an application specific integrated circuit Application Specific Integrated Circuit, ASIC
  • ASIC Application Specific Integrated Circuit
  • the memory 702 can be implemented in the form of a ROM (Read Only Memory, read-only memory), a RAM (Random Access Memory, random access memory), a static storage device, a dynamic storage device, and the like.
  • the memory 702 may store an operating system and other application programs. When implementing the technical solutions provided by the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 702 and invoked by the processor 701 for execution.
  • the input/output interface 703 is used to connect the input/output module to realize the input and output of information.
  • the input/output/module can be configured in the device as a component (not shown in the figure), or can be externally connected to the device to provide corresponding functions.
  • the input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc.
  • the output device may include a display, a speaker, a vibrator, an indicator light, and the like.
  • the communication interface 704 is used to connect a communication module (not shown in the figure), so as to realize the communication interaction between the device and other devices.
  • the communication module may implement communication through wired means (eg, USB, network cable, etc.), or may implement communication through wireless means (eg, mobile network, WIFI, Bluetooth, etc.).
  • Bus 705 includes a path to transfer information between the various components of the device (eg, processor 701, memory 702, input/output interface 703, and communication interface 704).
  • the above-mentioned device only shows the processor 701, the memory 702, the input/output interface 703, the communication interface 704 and the bus 705, in the specific implementation process, the device may also include necessary components for normal operation. other components.
  • the above-mentioned device may only include components necessary to implement the solutions of the embodiments of the present specification, rather than all the components shown in the figures.
  • an embodiment of the present disclosure further provides a movable platform 800 , which includes a housing 801 ; a point cloud collecting device 802 , which is arranged on the housing 801 and is used to collect a three-dimensional point cloud; and a three-dimensional point cloud.
  • the dividing device 803 is arranged in the casing 801 and is used for executing the method described in any embodiment of the present disclosure.
  • the movable platform 800 may be an unmanned aerial vehicle, an unmanned vehicle, an unmanned ship, a mobile robot, etc.
  • the point cloud collection device 802 may be a visual sensor (eg, a binocular vision sensor, a trinocular vision sensor, etc.) etc.) or lidar.
  • Embodiments of the present disclosure further provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, implements the steps executed by the second processing unit in the method described in any of the foregoing embodiments.
  • Computer-readable media includes both persistent and non-permanent, removable and non-removable media, and storage of information may be implemented by any method or technology.
  • Information may be computer readable instructions, data structures, modules of programs, or other data.
  • Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
  • computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.
  • a typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media player, navigation device, email sending and receiving device, game control desktop, tablet, wearable device, or a combination of any of these devices.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

L'invention porte sur un procédé et sur un appareil de segmentation de nuage de points tridimensionnel, ainsi que sur une plate-forme mobile, ledit procédé et ledit appareil étant utilisés pour effectuer une segmentation de nuage de points sur un nuage de points tridimensionnel collecté par une plate-forme mobile. Ledit procédé consiste : à acquérir une pluralité de points candidats dans le nuage de points tridimensionnel (301) ; à rechercher la pluralité de points candidats sur un plan de disparité en v pour déterminer des points candidats cibles, situés sur une surface de route d'entraînement de la plate-forme mobile, parmi la pluralité de points candidats (302) ; et à obtenir, par ajustement, un modèle de la surface de route d'entraînement sur la base des points candidats cibles, et sur le plan de disparité en u, à effectuer une seconde segmentation de nuage de points sur le nuage de points tridimensionnel sur la base du modèle de la surface de route d'entraînement de sorte à obtenir un résultat de segmentation de nuage de points (303).
PCT/CN2020/128711 2020-11-13 2020-11-13 Procédé et appareil de segmentation de nuage de points tridimensionnel et plate-forme mobile WO2022099620A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202080071116.8A CN114631124A (zh) 2020-11-13 2020-11-13 三维点云分割方法和装置、可移动平台
PCT/CN2020/128711 WO2022099620A1 (fr) 2020-11-13 2020-11-13 Procédé et appareil de segmentation de nuage de points tridimensionnel et plate-forme mobile

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/128711 WO2022099620A1 (fr) 2020-11-13 2020-11-13 Procédé et appareil de segmentation de nuage de points tridimensionnel et plate-forme mobile

Publications (1)

Publication Number Publication Date
WO2022099620A1 true WO2022099620A1 (fr) 2022-05-19

Family

ID=81602047

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/128711 WO2022099620A1 (fr) 2020-11-13 2020-11-13 Procédé et appareil de segmentation de nuage de points tridimensionnel et plate-forme mobile

Country Status (2)

Country Link
CN (1) CN114631124A (fr)
WO (1) WO2022099620A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116147653A (zh) * 2023-04-14 2023-05-23 北京理工大学 一种面向无人驾驶车辆的三维参考路径规划方法
CN116524472A (zh) * 2023-06-30 2023-08-01 广汽埃安新能源汽车股份有限公司 一种障碍物检测方法、装置、存储介质及设备

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105740802A (zh) * 2016-01-28 2016-07-06 北京中科慧眼科技有限公司 基于视差图的障碍物检测方法和装置及汽车驾驶辅助***
CN107977654A (zh) * 2017-12-25 2018-05-01 海信集团有限公司 一种道路区域检测方法、装置及终端
CN110879991A (zh) * 2019-11-26 2020-03-13 杭州光珀智能科技有限公司 一种障碍物识别方法及***

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105740802A (zh) * 2016-01-28 2016-07-06 北京中科慧眼科技有限公司 基于视差图的障碍物检测方法和装置及汽车驾驶辅助***
CN107977654A (zh) * 2017-12-25 2018-05-01 海信集团有限公司 一种道路区域检测方法、装置及终端
CN110879991A (zh) * 2019-11-26 2020-03-13 杭州光珀智能科技有限公司 一种障碍物识别方法及***

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
IMAD BENACER ET AL.: "A novel stereovision algorithm for obstacles detection based on U-V- disparity approach", INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS (ISCAS, 27 May 2015 (2015-05-27), pages 369 - 372, XP033183134, DOI: 10.1109/ISCAS.2015.7168647 *
ZHENCHENG HU ET AL.: "U-V-disparity: an efficient algorithm for stereovision based scene analysis", PROCEEDINGS. INTELLIGENT VEHICLES SYMPOSIUM, 8 June 2005 (2005-06-08), pages 48 - 54, XP010833942, ISSN: 1931-0587, DOI: 10.1109/IVS.2005.1505076 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116147653A (zh) * 2023-04-14 2023-05-23 北京理工大学 一种面向无人驾驶车辆的三维参考路径规划方法
CN116147653B (zh) * 2023-04-14 2023-08-22 北京理工大学 一种面向无人驾驶车辆的三维参考路径规划方法
CN116524472A (zh) * 2023-06-30 2023-08-01 广汽埃安新能源汽车股份有限公司 一种障碍物检测方法、装置、存储介质及设备
CN116524472B (zh) * 2023-06-30 2023-09-22 广汽埃安新能源汽车股份有限公司 一种障碍物检测方法、装置、存储介质及设备

Also Published As

Publication number Publication date
CN114631124A (zh) 2022-06-14

Similar Documents

Publication Publication Date Title
US11506769B2 (en) Method and device for detecting precision of internal parameter of laser radar
JP2023523243A (ja) 障害物検出方法及び装置、コンピュータデバイス、並びにコンピュータプログラム
US8199977B2 (en) System and method for extraction of features from a 3-D point cloud
CN113593017B (zh) 露天矿地表三维模型构建方法、装置、设备及存储介质
CN111542860A (zh) 用于自主车辆的高清地图的标志和车道创建
CN107677279A (zh) 一种定位建图的方法及***
CN111968229A (zh) 高精度地图制图方法及装置
WO2022141116A1 (fr) Procédé et appareil de segmentation de nuage de points tridimensionnel et plateforme mobile
US20230042968A1 (en) High-definition map creation method and device, and electronic device
WO2022099620A1 (fr) Procédé et appareil de segmentation de nuage de points tridimensionnel et plate-forme mobile
WO2024149060A1 (fr) Procédé et appareil de détection d'espace libre et de bord de route, et dispositif associé
CN114325634A (zh) 一种基于激光雷达的高鲁棒性野外环境下可通行区域提取方法
CN113804100B (zh) 确定目标对象的空间坐标的方法、装置、设备和存储介质
WO2022126380A1 (fr) Procédé et appareil de segmentation de nuage de points tridimensionnel et plateforme mobile
CN113822332A (zh) 路沿数据标注方法及相关***、存储介质
CN113838129A (zh) 一种获得位姿信息的方法、装置以及***
CN113111787A (zh) 目标检测方法、装置、设备以及存储介质
CN114384486A (zh) 一种数据处理方法及装置
CN115239899B (zh) 位姿图生成方法、高精地图生成方法和装置
CN115937449A (zh) 高精地图生成方法、装置、电子设备和存储介质
CN112835063B (zh) 物体动静属性的确定方法、装置、设备及存储介质
Madake et al. Visualization of 3D Point Clouds for Vehicle Detection Based on LiDAR and Camera Fusion
CN112907659B (zh) 移动设备定位***、方法及设备
CN112183378A (zh) 一种基于颜色和深度图像的道路坡度估计方法及装置
EP3944137A1 (fr) Procédé et appareil de positionnement

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20961176

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20961176

Country of ref document: EP

Kind code of ref document: A1