WO2022126380A1 - Three-dimensional point cloud segmentation method and apparatus, and movable platform - Google Patents

Three-dimensional point cloud segmentation method and apparatus, and movable platform Download PDF

Info

Publication number
WO2022126380A1
WO2022126380A1 PCT/CN2020/136546 CN2020136546W WO2022126380A1 WO 2022126380 A1 WO2022126380 A1 WO 2022126380A1 CN 2020136546 W CN2020136546 W CN 2020136546W WO 2022126380 A1 WO2022126380 A1 WO 2022126380A1
Authority
WO
WIPO (PCT)
Prior art keywords
point cloud
motion
frame
grid
movable platform
Prior art date
Application number
PCT/CN2020/136546
Other languages
French (fr)
Chinese (zh)
Inventor
李星河
葛宏斌
邱凡
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2020/136546 priority Critical patent/WO2022126380A1/en
Priority to CN202080070567.XA priority patent/CN114556419A/en
Publication of WO2022126380A1 publication Critical patent/WO2022126380A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/215—Motion-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds

Definitions

  • the present disclosure relates to the technical field of computer vision, and in particular, to a three-dimensional point cloud segmentation method and device, and a movable platform.
  • a path planning module on the movable platform can perform decision planning on the traveling state (eg, pose and speed) of the movable platform.
  • the point cloud acquisition device on the movable platform needs to collect the 3D point cloud of the surrounding environment, and perform point cloud segmentation to distinguish the ground and obstacles in the 3D point cloud, and further Distinguish dynamic and static objects from obstacles. Therefore, point cloud segmentation is an important part of decision planning for the driving state of the mobile platform.
  • the traditional point cloud segmentation method is generally to first identify the point cloud to determine the category to which the point cloud belongs, and then determine the point cloud that may move based on the category to which the point cloud belongs, and track the point cloud that may move. Distinguish between moving point clouds and stationary point clouds. However, the reliability of point cloud segmentation in this way is low.
  • the embodiments of the present disclosure propose a three-dimensional point cloud segmentation method and device, and a movable platform, so as to reliably perform point cloud segmentation on the three-dimensional point clouds of various objects.
  • a method for segmenting a 3D point cloud is provided, which is used for segmenting a 3D point cloud collected by a movable platform.
  • the method includes: based on a pre-established motion hypothesis model Motion hypothesis, project the multi-frame 3D point cloud collected by the movable platform to a preset coordinate system, and obtain the projection density corresponding to the motion hypothesis; A matching motion hypothesis is determined in the motion hypothesis; point cloud segmentation is performed on the first 3D point cloud in the multi-frame 3D point cloud based on the matching motion hypothesis.
  • a three-dimensional point cloud segmentation device including a processor, the three-dimensional point cloud segmentation device is configured to perform point cloud segmentation on a three-dimensional point cloud collected by a movable platform, and the processing The device is configured to perform the following steps: based on the motion hypothesis in the pre-established motion hypothesis model, project the multi-frame three-dimensional point cloud collected by the movable platform to a preset coordinate system, and obtain the projection density corresponding to the motion hypothesis; Determine a matching motion hypothesis from the plurality of motion hypotheses based on the projection densities corresponding to the plurality of motion hypotheses; perform point cloud segmentation on the first 3D point cloud in the multi-frame 3D point cloud based on the matching motion hypothesis .
  • a movable platform comprising: a casing; a point cloud collecting device, provided on the casing, for collecting a three-dimensional point cloud; and a three-dimensional point cloud segmentation device, set Inside the casing, the method is used to execute the method described in any embodiment of the present disclosure.
  • a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the method described in any of the embodiments of the present disclosure.
  • Figure 1 is a schematic diagram of a point cloud segmentation process of some embodiments.
  • FIG. 2 is a schematic diagram of a decision-making planning process during travel of a mobile platform according to some embodiments.
  • FIG. 3 is a flowchart of a point cloud segmentation method according to an embodiment of the present disclosure.
  • FIG. 4 is a schematic diagram of a motion hypothesis of an embodiment of the present disclosure.
  • FIG. 5A is a schematic diagram of a grid weight map according to an embodiment of the present disclosure.
  • FIG. 5B is a schematic diagram of a mask diagram of an embodiment of the present disclosure.
  • FIG. 6 is an overall flowchart of a point cloud segmentation process according to an embodiment of the present disclosure.
  • FIG. 7 is a schematic diagram of a point cloud segmentation apparatus according to an embodiment of the present disclosure.
  • FIG. 8 is a schematic diagram of a movable platform according to an embodiment of the present disclosure.
  • first, second, third, etc. may be used in this disclosure to describe various pieces of information, such information should not be limited by these terms. These terms are only used to distinguish the same type of information from each other.
  • first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information, without departing from the scope of the present disclosure.
  • word "if” as used herein can be interpreted as "at the time of” or "when” or "in response to determining.”
  • a path planning module on the movable platform can be used to make decision planning on the traveling state of the movable platform.
  • point cloud segmentation is an important part of decision-making planning for the driving state of the mobile platform.
  • FIG. 1 it is a schematic diagram of a point cloud segmentation process in some embodiments.
  • the point cloud collection device on the movable platform can collect the 3D point cloud, and then, in step 102, for the movable platform (such as an unmanned vehicle) driving on the ground, the collected 3D point cloud can be collected.
  • the point cloud is divided into ground points and non-ground points.
  • the collected 3D point cloud can be segmented to segment the 3D points in the 3D point cloud into points on the road that the mobile platform travels on and those not driving on the mobile platform. point on the road.
  • the following description will be given by taking the driving road as the ground.
  • step 103 if a three-dimensional point is a ground point, step 104 is performed to add a ground point label to the three-dimensional point, otherwise step 105 is performed to perform dynamic and static segmentation of the three-dimensional point, that is, the three-dimensional point is divided into stationary The static point and the dynamic point where motion occurs.
  • step 106 if a 3D point is a static point, step 107 is performed to add a static point label to the 3D point; otherwise, step 108 is performed to add a dynamic point label to the 3D point, and in step 109, output the labeled 3D point cloud to downstream modules.
  • all or part of the three-dimensional points in the three-dimensional point cloud can be labeled.
  • the label may include at least one of a first label used to characterize whether the 3D point is a ground point and a second label used to characterize whether the 3D point is a static point, and may also include a label used to characterize other information of the 3D point. Label.
  • the downstream module may be a planning module on a movable platform, such as an electronic control unit (Electronic Control Unit, ECU), a central processing unit (Central Processing Unit, CPU) and the like.
  • ECU Electronic Control Unit
  • CPU Central Processing Unit
  • the Planning module can make decision planning on the driving state of the movable platform based on the label of the 3D point.
  • the driving state may include at least one of a pose and a speed of the movable platform.
  • FIG. 2 it is a schematic diagram of the decision planning process of some embodiments.
  • the planning module can receive the 3D point cloud and read the tags carried in the 3D point cloud.
  • step 203 it may be determined whether the three-dimensional point in the three-dimensional point cloud is a point on the road (eg, ground) on which the movable platform travels based on the label.
  • the road eg, ground
  • step 204 identify the three-dimensional point belonging to the lane line from the ground point, and determine the posture of the movable platform according to the direction of the lane line, so that the movable platform can follow the direction of the lane line. drive in the direction.
  • step 205 is executed to determine whether the non-ground point is a static point. If yes, step 206 is executed to determine the pose of the movable platform according to the orientation of the static point.
  • step 207 is executed to determine at least one of the attitude and speed of the movable platform according to the orientation and speed of the static point. For example, if the dynamic point is on the pre-planned travel path of the movable platform, and the moving speed of the dynamic point is less than or equal to the moving speed of the movable platform, control the movable platform to slow down, or adjust the posture of the movable platform, so that the movable platform bypasses the dynamic point.
  • the movable platform can be controlled to travel at the same speed as the dynamic point.
  • point cloud segmentation is an important part of decision-making and planning for the driving state of the mobile platform, and accurate point cloud segmentation is helpful for accurate decision-making and planning of the driving state of the mobile platform.
  • the traditional point cloud segmentation method is generally to first detect and identify the point cloud to determine the category to which the point cloud belongs, then determine the point cloud that may move based on the category to which the point cloud belongs, and track the point cloud that may move. In this way, moving point clouds and stationary point clouds are distinguished. This method is called detection-based point cloud segmentation.
  • the target detection in the image space which represents the detection result as a two-dimensional bounding box, and finally projects the three-dimensional point cloud to the image to determine whether the three-dimensional point cloud is in the two-dimensional bounding box of the image.
  • the other is the target detection in the point cloud space, which expresses the detection result as a three-dimensional bounding box, and directly judges whether the three-dimensional point cloud is in the detected three-dimensional bounding box in the three-dimensional space.
  • the above two methods are data-driven, and both need to train the detection model through the training set, and perform target detection through the detection model.
  • the detection model When encountering objects such as special-shaped cars outside the training set, the detection model often fails, thus affecting the reliability of point cloud segmentation.
  • the image-based detection method needs to additionally rely on the image for detection, which is not suitable for the use of lidar smart sensors. Once the camera and lidar are arranged far away, the projection will have a front-background deviation when the object is occluded, and the front-background will appear in the near distance. The problem of bias is particularly pronounced.
  • the accuracy of space-based detection methods is generally lower than that of image-based detection methods, and it is especially obvious in areas with sparse point clouds in the distance. To sum up, the reliability of traditional point cloud segmentation methods is low.
  • the present disclosure provides a three-dimensional point cloud segmentation method, which is used to perform point cloud segmentation on a three-dimensional point cloud collected by a movable platform. As shown in FIG. 3 , the method includes:
  • Step 301 Based on the motion hypothesis in the pre-established motion hypothesis model, project the multi-frame 3D point cloud collected by the movable platform to a preset coordinate system, and obtain the projection density corresponding to the motion hypothesis;
  • Step 302 Determine a matching motion hypothesis from the plurality of motion hypotheses based on the projection densities corresponding to the plurality of motion hypotheses;
  • Step 303 Perform point cloud segmentation on the first three-dimensional point cloud in the multi-frame three-dimensional point cloud based on the matching motion hypothesis.
  • the present disclosure utilizes the Multiple Hypothesis Tracking (MHT) technology.
  • MHT Multiple Hypothesis Tracking
  • the MHT establishes a potential tracking hypothesis tree for each candidate target, and then calculates the probability of each tracking to select the most likely tracking combination.
  • the present disclosure only relies on the three-dimensional point cloud itself for point cloud segmentation, and does not need to detect the category to which the three-dimensional point cloud belongs, so there is no dependence on image target detection, and there is no need to align with the origin of the image coordinate system to reduce occlusion deviation;
  • the disclosure does not rely on data-driven methods, so there is no risk of missing detection of shaped objects outside the data training set; the disclosure has very low requirements for point cloud density, so in areas with sparse point clouds in the distance, the method of this disclosure can also be used. It can achieve more accurate point cloud segmentation.
  • the present disclosure can process each 3D point in the 3D point cloud collected by the point cloud collection device to perform point cloud segmentation on each 3D point, or pre-segment the collected 3D point cloud (also known as ground segmentation), to determine the 3D points on the road where the mobile platform travels and the 3D points outside the road where the mobile platform travels, and then perform point cloud segmentation on the 3D points outside the road where the mobile platform travels.
  • the traveling road surface may be the ground on which the vehicle travels or the glass plane on which the mobile robot travels, or the like.
  • the pre-segmentation may be implemented by means of RANSAC ground model fitting, etc., which is not limited in the present disclosure.
  • a first label may be added to each 3D point in the 3D point cloud, which is used to indicate the 3D point in the 3D point cloud of each frame that belongs to the 3D point outside the driving surface of the movable platform Then, only the three-dimensional point carrying the first label in the three-dimensional point cloud of each frame is projected to the preset coordinate system. For three-dimensional points that do not carry the first label, processing may not be performed.
  • the motion of 3D points in two adjacent frames of 3D point clouds is not significant enough.
  • Perform frequency division processing to obtain multi-frame 3D point clouds that need to be segmented.
  • the motion saliency of the three-dimensional points in the three-dimensional point cloud of each frame for point cloud segmentation can be improved, and on the other hand, the consumption of computing power can be reduced and system resources can be saved.
  • a divide-by-two frequency can be used.
  • the timestamp of the 3D point cloud can be modulo 200ms. If the result is 0, point cloud segmentation is performed on the 3D point cloud of the frame.
  • point cloud segmentation is not performed on the 3D point cloud of the frame. Since the determination of motion attributes cannot be completed with a single frame of point cloud, it is necessary to determine which point clouds are moving in time series. Therefore, the multi-frame 3D point cloud that needs to be segmented can be added to the point cloud queue, and the matching motion hypothesis can be determined for the multi-frame 3D point cloud in the queue, so as to perform point cloud segmentation. Further, in order to improve the significance of the observation evidence, the point cloud segmentation process of the present disclosure may be performed after the point cloud queue is accumulated for a certain period of time (for example, 3 seconds). If the point cloud queue is not accumulated for a certain period of time, it will continue to accumulate.
  • a certain period of time for example, 3 seconds
  • a three-dimensional point cloud may be collected by a point cloud collection device (eg, lidar, vision sensor, etc.) on the movable platform.
  • the movable platform may be an unmanned vehicle, an unmanned aerial vehicle, an unmanned ship, a movable robot, and the like.
  • the preset coordinate system may be the current vehicle body coordinate system of the movable platform, and the coordinate system takes the current position of the movable platform as the coordinate origin. Alternatively, the preset coordinate system may also be the world coordinate system or other pre-selected coordinate systems.
  • the motion hypothesis model is used to make assumptions about the motion speed of the movable platform.
  • a motion hypothesis model may include one or more motion hypotheses, and each motion hypothesis may correspond to a motion speed vector, that is, different motion hypotheses may have different motion hypotheses.
  • the magnitude of the movement speed and/or the movement direction As shown in FIG. 4, it is a schematic diagram of the motion hypothesis model of some embodiments. Among them, each ray with an arrow represents a motion hypothesis, the length of the ray represents the magnitude of the velocity, and the direction of the ray represents the direction of the velocity.
  • rays 401 to 404 in the first quadrant indicate that the velocity direction is 0° to 90°, and the velocity magnitude is positive;
  • rays 405 to 409 in the second quadrant indicate that the velocity direction is -90° to 0°, and the velocity magnitude is positive;
  • Rays in the three quadrants (not shown in the figure) represent a velocity direction of -90° to 0° and a negative velocity magnitude;
  • a ray 410 in the fourth quadrant represents a velocity direction of 0° to 90° and a negative velocity magnitude.
  • the ray 407 and the ray 408 have the same direction but different lengths, representing two motion hypotheses with the same velocity direction but different velocity magnitudes.
  • the movement in the vertical direction is considered to be zero.
  • the movement speed in the vertical direction may not be 0.
  • the motion hypothesis models in the above embodiments are only illustrative. In practical applications, the number of motion hypotheses included in the motion hypothesis model and the speed direction and size corresponding to each motion hypothesis can be determined according to actual needs. (For example, the accuracy requirements of point cloud segmentation, system computing power, etc.) are determined, which is not limited in the present disclosure. In some embodiments, at least one of the range of motion speed and the range of motion direction of each motion hypothesis in the motion hypothesis model may be determined based on characteristics of the environment in which the movable platform is located.
  • the environmental features may include features that characterize the type of environment (eg, urban environments, highway environments), features that characterize ambient lighting conditions (eg, day, night), and/or features that characterize ambient climate (eg, , sunny, foggy, blizzard) characteristics.
  • the environmental features may be determined based on road semantic information collected by the movable platform, location information of the movable platform, information received by the movable platform, and the like.
  • the speed and/or direction of the motion hypothesis included in the adopted motion hypothesis model may also be different. For example, in severe weather conditions such as heavy fog and blizzard, the velocity of the motion assumption is generally small. For another example, in a highway environment, the speed range is generally small. In other embodiments, the range of speed and direction of motion assumptions may also be determined based on the type of movable platform (eg, vehicle, drone, mobile robot, etc.).
  • each motion assumption in the motion hypothesis model has a motion speed in the range of [-40m/s, 40m/s] in the travel direction of the movable platform, perpendicular to the travel direction of the movable platform
  • the motion speed in the direction of the motion hypothesis is in the range of [-10m/s, 10m/s]; in other embodiments, the motion direction of each motion hypothesis in the motion hypothesis model is in the range of [-90°, 90°).
  • the above range can cover most motion scenarios when the movable platform is a vehicle. Of course, different motion assumptions can also be used in different scenarios.
  • the traveling process of the movable platform can be divided into several segments by time, and when the corresponding time of each segment is short enough (for example, less than or equal to 3 seconds), the movable platform can be placed in the The motion process in each time period is regarded as a uniform linear motion.
  • the uniform linear motion model the horizontal and vertical speeds of the movable platform are sampled at certain time intervals, and the motion hypothesis model shown in Figure 4 can be obtained.
  • other motion assumption models such as uniform acceleration motion model and uniform deceleration motion model can also be used to simulate the motion process of the movable platform.
  • the multi-frame 3D point cloud collected by the movable platform can be projected into a preset coordinate system based on the motion hypothesis in the motion hypothesis model to obtain the projection density corresponding to the motion hypothesis. Further, it is also possible to project the multi-frame 3D point cloud to a preset coordinate system based on the motion assumption and the pose when the movable platform collects each frame of the 3D point cloud in the multi-frame 3D point cloud.
  • Pin ,k represents the position of the projection point of the 3D point cloud of the in -th frame under the k-th motion hypothesis
  • Pin is the position of the 3D point in the 3-D point cloud of the in-th frame
  • the preset coordinate system is the acquisition of the i-th 3D point cloud.
  • odom in and odom i respectively represent the pose of the movable platform when collecting the 3D point cloud of the in-th frame and the pose of the movable platform when collecting the 3-D point cloud of the i-th frame
  • ⁇ t represents The time interval between the acquisition of the 3D point cloud of the in-th frame and the acquisition of the 3-D point cloud of the i-th frame
  • v h represents the velocity corresponding to the k-th motion hypothesis
  • -1 represents the matrix inversion operation.
  • the above process converts P in to the world coordinate system through odom in , and then passes Convert the point in the world coordinate system to the preset coordinate system, and compensate the movement of the movable platform itself through n* ⁇ t*v h , so as to obtain the position of the projected point.
  • the message synchronization mechanism commonly used in the vehicle system can be used to receive the 3D point cloud and odometry data with the same timestamp (that is, the pose data of the vehicle) synchronously, so as to provide the data for the multi-frame 3D point cloud.
  • the projection between them provides a position and attitude reference, compensating for the deviation caused by the vehicle's own motion.
  • the above transformations can be made separately for all 3D point cloud frames and all motion hypotheses, thereby completing the injection of motion hypotheses.
  • each projected point corresponds to the same point if the motion is assumed to be the same as the real motion of the movable platform.
  • the motion process of the movable platform is not completely equivalent to the uniform linear model, and due to the differences between the noise and motion assumptions and the movement mode of the movable platform, there may be certain differences between the projection points.
  • the deviation between the projection points should be small, that is, the positions of the projection points are relatively close. Therefore, a match between the motion hypothesis and the motion pattern of the movable platform can be determined based on the projected density. Therefore, in step 302, a matching motion hypothesis may be determined from the plurality of motion hypotheses based on the corresponding projection densities of the plurality of motion hypotheses.
  • the motion hypothesis with the highest projection density can be determined as the matching motion hypothesis. Further, it can also be determined whether the difference between the motion hypothesis with the largest projection density and any other projection density is greater than a preset value. If so, the motion hypothesis corresponding to the maximum projected density is determined as a matching motion hypothesis. Further, it can also be judged whether the maximum projection density is greater than the preset projection density threshold, if the maximum projection density is greater than the preset projection density threshold, and the difference between the motion hypothesis with the largest projection density and any other projection density is greater than the preset value. , the motion hypothesis corresponding to the maximum projection density is determined as a matching motion hypothesis. In this way, the saliency of the motion hypothesis can be increased, thereby improving the accuracy and reliability of point cloud segmentation. If the difference between the maximum weight and at least one other weight is not greater than a preset value, or the maximum projected density is not greater than a preset projected density threshold, it is determined that the matching motion hypothesis does not exist.
  • the preset coordinate system may be pre-divided into multiple grids, and the area and/or shape of each grid may be the same or different.
  • the preset coordinate system may be pre-divided into multiple rectangular grids with the same size.
  • the corresponding projection densities of the motion hypothesis A in each grid are obtained respectively.
  • each square represents a grid, and each number in the square represents the number of frames of the 3D point cloud with projected points in the grid.
  • the graph shown in Figure 5A is called grid weight picture.
  • the projected density can be determined based on the ratio of the number of frames of the 3D point cloud where projected points exist within the grid to the grid area. Since each grid has the same area, the matching motion hypothesis can be determined directly based on the number of frames of the 3D point cloud where the projected points exist within the grid. It should be noted that a frame of 3D point cloud may have multiple projection points in a grid. As long as there are projection points in the grid for this frame of 3D point cloud, no matter the number of projections, the grid will exist in the grid. The frame number of the 3D point cloud of the projected point is incremented by 1.
  • the projected points of 3D point cloud 1, 3D point cloud 2 and 3D point cloud 3 in grid 1 are 1, 3 and 0 respectively, then the number of frames of the 3D point cloud with projected points in grid 1 is recorded as 2.
  • counting the number of frames instead of counting the number of projection points falling into the grid, errors caused by uneven distribution of the number of 3D points in different scanned regions can be reduced.
  • a grid weight map can be generated for each motion hypothesis, and if there are H motion hypotheses, H grid weight maps are generated. For each motion hypothesis h, all 3D point cloud frames in the 3D point cloud queue can be transformed to the current frame according to the motion hypothesis h, and the number of historical 3D point cloud frames falling into each grid can be counted.
  • the motion hypothesis h is compared Close to the real motion mode of the 3D point cloud in the grid, the historical 3D point cloud corresponding to the motion hypothesis h will overlap in a similar area with a high probability, and the corresponding grid weight will become higher; otherwise, if the motion hypothesis h is equal to If the real movement mode is quite different, the historical point cloud injected by the movement hypothesis will overlap with a small probability, and the corresponding weight will be very low.
  • the matching motion hypothesis of the grid can be determined as the corresponding motion of each point in the grid.
  • the matching motion hypothesis of the 3D points can be determined as the corresponding motion of each point in the grid.
  • a mask map can also be generated based on the matching motion hypothesis.
  • the mask map and the grid weight map are of equal size.
  • Each grid in the mask map includes a grid parameter, which is used to record the matching motion hypothesis of the grid. .
  • h1 to h4 in the figure respectively represent the matching motion hypothesis of the corresponding grid, and null indicates that there is no matching motion hypothesis for the grid.
  • point cloud segmentation may be performed on the first three-dimensional point cloud in the multi-frame three-dimensional point cloud based on the mask image, that is, it is determined whether the three-dimensional point in the first three-dimensional point cloud is a dynamic point or a static point.
  • the dynamic point represents the three-dimensional point whose motion speed is not 0, and the static point represents the three-dimensional point whose motion speed is 0. If the velocity assumed by the matching motion in a grid is 0, the 3D points projected into the grid from the first 3D point cloud are divided into static points. If the speed assumed by the matching motion in a grid is not 0, the 3D points projected into the grid from the first 3D point cloud are divided into dynamic points.
  • the first three-dimensional point cloud may include some or all of the three-dimensional point cloud frames in the multi-frame three-dimensional point cloud. If there is no matching motion hypothesis in a grid, the 3D points projected into the grid from the first 3D point cloud may be divided into 3D points with unknown attributes.
  • each three-dimensional point in the first three-dimensional point cloud may be labeled.
  • the label may include at least one of numbers, letters, and symbols. Taking the label including numbers as an example, bit 1 can be used to represent a dynamic point, bit 0 can be used to represent a static point, and bit 01 can also be used to represent a 3D point with unknown properties.
  • the point cloud segmentation result can be used by the planning unit on the movable platform to plan the driving state of the movable platform.
  • the planning unit can determine the moving speed of obstacles on the driving path based on the labels obtained from the segmentation results of the point cloud, so as to decide whether to control the speed and attitude of the movable platform to avoid obstacles.
  • the point cloud segmentation result can also be output to a multimedia system on the movable platform, such as a display screen, a voice playback system, etc., for outputting multimedia prompt information to the user.
  • FIG. 6 it is an overall flowchart of a point cloud segmentation method according to an embodiment of the present disclosure.
  • multi-motion hypotheses may be generated according to scenes such as highways and city streets.
  • step 602 a frame of the three-dimensional point cloud and the pose data of the movable platform at the moment when the frame of the three-dimensional point cloud is collected may be received synchronously.
  • step 603 points on the road surface (eg, the ground) on which the movable platform travels may be removed from the received three-dimensional point cloud.
  • step 604 whether to use the 3D point cloud of this frame can be determined according to the frequency division setting. For example, if the result obtained by taking the modulo of the timestamp of the 3D point cloud of this frame and 200ms is 0, then the 3D point cloud of this frame is used, and the steps are executed. 605; otherwise, the 3D point cloud of this frame is not used, and step 606 is executed.
  • step 605 the 3D point cloud can be added to the point cloud queue.
  • step 606 it is determined whether the point cloud queue reaches the minimum computable length. If yes, go to step 607, otherwise go back to step 602.
  • step 607 H motion hypotheses may be injected for the current queue.
  • each motion hypothesis is used to project the 3D point on the current frame into the preset coordinate system, and it is judged whether each grid in the preset coordinate system has a matching motion hypothesis, and based on the matching motion hypothesis of each grid, generate mask map.
  • step 609 the 3D point cloud can be traversed, and the motion mask image can be queried, so as to generate a label for each 3D point in the 3D point cloud, which is used to indicate that each 3D point is a dynamic point or a static point.
  • the labeled 3D point cloud can be output to downstream modules, eg, the planning module of the mobile platform and the multimedia system.
  • step 611 it is determined whether or not the program ends. If the procedure is not over, return to step 602 to continue point cloud segmentation.
  • the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the specific execution order of each step should be based on its function and possible Internal logic is determined.
  • An embodiment of the present disclosure further provides a point cloud segmentation device, including a processor, where the processor is configured to perform the following steps:
  • Point cloud segmentation is performed on the first three-dimensional point cloud in the multi-frame three-dimensional point cloud based on the matching motion hypothesis.
  • different motion hypotheses correspond to different motion speeds and/or motion directions.
  • the range of motion speed and/or motion direction of the motion assumption is determined based on environmental characteristics of the environment in which the movable platform is located.
  • the movement assumes that the movement speed in the travel direction of the movable platform is in the range of [-40m/s, 40m/s], and the movement speed in the direction perpendicular to the travel direction of the movable platform The movement speed is in the range of [-10m/s, 10m/s].
  • the assumed direction of motion of the motion is in the range [-90°, 90°).
  • the processor is configured to: based on the motion hypothesis and the pose when the movable platform collects each frame of the 3D point cloud in the multi-frame 3D point cloud, convert the multi-frame 3D point cloud The cloud is projected into the preset coordinate system.
  • each frame of the 3D point cloud in the multi-frame 3D point cloud includes a first label, which is used to indicate a 3D point in the 3D point cloud of each frame that belongs to the outside of the road surface of the movable platform.
  • the processor is configured to: project the three-dimensional point carrying the first label in the three-dimensional point cloud of each frame to a preset coordinate system.
  • the processor is further configured to: acquire the multi-frame original 3D point cloud collected by the movable platform; perform frequency division processing on the multi-frame original 3D point cloud to obtain the multi-frame 3D point cloud point cloud.
  • the processor is configured to: if the difference between the maximum projected density and any other projected density is greater than a preset value, determine the motion hypothesis corresponding to the maximum projected density as a matching motion hypothesis.
  • the processor is configured to determine that the matching motion hypothesis does not exist if the difference between the maximum weight and the at least one other weight is not greater than a preset value.
  • the processor is configured to: obtain the corresponding projection densities of the motion hypotheses in each grid.
  • the processor is further configured to: if a matching motion hypothesis exists in the grid, determine the matching motion hypothesis of the grid as the matching motion of the three-dimensional point corresponding to each point in the grid Suppose.
  • the processor is configured to: after projecting the multi-frame 3D point cloud into a preset coordinate system based on the motion hypothesis, obtain a 3D point cloud with projected points in the grid. number of frames; determining the ratio of the number of frames to the area of the grid as the corresponding projected density of the motion hypothesis within the grid.
  • the processor is configured to: if the velocity of the matching motion hypothesis in a grid is 0, segment the 3D points projected into the grid from the first 3D point cloud into static points; And/or if the velocity assumed by the matching motion in a grid is not 0, the 3D points projected into the grid from the first 3D point cloud are divided into dynamic points.
  • the processor is configured to: if there is no matching motion hypothesis in a grid, segment the 3D points from the first 3D point cloud projected into the grid into points with unknown attributes.
  • the processor is further configured to: label each 3D point in the first 3D point cloud based on the point cloud segmentation result.
  • the three-dimensional point cloud is acquired based on a visual sensor or lidar installed on the movable platform; and/or a point cloud segmentation result obtained by performing point cloud segmentation on the first three-dimensional point cloud
  • the planning unit on the movable platform plans the driving state of the movable platform.
  • FIG. 7 shows a schematic diagram of a more specific hardware structure of a data processing apparatus provided by an embodiment of this specification.
  • the apparatus may include: a processor 701 , a memory 702 , an input/output interface 703 , a communication interface 704 and a bus 705 .
  • the processor 701 , the memory 702 , the input/output interface 703 and the communication interface 704 realize the communication connection among each other within the device through the bus 705 .
  • the processor 701 can be implemented by a general-purpose CPU (Central Processing Unit, central processing unit), a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc. program to implement the technical solutions provided by the embodiments of this specification.
  • a general-purpose CPU Central Processing Unit, central processing unit
  • a microprocessor central processing unit
  • an application specific integrated circuit Application Specific Integrated Circuit, ASIC
  • ASIC Application Specific Integrated Circuit
  • the memory 702 can be implemented in the form of a ROM (Read Only Memory, read-only memory), a RAM (Random Access Memory, random access memory), a static storage device, a dynamic storage device, and the like.
  • the memory 702 may store the operating system and other application programs. When implementing the technical solutions provided by the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 702 and invoked by the processor 701 for execution.
  • the input/output interface 703 is used to connect the input/output module to realize the input and output of information.
  • the input/output/module can be configured in the device as a component (not shown in the figure), or can be externally connected to the device to provide corresponding functions.
  • the input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc.
  • the output device may include a display, a speaker, a vibrator, an indicator light, and the like.
  • the communication interface 704 is used to connect a communication module (not shown in the figure), so as to realize the communication interaction between the device and other devices.
  • the communication module may implement communication through wired means (eg, USB, network cable, etc.), or may implement communication through wireless means (eg, mobile network, WIFI, Bluetooth, etc.).
  • Bus 705 includes a path to transfer information between the various components of the device (eg, processor 701, memory 702, input/output interface 703, and communication interface 704).
  • the above-mentioned device only shows the processor 701, the memory 702, the input/output interface 703, the communication interface 704 and the bus 705, in the specific implementation process, the device may also include necessary components for normal operation. other components.
  • the above-mentioned device may only include components necessary to implement the solutions of the embodiments of the present specification, rather than all the components shown in the figures.
  • an embodiment of the present disclosure further provides a movable platform 800 , which includes a housing 801 ; a point cloud collecting device 802 , which is arranged on the housing 801 and is used to collect a three-dimensional point cloud; and a three-dimensional point cloud.
  • the dividing device 803 is arranged in the casing 801 and is used for executing the method described in any embodiment of the present disclosure.
  • the movable platform 800 may be an unmanned aerial vehicle, an unmanned vehicle, an unmanned ship, a mobile robot, etc.
  • the point cloud collection device 802 may be a visual sensor (eg, a binocular vision sensor, a trinocular vision sensor, etc.) etc.) or lidar.
  • Embodiments of the present disclosure further provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, implements the steps executed by the second processing unit in the method described in any of the foregoing embodiments.
  • Computer-readable media includes both persistent and non-permanent, removable and non-removable media, and storage of information may be implemented by any method or technology.
  • Information may be computer readable instructions, data structures, modules of programs, or other data.
  • Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
  • computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.
  • a typical implementing device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media player, navigation device, email sending and receiving device, game control desktop, tablet, wearable device, or a combination of any of these devices.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

Provided are a three-dimensional point cloud segmentation method and apparatus, and a movable platform. The three-dimensional point cloud segmentation method and apparatus are used for performing point cloud segmentation on a three-dimensional point cloud collected by a movable platform. The method comprises: on the basis of motion hypotheses in a pre-established motion hypothesis model, projecting, into a pre-set coordinate system, a plurality of frames of three-dimensional point clouds collected by a movable platform, and acquiring a projection density corresponding to the motion hypotheses; on the basis of the projection density corresponding to the plurality of motion hypotheses, determining a matching motion hypothesis from the plurality of motion hypotheses; and on the basis of the matching motion hypothesis, performing point cloud segmentation on a first three-dimensional point cloud in the plurality of frames of three-dimensional point clouds.

Description

三维点云分割方法和装置、可移动平台Three-dimensional point cloud segmentation method and device, and movable platform 技术领域technical field
本公开涉及计算机视觉技术领域，具体而言，涉及三维点云分割方法和装置、可移动平台。The present disclosure relates to the technical field of computer vision, and in particular, to a three-dimensional point cloud segmentation method and device, and a movable platform.
背景技术Background technique
可移动平台在行驶过程中，可以通过可移动平台上的路径规划(planning)模块来对可移动平台的行驶状态(例如，位姿和速度)进行决策规划。为了使planning模块能够完成决策规划，需要由可移动平台上的点云采集装置来采集周围环境的三维点云，并进行点云分割，以区分出三维点云中的地面和障碍物，并进一步从障碍物中区分出动态对象和静态对象。因此，点云分割是对可移动平台的行驶状态进行决策规划的重要环节。During the traveling process of the movable platform, a path planning module on the movable platform can perform decision planning on the traveling state (eg, pose and speed) of the movable platform. In order to enable the planning module to complete the decision-making planning, the point cloud acquisition device on the movable platform needs to collect the 3D point cloud of the surrounding environment, and perform point cloud segmentation to distinguish the ground and obstacles in the 3D point cloud, and further Distinguish dynamic and static objects from obstacles. Therefore, point cloud segmentation is an important part of decision planning for the driving state of the mobile platform.
传统的点云分割方式一般是先对点云进行识别，以确定点云所属的类别，再基于点云所属的类别确定可能发生运动的点云，并对可能发生运动的点云进行跟踪，从而区分出运动的点云和静止的点云。然而，这种方式进行点云分割的可靠性较低。The traditional point cloud segmentation method is generally to first identify the point cloud to determine the category to which the point cloud belongs, and then determine the point cloud that may move based on the category to which the point cloud belongs, and track the point cloud that may move. Distinguish between moving point clouds and stationary point clouds. However, the reliability of point cloud segmentation in this way is low.
发明内容SUMMARY OF THE INVENTION
有鉴于此，本公开的实施例提出了三维点云分割方法和装置、可移动平台，以可靠地对各种物体的三维点云进行点云分割。In view of this, the embodiments of the present disclosure propose a three-dimensional point cloud segmentation method and device, and a movable platform, so as to reliably perform point cloud segmentation on the three-dimensional point clouds of various objects.
根据本公开实施例的第一方面，提供一种三维点云分割方法，用于对可移动平台采集到的三维点云进行点云分割，所述方法包括：基于预先建立的运动假设模型中的运动假设，将所述可移动平台采集到的多帧三维点云投影到预设坐标系下，获取所述运动假设对应的投影密度；基于多个所述运动假设对应的投影密度，从多个所述运动假设中确定匹配运动假设；基于所述匹配运动假设对所述多帧三维点云中的第一三维点云进行点云分割。According to a first aspect of the embodiments of the present disclosure, a method for segmenting a 3D point cloud is provided, which is used for segmenting a 3D point cloud collected by a movable platform. The method includes: based on a pre-established motion hypothesis model Motion hypothesis, project the multi-frame 3D point cloud collected by the movable platform to a preset coordinate system, and obtain the projection density corresponding to the motion hypothesis; A matching motion hypothesis is determined in the motion hypothesis; point cloud segmentation is performed on the first 3D point cloud in the multi-frame 3D point cloud based on the matching motion hypothesis.
根据本公开实施例的第二方面，提供一种三维点云分割装置，包括处理器，所述三维点云分割装置用于对可移动平台采集到的三维点云进行点云分割，所述处理器 用于执行以下步骤：基于预先建立的运动假设模型中的运动假设，将所述可移动平台采集到的多帧三维点云投影到预设坐标系下，获取所述运动假设对应的投影密度；基于多个所述运动假设对应的投影密度，从多个所述运动假设中确定匹配运动假设；基于所述匹配运动假设对所述多帧三维点云中的第一三维点云进行点云分割。According to a second aspect of the embodiments of the present disclosure, there is provided a three-dimensional point cloud segmentation device, including a processor, the three-dimensional point cloud segmentation device is configured to perform point cloud segmentation on a three-dimensional point cloud collected by a movable platform, and the processing The device is configured to perform the following steps: based on the motion hypothesis in the pre-established motion hypothesis model, project the multi-frame three-dimensional point cloud collected by the movable platform to a preset coordinate system, and obtain the projection density corresponding to the motion hypothesis; Determine a matching motion hypothesis from the plurality of motion hypotheses based on the projection densities corresponding to the plurality of motion hypotheses; perform point cloud segmentation on the first 3D point cloud in the multi-frame 3D point cloud based on the matching motion hypothesis .
根据本公开实施例的第三方面，提供一种可移动平台，包括：壳体；点云采集装置，设于所述壳体上，用于采集三维点云；以及三维点云分割装置，设于所述壳体内，用于执行本公开任一实施例所述的方法。According to a third aspect of the embodiments of the present disclosure, there is provided a movable platform, comprising: a casing; a point cloud collecting device, provided on the casing, for collecting a three-dimensional point cloud; and a three-dimensional point cloud segmentation device, set Inside the casing, the method is used to execute the method described in any embodiment of the present disclosure.
根据本公开实施例的第四方面，提供一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现本公开任一实施例所述的方法。According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the method described in any of the embodiments of the present disclosure.
应用本公开实施例方案，先为三维点云建立运动假设，再基于运动假设将采集到的多帧三维点云投影到预设坐标系下，即基于运动假设对三维点云的运动过程进行模拟，然后基于运动假设对应的投影密度判断建立的运动假设是否与三维点云的真实运动方式相同，从而确定匹配运动假设，进而基于匹配运动假设进行点云分割。上述方式无需识别出三维点云所属的类别，无需训练数据驱动，从而能够对任意形态的三维点云进行点云分割，提高了点云分割的可靠性。Applying the solution of the embodiments of the present disclosure, first establish a motion hypothesis for the 3D point cloud, and then project the collected multi-frame 3D point cloud into a preset coordinate system based on the motion hypothesis, that is, simulate the motion process of the 3D point cloud based on the motion hypothesis. Then, based on the projection density corresponding to the motion hypothesis, it is judged whether the established motion hypothesis is the same as the real motion of the 3D point cloud, so as to determine the matching motion hypothesis, and then perform point cloud segmentation based on the matching motion hypothesis. The above method does not need to identify the category to which the 3D point cloud belongs, and does not need to be driven by training data, so that point cloud segmentation can be performed on 3D point clouds of any shape, and the reliability of point cloud segmentation is improved.
附图说明Description of drawings
为了更清楚地说明本公开实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。In order to illustrate the technical solutions in the embodiments of the present disclosure more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative labor.
图1是一些实施例的点云分割过程的示意图。Figure 1 is a schematic diagram of a point cloud segmentation process of some embodiments.
图2是一些实施例的可移动平台行驶过程中的决策规划过程的示意图。FIG. 2 is a schematic diagram of a decision-making planning process during travel of a mobile platform according to some embodiments.
图3是本公开实施例的点云分割方法的流程图。FIG. 3 is a flowchart of a point cloud segmentation method according to an embodiment of the present disclosure.
图4是本公开实施例的运动假设的示意图。FIG. 4 is a schematic diagram of a motion hypothesis of an embodiment of the present disclosure.
图5A是本公开实施例的栅格权重图的示意图。FIG. 5A is a schematic diagram of a grid weight map according to an embodiment of the present disclosure.
图5B是本公开实施例的mask图的示意图。FIG. 5B is a schematic diagram of a mask diagram of an embodiment of the present disclosure.
图6是本公开实施例的点云分割过程的总体流程图。FIG. 6 is an overall flowchart of a point cloud segmentation process according to an embodiment of the present disclosure.
图7是本公开实施例的点云分割装置的示意图。FIG. 7 is a schematic diagram of a point cloud segmentation apparatus according to an embodiment of the present disclosure.
图8是本公开实施例的可移动平台的示意图。FIG. 8 is a schematic diagram of a movable platform according to an embodiment of the present disclosure.
具体实施方式Detailed ways
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本公开相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本公开的一些方面相一致的装置和方法的例子。Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with this disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as recited in the appended claims.
在本公开使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本公开。在本公开说明书和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to limit the present disclosure. As used in this disclosure and the appended claims, the singular forms "a," "the," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.
应当理解，尽管在本公开可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本公开范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，如在此所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various pieces of information, such information should not be limited by these terms. These terms are only used to distinguish the same type of information from each other. For example, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information, without departing from the scope of the present disclosure. Depending on the context, the word "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determining."
可移动平台在行驶过程中，可以通过可移动平台上的路径规划(planning)模块来对可移动平台的行驶状态进行决策规划。其中，点云分割是对可移动平台的行驶状态进行决策规划的重要环节。如图1所示，是一些实施例的点云分割过程的示意图。在步骤101中，可以由可移动平台上的点云采集装置采集三维点云，然后，在步骤102中，对于行驶在地面上的可移动平台(例如无人车)，可以对采集到的三维点云进行地面分割，即将三维点云中的三维点分割为地面点和非地面点。对于其他类型的可移动平台(例如可移动机器人)，可以对采集到的三维点云进行分割，以将三维点云中的三维点分割为可移动平台行驶路面上的点和不在可移动平台行驶路面上的点。为了便于描述，下文以行驶路面为地面进行说明。在步骤103中，如果一个三维点为地面点，则执行步骤104，为该三维点添加地面点标签，否则执行步骤105，对该三维点进行动静态分割，即将该三维点分割为静止不动的静态点和发生运动的动态点。在步骤 106中，如果一个三维点为静态点，则执行步骤107，为该三维点添加静态点标签，否则执行步骤108，为该三维点添加动态点标签，并在步骤109中输出带标签的三维点云至下游模块。其中，可以为三维点云中的全部或者部分三维点打标签。所述标签可以包括用于表征三维点是否为地面点的第一标签和用于表征三维点是否为静态点的第二标签中的至少一者，还可以包括用于表征三维点的其他信息的标签。During the traveling process of the movable platform, a path planning module on the movable platform can be used to make decision planning on the traveling state of the movable platform. Among them, point cloud segmentation is an important part of decision-making planning for the driving state of the mobile platform. As shown in FIG. 1 , it is a schematic diagram of a point cloud segmentation process in some embodiments. In step 101, the point cloud collection device on the movable platform can collect the 3D point cloud, and then, in step 102, for the movable platform (such as an unmanned vehicle) driving on the ground, the collected 3D point cloud can be collected. The point cloud is divided into ground points and non-ground points. For other types of mobile platforms (such as mobile robots), the collected 3D point cloud can be segmented to segment the 3D points in the 3D point cloud into points on the road that the mobile platform travels on and those not driving on the mobile platform. point on the road. For the convenience of description, the following description will be given by taking the driving road as the ground. In step 103, if a three-dimensional point is a ground point, step 104 is performed to add a ground point label to the three-dimensional point, otherwise step 105 is performed to perform dynamic and static segmentation of the three-dimensional point, that is, the three-dimensional point is divided into stationary The static point and the dynamic point where motion occurs. In step 106, if a 3D point is a static point, step 107 is performed to add a static point label to the 3D point; otherwise, step 108 is performed to add a dynamic point label to the 3D point, and in step 109, output the labeled 3D point cloud to downstream modules. Among them, all or part of the three-dimensional points in the three-dimensional point cloud can be labeled. The label may include at least one of a first label used to characterize whether the 3D point is a ground point and a second label used to characterize whether the 3D point is a static point, and may also include a label used to characterize other information of the 3D point. Label.
所述下游模块可以是可移动平台上的planning模块，例如电子控制单元(Electronic Control Unit，ECU)、中央处理器(Central Processing Unit，CPU)等。Planning模块在接收到带标签的三维点云之后，可以基于三维点的标签对可移动平台的行驶状态进行决策规划。所述行驶状态可以包括可移动平台的位姿和速度中的至少一者。如图2所示，是一些实施例的决策规划过程的示意图。在步骤201和步骤202中，planning模块可以接收三维点云并读取三维点云中携带的标签。在步骤203中，可以基于标签确定三维点云中的三维点是否为可移动平台行驶路面(例如地面)上的点。以地面点为例，如果是，则执行步骤204，从地面点中识别出属于车道线的三维点，并根据车道线的方向确定可移动平台的姿态，以使可移动平台沿着车道线的方向行驶。如果是非地面点，则执行步骤205，判断该非地面点是否为静态点。如果是，则执行步骤206，根据静态点的方位确定可移动平台的位姿。例如，判断该静态点是否处于预先规划的行驶路径上，如果是，则重新规划路径，以避免可移动平台与静态点相撞。如果该非地面点为动态点，则执行步骤207，根据该静态点的方位和速度确定可移动平台的姿态和速度中的至少一者。例如，若该动态点处于可移动平台预先规划的行驶路径上，且该动态点的移动速度小于或等于可移动平台的移动速度，则控制可移动平台减速行驶，或者调整可移动平台的姿态，以使可移动平台绕过该动态点。又例如，可以控制可移动平台按照与动态点相同的速度行驶。The downstream module may be a planning module on a movable platform, such as an electronic control unit (Electronic Control Unit, ECU), a central processing unit (Central Processing Unit, CPU) and the like. After receiving the labeled 3D point cloud, the Planning module can make decision planning on the driving state of the movable platform based on the label of the 3D point. The driving state may include at least one of a pose and a speed of the movable platform. As shown in FIG. 2 , it is a schematic diagram of the decision planning process of some embodiments. In step 201 and step 202, the planning module can receive the 3D point cloud and read the tags carried in the 3D point cloud. In step 203, it may be determined whether the three-dimensional point in the three-dimensional point cloud is a point on the road (eg, ground) on which the movable platform travels based on the label. Taking the ground point as an example, if it is, go to step 204, identify the three-dimensional point belonging to the lane line from the ground point, and determine the posture of the movable platform according to the direction of the lane line, so that the movable platform can follow the direction of the lane line. drive in the direction. If it is a non-ground point, step 205 is executed to determine whether the non-ground point is a static point. If yes, step 206 is executed to determine the pose of the movable platform according to the orientation of the static point. For example, it is judged whether the static point is on the pre-planned travel path, and if so, the path is re-planned to avoid the movable platform colliding with the static point. If the non-ground point is a dynamic point, step 207 is executed to determine at least one of the attitude and speed of the movable platform according to the orientation and speed of the static point. For example, if the dynamic point is on the pre-planned travel path of the movable platform, and the moving speed of the dynamic point is less than or equal to the moving speed of the movable platform, control the movable platform to slow down, or adjust the posture of the movable platform, so that the movable platform bypasses the dynamic point. As another example, the movable platform can be controlled to travel at the same speed as the dynamic point.
由此可知，点云分割是对可移动平台的行驶状态进行决策规划的重要环节，准确地进行点云分割有助于对可移动平台的行驶状态进行准确的决策规划。传统的点云分割方式一般是先对点云进行检测识别，以确定点云所属的类别，再基于点云所属的类别确定可能发生运动的点云，并对可能发生运动的点云进行跟踪，从而区分出运动的点云和静止的点云。这种方式称为基于检测(detection)的点云分割方式。It can be seen that point cloud segmentation is an important part of decision-making and planning for the driving state of the mobile platform, and accurate point cloud segmentation is helpful for accurate decision-making and planning of the driving state of the mobile platform. The traditional point cloud segmentation method is generally to first detect and identify the point cloud to determine the category to which the point cloud belongs, then determine the point cloud that may move based on the category to which the point cloud belongs, and track the point cloud that may move. In this way, moving point clouds and stationary point clouds are distinguished. This method is called detection-based point cloud segmentation.
目前，基于检测的点云分割方式主要有两种。一种是图像空间的目标检测，将检测结果表示为二维的包围盒，最后将三维点云投影到图像，判断三维点云是否在图像的二维包围盒中。另一种是点云空间的目标检测，将检测结果表示为三维包围盒， 直接在三维空间中判断三维点云是否在检测出的三维包围盒中。然而，上述两种方式是基于数据驱动的，都需要通过训练集来训练检测模型，并通过检测模型进行目标检测。在遇到训练集之外的异形车等物体时，检测模型往往会失效，从而影响点云分割的可靠性。此外，基于图像的检测方式需要额外依赖图像进行检测，不适用于激光雷达智能传感器使用，且相机和激光雷达一旦布置位置较远，物体遮挡时投影会出现前背景偏差，且在近处前背景偏差问题尤为显著。而基于空间的检测方式准确度一般低于基于图像的检测方式，并且在远处点云稀疏的区域尤其明显。综上所述，传统的点云分割方式的可靠性较低。At present, there are two main methods of point cloud segmentation based on detection. One is the target detection in the image space, which represents the detection result as a two-dimensional bounding box, and finally projects the three-dimensional point cloud to the image to determine whether the three-dimensional point cloud is in the two-dimensional bounding box of the image. The other is the target detection in the point cloud space, which expresses the detection result as a three-dimensional bounding box, and directly judges whether the three-dimensional point cloud is in the detected three-dimensional bounding box in the three-dimensional space. However, the above two methods are data-driven, and both need to train the detection model through the training set, and perform target detection through the detection model. When encountering objects such as special-shaped cars outside the training set, the detection model often fails, thus affecting the reliability of point cloud segmentation. In addition, the image-based detection method needs to additionally rely on the image for detection, which is not suitable for the use of lidar smart sensors. Once the camera and lidar are arranged far away, the projection will have a front-background deviation when the object is occluded, and the front-background will appear in the near distance. The problem of bias is particularly pronounced. The accuracy of space-based detection methods is generally lower than that of image-based detection methods, and it is especially obvious in areas with sparse point clouds in the distance. To sum up, the reliability of traditional point cloud segmentation methods is low.
基于此，本公开提供一种三维点云分割方法，用于对可移动平台采集到的三维点云进行点云分割，如图3所示，所述方法包括：Based on this, the present disclosure provides a three-dimensional point cloud segmentation method, which is used to perform point cloud segmentation on a three-dimensional point cloud collected by a movable platform. As shown in FIG. 3 , the method includes:
步骤301：基于预先建立的运动假设模型中的运动假设，将所述可移动平台采集到的多帧三维点云投影到预设坐标系下，获取所述运动假设对应的投影密度；Step 301: Based on the motion hypothesis in the pre-established motion hypothesis model, project the multi-frame 3D point cloud collected by the movable platform to a preset coordinate system, and obtain the projection density corresponding to the motion hypothesis;
步骤302：基于多个所述运动假设对应的投影密度，从多个所述运动假设中确定匹配运动假设；Step 302: Determine a matching motion hypothesis from the plurality of motion hypotheses based on the projection densities corresponding to the plurality of motion hypotheses;
步骤303：基于所述匹配运动假设对所述多帧三维点云中的第一三维点云进行点云分割。Step 303: Perform point cloud segmentation on the first three-dimensional point cloud in the multi-frame three-dimensional point cloud based on the matching motion hypothesis.
本公开利用了多假设跟踪(Multiple Hypothesis Tracking，MHT)技术，MHT会为每个候选目标建立一个潜在的跟踪假设树，然后，计算每一个跟踪的概率，选出最有可能的跟踪组合。本公开只依赖三维点云本身进行点云分割，不需要检测出三维点云所属的类别，因此没有对图像目标检测的依赖，更不存在和图像坐标系原点对齐减少遮挡偏差的需求；由于本公开不依赖数据驱动的方法，因此也不存在数据训练集之外异形物体漏检的风险；本公开对点云密度要求很低，因此在远处点云稀疏的区域，利用本公开的方法也能够实现较为准确的点云分割。The present disclosure utilizes the Multiple Hypothesis Tracking (MHT) technology. The MHT establishes a potential tracking hypothesis tree for each candidate target, and then calculates the probability of each tracking to select the most likely tracking combination. The present disclosure only relies on the three-dimensional point cloud itself for point cloud segmentation, and does not need to detect the category to which the three-dimensional point cloud belongs, so there is no dependence on image target detection, and there is no need to align with the origin of the image coordinate system to reduce occlusion deviation; The disclosure does not rely on data-driven methods, so there is no risk of missing detection of shaped objects outside the data training set; the disclosure has very low requirements for point cloud density, so in areas with sparse point clouds in the distance, the method of this disclosure can also be used. It can achieve more accurate point cloud segmentation.
本公开可以对点云采集装置采集到的三维点云中的每个三维点进行处理，以对所述每个三维点进行点云分割，也可以对采集到的三维点云先进行预分割(也称为地面分割)，以确定可移动平台行驶路面上的三维点和可移动平台行驶路面以外的三维点，再对可移动平台行驶路面以外的三维点进行点云分割。其中，所述行驶路面可以是车辆行驶的地面或者可移动机器人行驶的玻璃平面等。预分割可以采用RANSAC地面模型拟合等方式实现，本公开对此不做限制。对于后一种情况，可以在预分割之后， 为三维点云中的每个三维点添加第一标签，用于指示所述每帧三维点云中属于所述可移动平台行驶路面之外的三维点，然后，仅将每帧三维点云中携带所述第一标签的三维点投影到预设坐标系下。对于不携带第一标签的三维点，可以不进行处理。The present disclosure can process each 3D point in the 3D point cloud collected by the point cloud collection device to perform point cloud segmentation on each 3D point, or pre-segment the collected 3D point cloud ( Also known as ground segmentation), to determine the 3D points on the road where the mobile platform travels and the 3D points outside the road where the mobile platform travels, and then perform point cloud segmentation on the 3D points outside the road where the mobile platform travels. Wherein, the traveling road surface may be the ground on which the vehicle travels or the glass plane on which the mobile robot travels, or the like. The pre-segmentation may be implemented by means of RANSAC ground model fitting, etc., which is not limited in the present disclosure. In the latter case, after pre-segmentation, a first label may be added to each 3D point in the 3D point cloud, which is used to indicate the 3D point in the 3D point cloud of each frame that belongs to the 3D point outside the driving surface of the movable platform Then, only the three-dimensional point carrying the first label in the three-dimensional point cloud of each frame is projected to the preset coordinate system. For three-dimensional points that do not carry the first label, processing may not be performed.
在一些实施例中，由于点云采集装置采集三维点云的频率较高，相邻两帧三维点云中的三维点的运动不够显著，可以对可移动平台采集到的多帧原始三维点云进行分频处理，得到需要进行点云分割的多帧三维点云。这样，一方面可以提高进行点云分割的各帧三维点云中三维点的运动显著性，另一方面可以降低算力消耗，节约***资源。可选地，可以采用二分频。例如，可以将三维点云的时间戳对200ms取模，如果结果为0，则对该帧三维点云进行点云分割，如果结果不为0则对该帧三维点云不进行点云分割。由于运动属性的判定利用单帧点云是无法完成的，需要在时间序列上才能判定哪些点云是运动的。因此，可以需要进行点云分割的多帧三维点云加入点云队列，并为队列中的多帧三维点云进确定匹配运动假设，从而进行点云分割。进一步地，为了提高观测证据的显著性，还可以待点云队列累积到一定时长(例如，3秒)，再执行本公开的点云分割过程。如果点云队列未累积到一定时长，则继续累积。In some embodiments, due to the high frequency of collecting 3D point clouds by the point cloud collection device, the motion of 3D points in two adjacent frames of 3D point clouds is not significant enough. Perform frequency division processing to obtain multi-frame 3D point clouds that need to be segmented. In this way, on the one hand, the motion saliency of the three-dimensional points in the three-dimensional point cloud of each frame for point cloud segmentation can be improved, and on the other hand, the consumption of computing power can be reduced and system resources can be saved. Alternatively, a divide-by-two frequency can be used. For example, the timestamp of the 3D point cloud can be modulo 200ms. If the result is 0, point cloud segmentation is performed on the 3D point cloud of the frame. If the result is not 0, point cloud segmentation is not performed on the 3D point cloud of the frame. Since the determination of motion attributes cannot be completed with a single frame of point cloud, it is necessary to determine which point clouds are moving in time series. Therefore, the multi-frame 3D point cloud that needs to be segmented can be added to the point cloud queue, and the matching motion hypothesis can be determined for the multi-frame 3D point cloud in the queue, so as to perform point cloud segmentation. Further, in order to improve the significance of the observation evidence, the point cloud segmentation process of the present disclosure may be performed after the point cloud queue is accumulated for a certain period of time (for example, 3 seconds). If the point cloud queue is not accumulated for a certain period of time, it will continue to accumulate.
在步骤301中，可以由可移动平台上的点云采集装置(例如激光雷达、视觉传感器等)采集三维点云。所述可移动平台可以是无人车、无人机、无人船、可移动机器人等。预设坐标系可以是可移动平台当前的车体坐标系，该坐标系以可移动平台当前的位置为坐标原点。或者，预设坐标系也可以是世界坐标系或者预先选定的其他坐标系。In step 301, a three-dimensional point cloud may be collected by a point cloud collection device (eg, lidar, vision sensor, etc.) on the movable platform. The movable platform may be an unmanned vehicle, an unmanned aerial vehicle, an unmanned ship, a movable robot, and the like. The preset coordinate system may be the current vehicle body coordinate system of the movable platform, and the coordinate system takes the current position of the movable platform as the coordinate origin. Alternatively, the preset coordinate system may also be the world coordinate system or other pre-selected coordinate systems.
运动假设模型用于对可移动平台的运动速度进行假设，一个运动假设模型中可以包括一个或多个运动假设，每种运动假设可对应一种运动速度矢量，即，不同的运动假设可以具有不同的运动速度大小和/或运动方向。如图4所示，是一些实施例的运动假设模型的示意图。其中，每个带箭头的射线表示一个运动假设，射线的长度表示速度的大小，射线的方向表示速度的方向。例如，第一象限的射线401至404表示速度方向为0°至90°，速度大小为正；第二象限的射线405至409表示速度方向为-90°至0°，速度大小为正；第三象限的射线(图中未示出)表示速度方向为-90°至0°，速度大小为负；第四象限的射线410表示速度方向为0°至90°，速度大小为负。其中，射线407和射线408的方向相同但长度不同，表示速度方向相同但速度大小不同的两种运动假设。在可移动平台为车辆的情况下，由于车辆都是在平地上运动，因此，所有假设中，竖直方向上的运动都认为速度为0。在可移动平台为可移动机器人或者 无人机等情况下，竖直方向上的运动速度也可能不为0。The motion hypothesis model is used to make assumptions about the motion speed of the movable platform. A motion hypothesis model may include one or more motion hypotheses, and each motion hypothesis may correspond to a motion speed vector, that is, different motion hypotheses may have different motion hypotheses. The magnitude of the movement speed and/or the movement direction. As shown in FIG. 4, it is a schematic diagram of the motion hypothesis model of some embodiments. Among them, each ray with an arrow represents a motion hypothesis, the length of the ray represents the magnitude of the velocity, and the direction of the ray represents the direction of the velocity. For example, rays 401 to 404 in the first quadrant indicate that the velocity direction is 0° to 90°, and the velocity magnitude is positive; rays 405 to 409 in the second quadrant indicate that the velocity direction is -90° to 0°, and the velocity magnitude is positive; Rays in the three quadrants (not shown in the figure) represent a velocity direction of -90° to 0° and a negative velocity magnitude; a ray 410 in the fourth quadrant represents a velocity direction of 0° to 90° and a negative velocity magnitude. Among them, the ray 407 and the ray 408 have the same direction but different lengths, representing two motion hypotheses with the same velocity direction but different velocity magnitudes. In the case where the movable platform is a vehicle, since the vehicle moves on the flat ground, in all assumptions, the movement in the vertical direction is considered to be zero. In the case where the movable platform is a movable robot or a drone, the movement speed in the vertical direction may not be 0.
本领域技术人员可以理解，以上实施例中的运动假设模型仅为示例性说明，在实际应用中，运动假设模型中包括的运动假设的数量以及各个运动假设对应的速度方向和大小可以根据实际需要(例如，点云分割的准确度要求、***算力等)确定，本公开对此不做限制。在一些实施例中，运动假设模型中各个运动假设的运动速度的范围和运动方向的范围中的至少一者可以基于可移动平台所处的环境特征而确定。所述环境特征可包括用于表征环境类型(例如，城市环境、高速公路环境)的特征，用于表征环境光照情况(例如，白天、夜晚)的特征，和/或用于表征环境气候(例如，晴天、大雾、暴雪)的特征。环境特征可基于可移动平台采集到的道路语义信息、可移动平台的位置信息、可移动平台接收到的信息等确定。在可移动平台所处环境的环境特征不同的情况下，所采用的运动假设模型中包括的运动假设的速度和/或方向也可以不同。例如，在大雾、暴雪等恶劣气候环境下，运动假设的速度一般较小。又例如，在高速公路环境下，速度的范围一般较小。在另一些实施例中，还可以根据可移动平台的类型(例如，车辆、无人机、可移动机器人等)来确定运动假设的速度和方向的范围。Those skilled in the art can understand that the motion hypothesis models in the above embodiments are only illustrative. In practical applications, the number of motion hypotheses included in the motion hypothesis model and the speed direction and size corresponding to each motion hypothesis can be determined according to actual needs. (For example, the accuracy requirements of point cloud segmentation, system computing power, etc.) are determined, which is not limited in the present disclosure. In some embodiments, at least one of the range of motion speed and the range of motion direction of each motion hypothesis in the motion hypothesis model may be determined based on characteristics of the environment in which the movable platform is located. The environmental features may include features that characterize the type of environment (eg, urban environments, highway environments), features that characterize ambient lighting conditions (eg, day, night), and/or features that characterize ambient climate (eg, , sunny, foggy, blizzard) characteristics. The environmental features may be determined based on road semantic information collected by the movable platform, location information of the movable platform, information received by the movable platform, and the like. When the environmental characteristics of the environment in which the movable platform is located are different, the speed and/or direction of the motion hypothesis included in the adopted motion hypothesis model may also be different. For example, in severe weather conditions such as heavy fog and blizzard, the velocity of the motion assumption is generally small. For another example, in a highway environment, the speed range is generally small. In other embodiments, the range of speed and direction of motion assumptions may also be determined based on the type of movable platform (eg, vehicle, drone, mobile robot, etc.).
在一些实施例中，运动假设模型中的各个运动假设在所述可移动平台行驶方向上的运动速度在[-40m/s，40m/s]范围内，在与所述可移动平台行驶方向垂直的方向上的运动速度在[-10m/s，10m/s]范围内；在另一些实施例中，运动假设模型中的各个运动假设的运动方向在[-90°，90°)范围内。上述范围可以覆盖可移动平台为车辆时的大多数运动场景，当然，在不同的场景下也可以采用不同的运动假设。In some embodiments, each motion assumption in the motion hypothesis model has a motion speed in the range of [-40m/s, 40m/s] in the travel direction of the movable platform, perpendicular to the travel direction of the movable platform The motion speed in the direction of the motion hypothesis is in the range of [-10m/s, 10m/s]; in other embodiments, the motion direction of each motion hypothesis in the motion hypothesis model is in the range of [-90°, 90°). The above range can cover most motion scenarios when the movable platform is a vehicle. Of course, different motion assumptions can also be used in different scenarios.
应当说明的是，在实际应用中，可以将可移动平台的行驶过程按时间划分为若干段，当每一段对应的时间足够短(例如，小于或等于3秒)，则可以将可移动平台在每个时间段内的运动过程视为匀速直线运动。在匀速直线运动模型下，将可移动平台的横向速度和纵向速度按照一定时间间隔进行采样，可以得到图4所示的运动假设模型。在其他情况下，也可以采用匀加速运动模型、匀减速运动模型等其他运动假设模型对可移动平台的运动过程进行模拟。It should be noted that, in practical applications, the traveling process of the movable platform can be divided into several segments by time, and when the corresponding time of each segment is short enough (for example, less than or equal to 3 seconds), the movable platform can be placed in the The motion process in each time period is regarded as a uniform linear motion. Under the uniform linear motion model, the horizontal and vertical speeds of the movable platform are sampled at certain time intervals, and the motion hypothesis model shown in Figure 4 can be obtained. In other cases, other motion assumption models such as uniform acceleration motion model and uniform deceleration motion model can also be used to simulate the motion process of the movable platform.
在建立好运动假设模型之后，可以基于运动假设模型中的运动假设，将所述可移动平台采集到的多帧三维点云投影到预设坐标系下，获取所述运动假设对应的投影密度。进一步地，还可以基于所述运动假设以及所述可移动平台采集所述多帧三维点云中的每帧三维点云时的位姿，将所述多帧三维点云投影到预设坐标系下：After the motion hypothesis model is established, the multi-frame 3D point cloud collected by the movable platform can be projected into a preset coordinate system based on the motion hypothesis in the motion hypothesis model to obtain the projection density corresponding to the motion hypothesis. Further, it is also possible to project the multi-frame 3D point cloud to a preset coordinate system based on the motion assumption and the pose when the movable platform collects each frame of the 3D point cloud in the multi-frame 3D point cloud. Down:
Figure PCTCN2020136546-appb-000001
Figure PCTCN2020136546-appb-000001
其中，P i-n,k表示第i-n帧三维点云在第k个运动假设下的投影点的位置，P i-n为第i-n帧三维点云中的三维点的位置，预设坐标系为采集第i帧三维点云时的车体坐标系，odom i-n和odom i分别表示采集第i-n帧三维点云时可移动平台的位姿和采集第i帧三维点云时可移动平台的位姿，Δt表示采集第i-n帧三维点云与采集第i帧三维点云之间的时间间隔，v h表示第k个运动假设对应的速度，-1表示矩阵求逆操作。上述过程通过odom i-n将P i-n转换到世界坐标系，再通过
Figure PCTCN2020136546-appb-000002
将世界坐标系下的点转换到预设坐标系，并通过n*Δt*v h补偿可移动平台自身运动，从而得到投影点的位置。对于可移动平台是车辆的情况下，可以利用车载***中常使用的消息同步机制，同步接收相同时间戳的三维点云和odometry数据(即车辆的位姿数据)，从而为多帧三维点云之间的投影提供位置姿态参考，补偿车辆自身运动造成的偏差。可以对所有三维点云帧和所有运动假设分别作出上述变换，从而完成运动假设的注入。 Among them, Pin ,k represents the position of the projection point of the 3D point cloud of the in -th frame under the k-th motion hypothesis, Pin is the position of the 3D point in the 3-D point cloud of the in-th frame, and the preset coordinate system is the acquisition of the i-th 3D point cloud. The vehicle body coordinate system when the 3D point cloud is framed, odom in and odom i respectively represent the pose of the movable platform when collecting the 3D point cloud of the in-th frame and the pose of the movable platform when collecting the 3-D point cloud of the i-th frame, Δt represents The time interval between the acquisition of the 3D point cloud of the in-th frame and the acquisition of the 3-D point cloud of the i-th frame, v h represents the velocity corresponding to the k-th motion hypothesis, and -1 represents the matrix inversion operation. The above process converts P in to the world coordinate system through odom in , and then passes
Figure PCTCN2020136546-appb-000002
Convert the point in the world coordinate system to the preset coordinate system, and compensate the movement of the movable platform itself through n*Δt*v h , so as to obtain the position of the projected point. For the case where the movable platform is a vehicle, the message synchronization mechanism commonly used in the vehicle system can be used to receive the 3D point cloud and odometry data with the same timestamp (that is, the pose data of the vehicle) synchronously, so as to provide the data for the multi-frame 3D point cloud. The projection between them provides a position and attitude reference, compensating for the deviation caused by the vehicle's own motion. The above transformations can be made separately for all 3D point cloud frames and all motion hypotheses, thereby completing the injection of motion hypotheses.
理想情况下，如果运动假设与可移动平台的真实运动方式相同，则各个投影点对应的是同一个点。在实际情况下，由于可移动平台的运动过程并不能完全等价于匀速直线模型，以及由于噪声和运动假设与可移动平台的运动方式之间存在差异等原因，各个投影点之间可能存在一定的偏差，但只要运动假设与可移动平台的运动方式足够接近，各个投影点之间的偏差应该是较小的，也就是说，各个投影点之间的位置是比较接近的。因此，可以根据投影密度来确定运动假设与可移动平台的运动方式之间是否匹配。因此，在步骤302中，可以基于多个所述运动假设对应的投影密度，从多个所述运动假设中确定匹配运动假设。Ideally, each projected point corresponds to the same point if the motion is assumed to be the same as the real motion of the movable platform. In practical situations, since the motion process of the movable platform is not completely equivalent to the uniform linear model, and due to the differences between the noise and motion assumptions and the movement mode of the movable platform, there may be certain differences between the projection points. However, as long as the movement assumption is close enough to the movement of the movable platform, the deviation between the projection points should be small, that is, the positions of the projection points are relatively close. Therefore, a match between the motion hypothesis and the motion pattern of the movable platform can be determined based on the projected density. Therefore, in step 302, a matching motion hypothesis may be determined from the plurality of motion hypotheses based on the corresponding projection densities of the plurality of motion hypotheses.
具体来说，可以将投影密度最大的运动假设确定为匹配运动假设。进一步地，还可以判断投影密度最大的运动假设与其他任一投影密度之差是否均大于预设值。如果是，则将所述最大投影密度对应的运动假设确定为匹配运动假设。进一步地，还可以判断最大投影密度是否大于预设的投影密度阈值，如果最大投影密度大于预设的投影密度阈值，且投影密度最大的运动假设与其他任一投影密度之差均大于预设值，则将所述最大投影密度对应的运动假设确定为匹配运动假设。通过这种方式，可以提高运动假设的显著性，从而提高点云分割的准确性和可靠性。如果最大权重与至少一个其他权重之差不大于预设值，或者最大投影密度不大于预设的投影密度阈值，则确定所述匹配运动假设不存在。Specifically, the motion hypothesis with the highest projection density can be determined as the matching motion hypothesis. Further, it can also be determined whether the difference between the motion hypothesis with the largest projection density and any other projection density is greater than a preset value. If so, the motion hypothesis corresponding to the maximum projected density is determined as a matching motion hypothesis. Further, it can also be judged whether the maximum projection density is greater than the preset projection density threshold, if the maximum projection density is greater than the preset projection density threshold, and the difference between the motion hypothesis with the largest projection density and any other projection density is greater than the preset value. , the motion hypothesis corresponding to the maximum projection density is determined as a matching motion hypothesis. In this way, the saliency of the motion hypothesis can be increased, thereby improving the accuracy and reliability of point cloud segmentation. If the difference between the maximum weight and at least one other weight is not greater than a preset value, or the maximum projected density is not greater than a preset projected density threshold, it is determined that the matching motion hypothesis does not exist.
下面以运动假设A为例，对获取投影密度的方式进行说明，其他运动假设对应的投影密度的获取方式可参见获取运动假设A的投影密度的方式。可以将预设坐标系预先划分为多个栅格，各个栅格的面积和/或形状可以相同，也可以不同。例如，为了便于处理，可以将预设坐标系预先划分为多个尺寸相同的矩形栅格。然后，分别获取运动假设A在各个栅格内对应的投影密度。如图5A所示，每个方格代表一个栅格，方格中的每个数字代表在该栅格内存在投影点的三维点云的帧数，图5A所示的图称为栅格权重图。投影密度可以基于在栅格内存在投影点的三维点云的帧数与栅格面积之比来确定。由于各个栅格的面积都相同，因此可以直接基于在栅格内存在投影点的三维点云的帧数来确定匹配运动假设。应当说明的是，一帧三维点云在一个栅格内可能存在多个投影点，只要这帧三维点云在栅格内存在投影点，无论投影的数量是多少，都将该栅格内存在投影点的三维点云的帧数加1。例如，三维点云1、三维点云2和三维点云3在栅格1内的投影点数分别为1、3和0，则将栅格1内存在投影点的三维点云的帧数记为2。通过统计帧数而不是统计落入栅格内的投影点的点数，能够减少因为扫描到的不同区域内三维点的数量分布不均匀导致的误差。The following takes the motion hypothesis A as an example to describe the way of obtaining the projection density. For the ways of obtaining the projection density corresponding to other motion hypotheses, please refer to the way of obtaining the projection density of the motion hypothesis A. The preset coordinate system may be pre-divided into multiple grids, and the area and/or shape of each grid may be the same or different. For example, to facilitate processing, the preset coordinate system may be pre-divided into multiple rectangular grids with the same size. Then, the corresponding projection densities of the motion hypothesis A in each grid are obtained respectively. As shown in Figure 5A, each square represents a grid, and each number in the square represents the number of frames of the 3D point cloud with projected points in the grid. The graph shown in Figure 5A is called grid weight picture. The projected density can be determined based on the ratio of the number of frames of the 3D point cloud where projected points exist within the grid to the grid area. Since each grid has the same area, the matching motion hypothesis can be determined directly based on the number of frames of the 3D point cloud where the projected points exist within the grid. It should be noted that a frame of 3D point cloud may have multiple projection points in a grid. As long as there are projection points in the grid for this frame of 3D point cloud, no matter the number of projections, the grid will exist in the grid. The frame number of the 3D point cloud of the projected point is incremented by 1. For example, the projected points of 3D point cloud 1, 3D point cloud 2 and 3D point cloud 3 in grid 1 are 1, 3 and 0 respectively, then the number of frames of the 3D point cloud with projected points in grid 1 is recorded as 2. By counting the number of frames instead of counting the number of projection points falling into the grid, errors caused by uneven distribution of the number of 3D points in different scanned regions can be reduced.
可以为每个运动假设分别生成一个栅格权重图，假设有H个运动假设，则生成H个栅格权重图。每一个运动假设h，可以将三维点云队列中的全部三维点云帧按照运动假设h变换到当前帧，统计落入每个栅格中的历史三维点云的帧数，如果运动假设h比较接近该栅格内三维点云的真实运动方式，则对应运动假设h的历史三维点云将会大概率重叠在相似的区域，对应的该栅格权重就会变高；反之如果运动假设h与真实运动方式差异较大，则经过运动假设注入的历史点云将小概率重叠，其对应的权重就会很低。A grid weight map can be generated for each motion hypothesis, and if there are H motion hypotheses, H grid weight maps are generated. For each motion hypothesis h, all 3D point cloud frames in the 3D point cloud queue can be transformed to the current frame according to the motion hypothesis h, and the number of historical 3D point cloud frames falling into each grid can be counted. If the motion hypothesis h is compared Close to the real motion mode of the 3D point cloud in the grid, the historical 3D point cloud corresponding to the motion hypothesis h will overlap in a similar area with a high probability, and the corresponding grid weight will become higher; otherwise, if the motion hypothesis h is equal to If the real movement mode is quite different, the historical point cloud injected by the movement hypothesis will overlap with a small probability, and the corresponding weight will be very low.
由于邻近区域内的多个三维点的运动方式一般是类似的，因此，若所述栅格内存在匹配运动假设，可以将所述栅格的匹配运动假设确定为所述栅格中各个点对应的三维点的匹配运动假设。Since the motion modes of multiple three-dimensional points in the adjacent area are generally similar, if there is a matching motion hypothesis in the grid, the matching motion hypothesis of the grid can be determined as the corresponding motion of each point in the grid. The matching motion hypothesis of the 3D points.
然后，还可以基于匹配运动假设生成掩膜(mask)图，mask图与栅格权重图等尺寸，mask图中的每个栅格包括一个栅格参数，用于记录该栅格的匹配运动假设。如图5B所示，图中的h1至h4分别表示对应栅格的匹配运动假设，null表示栅格不存在匹配运动假设。Then, a mask map can also be generated based on the matching motion hypothesis. The mask map and the grid weight map are of equal size. Each grid in the mask map includes a grid parameter, which is used to record the matching motion hypothesis of the grid. . As shown in FIG. 5B , h1 to h4 in the figure respectively represent the matching motion hypothesis of the corresponding grid, and null indicates that there is no matching motion hypothesis for the grid.
在步骤303中，可以基于mask图，对所述多帧三维点云中的第一三维点云进行点云分割，即确定第一三维点云中的三维点是动态点还是静态点。其中，动态点表 示运动速度不为0的三维点，静态点表示运动速度为0的三维点。若一个栅格中的匹配运动假设的速度为0，将所述第一三维点云中投影到所述栅格中的三维点分割为静态点。若一个栅格中的匹配运动假设的速度不为0，将所述第一三维点云中投影到所述栅格中的三维点分割为动态点。所述第一三维点云可以包括所述多帧三维点云中的部分或全部三维点云帧。若一个栅格中不存在匹配运动假设，可以将所述第一三维点云中投影到所述栅格中的三维点分割为属性未知的三维点。In step 303, point cloud segmentation may be performed on the first three-dimensional point cloud in the multi-frame three-dimensional point cloud based on the mask image, that is, it is determined whether the three-dimensional point in the first three-dimensional point cloud is a dynamic point or a static point. Among them, the dynamic point represents the three-dimensional point whose motion speed is not 0, and the static point represents the three-dimensional point whose motion speed is 0. If the velocity assumed by the matching motion in a grid is 0, the 3D points projected into the grid from the first 3D point cloud are divided into static points. If the speed assumed by the matching motion in a grid is not 0, the 3D points projected into the grid from the first 3D point cloud are divided into dynamic points. The first three-dimensional point cloud may include some or all of the three-dimensional point cloud frames in the multi-frame three-dimensional point cloud. If there is no matching motion hypothesis in a grid, the 3D points projected into the grid from the first 3D point cloud may be divided into 3D points with unknown attributes.
基于点云分割结果，可以为所述第一三维点云中的各个三维点打标签。所述标签可以包括数字、字母、符号中的至少一者。以标签包括数字为例，可以用比特1表示动态点，用比特0表示静态点，还可以用比特01表示属性未知的三维点。Based on the point cloud segmentation result, each three-dimensional point in the first three-dimensional point cloud may be labeled. The label may include at least one of numbers, letters, and symbols. Taking the label including numbers as an example, bit 1 can be used to represent a dynamic point, bit 0 can be used to represent a static point, and bit 01 can also be used to represent a 3D point with unknown properties.
在实际应用中，点云分割结果可用于所述可移动平台上的规划单元对所述可移动平台的行驶状态进行规划。例如，规划单元可以基于点云分割结果得到的标签，确定行驶路径上的障碍物的运动速度，从而决定是否需要控制可移动平台的速度和姿态以躲避障碍物。点云分割结果还可以输出至可移动平台上的多媒体***，例如，显示屏、语音播放***等，用于向用户输出多媒体提示信息。In practical applications, the point cloud segmentation result can be used by the planning unit on the movable platform to plan the driving state of the movable platform. For example, the planning unit can determine the moving speed of obstacles on the driving path based on the labels obtained from the segmentation results of the point cloud, so as to decide whether to control the speed and attitude of the movable platform to avoid obstacles. The point cloud segmentation result can also be output to a multimedia system on the movable platform, such as a display screen, a voice playback system, etc., for outputting multimedia prompt information to the user.
如图6所示，是本公开实施例的点云分割方法的总体流程图。As shown in FIG. 6 , it is an overall flowchart of a point cloud segmentation method according to an embodiment of the present disclosure.
在步骤601中，可以根据高速公路、城市街道等场景生成多运动假设。In step 601, multi-motion hypotheses may be generated according to scenes such as highways and city streets.
在步骤602中，可以同步接收一帧三维点云以及采集到该帧三维点云的时刻可移动平台的位姿数据。In step 602, a frame of the three-dimensional point cloud and the pose data of the movable platform at the moment when the frame of the three-dimensional point cloud is collected may be received synchronously.
在步骤603中，可以从接收到的三维点云中去除可移动平台行驶路面(例如，地面)上的点。In step 603, points on the road surface (eg, the ground) on which the movable platform travels may be removed from the received three-dimensional point cloud.
在步骤604中，可以按照分频设置判断是否使用本帧三维点云，例如，本帧三维点云的时间戳与200ms进行取模得到的结果为0，则使用本帧三维点云，执行步骤605；否则不使用本帧三维点云，执行步骤606。In step 604, whether to use the 3D point cloud of this frame can be determined according to the frequency division setting. For example, if the result obtained by taking the modulo of the timestamp of the 3D point cloud of this frame and 200ms is 0, then the 3D point cloud of this frame is used, and the steps are executed. 605; otherwise, the 3D point cloud of this frame is not used, and step 606 is executed.
在步骤605中，可以将三维点云加入点云队列。In step 605, the 3D point cloud can be added to the point cloud queue.
在步骤606中，判断点云队列是否达到最小可计算长度。如果是，执行步骤607，否则返回步骤602。In step 606, it is determined whether the point cloud queue reaches the minimum computable length. If yes, go to step 607, otherwise go back to step 602.
在步骤607中，可以为当前队列注入H个运动假设。In step 607, H motion hypotheses may be injected for the current queue.
在步骤608中，可以验证假设并生成mask图。即，分别用每个运动假设将当 前帧上的三维点投影到预设坐标系下，判断预设坐标系中的各个栅格是否存在匹配运动假设，并基于每个栅格的匹配运动假设生成mask图。In step 608, the hypothesis can be verified and a mask map generated. That is, each motion hypothesis is used to project the 3D point on the current frame into the preset coordinate system, and it is judged whether each grid in the preset coordinate system has a matching motion hypothesis, and based on the matching motion hypothesis of each grid, generate mask map.
在步骤609中，可以遍历三维点云，并查询运动mask图，从而为三维点云中的各个三维点生成标签，用于表示各个三维点为动态点或者静态点。In step 609, the 3D point cloud can be traversed, and the motion mask image can be queried, so as to generate a label for each 3D point in the 3D point cloud, which is used to indicate that each 3D point is a dynamic point or a static point.
在步骤610中，可以将带标签的三维点云输出至下游模块，例如，可移动平台的planning模块和多媒体***。In step 610, the labeled 3D point cloud can be output to downstream modules, eg, the planning module of the mobile platform and the multimedia system.
在步骤611中，判断程序是否结束。如果程序未结束，则返回步骤602继续进行点云分割。In step 611, it is determined whether or not the program ends. If the procedure is not over, return to step 602 to continue point cloud segmentation.
本领域技术人员可以理解，在具体实施方式的上述方法中，各步骤的撰写顺序并不意味着严格的执行顺序而对实施过程构成任何限定，各步骤的具体执行顺序应当以其功能和可能的内在逻辑确定。Those skilled in the art can understand that in the above method of the specific implementation, the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the specific execution order of each step should be based on its function and possible Internal logic is determined.
本公开实施例还提供一种点云分割装置，包括处理器，所述处理器用于执行以下步骤：An embodiment of the present disclosure further provides a point cloud segmentation device, including a processor, where the processor is configured to perform the following steps:
基于预先建立的运动假设模型中的运动假设，将所述可移动平台采集到的多帧三维点云投影到预设坐标系下，获取所述运动假设对应的投影密度；Based on the motion hypothesis in the pre-established motion hypothesis model, project the multi-frame 3D point cloud collected by the movable platform to a preset coordinate system, and obtain the projection density corresponding to the motion hypothesis;
基于多个所述运动假设对应的投影密度，从多个所述运动假设中确定匹配运动假设；determining a matching motion hypothesis from the plurality of motion hypotheses based on projection densities corresponding to the plurality of motion hypotheses;
基于所述匹配运动假设对所述多帧三维点云中的第一三维点云进行点云分割。Point cloud segmentation is performed on the first three-dimensional point cloud in the multi-frame three-dimensional point cloud based on the matching motion hypothesis.
在一些实施例中，不同的运动假设对应不同的运动速度和/或运动方向。In some embodiments, different motion hypotheses correspond to different motion speeds and/or motion directions.
在一些实施例中，所述运动假设的运动速度的范围和/或运动方向的范围基于所述可移动平台所处环境的环境特征而确定。In some embodiments, the range of motion speed and/or motion direction of the motion assumption is determined based on environmental characteristics of the environment in which the movable platform is located.
在一些实施例中，所述运动假设在所述可移动平台行驶方向上的运动速度在[-40m/s，40m/s]范围内，在与所述可移动平台行驶方向垂直的方向上的运动速度在[-10m/s，10m/s]范围内。In some embodiments, the movement assumes that the movement speed in the travel direction of the movable platform is in the range of [-40m/s, 40m/s], and the movement speed in the direction perpendicular to the travel direction of the movable platform The movement speed is in the range of [-10m/s, 10m/s].
在一些实施例中，所述运动假设的运动方向在[-90°，90°)范围内。In some embodiments, the assumed direction of motion of the motion is in the range [-90°, 90°).
在一些实施例中，所述处理器用于：基于所述运动假设以及所述可移动平台采集所述多帧三维点云中的每帧三维点云时的位姿，将所述多帧三维点云投影到预设坐 标系下。In some embodiments, the processor is configured to: based on the motion hypothesis and the pose when the movable platform collects each frame of the 3D point cloud in the multi-frame 3D point cloud, convert the multi-frame 3D point cloud The cloud is projected into the preset coordinate system.
在一些实施例中，所述多帧三维点云中的每帧三维点云中包括第一标签，用于指示所述每帧三维点云中属于所述可移动平台行驶路面之外的三维点；所述处理器用于：将所述每帧三维点云中携带所述第一标签的三维点投影到预设坐标系下。In some embodiments, each frame of the 3D point cloud in the multi-frame 3D point cloud includes a first label, which is used to indicate a 3D point in the 3D point cloud of each frame that belongs to the outside of the road surface of the movable platform. ; the processor is configured to: project the three-dimensional point carrying the first label in the three-dimensional point cloud of each frame to a preset coordinate system.
在一些实施例中，所述处理器还用于：获取所述可移动平台采集到的多帧原始三维点云；对所述多帧原始三维点云进行分频处理，得到所述多帧三维点云。In some embodiments, the processor is further configured to: acquire the multi-frame original 3D point cloud collected by the movable platform; perform frequency division processing on the multi-frame original 3D point cloud to obtain the multi-frame 3D point cloud point cloud.
在一些实施例中，所述处理器用于：若最大投影密度与其他任一投影密度之差均大于预设值，将所述最大投影密度对应的运动假设确定为匹配运动假设。In some embodiments, the processor is configured to: if the difference between the maximum projected density and any other projected density is greater than a preset value, determine the motion hypothesis corresponding to the maximum projected density as a matching motion hypothesis.
在一些实施例中，所述处理器用于：若最大权重与至少一个其他权重之差不大于预设值，确定所述匹配运动假设不存在。In some embodiments, the processor is configured to determine that the matching motion hypothesis does not exist if the difference between the maximum weight and the at least one other weight is not greater than a preset value.
在一些实施例中，所述处理器用于：获取所述运动假设在各个栅格内对应的投影密度。In some embodiments, the processor is configured to: obtain the corresponding projection densities of the motion hypotheses in each grid.
在一些实施例中，所述处理器还用于：若所述栅格内存在匹配运动假设，将所述栅格的匹配运动假设确定为所述栅格中各个点对应的三维点的匹配运动假设。In some embodiments, the processor is further configured to: if a matching motion hypothesis exists in the grid, determine the matching motion hypothesis of the grid as the matching motion of the three-dimensional point corresponding to each point in the grid Suppose.
在一些实施例中，所述处理器用于：在基于所述运动假设将所述多帧三维点云投影到预设坐标系下之后，获取在所述栅格内存在投影点的三维点云的帧数；将所述帧数与所述栅格的面积之比确定为所述运动假设在所述栅格内对应的投影密度。In some embodiments, the processor is configured to: after projecting the multi-frame 3D point cloud into a preset coordinate system based on the motion hypothesis, obtain a 3D point cloud with projected points in the grid. number of frames; determining the ratio of the number of frames to the area of the grid as the corresponding projected density of the motion hypothesis within the grid.
在一些实施例中，所述处理器用于：若一个栅格中的匹配运动假设的速度为0，将所述第一三维点云中投影到所述栅格中的三维点分割为静态点；和/或若一个栅格中的匹配运动假设的速度不为0，将所述第一三维点云中投影到所述栅格中的三维点分割为动态点。In some embodiments, the processor is configured to: if the velocity of the matching motion hypothesis in a grid is 0, segment the 3D points projected into the grid from the first 3D point cloud into static points; And/or if the velocity assumed by the matching motion in a grid is not 0, the 3D points projected into the grid from the first 3D point cloud are divided into dynamic points.
在一些实施例中，所述处理器用于：若一个栅格中不存在匹配运动假设，将所述第一三维点云中投影到所述栅格中的三维点分割为属性未知的点。In some embodiments, the processor is configured to: if there is no matching motion hypothesis in a grid, segment the 3D points from the first 3D point cloud projected into the grid into points with unknown attributes.
在一些实施例中，所述处理器还用于：基于点云分割结果，为所述第一三维点云中的各个三维点打标签。In some embodiments, the processor is further configured to: label each 3D point in the first 3D point cloud based on the point cloud segmentation result.
在一些实施例中，所述三维点云基于安装于所述可移动平台上的视觉传感器或者激光雷达采集得到；和/或对所述第一三维点云进行点云分割得到的点云分割结果用于所述可移动平台上的规划单元对所述可移动平台的行驶状态进行规划。In some embodiments, the three-dimensional point cloud is acquired based on a visual sensor or lidar installed on the movable platform; and/or a point cloud segmentation result obtained by performing point cloud segmentation on the first three-dimensional point cloud The planning unit on the movable platform plans the driving state of the movable platform.
本公开实施例的点云分割装置中处理器所执行的方法的具体实施例可参见前述方法实施例，此处不再赘述。For specific embodiments of the method executed by the processor in the point cloud segmentation apparatus according to the embodiment of the present disclosure, reference may be made to the foregoing method embodiments, which will not be repeated here.
图7示出了本说明书实施例所提供的一种更为具体的数据处理装置硬件结构示意图，该设备可以包括：处理器701、存储器702、输入/输出接口703、通信接口704和总线705。其中处理器701、存储器702、输入/输出接口703和通信接口704通过总线705实现彼此之间在设备内部的通信连接。FIG. 7 shows a schematic diagram of a more specific hardware structure of a data processing apparatus provided by an embodiment of this specification. The apparatus may include: a processor 701 , a memory 702 , an input/output interface 703 , a communication interface 704 and a bus 705 . The processor 701 , the memory 702 , the input/output interface 703 and the communication interface 704 realize the communication connection among each other within the device through the bus 705 .
处理器701可以采用通用的CPU(Central Processing Unit，中央处理器)、微处理器、应用专用集成电路(Application Specific Integrated Circuit，ASIC)、或者一个或多个集成电路等方式实现，用于执行相关程序，以实现本说明书实施例所提供的技术方案。The processor 701 can be implemented by a general-purpose CPU (Central Processing Unit, central processing unit), a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc. program to implement the technical solutions provided by the embodiments of this specification.
存储器702可以采用ROM(Read Only Memory，只读存储器)、RAM(Random Access Memory，随机存取存储器)、静态存储设备，动态存储设备等形式实现。存储器702可以存储操作***和其他应用程序，在通过软件或者固件来实现本说明书实施例所提供的技术方案时，相关的程序代码保存在存储器702中，并由处理器701来调用执行。The memory 702 can be implemented in the form of a ROM (Read Only Memory, read-only memory), a RAM (Random Access Memory, random access memory), a static storage device, a dynamic storage device, and the like. The memory 702 may store the operating system and other application programs. When implementing the technical solutions provided by the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 702 and invoked by the processor 701 for execution.
输入/输出接口703用于连接输入/输出模块，以实现信息输入及输出。输入输出/模块可以作为组件配置在设备中(图中未示出)，也可以外接于设备以提供相应功能。其中输入设备可以包括键盘、鼠标、触摸屏、麦克风、各类传感器等，输出设备可以包括显示器、扬声器、振动器、指示灯等。The input/output interface 703 is used to connect the input/output module to realize the input and output of information. The input/output/module can be configured in the device as a component (not shown in the figure), or can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, and the like.
通信接口704用于连接通信模块(图中未示出)，以实现本设备与其他设备的通信交互。其中通信模块可以通过有线方式(例如USB、网线等)实现通信，也可以通过无线方式(例如移动网络、WIFI、蓝牙等)实现通信。The communication interface 704 is used to connect a communication module (not shown in the figure), so as to realize the communication interaction between the device and other devices. The communication module may implement communication through wired means (eg, USB, network cable, etc.), or may implement communication through wireless means (eg, mobile network, WIFI, Bluetooth, etc.).
总线705包括一通路，在设备的各个组件(例如处理器701、存储器702、输入/输出接口703和通信接口704)之间传输信息。 Bus 705 includes a path to transfer information between the various components of the device (eg, processor 701, memory 702, input/output interface 703, and communication interface 704).
需要说明的是，尽管上述设备仅示出了处理器701、存储器702、输入/输出接口703、通信接口704以及总线705，但是在具体实施过程中，该设备还可以包括实现正常运行所必需的其他组件。此外，本领域的技术人员可以理解的是，上述设备中也可以仅包含实现本说明书实施例方案所必需的组件，而不必包含图中所示的全部组件。It should be noted that, although the above-mentioned device only shows the processor 701, the memory 702, the input/output interface 703, the communication interface 704 and the bus 705, in the specific implementation process, the device may also include necessary components for normal operation. other components. In addition, those skilled in the art can understand that, the above-mentioned device may only include components necessary to implement the solutions of the embodiments of the present specification, rather than all the components shown in the figures.
如图8所示，本公开实施例还提供一种可移动平台800，包括壳体801；点云 采集装置802，设于所述壳体801上，用于采集三维点云；以及三维点云分割装置803，设于所述壳体801内，用于执行本公开任一实施例所述的方法。其中，所述可移动平台800可以是无人机、无人车、无人船、可移动机器人等设备，所述点云采集装置802可以是视觉传感器(例如双目视觉传感器、三目视觉传感器等)或者激光雷达。As shown in FIG. 8 , an embodiment of the present disclosure further provides a movable platform 800 , which includes a housing 801 ; a point cloud collecting device 802 , which is arranged on the housing 801 and is used to collect a three-dimensional point cloud; and a three-dimensional point cloud. The dividing device 803 is arranged in the casing 801 and is used for executing the method described in any embodiment of the present disclosure. Wherein, the movable platform 800 may be an unmanned aerial vehicle, an unmanned vehicle, an unmanned ship, a mobile robot, etc., and the point cloud collection device 802 may be a visual sensor (eg, a binocular vision sensor, a trinocular vision sensor, etc.) etc.) or lidar.
本公开实施例还提供一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现前述任一实施例所述的方法中由第二处理单元执行的步骤。Embodiments of the present disclosure further provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, implements the steps executed by the second processing unit in the method described in any of the foregoing embodiments.
计算机可读介质包括永久性和非永久性、可移动和非可移动媒体可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括，但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带，磁带磁磁盘存储或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。按照本文中的界定，计算机可读介质不包括暂存电脑可读媒体(transitory media)，如调制的数据信号和载波。Computer-readable media includes both persistent and non-permanent, removable and non-removable media, and storage of information may be implemented by any method or technology. Information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.
通过以上的实施方式的描述可知，本领域的技术人员可以清楚地了解到本说明书实施例可借助软件加必需的通用硬件平台的方式来实现。基于这样的理解，本说明书实施例的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本说明书实施例各个实施例或者实施例的某些部分所述的方法。From the description of the above embodiments, those skilled in the art can clearly understand that the embodiments of the present specification can be implemented by means of software plus a necessary general hardware platform. Based on such understanding, the technical solutions of the embodiments of this specification or the parts that make contributions to the prior art may be embodied in the form of software products, and the computer software products may be stored in storage media, such as ROM/RAM, A magnetic disk, an optical disk, etc., includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments in this specification.
上述实施例阐明的***、装置、模块或单元，具体可以由计算机芯片或实体实现，或者由具有某种功能的产品来实现。一种典型的实现设备为计算机，计算机的具体形式可以是个人计算机、膝上型计算机、蜂窝电话、相机电话、智能电话、个人数字助理、媒体播放器、导航设备、电子邮件收发设备、游戏控制台、平板计算机、可穿戴设备或者这些设备中的任意几种设备的组合。The systems, devices, modules or units described in the above embodiments may be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementing device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media player, navigation device, email sending and receiving device, game control desktop, tablet, wearable device, or a combination of any of these devices.
以上实施例中的各种技术特征可以任意进行组合，只要特征之间的组合不存在冲突或矛盾，但是限于篇幅，未进行一一描述，因此上述实施方式中的各种技术特征的任意进行组合也属于本公开的范围。Various technical features in the above embodiments can be combined arbitrarily, as long as there is no conflict or contradiction between the combinations of features, but due to space limitations, they are not described one by one, so the various technical features in the above embodiments can be combined arbitrarily It is also within the scope of this disclosure.
本领域技术人员在考虑公开及实践这里公开的说明书后，将容易想到本公开的其它实施方案。本公开旨在涵盖本公开的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本公开的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本公开的真正范围和精神由下面的权利要求指出。Other embodiments of the present disclosure will readily occur to those skilled in the art upon consideration of the disclosure and practice of the specification disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common general knowledge or techniques in the technical field not disclosed by this disclosure . The specification and examples are to be regarded as exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.
应当理解的是，本公开并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本公开的范围仅由所附的权利要求来限制。It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
以上所述仅为本公开的较佳实施例而已，并不用以限制本公开，凡在本公开的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本公开保护的范围之内。The above descriptions are only preferred embodiments of the present disclosure, and are not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the present disclosure. within the scope of protection.

Claims (34)

  1. 一种三维点云分割方法，其特征在于，用于对可移动平台采集到的三维点云进行点云分割，所述方法包括：A three-dimensional point cloud segmentation method, characterized in that it is used for point cloud segmentation on a three-dimensional point cloud collected by a movable platform, the method comprising:
    基于预先建立的运动假设模型中的运动假设，将所述可移动平台采集到的多帧三维点云投影到预设坐标系下，获取所述运动假设对应的投影密度；Based on the motion hypothesis in the pre-established motion hypothesis model, project the multi-frame 3D point cloud collected by the movable platform to a preset coordinate system, and obtain the projection density corresponding to the motion hypothesis;
    基于多个所述运动假设对应的投影密度，从多个所述运动假设中确定匹配运动假设；determining a matching motion hypothesis from the plurality of motion hypotheses based on projection densities corresponding to the plurality of motion hypotheses;
    基于所述匹配运动假设对所述多帧三维点云中的第一三维点云进行点云分割。Point cloud segmentation is performed on the first three-dimensional point cloud in the multi-frame three-dimensional point cloud based on the matching motion hypothesis.
  2. 根据权利要求1所述的方法，其特征在于，不同的运动假设对应不同的运动速度和/或运动方向。The method according to claim 1, wherein different motion assumptions correspond to different motion speeds and/or motion directions.
  3. 根据权利要求2所述的方法，其特征在于，所述运动假设的运动速度的范围和/或运动方向的范围基于所述可移动平台所处环境的环境特征而确定。The method according to claim 2, wherein the range of the movement speed and/or the range of the movement direction of the movement assumption is determined based on environmental characteristics of the environment where the movable platform is located.
  4. 根据权利要求3所述的方法，其特征在于，所述运动假设在所述可移动平台行驶方向上的运动速度在[-40m/s，40m/s]范围内，在与所述可移动平台行驶方向垂直的方向上的运动速度在[-10m/s，10m/s]范围内；和/或The method according to claim 3, wherein the movement assumes that the movement speed in the traveling direction of the movable platform is in the range of [-40m/s, 40m/s], and the movement speed is within the range of [-40m/s, 40m/s]. The speed of movement in the direction perpendicular to the direction of travel is in the range [-10m/s, 10m/s]; and/or
    所述运动假设的运动方向在[-90°，90°)范围内。The movement assumes that the movement direction is in the range of [-90°, 90°).
  5. 根据权利要求1所述的方法，其特征在于，所述基于预先建立的运动假设模型中的运动假设，将所述可移动平台采集到的多帧三维点云投影到预设坐标系下，包括：The method according to claim 1, wherein the multi-frame 3D point cloud collected by the movable platform is projected to a preset coordinate system based on the motion hypothesis in the pre-established motion hypothesis model, comprising: :
    基于所述运动假设以及所述可移动平台采集所述多帧三维点云中的每帧三维点云时的位姿，将所述多帧三维点云投影到预设坐标系下。Based on the motion assumption and the pose when the movable platform collects each frame of the 3D point cloud in the multi-frame 3D point cloud, the multi-frame 3D point cloud is projected to a preset coordinate system.
  6. 根据权利要求1所述的方法，其特征在于，所述多帧三维点云中的每帧三维点云中包括第一标签，用于指示所述每帧三维点云中属于所述可移动平台行驶路面之外的三维点；所述将所述可移动平台采集到的多帧三维点云投影到预设坐标系下，包括：The method according to claim 1, wherein each frame of the three-dimensional point cloud in the multi-frame three-dimensional point cloud includes a first label, which is used to indicate that the three-dimensional point cloud of each frame belongs to the movable platform Three-dimensional points outside the driving road surface; the projecting the multi-frame three-dimensional point cloud collected by the movable platform to a preset coordinate system includes:
    将所述每帧三维点云中携带所述第一标签的三维点投影到预设坐标系下。Projecting the three-dimensional point carrying the first tag in the three-dimensional point cloud of each frame into a preset coordinate system.
  7. 根据权利要求1所述的方法，其特征在于，所述方法还包括：The method according to claim 1, wherein the method further comprises:
    获取所述可移动平台采集到的多帧原始三维点云；acquiring the multi-frame original 3D point cloud collected by the movable platform;
    对所述多帧原始三维点云进行分频处理，得到所述多帧三维点云。Frequency division processing is performed on the multi-frame original 3D point cloud to obtain the multi-frame 3D point cloud.
  8. 根据权利要求1所述的方法，其特征在于，所述基于多个所述运动假设对应的投影密度，从多个所述运动假设中确定匹配运动假设，包括：The method according to claim 1, wherein the determining a matching motion hypothesis from the plurality of motion hypotheses based on projection densities corresponding to the plurality of motion hypotheses comprises:
    若最大投影密度与其他任一投影密度之差均大于预设值，将所述最大投影密度对 应的运动假设确定为匹配运动假设。If the difference between the maximum projected density and any other projected density is greater than the preset value, the motion hypothesis corresponding to the maximum projected density is determined as a matching motion hypothesis.
  9. 根据权利要求8所述的方法，其特征在于，所述基于多个所述运动假设对应权重确定所述匹配运动假设，包括：The method according to claim 8, wherein the determining the matching motion hypothesis based on the corresponding weights of the plurality of motion hypotheses comprises:
    若最大权重与至少一个其他权重之差不大于预设值，确定所述匹配运动假设不存在。If the difference between the maximum weight and the at least one other weight is not greater than a preset value, it is determined that the matching motion hypothesis does not exist.
  10. 根据权利要求1所述的方法，其特征在于，所述预设坐标系包括多个栅格；所述获取所述运动假设对应的投影密度，包括：The method according to claim 1, wherein the preset coordinate system comprises a plurality of grids; and the acquiring the projection density corresponding to the motion hypothesis comprises:
    获取所述运动假设在各个栅格内对应的投影密度。Obtain the corresponding projected density of the motion hypothesis in each grid.
  11. 根据权利要求10所述的方法，其特征在于，所述方法还包括：The method of claim 10, wherein the method further comprises:
    若所述栅格内存在匹配运动假设，将所述栅格的匹配运动假设确定为所述栅格中各个点对应的三维点的匹配运动假设。If there is a matching motion hypothesis in the grid, the matching motion hypothesis of the grid is determined as the matching motion hypothesis of the three-dimensional point corresponding to each point in the grid.
  12. 根据权利要求10所述的方法，其特征在于，所述运动假设在所述栅格内对应的投影密度基于以下方式确定：The method according to claim 10, wherein the projection density corresponding to the motion hypothesis in the grid is determined based on the following method:
    在基于所述运动假设将所述多帧三维点云投影到预设坐标系下之后，获取在所述栅格内存在投影点的三维点云的帧数；After projecting the multi-frame 3D point cloud into a preset coordinate system based on the motion hypothesis, acquiring the number of frames of the 3D point cloud with projected points in the grid;
    将所述帧数与所述栅格的面积之比确定为所述运动假设在所述栅格内对应的投影密度。The ratio of the number of frames to the area of the grid is determined as the corresponding projected density of the motion hypothesis within the grid.
  13. 根据权利要求10所述的方法，其特征在于，所述基于所述匹配运动假设对所述多帧三维点云中的第一三维点云进行点云分割，包括：The method according to claim 10, wherein the performing point cloud segmentation on the first three-dimensional point cloud in the multi-frame three-dimensional point cloud based on the matching motion hypothesis comprises:
    若一个栅格中的匹配运动假设的速度为0，将所述第一三维点云中投影到所述栅格中的三维点分割为静态点；和/或If the velocity assumed by the matching motion in a grid is 0, segment the 3D points projected into the grid from the first 3D point cloud into static points; and/or
    若一个栅格中的匹配运动假设的速度不为0，将所述第一三维点云中投影到所述栅格中的三维点分割为动态点。If the speed assumed by the matching motion in a grid is not 0, the 3D points projected into the grid from the first 3D point cloud are divided into dynamic points.
  14. 根据权利要求13所述的方法，其特征在于，所述基于所述匹配运动假设对所述多帧三维点云中的第一三维点云进行点云分割，还包括：The method according to claim 13, wherein the performing point cloud segmentation on the first three-dimensional point cloud in the multi-frame three-dimensional point cloud based on the matching motion hypothesis, further comprising:
    若一个栅格中不存在匹配运动假设，将所述第一三维点云中投影到所述栅格中的三维点分割为属性未知的点。If there is no matching motion hypothesis in a grid, the 3D points projected into the grid from the first 3D point cloud are divided into points with unknown attributes.
  15. 根据权利要求1所述的方法，其特征在于，在基于所述匹配运动假设对所述多帧三维点云中的第一三维点云进行点云分割之后，所述方法还包括：The method according to claim 1, wherein after performing point cloud segmentation on the first three-dimensional point cloud in the multi-frame three-dimensional point cloud based on the matching motion hypothesis, the method further comprises:
    基于点云分割结果，为所述第一三维点云中的各个三维点打标签。Based on the point cloud segmentation result, each three-dimensional point in the first three-dimensional point cloud is labeled.
  16. 根据权利要求1所述的方法，其特征在于，所述三维点云基于安装于所述可 移动平台上的视觉传感器或者激光雷达采集得到；和/或The method according to claim 1, wherein the three-dimensional point cloud is acquired based on a vision sensor or lidar installed on the movable platform; and/or
    对所述第一三维点云进行点云分割得到的点云分割结果用于所述可移动平台上的规划单元对所述可移动平台的行驶状态进行规划。The point cloud segmentation result obtained by performing point cloud segmentation on the first three-dimensional point cloud is used by the planning unit on the movable platform to plan the driving state of the movable platform.
  17. 一种三维点云分割装置，包括处理器，其特征在于，所述三维点云分割装置用于对可移动平台采集到的三维点云进行点云分割，所述处理器用于执行以下步骤：A three-dimensional point cloud segmentation device, comprising a processor, characterized in that, the three-dimensional point cloud segmentation device is used to perform point cloud segmentation on a three-dimensional point cloud collected by a movable platform, and the processor is configured to perform the following steps:
    基于预先建立的运动假设模型中的运动假设，将所述可移动平台采集到的多帧三维点云投影到预设坐标系下，获取所述运动假设对应的投影密度；Based on the motion hypothesis in the pre-established motion hypothesis model, project the multi-frame 3D point cloud collected by the movable platform to a preset coordinate system, and obtain the projection density corresponding to the motion hypothesis;
    基于多个所述运动假设对应的投影密度，从多个所述运动假设中确定匹配运动假设；determining a matching motion hypothesis from the plurality of motion hypotheses based on projection densities corresponding to the plurality of motion hypotheses;
    基于所述匹配运动假设对所述多帧三维点云中的第一三维点云进行点云分割。Point cloud segmentation is performed on the first three-dimensional point cloud in the multi-frame three-dimensional point cloud based on the matching motion hypothesis.
  18. 根据权利要求17所述的装置，其特征在于，不同的运动假设对应不同的运动速度和/或运动方向。The device according to claim 17, wherein different motion assumptions correspond to different motion speeds and/or motion directions.
  19. 根据权利要求18所述的装置，其特征在于，所述运动假设的运动速度的范围和/或运动方向的范围基于所述可移动平台所处环境的环境特征而确定。The apparatus according to claim 18, wherein the range of the movement speed and/or the range of the movement direction of the movement assumption is determined based on environmental characteristics of the environment where the movable platform is located.
  20. 根据权利要求19所述的装置，其特征在于，所述运动假设在所述可移动平台行驶方向上的运动速度在[-40m/s，40m/s]范围内，在与所述可移动平台行驶方向垂直的方向上的运动速度在[-10m/s，10m/s]范围内；和/或The device according to claim 19, wherein the movement assumes that the movement speed in the travel direction of the movable platform is in the range of [-40m/s, 40m/s], and the movement speed is within the range of [-40m/s, 40m/s]. The speed of movement in the direction perpendicular to the direction of travel is in the range [-10m/s, 10m/s]; and/or
    所述运动假设的运动方向在[-90°，90°)范围内。The movement assumes that the movement direction is in the range of [-90°, 90°).
  21. 根据权利要求17所述的装置，其特征在于，所述处理器用于：The apparatus of claim 17, wherein the processor is configured to:
    基于所述运动假设以及所述可移动平台采集所述多帧三维点云中的每帧三维点云时的位姿，将所述多帧三维点云投影到预设坐标系下。Based on the motion assumption and the pose when the movable platform collects each frame of the 3D point cloud in the multi-frame 3D point cloud, the multi-frame 3D point cloud is projected to a preset coordinate system.
  22. 根据权利要求17所述的装置，其特征在于，所述多帧三维点云中的每帧三维点云中包括第一标签，用于指示所述每帧三维点云中属于所述可移动平台行驶路面之外的三维点；所述处理器用于：The apparatus according to claim 17, wherein each frame of the 3D point cloud in the multi-frame 3D point cloud includes a first label, which is used to indicate that the 3D point cloud of each frame belongs to the movable platform 3D points off the road surface; the processor is used to:
    将所述每帧三维点云中携带所述第一标签的三维点投影到预设坐标系下。Projecting the three-dimensional point carrying the first tag in the three-dimensional point cloud of each frame into a preset coordinate system.
  23. 根据权利要求17所述的装置，其特征在于，所述处理器还用于：The apparatus of claim 17, wherein the processor is further configured to:
    获取所述可移动平台采集到的多帧原始三维点云；obtaining the multi-frame original 3D point cloud collected by the movable platform;
    对所述多帧原始三维点云进行分频处理，得到所述多帧三维点云。Frequency division processing is performed on the multi-frame original 3D point cloud to obtain the multi-frame 3D point cloud.
  24. 根据权利要求17所述的装置，其特征在于，所述处理器用于：The apparatus of claim 17, wherein the processor is configured to:
    若最大投影密度与其他任一投影密度之差均大于预设值，将所述最大投影密度对应的运动假设确定为匹配运动假设。If the difference between the maximum projected density and any other projected density is greater than a preset value, the motion hypothesis corresponding to the maximum projected density is determined as a matching motion hypothesis.
  25. 根据权利要求24所述的装置，其特征在于，所述处理器用于：The apparatus of claim 24, wherein the processor is configured to:
    若最大权重与至少一个其他权重之差不大于预设值，确定所述匹配运动假设不存在。If the difference between the maximum weight and the at least one other weight is not greater than a preset value, it is determined that the matching motion hypothesis does not exist.
  26. 根据权利要求17所述的装置，其特征在于，所述处理器用于：The apparatus of claim 17, wherein the processor is configured to:
    获取所述运动假设在各个栅格内对应的投影密度。Obtain the corresponding projected density of the motion hypothesis in each grid.
  27. 根据权利要求26所述的装置，其特征在于，所述处理器还用于：The apparatus of claim 26, wherein the processor is further configured to:
    若所述栅格内存在匹配运动假设，将所述栅格的匹配运动假设确定为所述栅格中各个点对应的三维点的匹配运动假设。If there is a matching motion hypothesis in the grid, the matching motion hypothesis of the grid is determined as the matching motion hypothesis of the three-dimensional point corresponding to each point in the grid.
  28. 根据权利要求26所述的装置，其特征在于，所述处理器用于：The apparatus of claim 26, wherein the processor is configured to:
    在基于所述运动假设将所述多帧三维点云投影到预设坐标系下之后，获取在所述栅格内存在投影点的三维点云的帧数；After projecting the multi-frame 3D point cloud into a preset coordinate system based on the motion hypothesis, acquiring the number of frames of the 3D point cloud with projected points in the grid;
    将所述帧数与所述栅格的面积之比确定为所述运动假设在所述栅格内对应的投影密度。The ratio of the number of frames to the area of the grid is determined as the corresponding projected density of the motion hypothesis within the grid.
  29. 根据权利要求26所述的装置，其特征在于，所述处理器用于：The apparatus of claim 26, wherein the processor is configured to:
    若一个栅格中的匹配运动假设的速度为0，将所述第一三维点云中投影到所述栅格中的三维点分割为静态点；和/或If the velocity assumed by the matching motion in a grid is 0, segment the 3D points projected into the grid from the first 3D point cloud into static points; and/or
    若一个栅格中的匹配运动假设的速度不为0，将所述第一三维点云中投影到所述栅格中的三维点分割为动态点。If the speed assumed by the matching motion in a grid is not 0, the 3D points projected into the grid from the first 3D point cloud are divided into dynamic points.
  30. 根据权利要求29所述的装置，其特征在于，所述处理器用于：The apparatus of claim 29, wherein the processor is configured to:
    若一个栅格中不存在匹配运动假设，将所述第一三维点云中投影到所述栅格中的三维点分割为属性未知的点。If there is no matching motion hypothesis in a grid, the 3D points projected into the grid from the first 3D point cloud are divided into points with unknown attributes.
  31. 根据权利要求17所述的装置，其特征在于，所述处理器还用于：The apparatus of claim 17, wherein the processor is further configured to:
    基于点云分割结果，为所述第一三维点云中的各个三维点打标签。Based on the point cloud segmentation result, each three-dimensional point in the first three-dimensional point cloud is labeled.
  32. 根据权利要求17所述的装置，其特征在于，所述三维点云基于安装于所述可移动平台上的视觉传感器或者激光雷达采集得到；和/或The device according to claim 17, wherein the three-dimensional point cloud is acquired based on a vision sensor or lidar installed on the movable platform; and/or
    对所述第一三维点云进行点云分割得到的点云分割结果用于所述可移动平台上的规划单元对所述可移动平台的行驶状态进行规划。The point cloud segmentation result obtained by performing point cloud segmentation on the first three-dimensional point cloud is used by the planning unit on the movable platform to plan the driving state of the movable platform.
  33. 一种可移动平台，其特征在于，包括：A movable platform is characterized in that, comprising:
    壳体；case;
    点云采集装置，设于所述壳体上，用于采集三维点云；以及a point cloud collecting device, arranged on the casing, for collecting three-dimensional point clouds; and
    三维点云分割装置，设于所述壳体内，用于执行权利要求1至16任意一项所述的 方法。A three-dimensional point cloud segmentation device, arranged in the casing, is used to execute the method described in any one of claims 1 to 16.
  34. 一种计算机可读存储介质，其特征在于，其上存储有计算机指令，该指令被处理器执行时实现权利要求1至16任意一项所述的方法。A computer-readable storage medium, characterized in that computer instructions are stored thereon, and when the instructions are executed by a processor, the method of any one of claims 1 to 16 is implemented.
PCT/CN2020/136546 2020-12-15 2020-12-15 Three-dimensional point cloud segmentation method and apparatus, and movable platform WO2022126380A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2020/136546 WO2022126380A1 (en) 2020-12-15 2020-12-15 Three-dimensional point cloud segmentation method and apparatus, and movable platform
CN202080070567.XA CN114556419A (en) 2020-12-15 2020-12-15 Three-dimensional point cloud segmentation method and device and movable platform

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/136546 WO2022126380A1 (en) 2020-12-15 2020-12-15 Three-dimensional point cloud segmentation method and apparatus, and movable platform

Publications (1)

Publication Number Publication Date
WO2022126380A1 true WO2022126380A1 (en) 2022-06-23

Family

ID=81667890

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/136546 WO2022126380A1 (en) 2020-12-15 2020-12-15 Three-dimensional point cloud segmentation method and apparatus, and movable platform

Country Status (2)

Country Link
CN (1) CN114556419A (en)
WO (1) WO2022126380A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116612129A (en) * 2023-06-02 2023-08-18 清华大学 Low-power consumption automatic driving point cloud segmentation method and device suitable for severe environment
CN117475398A (en) * 2023-12-26 2024-01-30 苏州元脑智能科技有限公司 Ground segmentation optimization method and device based on voxel sampling
WO2024145861A1 (en) * 2023-01-05 2024-07-11 Huawei Technologies Co., Ltd. Automatic range-based point cloud density optimization for lidar perception

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103578117A (en) * 2012-08-01 2014-02-12 三菱电机株式会社 Method for determining poses of camera relative to environment
US20150154467A1 (en) * 2013-12-04 2015-06-04 Mitsubishi Electric Research Laboratories, Inc. Method for Extracting Planes from 3D Point Cloud Sensor Data
CN107918753A (en) * 2016-10-10 2018-04-17 腾讯科技（深圳）有限公司 Processing Method of Point-clouds and device
CN111699410A (en) * 2019-05-29 2020-09-22 深圳市大疆创新科技有限公司 Point cloud processing method, device and computer readable storage medium
CN111712731A (en) * 2019-07-25 2020-09-25 深圳市大疆创新科技有限公司 Target detection method and system and movable platform

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103578117A (en) * 2012-08-01 2014-02-12 三菱电机株式会社 Method for determining poses of camera relative to environment
US20150154467A1 (en) * 2013-12-04 2015-06-04 Mitsubishi Electric Research Laboratories, Inc. Method for Extracting Planes from 3D Point Cloud Sensor Data
CN107918753A (en) * 2016-10-10 2018-04-17 腾讯科技（深圳）有限公司 Processing Method of Point-clouds and device
CN111699410A (en) * 2019-05-29 2020-09-22 深圳市大疆创新科技有限公司 Point cloud processing method, device and computer readable storage medium
CN111712731A (en) * 2019-07-25 2020-09-25 深圳市大疆创新科技有限公司 Target detection method and system and movable platform

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024145861A1 (en) * 2023-01-05 2024-07-11 Huawei Technologies Co., Ltd. Automatic range-based point cloud density optimization for lidar perception
CN116612129A (en) * 2023-06-02 2023-08-18 清华大学 Low-power consumption automatic driving point cloud segmentation method and device suitable for severe environment
CN117475398A (en) * 2023-12-26 2024-01-30 苏州元脑智能科技有限公司 Ground segmentation optimization method and device based on voxel sampling
CN117475398B (en) * 2023-12-26 2024-02-23 苏州元脑智能科技有限公司 Ground segmentation optimization method and device based on voxel sampling

Also Published As

Publication number Publication date
CN114556419A (en) 2022-05-27

Similar Documents

Publication Publication Date Title
US10437252B1 (en) High-precision multi-layer visual and semantic map for autonomous driving
US10794710B1 (en) High-precision multi-layer visual and semantic map by autonomous units
US10339669B2 (en) Method, apparatus, and system for a vertex-based evaluation of polygon similarity
US10668921B2 (en) Enhanced vehicle tracking
KR101829556B1 (en) Lidar-based classification of object movement
CN110226186B (en) Method and device for representing map elements and method and device for positioning
KR20120118478A (en) Traffic signal mapping and detection
WO2022126380A1 (en) Three-dimensional point cloud segmentation method and apparatus, and movable platform
WO2021003487A1 (en) Training data generation for dynamic objects using high definition map data
WO2022141116A1 (en) Three-dimensional point cloud segmentation method and apparatus, and movable platform
CN112082553A (en) Indoor positioning method and positioning device based on WIFI and laser radar and robot
Shan et al. LiDAR-based stable navigable region detection for unmanned surface vehicles
Tschopp et al. Hough $^ 2$ Map–Iterative Event-Based Hough Transform for High-Speed Railway Mapping
WO2022099620A1 (en) Three-dimensional point cloud segmentation method and apparatus, and mobile platform
CN112381873B (en) Data labeling method and device
Wang et al. Improved LeGO-LOAM method based on outlier points elimination
Al-Azizi et al. DeepAutoMapping: low-cost and real-time geospatial map generation method using deep learning and video streams
CN113838129A (en) Method, device and system for obtaining pose information
Chen et al. Exploring the effect of 3D object removal using deep learning for LiDAR-based mapping and long-term vehicular localization
Notz et al. Extraction and assessment of naturalistic human driving trajectories from infrastructure camera and radar sensors
Kumar et al. Citywide reconstruction of traffic flow using the vehicle-mounted moving camera in the carla driving simulator
CN117132620A (en) Multi-target tracking method, system, storage medium and terminal for automatic driving scene
CN116337068A (en) Robot synchronous semantic positioning and mapping method and system based on humanoid thought
Pang et al. FLAME: Feature-likelihood based mapping and localization for autonomous vehicles
CN113433566A (en) Map construction system and map construction method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20965398

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20965398

Country of ref document: EP

Kind code of ref document: A1