WO2023087526A1

WO2023087526A1 - Point cloud denoising method, electronic device, and storage medium

Info

Publication number: WO2023087526A1
Application number: PCT/CN2022/071296
Authority: WO
Inventors: 黄超; 孟泽楠
Original assignee: 上海仙途智能科技有限公司
Priority date: 2021-11-18
Filing date: 2022-01-11
Publication date: 2023-05-25
Also published as: CN115249349B; CN115249349A

Abstract

A point cloud denoising method, an electronic device, and a storage medium. The method comprises: acquiring a three-dimensional point cloud to be denoised (S101); performing down-sampling on first point cloud features extracted from the three-dimensional point cloud (S102); collecting shape feature statistics according to the down-sampled first point cloud features, and determining first point cloud features belonging to a same obstacle (S103); performing up-sampling on the first point cloud features belonging to the same obstacle to obtain second point cloud features, the dimension of the second point cloud features being the same as the dimension of the first point cloud features (S104); fusing the first point cloud features and the second point cloud features, and identifying noise points by using the fused point cloud features (S105); and removing the noise points in the three-dimensional point cloud according to the noise point identification result (S106). The method allows for accurate identification and removal of noise points.

Description

用于点云去噪的方法、电子设备及存储介质Method, electronic device and storage medium for point cloud denoising

技术领域technical field

本申请涉及点云处理领域，尤其涉及用于点云去噪的方法、电子设备及存储介质。The present application relates to the field of point cloud processing, and in particular to a method, electronic equipment and storage medium for point cloud denoising.

背景技术Background technique

随着激光雷达、深度相机、毫米波雷达等设备的广泛应用，三维点云广泛应用于不同的领域中，如自动驾驶领域、安检安防领域、巡检领域或者灾难救援领域等。With the wide application of laser radar, depth camera, millimeter-wave radar and other equipment, 3D point cloud is widely used in different fields, such as the field of automatic driving, security inspection and security, patrol inspection or disaster rescue.

示例性的，三维点云成为了除图像数据之外另一种在自动驾驶领域广泛应用的数据形式。可以通过点云采集装置如激光雷达采集的三维点云进行障碍物检测，以便基于障碍物检测结果辅助车辆进行良好的路径规划或者驾驶控制。Exemplarily, 3D point cloud has become another data form widely used in the field of autonomous driving besides image data. Obstacle detection can be performed through the 3D point cloud collected by a point cloud acquisition device such as lidar, so as to assist the vehicle in good path planning or driving control based on the obstacle detection result.

然而，在一些场景中，点云采集装置如激光雷达采集的三维点云中可能存在噪声点，导致障碍物检测结果不准确。比如在环卫车执行洒水作业的场景中，三维点云中与水雾相关的三维点是一个需要去除的噪声点。又比如在雾霾或者沙尘天气中，三维点云中与雾霾、尘等细小颗粒相关的三维点是一个需要去除的噪声点。However, in some scenarios, there may be noise points in the 3D point cloud collected by point cloud collection devices such as lidar, resulting in inaccurate obstacle detection results. For example, in the scene where the sanitation vehicle performs watering operations, the 3D point related to the water mist in the 3D point cloud is a noise point that needs to be removed. For another example, in haze or dust weather, the 3D point in the 3D point cloud related to fine particles such as haze and dust is a noise point that needs to be removed.

因此，为了提高障碍物检测的准确性，需要对三维点云中的噪声点进行有效去除。Therefore, in order to improve the accuracy of obstacle detection, it is necessary to effectively remove the noise points in the 3D point cloud.

发明内容Contents of the invention

有鉴于此，本申请提供一种点云去噪方法、电子设备及存储介质。In view of this, the present application provides a point cloud denoising method, electronic equipment and a storage medium.

具体地，本申请是通过如下技术方案实现的：第一方面，本申请实施例提供了一种点云去噪方法，所述方法包括：获取待去噪的三维点云；将从所述三维点云中提取的第一点云特征进行下采样处理；根据下采样后的第一点云特征进行形状特征统计，确定属于同一障碍物的第一点云特征；对所述属于同一障碍物的第一点云特征进行上采样处理，获取第二点云特征；所述第二点云特征的维度与所述第一点云特征的维度相同；将所述第一点云特征和所述第二点云特征进行融合处理，并使用融合后的点云特征进行噪声点识别；根据噪声点识别结果去除所述三维点云中的所述噪声点。Specifically, the present application is achieved through the following technical solutions: In the first aspect, the embodiment of the present application provides a point cloud denoising method, the method includes: obtaining a 3D point cloud to be denoised; The first point cloud feature extracted in the point cloud is subjected to down-sampling processing; according to the first point cloud feature after down-sampling, the shape feature statistics are carried out to determine the first point cloud feature belonging to the same obstacle; The first point cloud feature performs upsampling processing to obtain the second point cloud feature; the dimension of the second point cloud feature is the same as the dimension of the first point cloud feature; the first point cloud feature and the first point cloud feature are combined The two point cloud features are fused, and the fused point cloud features are used to identify noise points; and the noise points in the three-dimensional point cloud are removed according to the noise point identification results.

第二方面，本申请实施例提供了一种电子设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述程序时实现如第一方面所述的方法。In the second aspect, the embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and operable on the processor. When the processor executes the program, the first aspect is implemented. the method described.

第三方面，本申请实施例提供了一种计算机可读存储介质，所述计算机可读存储介质存储有可执行指令，所述可执行指令被处理器执行时实现如第一方面所述的方法。In a third aspect, an embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores executable instructions, and when the executable instructions are executed by a processor, the method as described in the first aspect is implemented .

本申请实施例提供的点云去噪方法、电子设备及存储介质，在进行噪声点识别的过程中，为了节省电子设备的算力资源，在进行形状特征统计过程中，首先将从所述三维点云中提取的第一点云特征进行下采样处理，从而节省了形状特征统计过程的计算量，提高统计效率，进而考虑到下采样过程会造成低精度信息的损失，因此将第一点云特征和上采样得到的第二点云特征进行融合处理，利用所述第一点云特征弥补下采样过程中损失的低精度信息，进而使用融合后的点云特征进行噪声点识别，有利于提高噪声点识别的准确性，实现将所述三维点云中的所述噪声点进行良好去除。In the point cloud denoising method, electronic equipment, and storage medium provided by the embodiments of the present application, in the process of identifying noise points, in order to save computing power resources of the electronic equipment, in the process of performing shape feature statistics, firstly, the three-dimensional The first point cloud feature extracted from the point cloud is down-sampled, which saves the calculation amount of the shape feature statistical process and improves the statistical efficiency. Considering that the down-sampling process will cause the loss of low-precision information, the first point cloud feature and the second point cloud feature obtained by up-sampling are fused, and the first point cloud feature is used to make up for the low-precision information lost in the down-sampling process, and then the fused point cloud feature is used to identify noise points, which is beneficial to improve The accuracy of noise point identification enables good removal of the noise points in the three-dimensional point cloud.

附图说明Description of drawings

图1是本申请一示例性实施例示出的一种点云去噪方法的流程示意图。Fig. 1 is a schematic flowchart of a point cloud denoising method shown in an exemplary embodiment of the present application.

图2是本申请一示例性实施例示出的一种噪声点识别模型的架构图。Fig. 2 is a structural diagram of a noise point recognition model shown in an exemplary embodiment of the present application.

图3是本申请一示例性实施例示出的人工标注过程框选噪声点的运行轨迹的示意图。Fig. 3 is a schematic diagram of a running track of frame-selected noise points in a manual labeling process according to an exemplary embodiment of the present application.

图4是本申请一示例性实施例示出的一种电子设备的结构示意图。Fig. 4 is a schematic structural diagram of an electronic device shown in an exemplary embodiment of the present application.

具体实施方式Detailed ways

这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本申请相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本申请的一些方面相一致的装置和方法的例子。Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present application as recited in the appended claims.

在本申请使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本申请。在本申请和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。The terminology used in this application is for the purpose of describing particular embodiments only, and is not intended to limit the application. As used in this application and the appended claims, the singular forms "a", "the", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It should also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

应当理解，尽管在本申请可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本申请范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，如在此所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of the present application, first information may also be called second information, and similarly, second information may also be called first information. Depending on the context, the word "if" as used herein may be interpreted as "at" or "when" or "in response to a determination."

针对于相关技术中的问题，本申请实施例提供了一种点云去噪方法，实现对待去噪的三维点云进行有效的噪声点识别，并进行噪声点去除。In view of the problems in the related technologies, the embodiment of the present application provides a point cloud denoising method, which realizes effective noise point identification and noise point removal on the 3D point cloud to be denoised.

本申请实施例提供的点云去噪方法可应用于电子设备。示例性的，所述电子设备可以包括有执行所述点云去噪方法的程序。示例性的，所述电子设备至少包括有存储器和处理器，所述存储器存储有所述点云去噪方法的可执行指令，所述处理器可被配置为执行所述可执行指令。The point cloud denoising method provided in the embodiment of the present application can be applied to electronic equipment. Exemplarily, the electronic device may include a program for executing the point cloud denoising method. Exemplarily, the electronic device includes at least a memory and a processor, the memory stores executable instructions of the point cloud denoising method, and the processor can be configured to execute the executable instructions.

在一示例性的应用场景中，三维点云广泛应用于车辆驾驶领域中，可以通过点云采集装置如激光雷达采集的三维点云进行障碍物检测，以便基于障碍物检测结果辅助车辆进行良好的路径规划或者驾驶控制。考虑到点云采集装置采集的三维点云中存在噪声点导致障碍物检测结果不准确的问题，所述噪声点包括但不限于水雾、砂砾、灰尘或者等细颗粒物对应的三维点，因此，所述电子设备可以是车载终端，车载终端可以在接收点云采集装置采集的三维点云之后，使用本申请实施例提供的点云去噪方法去除所述三维点云中的噪声点，从而提高后续进行障碍物检测的准确性。In an exemplary application scenario, 3D point clouds are widely used in the field of vehicle driving. Obstacle detection can be performed on the 3D point clouds collected by a point cloud collection device such as lidar, so as to assist the vehicle to perform good driving based on the obstacle detection results. Path planning or driving control. Considering that there are noise points in the 3D point cloud collected by the point cloud collection device, which lead to inaccurate obstacle detection results, the noise points include but are not limited to 3D points corresponding to water mist, gravel, dust or other fine particles, therefore, The electronic device may be a vehicle-mounted terminal, and the vehicle-mounted terminal may use the point cloud denoising method provided in the embodiment of the present application to remove noise points in the 3D point cloud after receiving the 3D point cloud collected by the point cloud collection device, thereby improving The accuracy of subsequent obstacle detection.

在另一示例性的应用场景中，三维点云也广泛应用于巡检领域，例如在仓库或者大型工厂等场所中，巡检机器人可以搭载有点云采集装置，通过点云采集装置如激光雷达采集的三维点云进行障碍物检测，以便基于障碍物检测结果辅助巡检机器人进行良好的巡检路径规划或者进行避障。考虑到点云采集装置采集的三维点云中存在噪声点导致障碍物检测结果不准确的问题，所述噪声点包括但不限于灰尘或者粉尘等细颗粒物对应的三维点，因此，所述电子设备可以是巡检机器人或者安装于巡检机器人上的终端，巡检机器人或者安装于巡检机器人上的终端可以在接收点云采集装置采集的三维点云之后，使用本申请实施例提供的点云去噪方法去除所述三维点云中的噪声点，从而提高后续进行障碍物检测的准确性。In another exemplary application scenario, the 3D point cloud is also widely used in the field of inspection. For example, in warehouses or large factories, the inspection robot can be equipped with a point cloud collection device. Based on the obstacle detection results, it can assist the inspection robot in planning a good inspection path or avoiding obstacles. Considering that there are noise points in the three-dimensional point cloud collected by the point cloud collection device, which lead to inaccurate obstacle detection results, the noise points include but are not limited to three-dimensional points corresponding to fine particles such as dust or dust, therefore, the electronic device It can be an inspection robot or a terminal installed on the inspection robot. The inspection robot or the terminal installed on the inspection robot can use the point cloud provided by the embodiment of the present application after receiving the three-dimensional point cloud collected by the point cloud acquisition device. The denoising method removes noise points in the three-dimensional point cloud, thereby improving the accuracy of subsequent obstacle detection.

当然，电子设备也可以是其他类型的设备，本实施例对此不做任何限制，比如电子设备可以是服务器，可以从其他设备(如自动驾驶车辆、巡检机器人)接收待去噪的三维点云，使用本申请实施例提供的点云去噪方法去除所述三维点云中的噪声点之后，并去噪后的三维点云返回给其他设备。Of course, the electronic device can also be other types of devices, and this embodiment does not impose any restrictions on this. For example, the electronic device can be a server that can receive 3D points to be denoised from other devices (such as self-driving vehicles, inspection robots) cloud, use the point cloud denoising method provided by the embodiment of the present application to remove noise points in the 3D point cloud, and return the denoised 3D point cloud to other devices.

接下来对本申请实施例提供的点云去噪方法进行示例性说明，请参阅图1，图1为一种点云去噪方法的流程示意图，所述方法可应用于电子设备中，所述方法包括：在步骤S101中，获取待去噪的三维点云。Next, the point cloud denoising method provided by the embodiment of the present application is exemplified. Please refer to FIG. 1. FIG. 1 is a schematic flow chart of a point cloud denoising method, which can be applied to electronic devices. The method Including: in step S101, acquiring the 3D point cloud to be denoised.

在步骤S102中，将从所述三维点云中提取的第一点云特征进行下采样处理。In step S102, downsampling is performed on the first point cloud features extracted from the three-dimensional point cloud.

在步骤S103中，根据下采样后的第一点云特征进行形状特征统计，确定属于同一障碍物的第一点云特征。In step S103, the shape feature statistics are performed according to the down-sampled first point cloud features, and the first point cloud features belonging to the same obstacle are determined.

在步骤S104中，对所述属于同一障碍物的第一点云特征进行上采样处理，获取第二点云特征；所述第二点云特征的维度与所述第一点云特征的维度相同。In step S104, the first point cloud feature belonging to the same obstacle is subjected to upsampling processing to obtain a second point cloud feature; the dimension of the second point cloud feature is the same as the dimension of the first point cloud feature .

在步骤S105中，将所述第一点云特征和所述第二点云特征进行融合处理，并使用融合后的点云特征进行噪声点识别。In step S105, the first point cloud feature and the second point cloud feature are fused, and noise point recognition is performed using the fused point cloud feature.

在步骤S106中，根据噪声点识别结果去除所述三维点云中的所述噪声点。In step S106, the noise points in the 3D point cloud are removed according to the noise point identification result.

本实施例中，在进行噪声点识别的过程中，为了节省电子设备的算力资源，在进行形状特征统计过程中，首先将从所述三维点云中提取的第一点云特征进行下采样处理，从而节省了形状特征统计过程的计算量，提高统计效率，进而考虑到下采样过程会造成低精度信息的损失，因此将第一点云特征和上采样得到的第二点云特征进行融合处理，利用所述第一点云特征弥补下采样过程中损失的低精度信息，进而使用融合后的点云特征进行噪声点识别，有利于提高噪声点识别的准确性，实现将所述三维点云中的所述噪声点进行良好去除。In this embodiment, in order to save computing power resources of the electronic device during the noise point recognition process, the first point cloud feature extracted from the 3D point cloud is first down-sampled during the shape feature statistics process processing, thereby saving the calculation amount of the statistical process of shape features and improving statistical efficiency, and considering the loss of low-precision information caused by the downsampling process, the first point cloud feature and the second point cloud feature obtained by upsampling are fused processing, using the first point cloud feature to make up for the low-precision information lost in the down-sampling process, and then using the fused point cloud feature to identify noise points, which is conducive to improving the accuracy of noise point identification and realizing the three-dimensional point The noise points in the cloud are well removed.

在一些实施例中，所述待去噪的三维点云可由点云采集装置来采集，目前主流的点云采集装置分为两类：主动式和被动式。主动式传感器又可分为基于TOF(Time of Flight)***和三角测量***两种，其中TOF***通过测量所发射信号到达物体表面和返回接收器之间的时间间隔来确定传感器到物体表面的真实距离，而三角测量***则通过两个传感器在不同地点对物体同一点之间的测量关系计算点的空间位置。被动式传感器依赖图像对或图像序列并根据相机参数来从二维图像数据中复原出三维数据。典型的主动式的点云采集装置包括但不限于激光雷达、深度相机(如RGB-D相机)、毫米波雷达或者双目视觉传感器等等；典型的被动式的点云采集装置包括但不限于立体相机、SFM(structure from motion)***、SFS(shape from shading)***等。In some embodiments, the 3D point cloud to be denoised may be collected by a point cloud collection device. Currently, mainstream point cloud collection devices are divided into two categories: active and passive. Active sensors can be divided into two types based on TOF (Time of Flight) systems and triangulation systems. The TOF system determines the true distance from the sensor to the surface of the object by measuring the time interval between the transmitted signal reaching the surface of the object and returning to the receiver. The triangulation system calculates the spatial position of a point by measuring the relationship between two sensors at different locations on the same point of the object. Passive sensors rely on image pairs or image sequences to recover 3D data from 2D image data based on camera parameters. Typical active point cloud collection devices include, but are not limited to, lidar, depth cameras (such as RGB-D cameras), millimeter-wave radars, or binocular vision sensors, etc.; typical passive point cloud collection devices include, but are not limited to, stereo Camera, SFM (structure from motion) system, SFS (shape from shading) system, etc.

示例性的，所述电子设备为安装于车辆上的车载终端，该车载终端与点云采集装置(如安装于车辆上的激光雷达)通信连接，搭载于车辆上的点云采集装置在车辆驾驶过程中采集三维点云，车载终端可以获取点云采集装置采集的待去噪的三维点云，并利用如步骤S101～S106的方法进行噪声点去除。示例性的，所述电子设备为巡检机器人，所述巡检机器人安装有点云采集装置，巡检机器人可以获取点云采集装置采集的待去噪的三维点云，并利用如步骤S101～S106的方法进行噪声点去除。Exemplarily, the electronic device is a vehicle-mounted terminal installed on a vehicle, and the vehicle-mounted terminal communicates with a point cloud acquisition device (such as a laser radar installed on the vehicle). During the process of collecting 3D point clouds, the vehicle-mounted terminal can obtain the 3D point clouds to be denoised collected by the point cloud collection device, and use the methods such as steps S101 to S106 to remove noise points. Exemplarily, the electronic device is an inspection robot, the inspection robot is equipped with a point cloud collection device, and the inspection robot can obtain the 3D point cloud to be denoised collected by the point cloud collection device, and use steps S101-S106 method for noise point removal.

其中，所述三维点云中的噪声点包括但不限于至少一种细颗粒物对应的三维点：水雾、砂砾、灰尘或者粉尘等等。可以理解的是，本申请对于细颗粒物的类型不做任何限制。Wherein, the noise points in the 3D point cloud include, but are not limited to, 3D points corresponding to at least one fine particle: water mist, gravel, dust or dust, and the like. It can be understood that this application does not impose any limitation on the type of fine particles.

在一些实施例中，在获取待去噪的三维点云之后，可以从待去噪的三维点云提取第一点云特征，以便进行后续处理。In some embodiments, after the three-dimensional point cloud to be denoised is acquired, first point cloud features may be extracted from the three-dimensional point cloud to be denoised for subsequent processing.

作为一种可能的实现方式，所述第一点云特征包括所述三维点云中的若干三维点的特征，所述三维点的特征包括但不限于三维点的坐标、反射强度、三维点对应的光脉冲序列的标识(比如雷达线束ID)、深度信息(比如三维点与点云采集装置的距离)、高度信息或者角度信息(比如三维点与点云采集装置的连线的偏转角)。As a possible implementation, the first point cloud features include features of several 3D points in the 3D point cloud, and the features of the 3D points include but not limited to coordinates of 3D points, reflection intensity, and corresponding The identification of the light pulse sequence (such as the radar beam ID), depth information (such as the distance between the 3D point and the point cloud collection device), height information or angle information (such as the deflection angle of the line connecting the 3D point and the point cloud collection device).

作为另一种可能的实现方式，考虑到直接针对于三维点的特征获取第一点云特征的数据量比较大，需要耗费较多的计算资源，因此为了提高针对噪声点的识别效率，可以对待去噪的三维点云进行三维网格化处理，按照预设距离分割所述三维点云，获取三维网格化的三维点云；可以理解的是，所述预设距离可依据实际应用场景进行具体设置，本实施例对此不做任何限制，比如所述预设距离包括但不限于5cm、20cm或者1m等。在获取三维网格化的三维点云之后，可以对所述三维网格化的三维点云中的每个三维网格进行特征提取，比如可以根据该三维网格内的三维点的特征的统计值确定该三维网格的特征，从而基于所述三维点云中的所有三维网格的特征获取所述第一点云特征。其中，所述三维点的特征包括以下至少一种：三维点的坐标、反射强度、三维点对应的光脉冲序列的标识、深度信息、高度信息或者角度信息。所述统计值包括但不限于平均值、中位数、最大值或者最小值等等，例如可以根据该三维网格内的三维点的特征的平均值确定该三维网格的特征，可依据实际应用场景进行具体计算，本实施例对此不做任何限制。本实施例中，对待去噪的三维点云进行三维网格化处理，并提取三维网格的特征作为第一点云特征，有利于降低参与运算的数据量，提高了对于噪声点的识别效率。As another possible implementation method, considering that the data volume of the first point cloud feature obtained directly for the feature of the 3D point is relatively large, and it needs to consume more computing resources, so in order to improve the recognition efficiency of the noise point, it can be treated as Perform three-dimensional grid processing on the denoised three-dimensional point cloud, segment the three-dimensional point cloud according to a preset distance, and obtain a three-dimensional gridded three-dimensional point cloud; it can be understood that the preset distance can be determined according to the actual application scene The specific setting is not limited in this embodiment. For example, the preset distance includes but is not limited to 5cm, 20cm, or 1m. After obtaining the 3D gridded 3D point cloud, feature extraction can be performed on each 3D grid in the 3D gridded 3D point cloud, for example, according to the statistics of the features of the 3D points in the 3D grid The value determines the feature of the 3D mesh, so as to obtain the feature of the first point cloud based on the features of all the 3D mesh in the 3D point cloud. Wherein, the characteristics of the three-dimensional point include at least one of the following: coordinates of the three-dimensional point, reflection intensity, identification of the light pulse sequence corresponding to the three-dimensional point, depth information, height information or angle information. The statistical value includes but not limited to average value, median, maximum or minimum value, etc., for example, the characteristics of the three-dimensional grid can be determined according to the average value of the characteristics of the three-dimensional points in the three-dimensional grid, and can be determined according to the actual Specific calculations are performed in application scenarios, which is not limited in this embodiment. In this embodiment, the three-dimensional point cloud to be denoised is subjected to three-dimensional grid processing, and the features of the three-dimensional grid are extracted as the first point cloud feature, which is beneficial to reduce the amount of data involved in the calculation and improve the recognition efficiency of noise points .

在一些实施例中，在获取第一点云特征之后，为了节省后续进行形状特征统计过程的计算资源，另一方面也考虑到电子设备算力有限的场景，因此可以先对第一点云特征进行下采样处理，比如第一点云特征的维数为1024*1024*1024维，下采样后的第一点云特征的维数为256*256*256维，进而利用下采样后的第一点云特征进行形状特征统计，有利于节省计算资源，形状特征统计过程即检测三维点(或者三维网格)之间的相对关系，以确定是否属于同一障碍物，对下采样后的第一点云特征进行形状特征的划分统计可以确定属于各个障碍物的第一点云特征(或者说属于各个障碍物的各个三维点或者三维网格)，进而进行上采样处理得到第二点云特征，所述第二点云特征的维度与所述第一点云特征的维度相同，比如第二点云特征也是1024*1024*1024维，最后利用融合了所述第一点云特征和所述第二点云特征的点云特征进行噪声点识别，有利于提高噪声点识别的准确性。In some embodiments, after acquiring the first point cloud features, in order to save computing resources for the subsequent shape feature statistics process, and on the other hand, considering the scene of limited computing power of electronic devices, the first point cloud features can be calculated first Perform downsampling processing. For example, the dimension of the first point cloud feature is 1024*1024*1024 dimensions, and the dimension of the first point cloud feature after downsampling is 256*256*256 dimensions, and then use the first point cloud feature after downsampling The shape feature statistics of point cloud features is beneficial to save computing resources. The shape feature statistics process is to detect the relative relationship between 3D points (or 3D grids) to determine whether they belong to the same obstacle. The first point after downsampling Cloud features can determine the first point cloud features belonging to each obstacle (or each 3D point or 3D grid belonging to each obstacle) by dividing and counting the shape features of cloud features, and then perform up-sampling processing to obtain the second point cloud features, so The dimension of the second point cloud feature is the same as the dimension of the first point cloud feature, such as the second point cloud feature is also 1024*1024*1024 dimensions, and finally utilizes the fusion of the first point cloud feature and the second The point cloud feature of the point cloud feature is used for noise point recognition, which is beneficial to improve the accuracy of noise point recognition.

在一些实施例中，可以预先构建一噪声点识别模型，用于识别待去噪的三维点云中的噪声点，例如在获取待去噪的三维点云之后，可以将所述三维点云输入预先建立的噪声点识别模型中，通过所述噪声点识别模型对所述三维点云进行处理后得到所述噪声点识别结果。In some embodiments, a noise point recognition model can be pre-built to identify noise points in the 3D point cloud to be denoised, for example, after obtaining the 3D point cloud to be denoised, the 3D point cloud can be input In the pre-established noise point recognition model, the noise point recognition result is obtained after the three-dimensional point cloud is processed by the noise point recognition model.

请参阅图2，图2示出了噪声点识别模型的架构图，所述噪声点识别模型至少包括特征提取层22、形状识别网络23和噪声点识别网络24等，以及所述噪声点识别模型还可以包括输入层21和输出层25。Please refer to Fig. 2, Fig. 2 has shown the architectural diagram of noise point recognition model, described noise point recognition model at least comprises feature extraction layer 22, shape recognition network 23 and noise point recognition network 24 etc., and described noise point recognition model An input layer 21 and an output layer 25 may also be included.

所述输入层21用于获取输入的待去噪的三维点云。The input layer 21 is used to acquire the input 3D point cloud to be denoised.

所述特征提取层22用于对所述三维点云进行特征提取，例如可以按照预设距离分割所述三维点云，获取三维网格化的三维点云，然后对所述三维网格化的三维点云中的每个三维网格进行特征提取，获取所述第一点云特征；其中，每个三维网格的特征为该三维网格内的三维点的特征的统计值(比如平均值等)。本实施例中提取网格特征有利于减少后续参与运算的数据量，有利于提高运算效率。The feature extraction layer 22 is used to extract features from the 3D point cloud, for example, the 3D point cloud can be segmented according to a preset distance to obtain a 3D gridded 3D point cloud, and then the 3D gridded 3D point cloud can be obtained. Each three-dimensional grid in the three-dimensional point cloud performs feature extraction to obtain the first point cloud features; wherein, the feature of each three-dimensional grid is the statistical value (such as the average value) of the characteristics of the three-dimensional points in the three-dimensional grid wait). Extracting grid features in this embodiment is beneficial to reducing the amount of data involved in subsequent calculations and improving calculation efficiency.

所述形状识别网络23包括系数卷积层231、第一多层感知机网络232和上采样层233；所述系数卷积层231用于对所述第一点云特征进行下采样处理；所述第一多层感知机网络232用于根据下采样后的第一点云特征进行形状特征统计，比如根据三维网格之间的位置关系进行形状特征统计分析，确定属于同一障碍物的第一点云特征，即将所述第一点云特征按照障碍物进行划分统计；所述上采样层233用于将所述属于同一障碍物的第一点云特征进行上采样处理，获取第二点云特征。在所述形状识别网络23中，将第一点云特征进行下采样处理，从而节省了第一多层感知机网络232在进行形状特征统计过程的计算量，提高统计效率。The shape recognition network 23 includes a coefficient convolution layer 231, a first multi-layer perceptron network 232 and an upsampling layer 233; the coefficient convolution layer 231 is used for downsampling the first point cloud features; The first multi-layer perceptron network 232 is used to perform shape feature statistics according to the downsampled first point cloud features, such as performing statistical analysis of shape features according to the positional relationship between three-dimensional grids to determine the first point cloud belonging to the same obstacle. Point cloud features, that is, divide and count the first point cloud features according to obstacles; the upsampling layer 233 is used to upsample the first point cloud features belonging to the same obstacle to obtain a second point cloud feature. In the shape recognition network 23, the first point cloud features are down-sampled, thereby saving the calculation amount of the first multi-layer perceptron network 232 in the shape feature statistics process, and improving the statistical efficiency.

所述噪声点识别网络24包括特征融合层241和第二多层感知机网络242；所述特征融合层241用于融合所述第一点云特征和所述第二点云特征；所述第二多层感知机网络242用于对融合后的点云特征进行噪声点识别，以获取噪声点识别结果。在所述噪声点识别网络24，考虑到下采样过程会造成低精度信息的损失，因此将第一点云特征和上采样得到的第二点云特征进行融合处理，利用所述第一点云特征弥补下采样过程中损失的低精度信息，进而使用融合后的点云特征进行噪声点识别，有利于提高噪声点识别的准确性。The noise point recognition network 24 includes a feature fusion layer 241 and a second multilayer perceptron network 242; the feature fusion layer 241 is used to fuse the first point cloud feature and the second point cloud feature; the second point cloud feature The two-layer multi-layer perceptron network 242 is used to perform noise point recognition on the fused point cloud features to obtain noise point recognition results. In the noise point recognition network 24, considering that the downsampling process will cause the loss of low-precision information, the first point cloud feature and the second point cloud feature obtained by upsampling are fused, and the first point cloud feature is used The feature makes up for the low-precision information lost in the downsampling process, and then uses the fused point cloud features to identify noise points, which is conducive to improving the accuracy of noise point recognition.

所述输出层25用于输出所述噪声点识别结果。本实施例中，所述噪声点识别模型在进行形状特征统计时，为了进一步减低运算量，节省计算资源，通过系数卷积层231对第一点云特征进行下采样处理，进而可以适用于运算资源有限的设备中，使得所述噪声点识别模型适用于更多的设备，具有广泛的适用性；并且考虑到下采样过程会造成低精度信息的损失，因此噪声点识别模型在形状识别网络23后半部分设计了上采样结构将点云特征恢复成下采样之前的大小，并将上采样得到的第二点云特征与第一点云特征进行融合，弥补了下采样过程中损失的低精度信息，进而第二多层感知机网络242对融合后的点云特征进行噪声点识别，有利于提高噪声点识别的准确性，实现了计算资源和精准度的权衡。The output layer 25 is used to output the noise point identification result. In this embodiment, when the noise point recognition model performs shape feature statistics, in order to further reduce the amount of computation and save computing resources, the first point cloud feature is down-sampled through the coefficient convolution layer 231, and then it can be applied to the calculation In devices with limited resources, the noise point recognition model is suitable for more devices and has wide applicability; and considering that the downsampling process will cause the loss of low-precision information, the noise point recognition model is used in the shape recognition network23 In the second half, an upsampling structure is designed to restore the point cloud features to the size before downsampling, and fuse the second point cloud features obtained by upsampling with the first point cloud features to make up for the low precision lost in the downsampling process. Information, and then the second multi-layer perceptron network 242 performs noise point recognition on the fused point cloud features, which is conducive to improving the accuracy of noise point recognition and achieving a trade-off between computing resources and accuracy.

接下来对所述噪声点识别模型的训练过程进行示例性说明，使用有监督学习方式进行训练为例，所述噪声点识别模型基于若干标注有噪声点的三维点云样本训练得到。Next, the training process of the noise point recognition model is exemplified, using a supervised learning method as an example. The noise point recognition model is trained based on several 3D point cloud samples marked with noise points.

首先，针对于训练样本的获取，可以将多个单帧三维点云样本融合成第一稠密点云，其中，所述多个单帧三维点云样本由搭载于可移动平台上的点云采集装置在移动过程中采集得到，所述可移动平台可以是车辆或者移动机器人等，以车辆为例，应用本申请实施例的点云去噪方法的电子设备可以是安装于所述车辆上的车载终端，所述多个单帧三维点云样本由搭载于车辆上的点云采集装置在车辆驾驶过程中采集得到。First, for the acquisition of training samples, multiple single-frame 3D point cloud samples can be fused into the first dense point cloud, wherein the multiple single-frame 3D point cloud samples are collected by a point cloud mounted on a mobile platform The device is collected during the moving process, and the movable platform may be a vehicle or a mobile robot. Taking a vehicle as an example, the electronic device applying the point cloud denoising method of the embodiment of the present application may be a vehicle mounted on the vehicle. In the terminal, the multiple single-frame three-dimensional point cloud samples are collected by a point cloud collection device mounted on the vehicle during driving of the vehicle.

示例性的，在融合过程中，首先可以通过点云配准方式确定点云采集装置在采集每帧三维点云样本时的位姿，进而基于点云采集装置在采集每帧三维点云样本时的位姿将多个单帧三维点云样本变换到同一三维坐标系下，在该三维坐标系中实现多个单帧三维点云样本的融合过程。Exemplarily, in the fusion process, the pose of the point cloud acquisition device when collecting each frame of three-dimensional point cloud samples can be determined first through point cloud registration, and then based on the point cloud acquisition device when collecting each frame of three-dimensional point cloud samples The pose transforms multiple single-frame 3D point cloud samples into the same 3D coordinate system, and realizes the fusion process of multiple single-frame 3D point cloud samples in this 3D coordinate system.

其中，在将多个单帧三维点云样本融合成第一稠密点云，静态障碍物(如树木，停止的车)会在第一稠密点云中表现为更稠密的状态(因为累积了多帧的三维点)，而动态障碍物(如行人，运动的车)会在第一稠密点云中呈现一条运动的轨迹，对应其相对于自身动态运动的方向。而水雾、砂砾、灰尘或者粉尘等细颗粒物由于体积、重量等都很小，因此会根据可移动平台(如车辆)的行驶状态，风向风力等等因素呈现不同的运动状态，则在第一稠密点云中，水雾、砂砾、灰尘或者粉尘等细颗粒物也会呈现一条运动的轨迹，对应其相对于自身动态运动的方向。Among them, when multiple single-frame 3D point cloud samples are fused into the first dense point cloud, static obstacles (such as trees, stopped cars) will appear in a denser state in the first dense point cloud (because more 3D points of the frame), while dynamic obstacles (such as pedestrians, moving cars) will present a moving trajectory in the first dense point cloud, corresponding to their own dynamic motion direction. Fine particles such as water mist, gravel, dust or dust, due to their small size and weight, will present different motion states according to the driving state of the movable platform (such as a vehicle), wind direction and wind force, etc. In the dense point cloud, fine particles such as water mist, gravel, dust or dust will also present a movement trajectory, corresponding to the direction of its dynamic movement relative to itself.

在一种可能的实现方式中，在获取第一稠密点云之后，可以将所述第一稠密点云中属于地面的三维点和属于所述可移动平台(比如车辆)的三维点去除，获取第二稠密点云；其中，属于地面的三维点是高度为0的三维点，而由于点云采集装置搭载在所述可移动平台(比如车辆)上，点云采集装置的位置可知，则属于所述可移动平台(比如车辆)的三维点也是可以确定的；在去除了属于地面的三维点和属于所述可移动平台(比如车辆)的三维点之后，则所述第二稠密点云中在所述可移动平台(比如车辆)周围的具有运动轨迹的三维点可以确定为水雾、砂砾、灰尘或者粉尘等细颗粒物对应的三维点，可以将该类三维点确定为噪声点并进行标注，进而可以根据标注的噪声点将所述第一稠密点云拆分得到多个标注有噪声点的单帧三维点云样本。本实施例中，实现噪声点的自动标注过程，并且将多个单帧三维点云样本融合成第一稠密点云一次性进行标注，有利于提高标注效率。In a possible implementation manner, after obtaining the first dense point cloud, the 3D points belonging to the ground and the 3D points belonging to the movable platform (such as a vehicle) in the first dense point cloud may be removed to obtain The second dense point cloud; wherein, the three-dimensional point belonging to the ground is a three-dimensional point with a height of 0, and since the point cloud collection device is mounted on the movable platform (such as a vehicle), the position of the point cloud collection device is known, then it belongs to The three-dimensional points of the movable platform (such as a vehicle) can also be determined; after removing the three-dimensional points belonging to the ground and the three-dimensional points belonging to the movable platform (such as a vehicle), then in the second dense point cloud The three-dimensional points with motion tracks around the movable platform (such as vehicles) can be determined as three-dimensional points corresponding to fine particles such as water mist, gravel, dust or dust, and these three-dimensional points can be determined as noise points and marked , and then the first dense point cloud may be split according to the marked noise points to obtain a plurality of single-frame three-dimensional point cloud samples marked with noise points. In this embodiment, the automatic labeling process of noise points is realized, and multiple single-frame three-dimensional point cloud samples are fused into the first dense point cloud for one-time labeling, which is beneficial to improve labeling efficiency.

当然，另一种可能的实现方式中，为了提高噪声点标注的准确性，也可以由用户进行人工标注，比如将第二稠密点云投影到二维空间以得到标注图像，由用户根据经验在标注图像中框选水雾、砂砾、灰尘或者粉尘等细颗粒物的运动轨迹，比如可以简便地利用鼠标点按或者使用其他工具如触控笔的方式将细颗粒物的运动轨迹框住，比如如图3所示，图3中的黑色框示出了人工标注过程框选的噪声点的运行轨迹；并且设置固定的高度值，进而电子设备可以将标注图像中框选的区域反投影到三维空间，以获取标注的噪声点，最后根据标注的噪声点将所述第一稠密点云拆解得到多个标注有噪声点的单帧三维点云样本。本实施例中，人工标注过程有利于提高噪声点标注的准确性，并且将多个单帧三维点云样本融合成第一稠密点云进行标注，只需简单进行框选即可，无需每个点都进行标注，有利于提高人工标注效率。Of course, in another possible implementation, in order to improve the accuracy of noise point labeling, the user can also manually label, such as projecting the second dense point cloud into a two-dimensional space to obtain the labeled image, and the user can use experience to Mark the trajectory of fine particles such as water mist, gravel, dust or dust in the marked image. For example, you can easily use the mouse to click or use other tools such as a stylus to frame the trajectory of fine particles, such as the figure 3, the black box in Figure 3 shows the running track of the noise points framed by the manual labeling process; and a fixed height value is set, and then the electronic device can back-project the framed area in the labeled image to the three-dimensional space, to obtain marked noise points, and finally disassemble the first dense point cloud according to the marked noise points to obtain a plurality of single-frame three-dimensional point cloud samples marked with noise points. In this embodiment, the manual labeling process is conducive to improving the accuracy of noise point labeling, and multiple single-frame 3D point cloud samples are fused into the first dense point cloud for labeling, which only needs to be selected simply, without the need for each All points are labeled, which is beneficial to improve the efficiency of manual labeling.

进一步地，在另一种可能的实现方式中，在获取第一稠密点云之后，除了可以去除属于地面的三维点和属于所述可移动平台(比如车辆)的三维点，还可以根据所述第一稠密点云进行障碍物识别以获得障碍物识别结果，比如可以将所述第一稠密点云输入现有的障碍物检测模型，利用现有的障碍物检测模型进行障碍物识别，并获取障碍物检测模型输出的障碍物识别结果；然后根据所述障碍物识别结果将所述第一稠密点云中属于障碍物的三维点去除，获得所述第二稠密点云，本实施例中除了去除地面点和本车的三维点之外，还将属于其他障碍物(比如其他车辆、行人、摩托车、树)的三维点去除，进一步消除干扰因素，在可移动平台(比如车辆)的移动路径(比如道路)上的障碍物都被清除之后，则道路上剩下的具有运动轨迹的三维点可以确定为水雾、砂砾、灰尘或者粉尘等细颗粒物对应的三维点，因此可以将所述第二稠密点云中处于移动路径(比如道路)上的具有运动轨迹的三维点标注为噪声点，最后根据标注的所述噪声点将所述第一稠密点云拆分，得到多个标注有噪声点的单帧三维点云样本。本实施例中，将无关因素去除，消除了同样具有运动轨迹的障碍物的干扰，有利于进一步提高噪声点标注的准确性。Further, in another possible implementation manner, after obtaining the first dense point cloud, in addition to removing the 3D points belonging to the ground and the 3D points belonging to the mobile platform (such as a vehicle), it can also be based on the Obstacle identification is performed on the first dense point cloud to obtain an obstacle identification result. For example, the first dense point cloud can be input into an existing obstacle detection model, and the existing obstacle detection model is used to perform obstacle identification, and obtain The obstacle recognition result output by the obstacle detection model; then according to the obstacle recognition result, the three-dimensional points belonging to obstacles in the first dense point cloud are removed to obtain the second dense point cloud. In this embodiment, except In addition to removing ground points and 3D points of the vehicle, 3D points belonging to other obstacles (such as other vehicles, pedestrians, motorcycles, trees) will be removed to further eliminate interference factors, and the movement of mobile platforms (such as vehicles) After the obstacles on the path (such as the road) are removed, the remaining three-dimensional points with motion trajectories on the road can be determined as three-dimensional points corresponding to fine particles such as water mist, gravel, dust or dust, so the described In the second dense point cloud, the three-dimensional points with motion tracks on the moving path (such as a road) are marked as noise points, and finally the first dense point cloud is split according to the marked noise points to obtain a plurality of points marked with A single-frame 3D point cloud sample of noisy points. In this embodiment, the irrelevant factors are removed to eliminate the interference of obstacles that also have moving trajectories, which is conducive to further improving the accuracy of noise point labeling.

在一些实施例中，考虑到样本的数量也与模型训练的准确度有关，则为了提高模型训练的准确度，可以增加进行训练的样本的数量，则除了通过上述方式标注噪声点之外，还可以通过以下至少一种的数据增强方式来丰富训练样本集：在第一种数据增强方式中，可以将所述标注有噪声点的单帧三维点云样本中的噪声点叠加到其他三维点云样本中，获取额外的标注有噪声点的单帧三维点云样本。比如在车辆驾驶场景中，考虑到水雾、砂砾、灰尘或者粉尘等细颗粒物相对于车身的距离是相似的，因此可以将已标注的噪声点叠加到不同场景下运行的不同车辆对应的三维点云中，以扩充训练数据集，增强多样性。In some embodiments, considering that the number of samples is also related to the accuracy of model training, in order to improve the accuracy of model training, the number of samples for training can be increased. In addition to marking noise points in the above-mentioned way, also The training sample set can be enriched by at least one of the following data enhancement methods: in the first data enhancement method, the noise points in the single-frame 3D point cloud sample marked with noise points can be superimposed on other 3D point clouds In the sample, additional single-frame 3D point cloud samples labeled with noise points are obtained. For example, in a vehicle driving scene, considering that the distances of fine particles such as water mist, gravel, dust or dust to the vehicle body are similar, the marked noise points can be superimposed on the corresponding 3D points of different vehicles running in different scenes In the cloud, to expand the training data set and enhance the diversity.

在第二种数据增强方式中，可以移动所述标注有噪声点的单帧三维点云样本中的噪声点的位置，获取额外的标注有噪声点的单帧三维点云样本。比如在车辆驾驶场景中，考虑到水雾、砂砾、灰尘或者粉尘等细颗粒物相对于车身的距离是相似的，则在确定属于车辆的三维点之后，可以将标注的噪声点移动到相对于车辆的不同位置，比如可以通过对称反转，或者调整某一个坐标轴的噪声点的数据，将噪声点对称或平移到其他相对于车辆不同的位置，这样可满足有一些车辆在左侧，或在车头，或在车身中间存在水雾、砂砾、灰尘或者粉尘等不同情况。In the second data enhancement manner, the positions of the noise points in the single-frame 3D point cloud samples marked with noise points may be moved to obtain additional single-frame 3D point cloud samples marked with noise points. For example, in a vehicle driving scene, considering that the distances of fine particles such as water mist, gravel, dust or dust to the vehicle body are similar, after determining the 3D points belonging to the vehicle, the marked noise points can be moved to a distance relative to the vehicle body. For example, through symmetrical inversion, or adjusting the data of noise points on a certain coordinate axis, the noise points can be symmetrically or translated to other positions relative to the vehicle, so that some vehicles are on the left, or in the The front of the car, or in the middle of the car body, there are different situations such as water mist, gravel, dust or dust.

在第三种数据增强方式中，可以将所述标注有噪声点的单帧三维点云样本中的所有三维点旋转预设角度，获取额外的标注有噪声点的单帧三维点云样本。比如在车辆驾驶场景中，为了防止数据出现固定在东西南北某个固定方向行驶导致模型训练的结果可能出现的偏置，所述标注有噪声点的单帧三维点云样本中的所有三维点可以被顺时针或逆时针旋转预设角度以扩充训练数据集，增强多样性。In the third data enhancement method, all the 3D points in the single-frame 3D point cloud samples marked with noise points can be rotated by a preset angle to obtain additional single-frame 3D point cloud samples marked with noise points. For example, in a vehicle driving scene, in order to prevent the data from being fixed in a fixed direction of east, west, north, south, and causing possible bias in the model training results, all the 3D points in the single frame 3D point cloud sample marked with noise points can be Rotated by preset angles clockwise or counterclockwise to expand the training data set and enhance diversity.

在第四种数据增强方式中，可以将表征障碍物的三维点集合添加到所述标注有噪声点的单帧三维点云样本中的噪声点附近，获取额外的标注有噪声点的单帧三维点云样本。比如在车辆驾驶场景中，可以将表征摩托车或者行人等障碍物的三维点集合添加到噪声点附近，从而可以验证噪声点识别模型在噪声点与其他物体交叉时的分别能力。In the fourth data enhancement method, a set of 3D points representing obstacles can be added to the vicinity of the noise points in the single-frame 3D point cloud sample marked with noise points to obtain additional single-frame 3D point cloud samples marked with noise points. Point cloud sample. For example, in a vehicle driving scene, a set of three-dimensional points representing obstacles such as motorcycles or pedestrians can be added near noise points, so that the ability of the noise point recognition model to distinguish when noise points intersect with other objects can be verified.

在第五种数据增强方式中，可以将所述标注有噪声点的单帧三维点云样本中的所有三维点进行随机偏移，可以在任意方向上偏移所述三维点，实现扩充训练数据集，增强多样性，而训练样本的增多可以提高模型的鲁棒性。In the fifth data enhancement method, all the three-dimensional points in the single-frame three-dimensional point cloud sample marked with noise points can be randomly offset, and the three-dimensional points can be offset in any direction to realize the expansion of training data Set, enhance the diversity, and the increase of training samples can improve the robustness of the model.

在一些实施例中，在获取了若干标注有噪声点的三维点云样本之后，可以基于若干标注有噪声点的三维点云样本进行模型训练，以获得所述噪声点识别模型。示例性的，请参阅图2所示的模型结构，所述电子设备在获取若干标注有噪声点的三维点云样本之后，可以将若干标注有噪声点的三维点云样本输入如图2所示的预设模型中：In some embodiments, after acquiring several 3D point cloud samples marked with noise points, model training may be performed based on the several 3D point cloud samples marked with noise points to obtain the noise point recognition model. For example, please refer to the model structure shown in Figure 2, after the electronic device acquires several 3D point cloud samples marked with noise points, it can input several 3D point cloud samples marked with noise points as shown in Figure 2 In the default model of:

然后通过预设模型的特征提取层进行特征提取，例如将三维点云样本进行三维网格化处理，按照预设距离将点云空间分割成三维网格，并提取该三维网格的特征以得到第一点云特征，例如该三维网格的特征可以是该三维网格内的三维点的特征的平均值。Then feature extraction is performed through the feature extraction layer of the preset model, for example, the 3D point cloud sample is subjected to 3D grid processing, the point cloud space is divided into 3D grids according to the preset distance, and the features of the 3D grid are extracted to obtain The first point cloud feature, for example, the feature of the 3D grid may be the average value of the features of the 3D points in the 3D grid.

进而由预设模型中的形状识别网络对第一点云特征进行处理，由稀疏卷积层对第一点云特征进行下采样处理，由第一多层感知机网络根据下采样后的第一点云特征进行形状特征统计以确定属于同一障碍物的第一点云特征，由上采样层对所述属于同一障碍物的第一点云特征进行上采样处理以得到第二点云特征。Then, the shape recognition network in the preset model processes the first point cloud features, the sparse convolution layer downsamples the first point cloud features, and the first multi-layer perceptron network according to the downsampled first The point cloud features perform shape feature statistics to determine the first point cloud features belonging to the same obstacle, and the up-sampling layer performs up-sampling processing on the first point cloud features belonging to the same obstacle to obtain the second point cloud features.

接着由预设模型中的噪声点识别网络进行处理，由特征融合层融合所述第一点云特征和所述第二点云特征，由第二多层感知机网络对对融合后的点云特征进行噪声点识别以获取噪声点预测结果。Then, it is processed by the noise point recognition network in the preset model, the first point cloud feature and the second point cloud feature are fused by the feature fusion layer, and the fused point cloud is paired by the second multi-layer perceptron network The features are used to identify noise points to obtain noise point prediction results.

最后确定所述噪声点预测结果与所述三维点云样本标注好的噪声点之间的差异，根据所述差异反向调整用于构建所述预设模型的参数，获取训练好的噪声点识别模型。比如可以基于预设的损失函数计算所述噪声点预测结果与所述三维点云样本标注好的噪声点之间的差异对应的损失值，根据所述损失值反向调整用于构建所述指定模型的参数。Finally, determine the difference between the noise point prediction result and the marked noise point of the 3D point cloud sample, reversely adjust the parameters used to build the preset model according to the difference, and obtain the trained noise point recognition Model. For example, the loss value corresponding to the difference between the noise point prediction result and the marked noise point of the 3D point cloud sample can be calculated based on a preset loss function, and the loss value used to construct the specified The parameters of the model.

本实施例中，使用的第一点云特征为从网格化后的点云中提取的网格特征，有利于降低计算量；所述噪声点识别模型在进行形状特征统计时，为了进一步减低运算量，节省计算资源，通过稀疏卷积层对第一点云特征进行下采样处理，进而可以适用于运算资源有限的设备中，使得所述噪声点识别模型适用于更多的设备，具有广泛的适用性；并且考虑到下采样过程会造成低精度信息的损失，因此噪声点识别模型在形状识别网络后半部分设计了上采样结构将点云特征恢复成下采样之前的大小，并将上采样得到的第二点云特征与第一点云特征进行融合，弥补了下采样过程中损失的低精度信息，进而第二多层感知机网络对融合后的点云特征进行噪声点识别，有利于提高噪声点识别的准确性，实现了计算资源和精准度的权衡。In this embodiment, the first point cloud feature used is the grid feature extracted from the gridded point cloud, which is beneficial to reduce the amount of calculation; when the noise point recognition model performs shape feature statistics, in order to further reduce Calculation, saving computing resources, downsampling the first point cloud features through the sparse convolution layer, and then applicable to devices with limited computing resources, making the noise point recognition model applicable to more devices, with a wide range of and considering the loss of low-precision information caused by the downsampling process, the noise point recognition model designed an upsampling structure in the second half of the shape recognition network to restore the point cloud features to the size before downsampling, and the upsampling The second point cloud feature obtained by sampling is fused with the first point cloud feature to make up for the low-precision information lost in the downsampling process, and then the second multi-layer perceptron network performs noise point recognition on the fused point cloud feature, which has It is beneficial to improve the accuracy of noise point identification, and realizes the trade-off between computing resources and accuracy.

可以理解的是，所述噪声点识别模型的训练过程和应用过程可以由相同的电子设备来执行，也可以由不同的电子设备来执行，本实施例对此不做任何限制。It can be understood that the training process and the application process of the noise point recognition model may be performed by the same electronic device, or may be performed by different electronic devices, which is not limited in this embodiment.

与上述点云去噪方法相对应，请参阅图4，本申请实施例还提供了一种电子设备30，包括存储器32、处理器31及存储在存储器32上并可在处理器31上运行的计算机程序33，所述处理器31执行所述程序时用于执行上述方法。Corresponding to the above point cloud denoising method, please refer to FIG. 4 , the embodiment of the present application also provides an electronic device 30, including a memory 32, a processor 31, and an A computer program 33, used for executing the above method when the processor 31 executes the program.

所述处理器31执行所述存储器32中包括的可执行指令，所述处理器31可以是中央处理单元(Central Processing Unit，CPU)，还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。The processor 31 executes the executable instructions included in the memory 32, and the processor 31 can be a central processing unit (Central Processing Unit, CPU), and can also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.

所述存储器32存储点云去噪方法的可执行指令，所述存储器32可以包括至少一种类型的存储介质，存储介质包括闪存、硬盘、多媒体卡、卡型存储器(例如，SD或DX存储器等等)、随机访问存储器(RAM)、静态随机访问存储器(SRAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、可编程只读存储器(PROM)、磁性存储器、磁盘、光盘等等。而且，设备可以与通过网络连接执行存储器的存储功能的网络存储设备协作。存储器32可以是设备30的内部存储单元，例如设备30的硬盘或内存。存储器32也可以是设备30的外部存储设备，例如设备30上配备的插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(Secure Digital,SD)卡，闪存卡(Flash Card)等。进一步地，存储器32还可以既包括设备30的内部存储单元也包括外部存储设备。存储器32用于存储计算机程序33以及设备所需的其他程序和数据。存储器32还可以用于暂时地存储已经输出或者将要输出的数据。The memory 32 stores the executable instructions of the point cloud denoising method, and the memory 32 can include at least one type of storage medium, and the storage medium includes a flash memory, a hard disk, a multimedia card, a card memory (for example, SD or DX memory, etc. etc.), Random Access Memory (RAM), Static Random Access Memory (SRAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Programmable Read Only Memory (PROM), Magnetic Memory, Diskettes, CDs, etc. Also, the device may cooperate with a network storage device performing a storage function of the memory through a network connection. The storage 32 may be an internal storage unit of the device 30 , such as a hard disk or a memory of the device 30 . Memory 32 also can be the external storage device of equipment 30, for example equipped with plug-in type hard disk on equipment 30, smart memory card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card, flash memory card (Flash Card) wait. Further, the memory 32 may also include both an internal storage unit of the device 30 and an external storage device. The memory 32 is used to store the computer program 33 and other programs and data required by the device. The memory 32 can also be used to temporarily store data that has been output or will be output.

示例性的，所述电子设备包括车载终端，所述车载终端安装于车辆上，车辆还安装有点云采集装置，该车载终端与点云采集装置(如安装于车辆上的激光雷达)通信连接，搭载于车辆上的点云采集装置在车辆驾驶过程中采集三维点云，车载终端可以获取点云采集装置采集的待去噪的三维点云，并基于本申请实施例提供的点云去噪方法进行去噪处理。Exemplarily, the electronic equipment includes a vehicle-mounted terminal, the vehicle-mounted terminal is installed on a vehicle, and the vehicle is also equipped with a point cloud collection device, and the vehicle-mounted terminal is communicatively connected with the point cloud collection device (such as a laser radar installed on the vehicle), The point cloud acquisition device mounted on the vehicle collects the three-dimensional point cloud during the driving process of the vehicle, and the vehicle-mounted terminal can obtain the three-dimensional point cloud to be denoised collected by the point cloud acquisition device, and based on the point cloud denoising method provided in the embodiment of the present application Perform denoising processing.

示例性的，所述处理器执行所述程序时用于执行以下步骤：获取待去噪的三维点云；将从所述三维点云中提取的第一点云特征进行下采样处理；根据下采样后的第一点云特征进行形状特征统计，确定属于同一障碍物的第一点云特征；对所述属于同一障碍物的第一点云特征进行上采样处理，获取第二点云特征；所述第二点云特征的维度与所述第一点云特征的维度相同；将所述第一点云特征和所述第二点云特征进行融合处理，并使用融合后的点云特征进行噪声点识别；根据噪声点识别结果去除所述三维点云中的所述噪声点。Exemplarily, when the processor executes the program, it is used to perform the following steps: acquire the 3D point cloud to be denoised; perform downsampling processing on the first point cloud features extracted from the 3D point cloud; The first point cloud feature after sampling is subjected to shape feature statistics, and the first point cloud feature belonging to the same obstacle is determined; the first point cloud feature belonging to the same obstacle is upsampled to obtain the second point cloud feature; The dimension of the second point cloud feature is the same as the dimension of the first point cloud feature; the first point cloud feature and the second point cloud feature are fused, and the fused point cloud feature is used to perform Identifying noise points: removing the noise points in the 3D point cloud according to the noise point identification results.

可选地，在所述获取待去噪的三维点云之后，所述处理器还用于：按照预设距离分割所述三维点云，获取三维网格化的三维点云；对所述三维网格化的三维点云中的每个三维网格进行特征提取，获取所述第一点云特征。Optionally, after the acquisition of the 3D point cloud to be denoised, the processor is further configured to: segment the 3D point cloud according to a preset distance to obtain a 3D gridded 3D point cloud; Each 3D grid in the gridded 3D point cloud is subjected to feature extraction to obtain the features of the first point cloud.

可选地，所述第一点云特征包括所述三维点云中的所有三维网格的特征。每个网格的特征由该三维网格内的三维点的特征的统计值确定。所述三维点的特征包括以下至少一种：三维点的坐标、反射强度、三维点对应的光脉冲序列的标识、深度信息、高度信息或者角度信息。Optionally, the first point cloud features include features of all 3D meshes in the 3D point cloud. The characteristics of each grid are determined by statistical values of the characteristics of the three-dimensional points within the three-dimensional grid. The characteristics of the three-dimensional point include at least one of the following: coordinates of the three-dimensional point, reflection intensity, identification of the light pulse sequence corresponding to the three-dimensional point, depth information, height information or angle information.

可选地，所述噪声点识别结果为将所述三维点云输入预先建立的噪声点识别模型中，通过所述噪声点识别模型对所述三维点云进行处理后得到。其中，所述噪声点识别模型包括特征提取层、形状识别网络和噪声点识别网络。所述第一点云特征通过所述特征提取层对所述三维点云进行特征提取得到。所述形状识别网络包括用于对第一点云特征进行下采样处理的稀疏卷积层、用于根据下采样后的第一点云特征进行形状特征统计的第一多层感知机网络、和用于对所述属于同一障碍物的第一点云特征进行上采样处理的上采样层。所述噪声点识别网络包括用于融合所述第一点云特征和所述第二点云特征的特征融合层、和用于对融合后的点云特征进行噪声点识别的第二多层感知机网络。Optionally, the noise point recognition result is obtained by inputting the 3D point cloud into a pre-established noise point recognition model, and processing the 3D point cloud through the noise point recognition model. Wherein, the noise point recognition model includes a feature extraction layer, a shape recognition network and a noise point recognition network. The first point cloud feature is obtained by feature extraction of the three-dimensional point cloud by the feature extraction layer. The shape recognition network includes a sparse convolution layer for downsampling the first point cloud features, a first multi-layer perceptron network for performing shape feature statistics according to the downsampled first point cloud features, and An upsampling layer for performing upsampling processing on the first point cloud features belonging to the same obstacle. The noise point recognition network includes a feature fusion layer for fusing the first point cloud features and the second point cloud features, and a second multilayer perception layer for performing noise point recognition on the fused point cloud features machine network.

可选地，所述噪声点识别模型基于若干标注有噪声点的三维点云样本训练得到。Optionally, the noise point recognition model is trained based on several three-dimensional point cloud samples labeled with noise points.

可选地，所述处理器还用于：将多个单帧三维点云样本融合成第一稠密点云；其中，所述多个单帧三维点云样本由搭载于车辆上的点云采集装置在车辆驾驶过程中采集得到；将所述第一稠密点云中属于地面的三维点和属于所述车辆的三维点去除，获取第二稠密点云；将所述第二稠密点云中在所述车辆周围的具有运动轨迹的三维点标注为噪声点；根据标注的所述噪声点将所述第一稠密点云拆分得到多个标注有噪声点的单帧三维点云样本。Optionally, the processor is further configured to: fuse a plurality of single-frame three-dimensional point cloud samples into a first dense point cloud; wherein, the plurality of single-frame three-dimensional point cloud samples are collected by a point cloud mounted on a vehicle The device is collected during the driving of the vehicle; removing the 3D points belonging to the ground and the 3D points belonging to the vehicle in the first dense point cloud to obtain a second dense point cloud; The three-dimensional points with motion tracks around the vehicle are marked as noise points; the first dense point cloud is split according to the marked noise points to obtain a plurality of single-frame three-dimensional point cloud samples marked with noise points.

可选地，所述处理器还用于：根据所述第一稠密点云进行障碍物识别以获得障碍物识别结果；根据所述障碍物识别结果将所述第一稠密点云中属于障碍物的三维点去除，获得所述第二稠密点云；将所述第二稠密点云中处于道路上的具有运动轨迹的三维点标注为噪声点。Optionally, the processor is further configured to: perform obstacle recognition according to the first dense point cloud to obtain an obstacle recognition result; The 3D point removal of the second dense point cloud is obtained; and the 3D point with a motion track on the road in the second dense point cloud is marked as a noise point.

可选地，所述处理器还用于：在所述获得多个标注有噪声点的单帧三维点云样本之后，通过以下至少一种方式获取标注有噪声点的三维点云样本：将所述标注有噪声点的单帧三维点云样本中的噪声点叠加到其他三维点云样本中；或，移动所述标注有噪声点的单帧三维点云样本中的噪声点的位置；或，将所述标注有噪声点的单帧三维点云样本中的所有三维点旋转预设角度；或，将表征障碍物的三维点集合添加到所述标注有噪声点的单帧三维点云样本中的噪声点附近；或，将所述标注有噪声点的单帧三维点云样本中的所有三维点进行随机偏移。Optionally, the processor is further configured to: after obtaining a plurality of single-frame 3D point cloud samples marked with noise points, obtain the 3D point cloud samples marked with noise points in at least one of the following ways: The noise points in the single-frame three-dimensional point cloud samples marked with noise points are superimposed on other three-dimensional point cloud samples; or, the position of the noise points in the single-frame three-dimensional point cloud samples marked with noise points is moved; or, Rotating all 3D points in the single-frame 3D point cloud sample marked with noise points by a preset angle; or, adding a set of 3D points representing obstacles to the single-frame 3D point cloud sample marked with noise points Near the noise point; or, randomly offset all three-dimensional points in the single-frame three-dimensional point cloud sample marked with noise points.

可选地，所述待去噪的三维点云由搭载于车辆上的点云采集装置在车辆驾驶过程中采集得到；所述噪声点包括以下至少一种细颗粒物对应的三维点：水雾、砂砾、灰尘或者粉尘。Optionally, the 3D point cloud to be denoised is collected by a point cloud collection device mounted on the vehicle during vehicle driving; the noise points include 3D points corresponding to at least one of the following fine particles: water mist, grit, dirt or dust.

可以理解的是，可以包括与图3所示更多或更少的部件，或者组合某些部件，或者不同的部件，例如设备还可以包括输入输出设备、网络接入设备、总线等。It can be understood that more or less components than those shown in FIG. 3 may be included, some components may be combined, or different components may be included. For example, the device may also include input and output devices, network access devices, buses, and so on.

相应的，在示例性实施例中，还提供了一种包括指令的非临时性计算机可读存储介质，例如包括指令的存储器，上述指令可由装置的处理器执行以完成上述方法。例如，非临时性计算机可读存储介质可以是ROM、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。Correspondingly, in an exemplary embodiment, there is also provided a non-transitory computer-readable storage medium including instructions, such as a memory including instructions, the instructions can be executed by a processor of the device to complete the above method. For example, the non-transitory computer readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

一种非临时性计算机可读存储介质，当存储介质中的指令由终端的处理器执行时，使得终端能够执行上述方法。A non-transitory computer-readable storage medium, enabling the terminal to execute the above method when instructions in the storage medium are executed by a processor of the terminal.

相应的，本申请实施例还提供了一种计算机程序产品，包括如上述任一所述方法的计算机程序。Correspondingly, an embodiment of the present application further provides a computer program product, including the computer program according to any one of the methods described above.

本说明书中描述的主题及功能操作的实施例可以在以下中实现：数字电子电路、有形体现的计算机软件或固件、包括本说明书中公开的结构及其结构性等同物的计算机硬件、或者它们中的一个或多个的组合。本说明书中描述的主题的实施例可以实现为一个或多个计算机程序，即编码在有形非暂时性程序载体上以被数据处理装置执行或控制数据处理装置的操作的计算机程序指令中的一个或多个模块。可替代地或附加地，程序指令可以被编码在人工生成的传播信号上，例如机器生成的电、光或电磁信号，该信号被生成以将信息编码并传输到合适的接收机装置以由数据处理装置执行。计算机存储介质可以是机器可读存储设备、机器可读存储基板、随机或串行存取存储器设备、或它们中的一个或多个的组合。Embodiments of the subject matter and functional operations described in this specification can be implemented in digital electronic circuitry, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or in A combination of one or more of . Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, that is, one or more of computer program instructions encoded on a tangible, non-transitory program carrier for execution by or to control the operation of data processing apparatus. Multiple modules. Alternatively or additionally, the program instructions may be encoded on an artificially generated propagated signal, such as a machine-generated electrical, optical or electromagnetic signal, which is generated to encode and transmit information to a suitable receiver device for transmission by the data The processing means executes. A computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

本说明书中描述的处理及逻辑流程可以由执行一个或多个计算机程序的一个或多个可编程计算机执行，以通过根据输入数据进行操作并生成输出来执行相应的功能。所述处理及逻辑流程还可以由专用逻辑电路—例如FPGA(现场可编程门阵列)或ASIC(专用集成电路)来执行，并且装置也可以实现为专用逻辑电路。The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform corresponding functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

适合用于执行计算机程序的计算机包括，例如通用和/或专用微处理器，或任何其他类型的中央处理单元。通常，中央处理单元将从只读存储器和/或随机存取存储器接收指令和数据。计算机的基本组件包括用于实施或执行指令的中央处理单元以及用于存储指令和数据的一个或多个存储器设备。通常，计算机还将包括用于存储数据的一个或多个大容量存储设备，例如磁盘、磁光盘或光盘等，或者计算机将可操作地与此大容量存储设备耦接以从其接收数据或向其传送数据，抑或两种情况兼而有之。然而，计算机不是必须具有这样的设备。此外，计算机可以嵌入在另一设备中，例如自动驾驶车辆、移动机器人等，仅举几例。Computers suitable for the execution of a computer program include, for example, general and/or special purpose microprocessors, or any other type of central processing unit. Generally, a central processing unit will receive instructions and data from a read only memory and/or a random access memory. The essential components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include, or be operatively coupled to, one or more mass storage devices for storing data, such as magnetic or magneto-optical disks, or optical disks, to receive data therefrom or to It transmits data, or both. However, a computer is not required to have such a device. Additionally, a computer can be embedded in another device, such as an autonomous vehicle, mobile robot, etc., just to name a few.

适合于存储计算机程序指令和数据的计算机可读介质包括所有形式的非易失性存储器、媒介和存储器设备，例如包括半导体存储器设备(例如EPROM、EEPROM和闪存设备)、磁盘(例如内部硬盘或可移动盘)、磁光盘以及CD ROM和DVD-ROM盘。处理器和存储器可由专用逻辑电路补充或并入专用逻辑电路中。Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (such as EPROM, EEPROM, and flash memory devices), magnetic disks (such as internal hard disks or removable disks), magneto-optical disks, and CD ROM and DVD-ROM disks. The processor and memory can be supplemented by, or incorporated in, special purpose logic circuitry.

虽然本说明书包含许多具体实施细节，但是这些不应被解释为限制任何发明的范围或所要求保护的范围，而是主要用于描述特定发明的具体实施例的特征。本说明书内在多个实施例中描述的某些特征也可以在单个实施例中被组合实施。另一方面，在单个实施例中描述的各种特征也可以在多个实施例中分开实施或以任何合适的子组合来实施。此外，虽然特征可以如上所述在某些组合中起作用并且甚至最初如此要求保护，但是来自所要求保护的组合中的一个或多个特征在一些情况下可以从该组合中去除，并且所要求保护的组合可以指向子组合或子组合的变型。While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any inventions or of what may be claimed, but rather as primarily describing features of particular embodiments of particular inventions. Certain features that are described in this specification in multiple embodiments can also be implemented in combination in a single embodiment. On the other hand, various features that are described in a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may function in certain combinations as described above and even be initially so claimed, one or more features from a claimed combination may in some cases be removed from that combination and the claimed A protected combination can point to a subcombination or a variant of a subcombination.

类似地，虽然在附图中以特定顺序描绘了操作，但是这不应被理解为要求这些操作以所示的特定顺序执行或顺次执行、或者要求所有例示的操作被执行，以实现期望的结果。在某些情况下，多任务和并行处理可能是有利的。此外，上述实施例中的各种***模块和组件的分离不应被理解为在所有实施例中均需要这样的分离，并且应当理解，所描述的程序组件和***通常可以一起集成在单个软件产品中，或者封装成多个软件产品。Similarly, while operations are depicted in the figures in a particular order, this should not be construed as requiring that those operations be performed in the particular order shown, or sequentially, or that all illustrated operations be performed, to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can often be integrated together in a single software product in, or packaged into multiple software products.

由此，主题的特定实施例已被描述。其他实施例在所附权利要求书的范围以内。在某些情况下，权利要求书中记载的动作可以以不同的顺序执行并且仍实现期望的结果。此外，附图中描绘的处理并非必需所示的特定顺序或顺次顺序，以实现期望的结果。在某些实现中，多任务和并行处理可能是有利的。Thus, certain embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some implementations, multitasking and parallel processing may be advantageous.

以上所述仅为本申请的较佳实施例而已，并不用以限制本申请，凡在本申请的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本申请保护的范围之内。The above is only a preferred embodiment of the application, and is not intended to limit the application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the application should be included in the application. within the scope of protection.

Claims

一种点云去噪方法，包括：A point cloud denoising method, comprising:

获取待去噪的三维点云；Obtain the 3D point cloud to be denoised;

将从所述三维点云中提取的第一点云特征进行下采样处理；The first point cloud feature extracted from the three-dimensional point cloud is subjected to down-sampling processing;

根据下采样后的第一点云特征进行形状特征统计，确定属于同一障碍物的第一点云特征；Perform shape feature statistics according to the first point cloud feature after downsampling, and determine the first point cloud feature belonging to the same obstacle;

对所述属于同一障碍物的第一点云特征进行上采样处理，获取第二点云特征；所述第二点云特征的维度与所述第一点云特征的维度相同；Carrying out upsampling processing on the first point cloud feature belonging to the same obstacle to obtain a second point cloud feature; the dimension of the second point cloud feature is the same as the dimension of the first point cloud feature;

将所述第一点云特征和所述第二点云特征进行融合处理；performing fusion processing on the first point cloud feature and the second point cloud feature;

使用融合后的点云特征进行噪声点识别；Use the fused point cloud features for noise point recognition;

根据噪声点识别结果去除所述三维点云中的所述噪声点。The noise points in the three-dimensional point cloud are removed according to the noise point identification result.
根据权利要求1所述的方法，其特征在于，在所述获取待去噪的三维点云之后，还包括：The method according to claim 1, characterized in that, after said acquiring the 3D point cloud to be denoised, further comprising:

按照预设距离分割所述三维点云，获取三维网格化的三维点云；Segmenting the 3D point cloud according to a preset distance to obtain a 3D gridded 3D point cloud;

对所述三维网格化的三维点云中的每个三维网格进行特征提取，获取所述第一点云特征。Perform feature extraction on each 3D grid in the 3D gridded 3D point cloud to obtain the first point cloud feature.
根据权利要求2所述的方法，其特征在于，The method according to claim 2, characterized in that,

所述第一点云特征包括所述三维点云中的所有三维网格的特征；The first point cloud features include features of all 3D meshes in the 3D point cloud;

所述三维网格的特征由该三维网格内的三维点的特征的统计值确定；The characteristics of the three-dimensional grid are determined by statistical values of the characteristics of the three-dimensional points in the three-dimensional grid;

所述三维点的特征包括以下至少一种：三维点的坐标、反射强度、三维点对应的光脉冲序列的标识、深度信息、高度信息或者角度信息。The characteristics of the three-dimensional point include at least one of the following: coordinates of the three-dimensional point, reflection intensity, identification of the light pulse sequence corresponding to the three-dimensional point, depth information, height information or angle information.
根据权利要求1所述的方法，其特征在于，The method according to claim 1, characterized in that,

所述噪声点识别结果为将所述三维点云输入预先建立的噪声点识别模型中，通过所述噪声点识别模型对所述三维点云进行处理后得到；The noise point recognition result is obtained by inputting the 3D point cloud into a pre-established noise point recognition model, and processing the 3D point cloud through the noise point recognition model;

其中，所述噪声点识别模型包括特征提取层、形状识别网络和噪声点识别网络；Wherein, the noise point recognition model includes a feature extraction layer, a shape recognition network and a noise point recognition network;

所述第一点云特征通过所述特征提取层对所述三维点云进行特征提取得到；The first point cloud feature is obtained by performing feature extraction on the three-dimensional point cloud by the feature extraction layer;

所述形状识别网络包括用于对第一点云特征进行下采样处理的稀疏卷积层、用于根据下采样后的第一点云特征进行形状特征统计的第一多层感知机网络、和用于对所述属于同一障碍物的第一点云特征进行上采样处理的上采样层；The shape recognition network includes a sparse convolution layer for downsampling the first point cloud features, a first multi-layer perceptron network for performing shape feature statistics according to the downsampled first point cloud features, and An upsampling layer for performing upsampling processing on the first point cloud features belonging to the same obstacle;

所述噪声点识别网络包括用于融合所述第一点云特征和所述第二点云特征的特征融合层、和用于对融合后的点云特征进行噪声点识别的第二多层感知机网络。The noise point recognition network includes a feature fusion layer for fusing the first point cloud features and the second point cloud features, and a second multilayer perception layer for performing noise point recognition on the fused point cloud features machine network.
根据权利要求4所述的方法，其特征在于，所述噪声点识别模型基于若干标注有噪声点的三维点云样本训练得到。The method according to claim 4, wherein the noise point recognition model is trained based on several three-dimensional point cloud samples labeled with noise points.
根据权利要求5所述的方法，其特征在于，还包括：The method according to claim 5, further comprising:

将多个单帧三维点云样本融合成第一稠密点云；其中，所述多个单帧三维点云样本由搭载于车辆上的点云采集装置在车辆驾驶过程中采集得到；Fusing multiple single-frame three-dimensional point cloud samples into a first dense point cloud; wherein, the multiple single-frame three-dimensional point cloud samples are collected by a point cloud collection device mounted on the vehicle during vehicle driving;

将所述第一稠密点云中属于地面的三维点和属于所述车辆的三维点去除，获取第二稠密点云；removing the 3D points belonging to the ground and the 3D points belonging to the vehicle in the first dense point cloud to obtain a second dense point cloud;

将所述第二稠密点云中在所述车辆周围的具有运动轨迹的三维点标注为噪声点；Marking three-dimensional points with motion trajectories around the vehicle in the second dense point cloud as noise points;

根据标注的所述噪声点将所述第一稠密点云拆分，得到多个标注有噪声点的单帧三维点云样本。The first dense point cloud is split according to the marked noise points to obtain a plurality of single-frame three-dimensional point cloud samples marked with noise points.
根据权利要求6所述的方法，其特征在于，还包括：The method according to claim 6, further comprising:

根据所述第一稠密点云进行障碍物识别以获得障碍物识别结果；performing obstacle identification according to the first dense point cloud to obtain an obstacle identification result;

根据所述障碍物识别结果将所述第一稠密点云中属于障碍物的三维点去除，获得所述第二稠密点云；removing the 3D points belonging to obstacles in the first dense point cloud according to the obstacle identification result to obtain the second dense point cloud;

所述将所述第二稠密点云中在所述车辆周围的具有运动轨迹的三维点标注为噪声点，包括：The marking the three-dimensional points with motion tracks around the vehicle in the second dense point cloud as noise points includes:

将所述第二稠密点云中处于道路上的具有运动轨迹的三维点标注为噪声点。Marking the 3D points with motion tracks on the road in the second dense point cloud as noise points.
根据权利要求6所述的方法，其特征在于，在所述得到多个标注有噪声点的单帧三维点云样本之后，还通过以下至少一种方式获取标注有噪声点的三维点云样本：The method according to claim 6, wherein, after obtaining a plurality of single-frame 3D point cloud samples marked with noise points, the 3D point cloud samples marked with noise points are obtained by at least one of the following methods:

将所述标注有噪声点的单帧三维点云样本中的噪声点叠加到其他三维点云样本中；Superimposing noise points in the single-frame three-dimensional point cloud sample marked with noise points into other three-dimensional point cloud samples;

移动所述标注有噪声点的单帧三维点云样本中的噪声点的位置；Move the position of the noise point in the single-frame three-dimensional point cloud sample marked with the noise point;

将所述标注有噪声点的单帧三维点云样本中的所有三维点旋转预设角度；Rotating all three-dimensional points in the single-frame three-dimensional point cloud sample marked with noise points by a preset angle;

将表征障碍物的三维点集合添加到所述标注有噪声点的单帧三维点云样本中的噪声点附近；Adding a set of three-dimensional points representing obstacles to the vicinity of noise points in the single-frame three-dimensional point cloud sample marked with noise points;

将所述标注有噪声点的单帧三维点云样本中的所有三维点进行随机偏移。All the three-dimensional points in the single-frame three-dimensional point cloud sample marked with noise points are randomly offset.
根据权利要求1至8任意一项所述的方法，其特征在于，The method according to any one of claims 1 to 8, characterized in that,

所述待去噪的三维点云由搭载于车辆上的点云采集装置在车辆驾驶过程中采集得到；The three-dimensional point cloud to be denoised is collected by a point cloud collection device mounted on the vehicle during driving of the vehicle;

所述噪声点包括以下至少一种细颗粒物对应的三维点：水雾、砂砾、灰尘或者粉尘。The noise points include three-dimensional points corresponding to at least one of the following fine particles: water mist, gravel, dust or dust.
一种电子设备，包括：An electronic device comprising:

存储器；memory;

处理器；及processor; and

存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述程序时实现如权利要求1至9任一项所述的方法。A computer program stored on the memory and operable on the processor, the processor implements the method according to any one of claims 1 to 9 when executing the program.
根据权利要求10所述的设备，其特征在于，所述电子设备包括车载终端。The device according to claim 10, wherein the electronic device comprises a vehicle-mounted terminal.
一种计算机可读存储介质，其上存储有可执行指令，所述可执行指令被处理器执行时实现如权利要求1至9任一项所述的方法。A computer-readable storage medium, on which executable instructions are stored, and when the executable instructions are executed by a processor, the method according to any one of claims 1 to 9 is realized.