WO2022193335A1 - 点云数据处理方法、装置、计算机设备和存储介质 - Google Patents

点云数据处理方法、装置、计算机设备和存储介质 Download PDF

Info

Publication number
WO2022193335A1
WO2022193335A1 PCT/CN2021/082250 CN2021082250W WO2022193335A1 WO 2022193335 A1 WO2022193335 A1 WO 2022193335A1 CN 2021082250 W CN2021082250 W CN 2021082250W WO 2022193335 A1 WO2022193335 A1 WO 2022193335A1
Authority
WO
WIPO (PCT)
Prior art keywords
point cloud
hausdorff
data
distance
convolution
Prior art date
Application number
PCT/CN2021/082250
Other languages
English (en)
French (fr)
Inventor
黄惠
黄鹏頔
Original Assignee
深圳大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳大学 filed Critical 深圳大学
Priority to US17/735,888 priority Critical patent/US11875424B2/en
Publication of WO2022193335A1 publication Critical patent/WO2022193335A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Definitions

  • the present application relates to a point cloud data processing method, apparatus, computer equipment and storage medium.
  • point cloud data semantic classification technology extracts the features of the data points in the point cloud data, and performs semantic classification on the point cloud data according to the extracted features.
  • the point cloud data is regularly downsampled, and the point cloud data is regularized at a lower resolution, and then feature extraction and semantic classification are performed on the regularized point cloud data.
  • traditional methods lead to sparser point cloud data, resulting in lower accuracy of semantic classification.
  • a first aspect of the present application provides a point cloud data processing method, the method comprising:
  • Convolution calculation is performed on the neighborhood point cloud, the distance matrix and the network weight matrix through the Hausdorff convolution layer in the encoder to obtain high-dimensional point cloud features; the encoder and decoder are deep learning two parts in the network; and
  • Feature dimension reduction is performed on the high-dimensional point cloud features by the decoder, so that the classifier performs semantic classification on the point cloud data according to the target point cloud features obtained by the dimension reduction.
  • a second aspect of the present application provides a point cloud data processing device, the device comprising:
  • an acquisition module for acquiring point cloud data, and constructing a corresponding neighborhood point cloud for each data point in the point cloud data
  • a distance calculation module for calculating the Hausdorff distance between the neighborhood point cloud and the pre-built kernel point cloud to obtain a distance matrix
  • the convolution calculation module is used to perform convolution calculation on the neighborhood point cloud, the distance matrix and the network weight matrix through the Hausdorff convolution layer in the encoder to obtain high-dimensional point cloud features;
  • the coding The encoder and the decoder are two parts in a deep learning network;
  • a feature dimension reduction module configured to perform feature dimension reduction on the high-dimensional point cloud features through the decoder, so that the classifier can perform semantic classification on the point cloud data according to the target point cloud features obtained by the dimension reduction.
  • a third aspect of the present application provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the method of processing the point cloud data when the processor executes the computer program. step.
  • a fourth aspect of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the steps of the method for processing point cloud data.
  • FIG. 1 is an application environment diagram of a point cloud data processing method in one embodiment
  • FIG. 2 is a schematic flowchart of a method for processing point cloud data in one embodiment
  • 3a is a schematic diagram of a neighborhood point cloud in one embodiment
  • 3b is a schematic diagram of the convolution of the neighborhood point cloud and the kernel point cloud 1 in one embodiment
  • 3c is a schematic diagram of the convolution of the neighborhood point cloud and the kernel point cloud 2 in one embodiment
  • FIG. 5 is a schematic flowchart of a method for constructing a neighborhood point cloud in one embodiment
  • FIG. 6 is a schematic flowchart of a method for constructing a kernel point cloud in one embodiment
  • FIG. 7 is a schematic diagram of a kernel point cloud in one embodiment
  • FIG. 8 is a schematic diagram of the principle of a multi-core Hausdorff convolution method in one embodiment
  • Figure 9a is a schematic diagram of the scene label truth value of the S3DIS dataset in one embodiment
  • Figure 9b is a schematic diagram of the result of semantic classification of the S3DIS dataset by a deep learning neural network in one embodiment
  • Figure 9c is a schematic diagram of the ground truth value of the scene label of the SemanticKITTI dataset in one embodiment
  • Figure 9d is a schematic diagram of the result of semantic classification of the SemanticKITTI dataset by a deep learning neural network in one embodiment
  • Fig. 10 is a structural block diagram of an apparatus for processing point cloud data in one embodiment
  • FIG. 11 is a structural block diagram of a point cloud data processing apparatus in another embodiment
  • FIG. 13 is an internal structure diagram of a computer apparatus in another embodiment.
  • the point cloud data processing method provided in this application can be applied to the application environment shown in FIG. 1 .
  • the computer device 102 communicates with the point cloud acquisition device 104 through the network, acquires point cloud data from the point cloud acquisition device 104, and constructs a corresponding neighborhood point cloud for each data point in the point cloud data, and then the computer device 102 calculates Hausdorff distance between the neighborhood point cloud and the pre-built kernel point cloud, get the distance matrix, and pass the Hausdorff convolution layer in the encoder to the neighborhood point cloud, distance matrix and network weight matrix. Convolution calculation to obtain high-dimensional point cloud features.
  • the computer device 102 performs feature dimension reduction on the high-dimensional point cloud features through the decoder, so that the classifier performs semantic classification on the point cloud data according to the target point cloud features obtained by the dimension reduction.
  • the computer device 102 may be a terminal or a server.
  • the terminal may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices.
  • the server may be an independent server or multiple servers. A cluster of servers is implemented.
  • the point cloud collecting device 104 may be any device capable of collecting point cloud data, for example, a lidar, a three-dimensional laser scanner, an RGBD depth camera, and the like.
  • a point cloud data processing method is provided, and the method is applied to the computer device in FIG. 1 as an example to illustrate, including the following steps:
  • the computer device acquires point cloud data, and constructs a corresponding neighborhood point cloud for each data point in the point cloud data.
  • the point cloud data is the data obtained by the point cloud collection device by scanning objects in space.
  • the point cloud data includes a plurality of data points, and each data point includes the position coordinates of the data point and the characteristics of the data point.
  • the features of the data points include color features, intensity features, laser echo times, etc.
  • the color features can be, for example, color RGB values.
  • the point cloud data is three-dimensional point cloud data, in particular, it may be three-dimensional single-frame point cloud data.
  • the 3D single frame point cloud data is relatively sparse, and the interval between data lines is large.
  • the point cloud data may be data obtained by scanning objects in a road environment by a lidar on a driverless car.
  • the point cloud data may be data obtained by scanning objects in a space environment by a depth camera on an artificial intelligence robot.
  • the neighborhood point cloud corresponding to a certain data point is a point cloud composed of data points in an area centered on the data point.
  • the neighborhood point cloud is composed of data points in a circular area with the data point p 0 as the center and the distance from the center of the circle is less than the radius.
  • the data points in the neighborhood point cloud include the x, y, and z coordinates of the data points and the characteristics of the data points.
  • the computer device calculates the Hausdorff distance between the neighborhood point cloud and the pre-built kernel point cloud to obtain a distance matrix.
  • the core point cloud is a point cloud with a certain geometric shape.
  • a kernel point cloud can have basic geometric shapes, including point, line, surface, and 3D geometric shapes.
  • the core point cloud can have normalized size dimensions, for example, the radius of the core point cloud is 1.
  • the computer equipment can obtain the kernel point cloud by sampling the model of the basic geometry.
  • a computer device samples a line model, extracts a specific number of data points, and constructs a kernel point cloud from the extracted data points.
  • the line model can be, for example, a function representing a line.
  • the computer equipment can also sample the surface model, and construct a core point cloud through the extracted data points.
  • the solid point is the neighborhood point cloud
  • the hollow point is the core point cloud
  • Figure 3b and Figure 3c are the three-dimensional Hausdo of the neighborhood point cloud, the core point cloud 1 and the core point cloud 2, respectively. Schematic diagram of the convolution.
  • the Hausdorff distance is the distance between compact subsets in the metric space and is used to calculate the similarity between two point clouds.
  • the distance matrix is a matrix composed of the distance between the data points in the neighborhood point cloud and the data points in the core point cloud, which represents the similarity between the core point cloud and the neighborhood point cloud.
  • the distance matrix is M, m i, j ⁇ M, m i, j represents the distance between the i-th data point in the neighborhood point cloud and the j-th data point in the kernel point cloud, when m i, j If it is not the shortest distance from the i-th data point in the neighborhood point cloud to the core point cloud, nor is it the shortest distance from the j-th data point in the core point cloud to the neighborhood point cloud, set mi, j to 0 .
  • the computer device performs convolution calculation on the neighborhood point cloud, the distance matrix and the network weight matrix through the Hausdorff convolution layer in the encoder to obtain high-dimensional point cloud features; the encoder and the decoder are in the deep learning network. of the two parts.
  • the network weight matrix is a matrix that weights the result of the convolution of the neighborhood point cloud and the distance matrix.
  • the Hausdorff convolution layer is a volume base layer that performs convolution calculations through the Hausdorff convolution algorithm.
  • the Hausdorff convolution algorithm performs convolution calculations on the features of the input data points through the distance matrix and the network weight matrix.
  • the computer equipment uses the distance matrix and the neighborhood point cloud to convolve, which is equivalent to weighting the characteristics of the neighborhood point cloud with the distance matrix, and then weighting again through the network weight matrix, which can extract a more abstract high-dimensional point cloud. feature.
  • the encoder can include multiple Hausdorff convolution layers, and each Hausdorff convolutional layer inputs the extracted features into the next Hausdorff convolutional layer for feature extraction, so that through multiple Hausdorff convolutional layers
  • the Sdorf convolutional layer extracts high-dimensional semantic features.
  • each Hausdorff convolutional layer in the encoder includes multiple Hausdorff convolution modules, each Hausdorff convolution module has a different query scale, that is, each Hausdorff convolution module has a different query scale.
  • the Stoff convolution module performs convolution calculations, the query radius of the neighborhood point cloud is different, so that multi-scale deep feature extraction and abstraction can be performed.
  • the processing process of the decoder is opposite to that of the encoder.
  • the decoder reduces the dimension of high-dimensional semantic features, and inputs the reduced-dimensional features into the classifier for semantic classification.
  • the computer device performs feature dimension reduction on the high-dimensional point cloud features through the decoder, so that the classifier performs semantic classification on the point cloud data according to the target point cloud features obtained by the dimension reduction.
  • each Hausdorff convolutional layer may include two Hausdorff convolution modules, and after the convolution of the first Hausdorff convolution module, the output features The size of , remains the same as c in , and after the convolution of the second Hausdorff convolution module, the output is 2*c in , and the decoder reduces the feature of size 2*c in to c in .
  • the semantic classification is to add semantic labels to each data point in the point cloud data, so as to classify the data points into different semantic categories through the added semantic labels.
  • the point cloud data is the point cloud data of the space environment collected by the lidar on the unmanned vehicle, such as people, vehicles and houses in the space environment.
  • the computer equipment can semantically classify the point cloud data by adding red labels to the data points representing people in the point cloud data, adding blue labels to the data points representing vehicles in the point cloud data, and adding a blue label to the data points representing houses in the point cloud data. Click to add a green label.
  • the computer equipment obtains point cloud data, constructs a corresponding neighborhood point cloud for each data point in the point cloud data, and then calculates the Hausdorff between the neighborhood point cloud and the pre-built core point cloud. distance to get the distance matrix.
  • the computer equipment performs convolution calculation on the neighborhood point cloud, distance matrix and network weight matrix through the Hausdorff convolution layer in the encoder to obtain high-dimensional point cloud features, and features the high-dimensional point cloud features through the decoder. Dimensionality reduction, so that the classifier can semantically classify point cloud data according to the target point cloud features obtained by dimensionality reduction.
  • the computer equipment obtains the distance matrix by calculating the Hausdorff distance, and convolves the distance matrix and the neighboring point cloud, and then uses the network weight matrix to weight the result of the convolution calculation to obtain the final high-dimensional point cloud feature.
  • the extracted high-dimensional point cloud features have high precision, so the result of semantic classification performed by computer equipment on the basis of the obtained high-dimensional point cloud features has high accuracy.
  • the point cloud data P is input into the Hausdorff convolution layer, and the point cloud data includes the x, y, and z coordinates of each data point, and the features carried by each data point, and the features include color, intensity, number of laser echoes, etc.
  • the size of the data matrix formed by the point cloud data P is n p ⁇ (3+c raw ), where n p is the number of data points in P, and c raw is the feature dimension of the data points. For example, each data point carries If there are three features, the feature dimension is 3.
  • the neighborhood point cloud of the data point p i in the point cloud data is Q
  • the number of data points in Q is n q
  • the size of the data matrix formed by Q is n q ⁇ (3+c raw ).
  • the feature matrix formed by the features of each data point in Q is c in .
  • the pre-built core point cloud for the neighborhood point cloud Q is G
  • the distance matrix calculated according to the neighborhood point cloud Q and the core point cloud G is M min
  • the network weight matrix is W
  • the computer equipment calculates by formula (1) Get the features of the data point pi .
  • the size of M min is n g ⁇ n q
  • C out is the characteristic of the calculated data point p i
  • W is a matrix of size (3+c raw ) ⁇ c out .
  • the matrix is obtained after M min *C in *W convolution, and the computer device obtains the maximum value of the elements in the matrix through the max function, as the feature of the extracted data point p i .
  • the computer device may replace the max function in formula (1) with a min function or a sum function.
  • the distance matrix is first calculated according to the kernel point cloud and the neighborhood point cloud, and then the distance matrix is normalized. Then, the feature matrix corresponding to the neighborhood point cloud is convolved with the normalized distance matrix, and the result of the convolution is convolved with the network weight matrix again to obtain the weighted distance matrix.
  • the computer equipment combines the features of the weighted distance matrix with the weighted distance matrix output by other convolutional layers to obtain output features.
  • the number of Hausdorff convolutional layers in the encoder is not less than two; the computer device uses the Haussdorff convolutional layers in the encoder to perform a calculation on the neighborhood point cloud, the distance matrix and the network weight matrix. Perform convolution calculation to obtain high-dimensional point cloud features including: in the process of convolution calculation, for the first Hausdorff convolution layer, the neighborhood point cloud is used as the input feature to convolve the distance matrix and the network weight matrix with the distance matrix and the network weight matrix.
  • the computer equipment forms the input feature matrix of the features of the data points in the neighborhood point cloud of each data point, and inputs it into the first layer of Hausdorff convolution layer to extract the first layer features. Then input the features of the first layer into the second layer of Hausdorff convolutional layer, extract the second layer of features, until the last layer of Hausdorff convolutional layer, the output of the last layer of Hausdorff convolutional layer features as high-dimensional point cloud features.
  • the query radius corresponding to each Hausdorff convolutional layer is different. Therefore, for each Hausdorff convolutional layer, the neighborhood point cloud corresponding to each data point is recalculated. In one embodiment, the query radius corresponding to a Hausdorff convolutional layer doubles as the number of layers increases.
  • the encoder includes five Hausdorff convolutional layers.
  • the computer equipment performs feature extraction on the data points in the point cloud data layer by layer through the multi-layer Hausdorff convolution layer.
  • Each layer extracts higher-dimensional semantic features on the basis of the previous layer to extract more abstract high-level features.
  • dimensional semantic features which improves the accuracy of semantic classification.
  • the corresponding neighborhood point clouds are also different, which means that different Hausdorff convolutional layers in the encoder have different receptive fields, and the extracted The features utilize contextual semantic information in different 3D spaces. Therefore, the final extracted high-dimensional point cloud features fuse the features of different sizes of neighborhood point clouds, which is more accurate.
  • the computer equipment constructs a corresponding neighborhood point cloud for each data point in the point cloud data, including the following steps:
  • the computer device selects a query point from the data points of the point cloud data.
  • the computer device extracts, from the data points of the point cloud data, the target data points whose distance from the query point is smaller than the query radius.
  • the computer device constructs a neighborhood point cloud corresponding to the query point according to the target data point.
  • the computer equipment takes each data point in the point cloud data as a query point, and constructs a neighborhood point cloud of the query point through the query radius R.
  • the query point is p 0
  • the computer equipment takes p 0 as the center of the circle.
  • the target data points whose distance p 0 is within the query radius, and the neighborhood point cloud corresponding to p 0 is constructed according to the target data points.
  • Q, Q ⁇ pi
  • the computer device may also use the query point p 0 as the center to construct a shape region other than a circle, and extract the data points of the point cloud data in the shape region to construct a neighborhood point of the query point cloud.
  • the computer equipment constructs the neighborhood point cloud of each data point in the point cloud data, and the constructed neighborhood point cloud is equivalent to the receptive field of the pixels in the two-dimensional image, so the computer equipment can pass the neighborhood point cloud of each data point.
  • the feature convolution calculates the features of each data point, and performs feature extraction on the data points.
  • the Hausdorff distance includes the first shortest distance and the second shortest distance; the computer device calculates the Hausdorff distance between the neighborhood point cloud and the pre-built kernel point cloud, and the obtained distance matrix includes: Calculate the first shortest distance between each data point in the neighborhood point cloud and the core point cloud respectively, and form the first shortest distance set by each first shortest distance; calculate each data point and neighborhood point in the core point cloud separately The second shortest distance between clouds, and each second shortest distance constitutes a second shortest distance set; a distance matrix is obtained by calculating the first shortest distance set and the second shortest distance set.
  • the core point cloud is G
  • the first shortest distance from each point qi (q i ⁇ Q) in Q to the core point cloud G is: where g is the data point in the core point cloud.
  • the second shortest distance to the neighborhood point cloud Q is: The computer device constructs the first shortest distance set and the second shortest distance set respectively according to the acquired first shortest distance and the second shortest distance.
  • the first shortest distance set S QG is:
  • the second shortest distance set is:
  • the distance matrix M min calculated by the computer equipment according to the first shortest distance set and the second shortest distance set is: Among them, i and j are both positive integers greater than or equal to 1, which are the subscript indices of the data points in G and Q, respectively. Since S QG contains the shortest distance value of each data point in the neighborhood point cloud Q to G, and S GQ contains the shortest distance value of each data point in the kernel point cloud G to Q, the Hausdorff distance is from The bidirectional angles of the neighborhood point cloud and the core point cloud measure the distance between them. Since only the values subordinate to S QG and S GQ are retained in M min , and the other values are set to zero, M min retains valuable information in the process of point cloud data comparison, eliminates the influence of redundant data, and effectively avoids noise interference.
  • the computer device calculates the Hausdorff distance between the neighborhood point cloud and the pre-built kernel point cloud, and after obtaining the distance matrix, the method further includes: calculating the ratio of the non-zero elements of the distance matrix to the query radius ; Calculate the normalized distance matrix according to the ratio; Convolve the neighborhood point cloud, the normalized distance matrix and the network weight matrix through the Hausdorff convolution layer in the encoder.
  • the distance matrix is M min
  • the query radius is R
  • the ratio of non-zero elements in the distance matrix to the query radius is M min (i, j)/R
  • the computer equipment passes 1-M min (i, j)/R Normalize the distance matrix to the interval (0, 1).
  • the computer device before calculating the Hausdorff distance between the neighborhood point cloud and the pre-built kernel point cloud, the computer device further includes the following steps:
  • the computer device creates a spherical area, and constructs a three-dimensional parametric model in the spherical area.
  • the computer device samples the three-dimensional parameter model through the farthest point algorithm, and constructs the core point cloud.
  • the three-dimensional parametric model is a model representing a geometric shape.
  • it can be a function representing a geometric shape.
  • the geometric shapes can be geometric shapes such as points, lines, surfaces, and volumes.
  • a line is a straight line
  • a surface is a plane
  • a volume is a sphere. Therefore, three-dimensional parametric models can be functions representing points, lines, surfaces, and volumes.
  • These geometric shapes can be well embedded in the three-dimensional neighborhood space, and maintain the symmetry of the shape, and the sphere also satisfies the rotation invariance.
  • the three-dimensional parametric model is not limited to a model representing points, lines, surfaces, and volumes, but may also be models representing other geometric shapes.
  • the computer equipment can preset the number of samples sampled by the farthest point algorithm, and construct a kernel point cloud according to the sample points sampled from the three-dimensional parameter model. For example, as shown in Figure 7, from left to right are the kernel point clouds of point, line, surface, and body shape.
  • the size of the kernel point cloud constructed by the computer equipment is a normalized scale, which can be changed with the change of the query radius during the convolution calculation, which is consistent with the query half.
  • the Hausdorff convolution layer in the encoder includes a multi-kernel Hausdorff convolution sublayer, and each sublayer in the multi-kernel Hausdorff convolution sublayer corresponds to a different distance matrix; by encoding The Hausdorff convolution layer in the device performs the convolution calculation on the neighborhood point cloud, the distance matrix and the network weight matrix, and the obtained high-dimensional point cloud features include: through each sublayer in the multi-core Hausdorff convolution sublayer The convolution calculation is performed on the neighborhood point cloud, the network weight matrix and the distance matrix corresponding to each sub-layer to obtain the point cloud features of each sub-layer; the point cloud features of each sub-layer are fused to obtain high-dimensional point cloud features.
  • the Hausdorff convolution layer in the encoder includes multiple multi-core Hausdorff convolution sub-layers, and each multi-core Hausdorff convolution sublayer corresponds to a different kernel point cloud, that is, the convolution The distance matrix calculated by the layer based on the neighborhood point cloud and the core point cloud is different.
  • the encoder fuses the point cloud features of each sub-layer to obtain high-dimensional point cloud features.
  • the fusion of point cloud features may be addition or weighted addition of point cloud features, or fusion is performed after point cloud features are processed, and the processing process may be linear or non-linear processing.
  • the Hausdorff convolution layer includes n k multi-core Hausdorff convolution sublayers, and the multicore Hausdorff convolution sublayer performs convolution calculation to obtain the corresponding The convolutional response vector C out , the computer device according to the formula First, each convolution response vector is processed through the activation function RELU, and then the processed convolution vectors are accumulated to obtain the accumulated sum vector, and then the accumulated sum vector is processed according to the activation function to obtain the fused feature C out .
  • the activation function RELU can be a nonlinear function, and processing the convolution response vector through the activation function can increase the nonlinear component of a single convolution response vector.
  • Computer equipment is paired with nonlinear activation functions Processing is performed to non-linearly map the accumulated RELU(C out ) to construct a fused feature.
  • the computer device splices the convolution response vectors output by the multi-core Hausdorff convolution sublayer, and fuses the convolution response vectors through splicing.
  • the computer device maps the convolution response vectors output by the multiple multi-core Hausdorff convolution sublayers into one-dimensional data through a multilayer perceptron, so as to realize the fusion of the convolution response vectors.
  • the computer equipment obtains the distance matrix by calculating the Hausdorff distance, and the similarity between the neighborhood point cloud and the core point cloud can be extracted by the method of explicit geometric comparison, which is the sparse point.
  • the cloud data brings the complement of geometric information and increases the interpretability of the 3D point cloud convolution method.
  • the Hausdorff distance adopts a bidirectional measurement method, and compares the similarity of point clouds from the perspective of the neighborhood point cloud and the core point cloud, and finally combines the comparison results of the two to obtain a distance matrix.
  • the point cloud data are respectively the S3DIS dataset and the SemanticKITTI dataset
  • the deep learning neural network includes five Hausdorff convolution layers, and the query radius of each layer is twice that of the previous layer.
  • Figure 9a shows the ground truth of the scene labels of the S3DIS dataset
  • Figure 9b shows the result of semantic classification of the S3DIS dataset obtained by the deep learning neural network.
  • Figure 9c shows the ground truth of the SemanticKITTI dataset
  • Figure 9d shows the result of semantic classification of the SemanticKITTI dataset through a deep learning neural network.
  • the objects with the same semantics are represented by a unified color, and the same semantic objects should have the same color in the two pictures.
  • the deep learning neural network can better distinguish the main semantic objects in the scene, and maintain the segmentation integrity and accuracy of the main semantic objects.
  • deep learning neural networks are more effective for sparse or dense data. Deep learning neural networks can perform complete semantic classification of semantic objects according to the geometric shape of objects, which fully demonstrates the learning ability of deep learning neural networks.
  • the point cloud data is the S3DIS data set
  • the computer equipment uses sequences 1-4, 6 as the training set, sequence 5 as the test set, and uses the three-dimensional spherical kernel point cloud as the kernel of the single-kernel Hausdorff convolution
  • mIOU mean of Intersection Over Union
  • the Hausdorff convolution layer of the deep learning neural network is a single-kernel Hausdorff convolution layer
  • the mIOU of semantic classification results is 66.7%.
  • the Hausdorff convolutional layer of the deep learning neural network is a multi-kernel Hausdorff convolutional layer
  • the mIOU of the semantic classification result is 68.2%.
  • the point cloud data is the SemanticKITTI data set
  • the computer equipment uses sequences 0-7, 9-10 as the training set, and sequence 8 as the test set
  • the Hausdorff convolutional layer of the deep learning neural network is a single
  • kernel Hausdorff convolutional layer the mIOU of semantic classification result is 59.6%.
  • the Hausdorff convolutional layer of the deep learning neural network is a multi-kernel Hausdorff convolutional layer
  • the mIOU of the semantic classification result is 60.3%. Since the SemanticKITTI dataset is a dataset formed by scanning outdoor scenes, the semantic classification results of the deep learning neural network prove that the deep learning neural network including Hausdorff convolution is effective for the classification of outdoor single-frame point cloud data.
  • FIGS. 2 and 5-6 are shown in sequence according to the arrows, these steps are not necessarily executed in the sequence indicated by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited to the order, and these steps may be performed in other orders. Moreover, at least a part of the steps in FIGS. 2 and 5-6 may include multiple steps or multiple stages. These steps or stages are not necessarily executed and completed at the same time, but may be executed at different times. These steps or stages The order of execution of the steps is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a portion of the steps or stages in the other steps.
  • a point cloud data processing device including: an acquisition module, a distance calculation module, a convolution calculation module and a feature dimension reduction module, wherein:
  • an acquisition module 1002 configured to acquire point cloud data, and construct a corresponding neighborhood point cloud for each data point in the point cloud data;
  • the distance calculation module 1004 is used to calculate the Hausdorff distance between the neighborhood point cloud and the pre-built kernel point cloud to obtain a distance matrix
  • the convolution calculation module 1006 is used to perform convolution calculation on the neighborhood point cloud, the distance matrix and the network weight matrix through the Hausdorff convolution layer in the encoder to obtain high-dimensional point cloud features; the encoder and the decoder are two parts in a deep learning network; and
  • the feature dimension reduction module 1008 is configured to perform feature dimension reduction on the high-dimensional point cloud features through the decoder, so that the classifier can perform semantic classification on the point cloud data according to the target point cloud features obtained by the dimension reduction.
  • the computer equipment obtains point cloud data, constructs a corresponding neighborhood point cloud for each data point in the point cloud data, and then calculates the Hausdorff between the neighborhood point cloud and the pre-built core point cloud. distance to get the distance matrix.
  • the computer equipment performs convolution calculation on the neighborhood point cloud, distance matrix and network weight matrix through the Hausdorff convolution layer in the encoder to obtain high-dimensional point cloud features, and features the high-dimensional point cloud features through the decoder. Dimensionality reduction, so that the classifier can semantically classify point cloud data according to the target point cloud features obtained by dimensionality reduction.
  • the computer equipment obtains the distance matrix by calculating the Hausdorff distance, and convolves the distance matrix and the neighboring point cloud, and then uses the network weight matrix to weight the result of the convolution calculation to obtain the final high-dimensional point cloud feature.
  • the extracted high-dimensional point cloud features have high precision, so the result of semantic classification performed by computer equipment on the basis of the obtained high-dimensional point cloud features has high accuracy.
  • the number of Hausdorff convolution layers in the encoder is not less than two; the convolution calculation module 1006 is further used for:
  • the neighborhood point cloud is used as the input feature to perform convolution calculation with the distance matrix and the network weight matrix to obtain the output feature;
  • the output feature of the previous Haussdorff convolutional layer of the current non-first-layer Hausdorff convolutional layer is used as the input feature of this layer, and the distance matrix and network weight The matrix is convolved to obtain the output features; if the current non-first Hausdorff convolutional layer is the last Hausdorff convolutional layer, the output features of the last Hausdorff convolutional layer are used as high-dimensional Point cloud features.
  • the obtaining module 1002 is further configured to:
  • the neighborhood point cloud corresponding to the query point is constructed according to the target data point.
  • the Hausdorff distance includes the first shortest distance and the second shortest distance; the distance calculation module 1004 is further configured to:
  • the distance matrix is calculated according to the first shortest distance set and the second shortest distance set.
  • the apparatus further includes:
  • the sampling module 1012 is configured to sample the three-dimensional parameter model through the farthest point algorithm to construct a kernel point cloud.
  • the apparatus further includes:
  • a ratio calculation module 1014 for calculating the ratio of the non-zero elements of the distance matrix to the query radius
  • Obtaining module 1016 for obtaining the normalized distance matrix according to the ratio calculation
  • the convolution calculation module 1006 is also used for:
  • the neighborhood point cloud, normalized distance matrix, and network weight matrix are convolved by a Hausdorff convolutional layer in the encoder.
  • the Hausdorff convolution layer in the encoder includes a multi-kernel Hausdorff convolution sublayer, and each sublayer in the multi-kernel Hausdorff convolution sublayer corresponds to a different distance matrix; convolution
  • the computing module 1006 is also used to:
  • the point cloud features of each sub-layer are obtained by performing convolution calculation on the neighborhood point cloud, the network weight matrix and the distance matrix corresponding to each sub-layer by each sub-layer in the multi-core Hausdorff convolution sub-layer;
  • the point cloud features of each sub-layer are fused to obtain high-dimensional point cloud features.
  • Each module in the above-mentioned point cloud data processing device may be implemented in whole or in part by software, hardware and combinations thereof.
  • the above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
  • a computer device is provided, and the computer device may be a server, and its internal structure diagram may be as shown in FIG. 12 .
  • the computer device includes a processor, memory, and a network interface connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium, an internal memory.
  • the nonvolatile storage medium stores an operating system, a computer program, and a database.
  • the internal memory provides an environment for the execution of the operating system and computer programs in the non-volatile storage medium.
  • the database of the computer device is used to store the point cloud data processing data.
  • the network interface of the computer device is used to communicate with an external terminal through a network connection.
  • the computer program implements a point cloud data processing method when executed by the processor.
  • a computer device in one embodiment, the computer device may be a terminal, and its internal structure diagram may be as shown in FIG. 13 .
  • the computer equipment includes a processor, memory, a communication interface, a display screen, and an input device connected by a system bus.
  • the processor of the computer device is used to provide computing and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium, an internal memory.
  • the nonvolatile storage medium stores an operating system and a computer program.
  • the internal memory provides an environment for the execution of the operating system and computer programs in the non-volatile storage medium.
  • the communication interface of the computer device is used for wired or wireless communication with an external terminal, and the wireless communication can be realized by WIFI, operator network, NFC (Near Field Communication) or other technologies.
  • the computer program when executed by the processor, implements a point cloud data processing method.
  • the display screen of the computer equipment may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment may be a touch layer covered on the display screen, or a button, a trackball or a touchpad set on the shell of the computer equipment , or an external keyboard, trackpad, or mouse.
  • FIGS. 12 and 13 are only block diagrams of partial structures related to the solution of the present application, and do not constitute a limitation on the computer equipment to which the solution of the present application is applied.
  • a device may include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.
  • a computer device including a memory and a processor, a computer program is stored in the memory, and when the processor executes the computer program, the processor implements the following steps: acquiring point cloud data, and analyzing each element in the point cloud data.
  • the corresponding neighborhood point cloud is constructed from the data points; the Hausdorff distance between the neighborhood point cloud and the pre-built kernel point cloud is calculated, and the distance matrix is obtained;
  • the point cloud, distance matrix and network weight matrix are convolutional to obtain high-dimensional point cloud features;
  • the encoder and decoder are two parts of the deep learning network;
  • the feature dimension reduction is performed on the high-dimensional point cloud features through the decoder, In order to make the classifier perform semantic classification on the point cloud data according to the target point cloud features obtained by dimensionality reduction.
  • the number of Hausdorff convolutional layers in the encoder is not less than two; when the processor executes the computer program, the following steps are further implemented: in the process of performing the convolution calculation, for the first layer of Hausdorff convolutional layers For the non-first-layer Hausdorff convolutional layer, the current non-first-layer Hausdorff The output feature of the upper Hausdorff convolutional layer of the convolutional layer is used as the input feature of this layer, and the convolution calculation is performed with the distance matrix and the network weight matrix to obtain the output feature; if the current non-first layer Hausdorff convolution When the layer is the last Hausdorff convolutional layer, the output features of the last Hausdorff convolutional layer are used as high-dimensional point cloud features.
  • the processor further implements the following steps when executing the computer program: selecting a query point from the data points of the point cloud data; extracting target data whose distance from the query point is less than the query radius from the data points of the point cloud data point; construct the neighborhood point cloud corresponding to the query point according to the target data point.
  • the Hausdorff distance includes the first shortest distance and the second shortest distance; when the processor executes the computer program, the processor further implements the following steps: respectively calculating the distance between each data point in the neighborhood point cloud and the core point cloud The first shortest distance, and the first shortest distance set is composed of the first shortest distances; the second shortest distance between each data point in the core point cloud and the neighboring point cloud is calculated separately, and each second shortest distance is composed of the first shortest distance. Two shortest distance sets; a distance matrix is obtained by calculating according to the first shortest distance set and the second shortest distance set.
  • the processor further implements the following steps when executing the computer program: creating a spherical area, and constructing a three-dimensional parametric model in the spherical area; sampling the three-dimensional parametric model through the farthest point algorithm to construct a kernel point cloud.
  • the processor also implements the following steps when executing the computer program: calculating the ratio of the non-zero elements of the distance matrix to the query radius; calculating the normalized distance matrix according to the ratio;
  • the convolutional layer performs convolution calculations on the neighborhood point cloud, the normalized distance matrix, and the network weight matrix.
  • the Hausdorff convolution layer in the encoder includes a multi-core Hausdorff convolution sublayer, and each sublayer in the multi-core Hausdorff convolution sublayer corresponds to a different distance matrix; the processor When executing the computer program, the following steps are also implemented: perform convolution calculation on the neighborhood point cloud, the network weight matrix and the distance matrix corresponding to each sublayer through each sublayer in the multi-core Hausdorff convolution sublayer, and obtain each sublayer The point cloud features of each sub-layer are fused to obtain high-dimensional point cloud features.
  • a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: acquiring point cloud data, and constructing each data point in the point cloud data Corresponding neighborhood point cloud; calculate the Hausdorff distance between the neighborhood point cloud and the pre-built kernel point cloud to obtain the distance matrix; pass the Hausdorff convolution layer in the encoder to the neighborhood point cloud, The distance matrix and the network weight matrix are convolutional to obtain high-dimensional point cloud features; the encoder and the decoder are two parts in the deep learning network; the feature dimension reduction is performed on the high-dimensional point cloud features through the decoder to make the classification
  • the classifier performs semantic classification on the point cloud data according to the target point cloud features obtained by dimensionality reduction.
  • the number of Hausdorff convolutional layers in the encoder is not less than two; when the computer program is executed by the processor, the following steps are also implemented: in the process of performing the convolution calculation, for the first layer of Hausdorff convolutional layers
  • the Dove convolution layer uses the neighborhood point cloud as the input feature to perform convolution calculation with the distance matrix and the network weight matrix to obtain the output feature; for the non-first-layer Hausdorff convolutional layer, the current non-first-layer Hausdorff convolution layer
  • the output feature of the previous Hausdorff convolutional layer of the convolutional layer is used as the input feature of this layer, and the convolution calculation is performed with the distance matrix and the network weight matrix to obtain the output feature;
  • the stacking layer is the last Hausdorff convolutional layer
  • the output features of the last Hausdorff convolutional layer are used as high-dimensional point cloud features.
  • the following steps are further implemented: selecting a query point from the data points of the point cloud data; in the data points of the point cloud data, extracting a target whose distance from the query point is less than the query radius Data points; construct the neighborhood point cloud corresponding to the query point according to the target data point.
  • the Hausdorff distance includes the first shortest distance and the second shortest distance; when the computer program is executed by the processor, it further implements the following steps: separately calculating the distance between each data point in the neighborhood point cloud and the kernel point cloud
  • the first shortest distance is the first shortest distance
  • the first shortest distance set is composed of the first shortest distance
  • the second shortest distance between each data point in the core point cloud and the neighboring point cloud is calculated separately, and is composed of each second shortest distance.
  • the second shortest distance set is obtained by calculating according to the first shortest distance set and the second shortest distance set.
  • the computer program further implements the following steps when executed by the processor: creating a spherical area, and constructing a three-dimensional parametric model in the spherical area; sampling the three-dimensional parametric model through the farthest point algorithm to construct a kernel point cloud.
  • the computer program further implements the following steps when executed by the processor: calculating the ratio of the non-zero elements of the distance matrix to the query radius; calculating the normalized distance matrix according to the ratio;
  • the convolutional layer performs convolution calculations on the neighborhood point cloud, the normalized distance matrix, and the network weight matrix.
  • the Hausdorff convolution layer in the encoder includes multi-core Hausdorff convolution sublayers, each of which corresponds to a different distance matrix; computer program When executed by the processor, the following steps are also implemented: the convolution calculation is performed on the neighborhood point cloud, the network weight matrix and the distance matrix corresponding to each sublayer through each sublayer in the multi-core Hausdorff convolution sublayer, and each sublayer is obtained.
  • the point cloud features of the layer; the point cloud features of each sub-layer are fused to obtain high-dimensional point cloud features.
  • Non-volatile memory may include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, or optical memory, and the like.
  • Volatile memory may include random access memory (RAM) or external cache memory.
  • RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

一种点云数据处理方法、装置、计算机设备和存储介质。所述方法包括：获取点云数据，并对所述点云数据中的各数据点构建对应的邻域点云；计算所述邻域点云和预构建的核点云之间的豪斯多夫距离，得到距离矩阵；通过编码器中的豪斯多夫卷积层对所述邻域点云、所述距离矩阵和网络权重矩阵进行卷积计算，得到高维点云特征；所述编码器和解码器是深度学习网络中的两个部分；通过所述解码器对所述高维点云特征进行特征降维，以使分类器根据降维所得的目标点云特征对所述点云数据进行语义分类。

Description

点云数据处理方法、装置、计算机设备和存储介质
相关申请交叉引用
本申请要求2021年03月15日递交的、标题为“点云数据处理方法、装置、计算机设备和存储介质”、申请号为2021102744393的中国申请，其公开内容通过引用全部结合在本申请中。
技术领域
本申请涉及一种点云数据处理方法、装置、计算机设备和存储介质。
背景技术
随着计算机图形学的发展，出现了点云数据语义分类技术，点云数据语义分类技术通过对点云数据中数据点进行特征提取，并根据所提取的特征对点云数据进行语义分类。传统方法中，对点云数据进行规则下采样，在更低分辨率上实现点云数据的规则化，然后再对规则化的点云数据进行特征提取和语义分类。但是，传统方法会导致点云数据更加稀疏，从而语义分类的准确性较低。
发明内容
根据多个实施例，本申请第一方面提供一种点云数据处理方法，所述方法包括：
获取点云数据，并对所述点云数据中的各数据点构建对应的邻域点云；
计算所述邻域点云和预构建的核点云之间的豪斯多夫距离，得到距离矩阵；
通过编码器中的豪斯多夫卷积层对所述邻域点云、所述距离矩阵和网络权重矩阵进行卷积计算，得到高维点云特征；所述编码器和解码器是深度学习网络中的两个部分；以及
通过所述解码器对所述高维点云特征进行特征降维，以使分类器根据降维所得的目标点云特征对所述点云数据进行语义分类。
根据多个实施例，本申请第二方面提供一种点云数据处理装置，所述装置包括：
获取模块，用于获取点云数据，并对所述点云数据中的各数据点构建对应的邻域点云；
距离计算模块，用于计算所述邻域点云和预构建的核点云之间的豪斯多夫距离，得到距离矩阵；
卷积计算模块，用于通过编码器中的豪斯多夫卷积层对所述邻域点云、所述距离矩阵和网络权重矩阵进行卷积计算，得到高维点云特征；所述编码器和解码器是深度学习网络中的两个部分；以及
特征降维模块，用于通过所述解码器对所述高维点云特征进行特征降维，以使分类器根据降维所得的目标点云特征对所述点云数据进行语义分类。
根据多个实施例，本申请第三方面提供一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，所述处理器执行所述计算机程序时实现所述点云数据处理方法的步骤。
根据多个实施例，本申请第四方面提供一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现所述点云数据处理方法的步骤。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征和优点将从说明书、附图以及权利要求书变得明显。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其它的附图。
图1为一个实施例中点云数据处理方法的应用环境图；
图2为一个实施例中点云数据处理方法的流程示意图；
图3a为一个实施例中邻域点云示意图；
图3b为一个实施例中邻域点云与核点云1卷积示意图；
图3c为一个实施例中邻域点云与核点云2卷积示意图；
图4为一个实施例中卷积计算的流程示意图；
图5为一个实施例中构建邻域点云方法的流程示意图；
图6为一个实施例中构建核点云方法的流程示意图；
图7为一个实施例中核点云的示意图；
图8为一个实施例中多核豪斯多夫卷积方法的原理示意图；
图9a为一个实施例中S3DIS数据集的场景标签真值示意图；
图9b为一个实施例中深度学习神经网络对S3DIS数据集进行语义分类的结果示意图；
图9c为一个实施例中SemanticKITTI数据集的场景标签真值示意图；
图9d为一个实施例中深度学习神经网络对SemanticKITTI数据集进行语义分类的结果示意图；
图10为一个实施例中点云数据处理装置的结构框图；
图11为另一个实施例中点云数据处理装置的结构框图；
图12为一个实施例中计算机设备的内部结构图；
图13为另一个实施例中计算机设备的内部结构图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
需要说明的是，本发明的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本发明的实施例能够以除了在这里图示或描述的那些以外的顺序实施。
本申请提供的点云数据处理方法，可以应用于如图1所示的应用环境中。其中，计算机设备102通过网络与点云采集设备104进行通信，从点云采集设备104获取点云数据，并对点云数据中的各数据点构建对应的邻域点云，然后计算机设备102计算邻域点云和预构建的核点云之间的豪斯多夫距离，得到距离矩阵，并通过编码器中的豪斯多夫卷积层对邻域点云、距离矩阵和网络权重矩阵进行卷积计算，得到高维点云特征。最后，计算机设备102通过解码器对高维点云特征进行特征降维，以使分类器根据降维所得的目标点云特征对点云数据进行语义分类。其中，计算机设备102可以是终端，也可以是服务器，终端可以但不限于是各种个人计算机、笔记本电脑、智能手机、平板电脑和便携式可穿戴设备，服务器可以用独立的服务器或者是多个服务器组成的服务器集群来实现。点云采集设备104可以是任意的能够采集到点云数据的设备，例如可以是激光雷达、三维激光扫描仪、RGBD深度相机等。
在一个实施例中，如图2所示，提供了一种点云数据处理方法，以该方法应用于图1中的计算机设备为例进行说明，包括以下步骤：
S202，计算机设备获取点云数据，并对点云数据中的各数据点构建对应的邻域点云。
其中，点云数据是点云采集设备通过对空间中物体进行扫描所获取的数据。点云数据中包括多个数据点，每个数据点包括该数据点的位置坐标，以及该数据点的特征。数据点的特征包括颜色特征、强度特征、激光回波次数等，颜色特征例如可以是颜色RGB值。在一个实施例中，点云数据是三维点云数据，特别地，可以是三维单帧点云数据。三维单帧点云数据比较稀疏，数据线之间间隔较大。在一个实施例中，点云数据可以是无人驾驶汽车上的激光雷达对道路环境中的物体进行扫描所获取的数据。在另一个实施例中，点云数据可以是人工智能机器人上的深度相机对空间环境中物体进行扫描所获取的数据。
其中，某个数据点对应的邻域点云是以该数据点为中心的区域内的数据点组成的点云。例如，如图3a所示，邻域点云是由以数据点p 0为圆心，距离圆心的距离小于半径的圆形区域内的数据点构成的。邻域点云中数据点包括数据点的x、y、z坐标以及数据点的特征。
S204，计算机设备计算邻域点云和预构建的核点云之间的豪斯多夫距离，得到距离矩阵。
其中，核点云是具有确定的几何形状的点云。例如，核点云可以具有基本的几何形状，包括点状、线状、面状和三维几何体形状。核点云可以具有归一化的大小尺寸，例如，核点云的半径为1。计算机设备可以通过对基本几何形状的模型进行抽样得到核点云。例如，计算机设备对直线模型进行抽样，从中抽取特定个数的数据点，由抽取出的数据点构建出核点云，直线模型例如可以是表示直线的函数。计算机设备还可以对面模型进行抽样，通过抽取出的数据点构建出核点云。如图3b、图3c所示，实心点为邻域点云，空心点为核点云，图3b与图3c分别是邻域点云与核点云1以及核点云2的三维豪斯多夫卷积示意图。
其中，豪斯多夫距离是度量空间中紧子集间的距离，用于计算两个点云之间的相似度。距离矩阵是由邻域点云中数据点与核点云中数据点间的距离组成的矩阵，表示了核点云与邻域点云间的相似性。例如，距离矩阵为M，m i，j∈M，m i，j表示邻域点云中的第i 个数据点与核点云中的第j个数据点间的距离，当m i，j不是邻域点云中的第i个数据点至核点云的最短距离，也不是核点云中的第j个数据点至邻域点云的最短距离时，将m i，j置为0。
S206，计算机设备通过编码器中的豪斯多夫卷积层对邻域点云、距离矩阵和网络权重矩阵进行卷积计算，得到高维点云特征；编码器和解码器是深度学习网络中的两个部分。
其中，网络权重矩阵是对邻域点云和距离矩阵的卷积结果进行加权的矩阵。豪斯多夫卷积层是通过豪斯多夫卷积算法进行卷积计算的卷基层。豪斯多夫卷积算法通过距离矩阵和网络权重矩阵对输入的数据点的特征进行卷积计算。计算机设备用距离矩阵和邻域点云进行卷积，相当于用距离矩阵对邻域点云的特征进行加权，加权后再通过网络权重矩阵再次进行加权，可以提取出更抽象的高维点云特征。
其中，编码器中可以包括多个豪斯多夫卷积层，每个豪斯多夫卷积层将提取出的特征输入下一豪斯多夫卷积层进行特征提取，从而通过多个豪斯多夫卷积层提取出高维语义特征。在一个实施例中，编码器中的每个豪斯多夫卷积层包括多个豪斯多夫卷积模块，每个豪斯多夫卷积模块具有不同的查询尺度，也即每个豪斯多夫卷积模块在进行卷积计算时，邻域点云的查询半径不同，从而可以进行多尺度的深度特征提取和抽象。解码器的处理过程与编码器相反，解码器对高维语义特征进行降维，并将降维后的特征输入分类器中进行语义分类。
S208，计算机设备通过解码器对高维点云特征进行特征降维，以使分类器根据降维所得的目标点云特征对点云数据进行语义分类。
其中，特征降维是降低特征的数据量大小。在一个实施例中，在编码器中，每个豪斯多夫卷积层可以包括两个豪斯多夫卷积模块，经过第一个豪斯多夫卷积模块的卷积后，输出特征的大小保持不变，仍为c in，而经过第二个豪斯多夫卷积模块的卷积后，输出为2*c in，解码器将2*c in大小的特征降为c in。
其中，语义分类是对点云数据中每个数据点添加语义标签，以通过添加的语义标签将数据点划分为不同的语义类别。例如，点云数据是无人车上的激光雷达采集的空间环境的点云数据，空间环境中有人、车辆和房屋等。计算机设备对该点云数据进行语义分类可以是对点云数据中表示人的数据点添加红色标签，对点云数据中表示车辆的数据点添加蓝 色标签，对点云数据中表示房屋的数据点添加绿色标签。
上述实施例中，计算机设备获取点云数据，并对点云数据中的各数据点构建对应的邻域点云，然后计算邻域点云和预构建的核点云之间的豪斯多夫距离，得到距离矩阵。计算机设备通过编码器中的豪斯多夫卷积层对邻域点云、距离矩阵和网络权重矩阵进行卷积计算，得到高维点云特征，并通过解码器对高维点云特征进行特征降维，以使分类器根据降维所得的目标点云特征对点云数据进行语义分类。计算机设备通过计算豪斯多夫距离得到距离矩阵，并将距离矩阵和邻域点云进行卷积计算，然后再用网络权重矩阵对卷积计算的结果进行加权，得到最终的高维点云特征，从而实现对点云数据的特征提取，所提取的高维点云特征具有较高的精度，所以，计算机设备在所得到的高维点云特征的基础上进行的语义分类的结果具有较高的准确性。
在一个实施例中，将点云数据P输入豪斯多夫卷积层，点云数据中包括每个数据点的x、y、z坐标，以及每个数据点携带的特征，特征包括颜色、强度、激光回波次数等。点云数据P构成的数据矩阵的大小为n p×(3+c raw)，其中，n p为P中数据点的个数，c raw为数据点的特征维度，例如，每个数据点携带了三种特征，则特征维度为3。假设点云数据中数据点p i的邻域点云为Q，Q中数据点的个数为n q，Q构成的数据矩阵的大小为n q×(3+c raw)。Q中各数据点的特征构成的特征矩阵为c in。针对邻域点云Q预先构建的核点云为G，根据邻域点云Q和核点云G计算得到的距离矩阵为M min，网络权重矩阵为W，则计算机设备通过公式(1)计算得到数据点p i的特征。
Figure PCTCN2021082250-appb-000001
其中，M min的大小为n g×n q，C out为计算得到的数据点p i的特征。W为(3+c raw)×c out大小矩阵。M min*C in*W卷积后得到矩阵，计算机设备通过max函数获取矩阵中元素的最大值，作为所提取的数据点p i的特征。在一个实施例中，计算机设备可以将公式(1)中的max函数替换为min函数或者sum函数。
在一个实施例中，如图4所示，计算机设备在进行卷积计算时，首先根据核点云与邻域点云计算出距离矩阵，然后对距离矩阵进行归一化。然后使邻域点云对应的特征矩阵与归一化后的距离矩阵进行卷积，并将卷积的结果与网络权重矩阵再次进行卷积，得到加权距离矩阵。计算机设备将加权距离矩阵与其它卷积层输出的加权距离矩阵进行特征合 并，得到输出特征。
在一个实施例中，编码器中豪斯多夫卷积层的数量不少于两个；计算机设备通过编码器中的豪斯多夫卷积层对邻域点云、距离矩阵和网络权重矩阵进行卷积计算，得到高维点云特征包括：在进行卷积计算的过程中，针对首层豪斯多夫卷积层，将邻域点云作为输入特征与距离矩阵和网络权重矩阵进行卷积计算，得到输出特征；针对非首层豪斯多夫卷积层，将当前非首层豪斯多夫卷积层的上一层豪斯多夫卷积层的输出特征作为本层输入特征，与距离矩阵和网络权重矩阵进行卷积计算，得到输出特征；若当前非首层豪斯多夫卷积层为末层豪斯多夫卷积层时，将末层豪斯多夫卷积层的输出特征作为高维点云特征。
计算机设备将每个数据点的邻域点云中数据点的特征构成输入的特征矩阵输入首层豪斯多夫卷积层，提取出第一层特征。然后将第一层特征输入第二层豪斯多夫卷积层，提取出第二层特征，直到最后一层豪斯多夫卷积层，将最后一层豪斯多夫卷积层的输出特征作为高维点云特征。在一个实施例中，每一层豪斯多夫卷积层对应的查询半径不同，所以，对于每一层的豪斯多夫卷积层，重新计算每个数据点对应的邻域点云。在一个实施例中，豪斯多夫卷积层对应的查询半径随着层数的增加而加倍。在一个实施例中，编码器包括五层豪斯多夫卷积层。
计算机设备通过多层豪斯多夫卷积层逐层对点云数据中数据点进行特征提取，每一层在上一层的基础上提取更高维的语义特征，以提取出更抽象的高维语义特征，提高了语义分类的准确性。并且，对于查询半径不同的豪斯多夫卷积层，对应的邻域点云也不相同，也即相当于编码器中的不同豪斯多夫卷积层具有不同的感受野，所提取出的特征利用了不同的三维空间中的上下文语义信息。所以，最终所提取的高维点云特征融合了不同大小的邻域点云的特征，更加准确。
在一个实施例中，如图5所示，计算机设备对点云数据中的各数据点构建对应的邻域点云包括如下步骤：
S502，计算机设备从点云数据的数据点中选取查询点。
S504，计算机设备在点云数据的数据点中，提取与查询点的距离小于查询半径的目标数据点。
S506，计算机设备根据目标数据点构建查询点对应的邻域点云。
计算机设备分别将点云数据中的每个数据点作为查询点，通过查询半径R构建该查询点的邻域点云。例如，查询点为p 0，计算机设备以p 0为圆心，查询点云数据中，距离p 0在查询半径范围内的目标数据点，并根据目标数据点构建出p 0对应的邻域点云Q，Q＝{p i|||p i-p 0||＜R}。
在一个实施例中，计算机设备也可以以查询点p 0为中心，构建除圆形外的其它形状区域，并提取点云数据在该形状区域内的数据点，构建出查询点的邻域点云。
计算机设备构建点云数据中每个数据点的邻域点云，所构建的邻域点云相当于二维图像中像素的感受野，所以，计算机设备可以通过各数据点的邻域点云的特征卷积计算出各数据点的特征，对数据点进行特征提取。
在一个实施例中，豪斯多夫距离包括第一最短距离与第二最短距离；计算机设备计算邻域点云和预构建的核点云之间的豪斯多夫距离，得到距离矩阵包括：分别计算邻域点云中每个数据点与核点云间的第一最短距离，并由各第一最短距离组成第一最短距离集合；分别计算核点云中每个数据点与邻域点云间的第二最短距离，并由各第二最短距离组成第二最短距离集合；根据第一最短距离集合以及第二最短距离集合计算得到距离矩阵。
假设邻域点云为Q，核点云为G，Q中每个点q i(q i∈Q)到核点云G的第一最短距离为：
Figure PCTCN2021082250-appb-000002
其中g是核点云中的数据点。对于核点云G中的每个数据点g i(g i∈G)到邻域点云Q的第二最短距离为：
Figure PCTCN2021082250-appb-000003
计算机设备根据获取的第一最短距离和第二最短距离分别构建第一最短距离集合与第二最短距离集合。第一最短距离集合S QG为：
Figure PCTCN2021082250-appb-000004
第二最短距离集合为：
Figure PCTCN2021082250-appb-000005
计算机设备根据第一最短距离集合与第二最短距离集合计算出的距离矩阵M min为
Figure PCTCN2021082250-appb-000006
其中，i和j均大于等于1的正整 数，分别为G和Q中数据点的下标索引。由于S QG包含了邻域点云Q中每个数据点到G的最短距离值，而S GQ包含了核点云G中每个数据点到Q的最近距离值，从而豪斯多夫距离从邻域点云和核点云双向的角度对相互间的距离进行度量。由于M min中只保留了从属于S QG和S GQ的数值，其余数值置零，所以M min保留了点云数据比对过程中有价值的信息，消除了冗余数据的影响，有效避免了噪声的干扰。
在一个实施例中，计算机设备计算邻域点云和预构建的核点云之间的豪斯多夫距离，得到距离矩阵之后，方法还包括：计算距离矩阵的非零元素与查询半径的比值；根据比值计算得到归一化的距离矩阵；通过编码器中的豪斯多夫卷积层对邻域点云、归一化的距离矩阵和网络权重矩阵进行卷积计算。
其中，距离矩阵为M min，查询半径为R，距离矩阵中的非零元素与查询半径的比值为M min(i，j)/R，计算机设备通过1-M min(i，j)/R将距离矩阵归一化在(0，1)区间。
在一个实施例中，如图6所示，计算机设备计算所述邻域点云和预构建的核点云之间的豪斯多夫距离之前，还包括如下步骤：
S602，计算机设备创建球形区域，并在所述球形区域内构建三维参数模型。
S604，计算机设备通过最远点算法对所述三维参数模型进行采样，构建出所述核点云。
其中，三维参数模型是表示几何形状的模型。例如可以是表示几何形状的函数。几何形状可以是点、线、面、体等几何形状，例如，线为直线，面为平面，体为球体。所以，三维参数模型可以是表示点、线、面、体的函数。这些几何形状可以很好的嵌入三维邻域空间，并且保持形状上的对称性，并且球体还满足旋转不变性。其中，三维参数模型不限于是表示点、线、面、体的模型，也可以是表示其它几何形状的模型。
计算机设备可以预先设置通过最远点算法采样的样本个数，并根据从三维参数模型中采样所得的样本点构建出核点云。例如，如图7所示，从左至右分别是点、线、面、体形 状的核点云。计算机设备所构建的核点云的大小为归一化尺度，在进行卷积计算时可以随着查询半径的变化而变化，与查询半保持一致。
在一个实施例中，编码器中的豪斯多夫卷积层包括多核豪斯多夫卷积子层，多核豪斯多夫卷积子层中的每个子层对应不同的距离矩阵；通过编码器中的豪斯多夫卷积层对邻域点云、距离矩阵和网络权重矩阵进行卷积计算，得到高维点云特征包括：通过多核豪斯多夫卷积子层中的各子层对邻域点云、网络权重矩阵和各子层对应的距离矩阵进行卷积计算，得到各子层的点云特征；将各子层的点云特征进行融合，得到高维点云特征。
其中，编码器中的豪斯多夫卷积层包括多个多核豪斯多夫卷积子层，每个多核豪斯多夫卷积子层对应于不同的核点云，也即该卷积层根据邻域点云和核点云计算得到的距离矩阵不相同。各个多核豪斯多夫卷积子层计算得到对应的点云特征后，编码器对各子层的点云特征进行融合，得到高维点云特征。其中，对点云特征进行融合可以是对点云特征进行相加或者加权相加，或者在对点云特征进行处理后进行融合，处理的过程可以是线性或者非线性处理。
在一个实施例中，如图8所示，豪斯多夫卷积层包括n k个多核豪斯多夫卷积子层，多核豪斯多夫卷积子层进行卷积计算后得到对应的卷积响应向量C out，计算机设备根据公式
Figure PCTCN2021082250-appb-000007
首先通过激活函数RELU对各卷积响应向量进行处理，然后对处理后的卷积向量进行累加，得到累加和向量，再根据激活函数对累加和向量进行处理，得到融合的特征C out。其中，激活函数RELU可以是非线性函数，通过激活函数对卷积响应向量进行处理，可以增加单个卷积响应向量的非线性成分。计算机设备通过非线性激活函数对
Figure PCTCN2021082250-appb-000008
进行处理，以对累加的RELU(C out)进行非线性映射，从而构建出融合的特征。
在一个实施例中，计算机设备对多核豪斯多夫卷积子层输出的卷积响应向量进行拼接，通过拼接对卷积响应向量进行融合。在另一个实施例中，计算机设备通过多层感知器将多个多核豪斯多夫卷积子层输出的卷积响应向量映射为一维数据，以实现对卷积响应向量的融合。
由于实时稀疏点云数据缺少几何信息，计算机设备通过豪斯多夫距离计算得到距离矩阵，可以通过显式几何比较的方法，提取邻域点云与核点云之间的相似度，为稀疏点云数据带来几何信息的补充，增加了三维点云卷积方法的可解释性。豪斯多夫距离采用了双向度量的方式，分别从邻域点云以及核点云的角度进行点云的相似度比对，并最后综合两者的比对结果，得到距离矩阵。
在一个实施例中，点云数据分别为S3DIS数据集和SemanticKITTI数据集，深度学习神 经网络中包括五层豪斯多夫卷积层，每层的查询半径为上一层的两倍。如图9a所示为S3DIS数据集的场景标签真值(ground truth)，图9b所示为通过深度学习神经网络得到的对S3DIS数据集进行语义分类的结果。如图9c所示为SemanticKITTI数据集的场景标签真值(ground truth)，图9d所示为通过深度学习神经网络得到的对SemanticKITTI数据集进行语义分类的结果。其中，具有相同语义的目标用统一的颜色表示，相同语义目标在两幅图片中应具有相同颜色。从图9a至9d可以看出，深度学习神经网络可以较好地区分场景中的主要语义目标，并保持主要语义目标的分割完整性和精确性。此外，深度学习神经网络对于稀疏数据或稠密数据都较为有效。深度学习神经网络可以根据物体的几何形状将语义目标进行完整的语义分类，充分显示了深度学习神经网络的学习能力。
在一个实施例中，点云数据为S3DIS数据集，计算机设备采用序列1-4，6作为训练集，序列5作为测试集，采用三维球形核点云作为单核豪斯多夫卷积的核点云，采用mIOU(mean of Intersection Over Union)作为衡量深度学习神经网络语义分类结果准确度的指标，在深度学习神经网络的豪斯多夫卷积层为单核豪斯多夫卷积层时，语义分类结果的mIOU为66.7％。在深度学习神经网络的豪斯多夫卷积层为多核豪斯多夫卷积层时，语义分类结果的mIOU为68.2％。
在一个实施例中，点云数据为SemanticKITTI数据集，计算机设备采用序列0-7，9-10作为训练集，序列8作为测试集，在深度学习神经网络的豪斯多夫卷积层为单核豪斯多夫卷积层时，语义分类结果的mIOU为59.6％。在深度学习神经网络的豪斯多夫卷积层为多核豪斯多夫卷积层时，语义分类结果的mIOU为60.3％。由于SemanticKITTI数据集是对户外场景进行扫描形成的数据集，所以深度学习神经网络的语义分类结果证明包括豪斯多夫卷积的深度学习神经网络对于户外单帧点云数据的分类有效。
应该理解的是，虽然图2、5-6的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，图2、5-6中的至少一部分步骤可以包括多个步骤或者多个阶段，这些步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤中的步骤或者阶段的至少一部分轮流或者交替地执行。
在一个实施例中，如图10所示，提供了一种点云数据处理装置，包括：获取模块、距离计算模块、卷积计算模块和特征降维模块，其中：
获取模块1002，用于获取点云数据，并对点云数据中的各数据点构建对应的邻域点云；
距离计算模块1004，用于计算邻域点云和预构建的核点云之间的豪斯多夫距离，得到距离矩阵；
卷积计算模块1006，用于通过编码器中的豪斯多夫卷积层对邻域点云、距离矩阵和网络权重矩阵进行卷积计算，得到高维点云特征；编码器和解码器是深度学习网络中的两个部分；以及
特征降维模块1008，用于通过解码器对高维点云特征进行特征降维，以使分类器根据降维所得的目标点云特征对点云数据进行语义分类。
上述实施例中，计算机设备获取点云数据，并对点云数据中的各数据点构建对应的邻域点云，然后计算邻域点云和预构建的核点云之间的豪斯多夫距离，得到距离矩阵。计算机设备通过编码器中的豪斯多夫卷积层对邻域点云、距离矩阵和网络权重矩阵进行卷积计算，得到高维点云特征，并通过解码器对高维点云特征进行特征降维，以使分类器根据降维所得的目标点云特征对点云数据进行语义分类。计算机设备通过计算豪斯多夫距离得到距离矩阵，并将距离矩阵和邻域点云进行卷积计算，然后再用网络权重矩阵对卷积计算的结果进行加权，得到最终的高维点云特征，从而实现对点云数据的特征提取，所提取的高维点云特征具有较高的精度，所以，计算机设备在所得到的高维点云特征的基础上进行的语义分类的结果具有较高的准确性。
在一个实施例中，编码器中豪斯多夫卷积层的数量不少于两个；卷积计算模块1006，还用于：
在进行卷积计算的过程中，针对首层豪斯多夫卷积层，将邻域点云作为输入特征与距离矩阵和网络权重矩阵进行卷积计算，得到输出特征；
针对非首层豪斯多夫卷积层，将当前非首层豪斯多夫卷积层的上一层豪斯多夫卷积层的输出特征作为本层输入特征，与距离矩阵和网络权重矩阵进行卷积计算，得到输出特征；若当前非首层豪斯多夫卷积层为末层豪斯多夫卷积层时，将末层豪斯多夫卷积层的输出特 征作为高维点云特征。
在一个实施例中，获取模块1002，还用于：
从点云数据的数据点中选取查询点；
在点云数据的数据点中，提取与查询点的距离小于查询半径的目标数据点；以及
根据目标数据点构建查询点对应的邻域点云。
在一个实施例中，豪斯多夫距离包括第一最短距离与第二最短距离；距离计算模块1004，还用于：
分别计算邻域点云中每个数据点与核点云间的第一最短距离，并由各第一最短距离组成第一最短距离集合；
分别计算核点云中每个数据点与邻域点云间的第二最短距离，并由各第二最短距离组成第二最短距离集合；以及
根据第一最短距离集合以及第二最短距离集合计算得到距离矩阵。
在一个实施例中，如图11所示，装置还包括：
创建模块1010，用于创建球形区域，并在球形区域内构建三维参数模型；以及
采样模块1012，用于通过最远点算法对三维参数模型进行采样，构建出核点云。
在一个实施例中，装置还包括：
比值计算模块1014，用于计算距离矩阵的非零元素与查询半径的比值；以及
获得模块1016，用于根据比值计算得到归一化的距离矩阵；
卷积计算模块1006，还用于：
通过编码器中的豪斯多夫卷积层对邻域点云、归一化的距离矩阵和网络权重矩阵进行卷积计算。
在一个实施例中，编码器中的豪斯多夫卷积层包括多核豪斯多夫卷积子层，多核豪斯多夫卷积子层中的每个子层对应不同的距离矩阵；卷积计算模块1006，还用于：
通过多核豪斯多夫卷积子层中的各子层对邻域点云、网络权重矩阵和各子层对应的距离矩阵进行卷积计算，得到各子层的点云特征；以及
将各子层的点云特征进行融合，得到高维点云特征。
关于点云数据处理装置的具体限定可以参见上文中对于点云数据处理方法的限定，在此不再赘述。上述点云数据处理装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中，也可以以软件形式存储于计算机设备中的存储器中，以便于处理器调用执行以上各个模块对应的操作。
在一个实施例中，提供了一种计算机设备，该计算机设备可以是服务器，其内部结构图可以如图12所示。该计算机设备包括通过***总线连接的处理器、存储器和网络接口。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作***、计算机程序和数据库。该内存储器为非易失性存储介质中的操作***和计算机程序的运行提供环境。该计算机设备的数据库用于存储点云数据处理数据。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机程序被处理器执行时以实现一种点云数据处理方法。
在一个实施例中，提供了一种计算机设备，该计算机设备可以是终端，其内部结构图可以如图13所示。该计算机设备包括通过***总线连接的处理器、存储器、通信接口、显示屏和输入装置。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作***和计算机程序。该内存储器为非易失性存储介质中的操作***和计算机程序的运行提供环境。该计算机设备的通信接口用于与外部的终端进行有线或无线方式的通信，无线方式可通过WIFI、运营商网络、NFC(近场通信)或其他技术实现。该计算机程序被处理器执行时以实现一种点云数据处理方法。该计算机设备的显示屏可以是液晶显示屏或者电子墨水显示屏，该计算机设备的输入装置可以是显示屏上覆盖的触摸层，也可以是计算机设备外壳上设置的按键、轨迹球或触控板，还可以是外接的键盘、触控板或鼠标等。
本领域技术人员可以理解，图12、13中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
在一个实施例中，提供了一种计算机设备，包括存储器和处理器，存储器中存储有计 算机程序，该处理器执行计算机程序时实现以下步骤：获取点云数据，并对点云数据中的各数据点构建对应的邻域点云；计算邻域点云和预构建的核点云之间的豪斯多夫距离，得到距离矩阵；通过编码器中的豪斯多夫卷积层对邻域点云、距离矩阵和网络权重矩阵进行卷积计算，得到高维点云特征；编码器和解码器是深度学习网络中的两个部分；通过解码器对高维点云特征进行特征降维，以使分类器根据降维所得的目标点云特征对点云数据进行语义分类。
在一个实施例中，编码器中豪斯多夫卷积层的数量不少于两个；处理器执行计算机程序时还实现以下步骤：在进行卷积计算的过程中，针对首层豪斯多夫卷积层，将邻域点云作为输入特征与距离矩阵和网络权重矩阵进行卷积计算，得到输出特征；针对非首层豪斯多夫卷积层，将当前非首层豪斯多夫卷积层的上一层豪斯多夫卷积层的输出特征作为本层输入特征，与距离矩阵和网络权重矩阵进行卷积计算，得到输出特征；若当前非首层豪斯多夫卷积层为末层豪斯多夫卷积层时，将末层豪斯多夫卷积层的输出特征作为高维点云特征。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：从点云数据的数据点中选取查询点；在点云数据的数据点中，提取与查询点的距离小于查询半径的目标数据点；根据目标数据点构建查询点对应的邻域点云。
在一个实施例中，豪斯多夫距离包括第一最短距离与第二最短距离；处理器执行计算机程序时还实现以下步骤：分别计算邻域点云中每个数据点与核点云间的第一最短距离，并由各第一最短距离组成第一最短距离集合；分别计算核点云中每个数据点与邻域点云间的第二最短距离，并由各第二最短距离组成第二最短距离集合；根据第一最短距离集合以及第二最短距离集合计算得到距离矩阵。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：创建球形区域，并在球形区域内构建三维参数模型；通过最远点算法对三维参数模型进行采样，构建出核点云。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：计算距离矩阵的非零元素与查询半径的比值；根据比值计算得到归一化的距离矩阵；通过编码器中的豪斯多夫卷积层对邻域点云、归一化的距离矩阵和网络权重矩阵进行卷积计算。
在一个实施例中，编码器中的豪斯多夫卷积层包括多核豪斯多夫卷积子层，多核豪斯 多夫卷积子层中的每个子层对应不同的距离矩阵；处理器执行计算机程序时还实现以下步骤：通过多核豪斯多夫卷积子层中的各子层对邻域点云、网络权重矩阵和各子层对应的距离矩阵进行卷积计算，得到各子层的点云特征；将各子层的点云特征进行融合，得到高维点云特征。
在一个实施例中，提供了一种计算机可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现以下步骤：获取点云数据，并对点云数据中的各数据点构建对应的邻域点云；计算邻域点云和预构建的核点云之间的豪斯多夫距离，得到距离矩阵；通过编码器中的豪斯多夫卷积层对邻域点云、距离矩阵和网络权重矩阵进行卷积计算，得到高维点云特征；编码器和解码器是深度学习网络中的两个部分；通过解码器对高维点云特征进行特征降维，以使分类器根据降维所得的目标点云特征对点云数据进行语义分类。
在一个实施例中，编码器中豪斯多夫卷积层的数量不少于两个；计算机程序被处理器执行时还实现以下步骤：在进行卷积计算的过程中，针对首层豪斯多夫卷积层，将邻域点云作为输入特征与距离矩阵和网络权重矩阵进行卷积计算，得到输出特征；针对非首层豪斯多夫卷积层，将当前非首层豪斯多夫卷积层的上一层豪斯多夫卷积层的输出特征作为本层输入特征，与距离矩阵和网络权重矩阵进行卷积计算，得到输出特征；若当前非首层豪斯多夫卷积层为末层豪斯多夫卷积层时，将末层豪斯多夫卷积层的输出特征作为高维点云特征。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：从点云数据的数据点中选取查询点；在点云数据的数据点中，提取与查询点的距离小于查询半径的目标数据点；根据目标数据点构建查询点对应的邻域点云。
在一个实施例中，豪斯多夫距离包括第一最短距离与第二最短距离；计算机程序被处理器执行时还实现以下步骤：分别计算邻域点云中每个数据点与核点云间的第一最短距离，并由各第一最短距离组成第一最短距离集合；分别计算核点云中每个数据点与邻域点云间的第二最短距离，并由各第二最短距离组成第二最短距离集合；根据第一最短距离集合以及第二最短距离集合计算得到距离矩阵。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：创建球形区域，并在球形区域内构建三维参数模型；通过最远点算法对三维参数模型进行采样，构建出核点云。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：计算距离矩阵的非零元素与查询半径的比值；根据比值计算得到归一化的距离矩阵；通过编码器中的豪斯多夫卷积层对邻域点云、归一化的距离矩阵和网络权重矩阵进行卷积计算。
在一个实施例中，编码器中的豪斯多夫卷积层包括多核豪斯多夫卷积子层，多核豪斯多夫卷积子层中的每个子层对应不同的距离矩阵；计算机程序被处理器执行时还实现以下步骤：通过多核豪斯多夫卷积子层中的各子层对邻域点云、网络权重矩阵和各子层对应的距离矩阵进行卷积计算，得到各子层的点云特征；将各子层的点云特征进行融合，得到高维点云特征。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一非易失性计算机可读取存储介质中，该计算机程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和易失性存储器中的至少一种。非易失性存储器可包括只读存储器(Read-Only Memory,ROM)、磁带、软盘、闪存或光存储器等。易失性存储器可包括随机存取存储器(Random Access Memory,RAM)或外部高速缓冲存储器。作为说明而非局限，RAM可以是多种形式，比如静态随机存取存储器(Static Random Access Memory,SRAM)或动态随机存取存储器(Dynamic Random Access Memory,DRAM)等。
以上实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。

Claims (14)

  1. 一种点云数据处理方法，所述方法包括：
    获取点云数据，并对所述点云数据中的各数据点构建对应的邻域点云；
    计算所述邻域点云和预构建的核点云之间的豪斯多夫距离，得到距离矩阵；
    通过编码器中的豪斯多夫卷积层对所述邻域点云、所述距离矩阵和网络权重矩阵进行卷积计算，得到高维点云特征；所述编码器和解码器是深度学习网络中的两个部分；以及
    通过所述解码器对所述高维点云特征进行特征降维，以使分类器根据降维所得的目标点云特征对所述点云数据进行语义分类。
  2. 根据权利要求1所述的方法，其中所述编码器中豪斯多夫卷积层的数量不少于两个；所述通过编码器中的豪斯多夫卷积层对所述邻域点云、所述距离矩阵和网络权重矩阵进行卷积计算，得到高维点云特征包括：
    在进行卷积计算的过程中，针对首层豪斯多夫卷积层，将所述邻域点云作为输入特征与所述距离矩阵和网络权重矩阵进行卷积计算，得到输出特征；以及
    针对非首层豪斯多夫卷积层，将当前非首层豪斯多夫卷积层的上一层豪斯多夫卷积层的输出特征作为本层输入特征，与所述距离矩阵和网络权重矩阵进行卷积计算，得到输出特征；若所述当前非首层豪斯多夫卷积层为末层豪斯多夫卷积层时，将所述末层豪斯多夫卷积层的输出特征作为所述高维点云特征。
  3. 根据权利要求1所述的方法，其中所述对所述点云数据中的各数据点构建对应的邻域点云包括：
    从所述点云数据的数据点中选取查询点；
    在所述点云数据的数据点中，提取与所述查询点的距离小于查询半径的目标数据点；以及
    根据所述目标数据点构建所述查询点对应的邻域点云。
  4. 根据权利要求1所述的方法，其中所述豪斯多夫距离包括第一最短距离与第二最短距离；所述计算所述邻域点云和预构建的核点云之间的豪斯多夫距离，得到距离矩阵包括：
    分别计算所述邻域点云中每个数据点与所述核点云间的第一最短距离，并由各所述第 一最短距离组成第一最短距离集合；
    分别计算所述核点云中每个数据点与所述邻域点云间的第二最短距离，并由各所述第二最短距离组成第二最短距离集合；以及
    根据所述第一最短距离集合以及所述第二最短距离集合计算得到所述距离矩阵。
  5. 根据权利要求1所述的方法，其中所述计算所述邻域点云和预构建的核点云之间的豪斯多夫距离之前，所述方法还包括：
    创建球形区域，并在所述球形区域内构建三维参数模型；以及
    通过最远点算法对所述三维参数模型进行采样，构建出所述核点云。
  6. 根据权利要求1所述的方法，其中所述计算所述邻域点云和预构建的核点云之间的豪斯多夫距离，得到距离矩阵之后，所述方法还包括：
    计算所述距离矩阵的非零元素与查询半径的比值；
    根据所述比值计算得到归一化的距离矩阵；并且
    所述通过编码器中的豪斯多夫卷积层对所述邻域点云、所述距离矩阵和网络权重矩阵进行卷积计算包括：
    通过编码器中的豪斯多夫卷积层对所述邻域点云、所述归一化的距离矩阵和网络权重矩阵进行卷积计算。
  7. 根据权利要求1所述的方法，其中所述编码器中的豪斯多夫卷积层包括多核豪斯多夫卷积子层，所述多核豪斯多夫卷积子层中的每个子层对应不同的所述距离矩阵；所述通过编码器中的豪斯多夫卷积层对所述邻域点云、所述距离矩阵和网络权重矩阵进行卷积计算，得到高维点云特征包括：
    通过所述多核豪斯多夫卷积子层中的各子层对所述邻域点云、网络权重矩阵和各所述子层对应的距离矩阵进行卷积计算，得到各所述子层的点云特征；以及
    将各所述子层的点云特征进行融合，得到高维点云特征。
  8. 根据权利要求1所述的方法，其中所述编码器中的每个豪斯多夫卷积层包括多个豪斯多夫卷积模块，每个豪斯多夫卷积模块具有不同的查询尺度。
  9. 根据权利要求1所述的方法，其中所述点云数据是三维点云数据。
  10. 根据权利要求1所述的方法，其中所述点云数据是无人驾驶汽车上的激光雷达对 道路环境中的物体进行扫描所获取的数据。
  11. 根据权利要求1所述的方法，其中所述点云数据是人工智能机器人上的深度相机对空间环境中物体进行扫描所获取的数据。
  12. 一种点云数据处理装置，其中所述装置包括：
    获取模块，用于获取点云数据，并对所述点云数据中的各数据点构建对应的邻域点云；
    距离计算模块，用于计算所述邻域点云和预构建的核点云之间的豪斯多夫距离，得到距离矩阵；
    卷积计算模块，用于通过编码器中的豪斯多夫卷积层对所述邻域点云、所述距离矩阵和网络权重矩阵进行卷积计算，得到高维点云特征；所述编码器和解码器是深度学习网络中的两个部分；以及
    特征降维模块，用于通过所述解码器对所述高维点云特征进行特征降维，以使分类器根据降维所得的目标点云特征对所述点云数据进行语义分类。
  13. 一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，所述处理器执行所述计算机程序时实现权利要求1至11中任一项所述的方法的步骤。
  14. 一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现权利要求1至11中任一项所述的方法的步骤。
PCT/CN2021/082250 2021-03-15 2021-03-23 点云数据处理方法、装置、计算机设备和存储介质 WO2022193335A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/735,888 US11875424B2 (en) 2021-03-15 2022-05-03 Point cloud data processing method and device, computer device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110274439.3 2021-03-15
CN202110274439.3A CN112990010B (zh) 2021-03-15 2021-03-15 点云数据处理方法、装置、计算机设备和存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/735,888 Continuation US11875424B2 (en) 2021-03-15 2022-05-03 Point cloud data processing method and device, computer device, and storage medium

Publications (1)

Publication Number Publication Date
WO2022193335A1 true WO2022193335A1 (zh) 2022-09-22

Family

ID=76335482

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/082250 WO2022193335A1 (zh) 2021-03-15 2021-03-23 点云数据处理方法、装置、计算机设备和存储介质

Country Status (2)

Country Link
CN (1) CN112990010B (zh)
WO (1) WO2022193335A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116416586A (zh) * 2022-12-19 2023-07-11 香港中文大学（深圳） 基于rgb点云的地图元素感知方法、终端及存储介质
CN117808703A (zh) * 2024-02-29 2024-04-02 南京航空航天大学 一种多尺度大型部件装配间隙点云滤波方法

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113537390B (zh) * 2021-08-06 2023-10-10 国网新源水电有限公司富春江水力发电厂 一种基于图卷积网络的动态点云位置预测方法
CN114359089B (zh) * 2021-12-29 2022-09-27 江苏苏彩信息技术有限公司 一种基于点云滤波网络的三维点云数据去噪方法
CN114048845B (zh) * 2022-01-14 2022-06-03 深圳大学 点云修复方法、装置、计算机设备和存储介质
CN114596447A (zh) * 2022-02-17 2022-06-07 成都飞机工业（集团）有限责任公司 一种点云下采样方法、装置、设备及存储介质
CN114972695B (zh) * 2022-05-20 2024-03-15 北京大学深圳研究生院 一种点云生成方法、装置、电子设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109118564A (zh) * 2018-08-01 2019-01-01 湖南拓视觉信息技术有限公司 一种基于融合体素的三维点云标记方法和装置
CN109859256A (zh) * 2019-03-13 2019-06-07 大连理工大学 一种基于自动对应点匹配的三维点云配准方法
CN110321910A (zh) * 2018-03-29 2019-10-11 中国科学院深圳先进技术研究院 面向点云的特征提取方法、装置及设备
CN111028327A (zh) * 2019-12-10 2020-04-17 深圳先进技术研究院 一种三维点云的处理方法、装置及设备
CN112215231A (zh) * 2020-09-29 2021-01-12 浙江工业大学 一种结合空间深度卷积和残差结构的大尺度点云语义分割方法
WO2021009258A1 (en) * 2019-07-15 2021-01-21 Promaton Holding B.V. Object detection and instance segmentation of 3d point clouds based on deep learning

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112101278A (zh) * 2020-09-25 2020-12-18 湖南盛鼎科技发展有限责任公司 基于k近邻特征提取和深度学习的宅基地点云分类方法
CN112435343B (zh) * 2020-11-24 2024-07-16 杭州荷尖科技合伙企业(有限合伙) 点云数据处理方法和装置、电子设备及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110321910A (zh) * 2018-03-29 2019-10-11 中国科学院深圳先进技术研究院 面向点云的特征提取方法、装置及设备
CN109118564A (zh) * 2018-08-01 2019-01-01 湖南拓视觉信息技术有限公司 一种基于融合体素的三维点云标记方法和装置
CN109859256A (zh) * 2019-03-13 2019-06-07 大连理工大学 一种基于自动对应点匹配的三维点云配准方法
WO2021009258A1 (en) * 2019-07-15 2021-01-21 Promaton Holding B.V. Object detection and instance segmentation of 3d point clouds based on deep learning
CN111028327A (zh) * 2019-12-10 2020-04-17 深圳先进技术研究院 一种三维点云的处理方法、装置及设备
CN112215231A (zh) * 2020-09-29 2021-01-12 浙江工业大学 一种结合空间深度卷积和残差结构的大尺度点云语义分割方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"Chinese Master Thesis Tsinghua University, Beijing", 15 March 2017, TSINGHUA UNIVERSITY, BEIJING, CN, article WANG QINGCAN: "Research on Registration Algorithm of 3D Point Cloud Data)", pages: 1 - 73, XP055968239 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116416586A (zh) * 2022-12-19 2023-07-11 香港中文大学（深圳） 基于rgb点云的地图元素感知方法、终端及存储介质
CN116416586B (zh) * 2022-12-19 2024-04-02 香港中文大学（深圳） 基于rgb点云的地图元素感知方法、终端及存储介质
CN117808703A (zh) * 2024-02-29 2024-04-02 南京航空航天大学 一种多尺度大型部件装配间隙点云滤波方法
CN117808703B (zh) * 2024-02-29 2024-05-10 南京航空航天大学 一种多尺度大型部件装配间隙点云滤波方法

Also Published As

Publication number Publication date
CN112990010B (zh) 2023-08-18
CN112990010A (zh) 2021-06-18

Similar Documents

Publication Publication Date Title
WO2022193335A1 (zh) 点云数据处理方法、装置、计算机设备和存储介质
US11875424B2 (en) Point cloud data processing method and device, computer device, and storage medium
CN111627065B (zh) 一种视觉定位方法及装置、存储介质
CN111831844A (zh) 图像检索方法、图像检索装置、图像检索设备及介质
CN112529068B (zh) 一种多视图图像分类方法、***、计算机设备和存储介质
CN114565916B (zh) 目标检测模型训练方法、目标检测方法以及电子设备
US11682166B2 (en) Fitting 3D primitives to a high-resolution point cloud
TWI812888B (zh) 影像辨識方法及影像辨識系統
Ahmad et al. 3D capsule networks for object classification from 3D model data
CN114022858A (zh) 一种针对自动驾驶的语义分割方法、***、电子设备及介质
Qin et al. Depth estimation by parameter transfer with a lightweight model for single still images
US20240005541A1 (en) Image depth prediction method and electronic device
CN114612572A (zh) 一种基于深度学习的激光雷达与相机外参标定方法及装置
CN113139540B (zh) 背板检测方法及设备
Liang et al. Multimodal information fusion based on LSTM for 3D model retrieval
CN111414802B (zh) 蛋白质数据特征提取方法
CN114998604A (zh) 一种基于局部点云位置关系的点云特征提取方法
CN115544190A (zh) 语义地图更新方法、装置、计算机设备和存储介质
Li et al. Accurate visual localization with semantic masking and attention
Wang Motion recognition based on deep learning and human joint points
CN114821013B (zh) 基于点云数据的基元检测方法、装置和计算机设备
WO2023241372A1 (zh) 相机内参标定方法及相关设备
US20240061980A1 (en) Machine-learning for topologically-aware cad retrieval
CN116612287B (zh) 图像识别方法、装置、计算机设备和存储介质
CN116894802B (zh) 图像增强方法、装置、计算机设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21930924

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 16.01.2024)