CN114241110B - Point cloud semantic uncertainty sensing method based on neighborhood aggregation Monte Carlo inactivation - Google Patents
Point cloud semantic uncertainty sensing method based on neighborhood aggregation Monte Carlo inactivation Download PDFInfo
- Publication number
- CN114241110B CN114241110B CN202210165979.2A CN202210165979A CN114241110B CN 114241110 B CN114241110 B CN 114241110B CN 202210165979 A CN202210165979 A CN 202210165979A CN 114241110 B CN114241110 B CN 114241110B
- Authority
- CN
- China
- Prior art keywords
- point cloud
- uncertainty
- random
- points
- dropout
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/04—Inference or reasoning models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Graphics (AREA)
- Software Systems (AREA)
- Geometry (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a point cloud semantic uncertainty perception method based on neighborhood aggregation Monte Carlo inactivation, which comprises the steps of obtaining an original point cloud of a scene to be processed; the method comprises the steps that original point cloud is used as input, a PointNet (++) is used as a basic model to construct an NSA-MC-dropout framework, wherein the NSA-MC-dropout framework generates feature vectors with different granularities in a coding stage, the feature vectors are spliced to corresponding space points in a decoding stage, and random reasoning results of all disordered points are generated after reasoning is carried out by a multilayer perceptron with space sampling; the prediction distribution of each disordered point in the original point cloud is established by fusing the random reasoning results to realize single random reasoning; point cloud semantic uncertainty is quantified by capturing the amount of information contained in a predicted distribution or by modeling the degree of dispersion of the predicted distribution. On the basis of not increasing model parameters and reasoning times, the frame of semantic segmentation of the uncertainty perception point cloud is realized.
Description
Technical Field
The invention belongs to the field of three-dimensional visual pattern recognition.
Background
The robot captures, plans a path, automatically drives and other scenes, three-dimensional modeling is needed to be carried out on surrounding scenes by using a three-dimensional laser scanning device, point cloud data (environmental structure topological data formed by massive discrete points) are formed, and point cloud semantic segmentation (recognizing semantic labels of all points in the point cloud) is the basis for further decision making of the robot. However, the semantic segmentation technology at the present stage is difficult to accurately identify the label of each point in the point cloud. False identifications pose a risk to machine decisions. Therefore, the confidence degree of the prediction result needs to be output by adopting a point cloud semantic segmentation technology with uncertainty perception, and the decision machine can conveniently adopt the point cloud semantic segmentation technology.
The uncertainty-aware point cloud semantic segmentation is focused on two key tasks: uncertainty assessment of the segmentation results and uncertainty of the segmentation model guide learning. Uncertainty assessment helps the operator to understand how reliable the segmentation model gives the prediction, which is important for decision-making applications. The uncertainty guide model optimizes and utilizes uncertainty to realize better convergence of the model, thereby realizing the improvement of the segmentation performance of the model.
The key of the uncertainty assessment task is to model the predicted distribution of each target and establish different types of uncertainty by quantifying the dispersion degree of the distribution. The traditional target prediction distribution establishment method is based on a Bayesian model to learn the Gaussian posterior probability on the model weight, so that the prediction distribution of each target is established. The Monte Carlo inactivation (MC-dropout) method is proved to be used for approximating the Bayes learning process, the uncertainty of the target is established by summarizing the results of multiple random inferences, and the method is convenient to use, depends on multiple forward propagation of models and is poor in timeliness.
In uncertainty-guided learning, different optimizations are achieved by guiding the model through the use of uncertainty components (accidental uncertainty and cognitive uncertainty). The accidental uncertainty component reflects the inherent noise in the data set, some work improves the robustness of the model by introducing the component into a loss function, and a lower training weight is distributed to a target with higher accidental uncertainty in the loss function for punishing the inherent noise so as to obtain better model performance. In tasks such as semi-supervised learning, cognitive uncertainty is utilized to enhance the cognitive performance of the model on data. However, whatever uncertainty component is used for model optimization, it must be built from the predicted distributions. However, the traditional method for establishing the prediction distribution depends on the introduction of MC-dropout or an additional variance parameter, and the low-efficiency method is difficult to popularize.
In summary, the existing uncertainty assessment and uncertainty guided learning methods have the problem that the establishment of the target prediction distribution is time-consuming. For mass data containing millions or even tens of millions of points, such as point clouds, the traditional prediction distribution establishing method is difficult to apply.
Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the related art.
Therefore, the first purpose of the invention is to provide a point cloud semantic uncertainty perception method based on neighborhood aggregation Monte Carlo inactivation, which is used for realizing a framework of uncertainty perception point cloud semantic segmentation.
The invention also provides a point cloud semantic uncertainty sensing device based on neighborhood aggregation Monte Carlo inactivation.
A third object of the present invention is to provide a computer device, which is characterized by comprising a memory, a processor and a computer program stored in the memory and capable of running on the processor.
A fourth object of the invention is to propose a non-transitory computer-readable storage medium having a computer program stored thereon.
In order to achieve the above object, an embodiment of a first aspect of the present invention provides a point cloud semantic uncertainty sensing method based on neighborhood aggregation monte carlo deactivation, including: acquiring an original point cloud of a scene to be processed, wherein the original point cloud is composed of unordered points describing the geometric structure of the scene to be processed; the method comprises the steps that original point cloud is used as input, a PointNet (++) is used as a basic model to construct an NSA-MC-dropout framework, wherein the NSA-MC-dropout framework generates feature vectors with different granularities in a coding stage, the feature vectors are spliced to corresponding space points in a decoding stage, and random reasoning results of all disordered points are generated after reasoning is carried out by a multilayer perceptron with space sampling; the prediction distribution of each disordered point in the original point cloud is established by fusing the random reasoning results to realize single random reasoning; point cloud semantic uncertainty is quantified by capturing the amount of information contained in a predicted distribution or by modeling the degree of dispersion of the predicted distribution. On the basis of not increasing model parameters and reasoning times, the frame of semantic segmentation of the uncertainty perception point cloud is realized.
The point cloud semantic uncertainty perception method based on neighborhood aggregation Monte Carlo inactivation provided by the embodiment of the invention provides an efficient uncertainty perception point cloud semantic segmentation framework, improves segmentation performance by using uncertainty, and quantifies credibility of a prediction result, which is the first framework for realizing uncertainty perception point cloud semantic segmentation on the basis of not increasing model parameters and reasoning times. The NSA-MC-Dropout not only focuses on the efficient assessment of uncertainty, but also uses the brand-new method for uncertainty-guided learning, can realize efficient uncertainty quantification and punishment on inherent noise in the learning process, and can realize good model convergence.
In addition, the point cloud semantic uncertainty perception method based on neighborhood aggregation monte carlo inactivation according to the above embodiment of the present invention may further have the following additional technical features:
further, in one embodiment of the invention, the model sampling is accomplished by random inactivation factors after part of the trainable layer of the NSA-MC-dropout framework.
Further, in one embodiment of the present invention, the multi-layered perceptron includes a fully connected weight-sharing layer, and FC layers used to reason about different target points may exhibit different random weights by random inactivation factors at the weight-sharing layer, thereby achieving spatial sampling in a single random forward direction.
Further, in an embodiment of the present invention, the establishing prediction distribution of each disordered point in the original point cloud by single stochastic reasoning through fusing stochastic reasoning results includes:
for a set of contained target points
And it
Set of points of a neighborhood of points
Can be written as:
wherein, the point set
Probability output of each point in
Probability output of T times which can be similar to target point established by MC dropout
The distribution mean obtained by summarizing the T probability outputs can be expressed as:
if used in
Stochastic inference weights of neighbors
And establishing the model in the random forward propagation of the primary model, and realizing the establishment of the predictive distribution of each point in the point cloud through one-time reasoning.
Further, in an embodiment of the present invention, after obtaining the prediction distribution of each unordered point, the method further includes:
and (3) representing all possible predicted values by the prediction distribution, quantitatively representing noise points and non-noise points in a unified mode, and distributing lower weight to the noise points in an uncertainty guidance loss function to realize stronger robustness of the NSA-MC-dropout framework model to noise:
wherein N and M represent the number of spatial points and the number of categories, respectively,
the representation is based on a casual uncertainty,
a predicted variance of the characterization; when the predicted value is true
Otherwise, the value is 0; the first term of the uncertainty guiding loss function is used for suppressing noise points;
the predictor for the regularization term to prevent accidental uncertainty is infinite.
In order to achieve the above object, an embodiment of a second aspect of the present invention provides a point cloud semantic uncertainty sensing apparatus based on neighborhood aggregation monte carlo deactivation, including: the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring an original point cloud of a scene to be processed, and the original point cloud is composed of unordered points describing the geometric structure of the scene to be processed; the construction module is used for taking the original point cloud as input and constructing an NSA-MC-dropout framework by taking PointNet (++) as a basic model, wherein the NSA-MC-dropout framework generates feature vectors with different granularities in a coding stage, the feature vectors are spliced to corresponding space points in a decoding stage, and random reasoning results of all disordered points are generated after reasoning by using a multilayer perceptron with space sampling; the output module is used for realizing single random inference by fusing random inference results to establish prediction distribution of each disordered point in the original point cloud; a perception module to quantify a point cloud semantic uncertainty by capturing an amount of information contained in a predicted distribution or by modeling a degree of dispersion of the predicted distribution.
Further, in an embodiment of the present invention, the building module is further configured to implement model sampling by using random inactivation factors after part of the trainable layer of the NSA-MC-dropout framework.
Further, in an embodiment of the present invention, the building module is further configured to, the multilayer perceptron includes a weight sharing layer that is fully connected, and the FC layer for reasoning different target points may present different random weights by using a random inactivation factor at the weight sharing layer, so as to implement spatial sampling in a single random forward direction.
To achieve the above object, a third aspect of the present invention provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements a point cloud semantic uncertainty perception method based on neighborhood aggregation monte carlo deactivation when executing the computer program.
To achieve the above object, a non-transitory computer-readable storage medium is provided in a fourth embodiment of the present invention, and when the processor executes the computer program, the processor stores thereon the computer program to implement a point cloud semantic uncertainty perception method based on neighborhood aggregation monte carlo deactivation.
Drawings
The foregoing and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is a schematic flow diagram of a point cloud semantic uncertainty sensing method based on neighborhood aggregation monte carlo inactivation according to an embodiment of the present invention.
Fig. 2 is a schematic flow chart of a point cloud semantic uncertainty sensing device based on neighborhood aggregation monte carlo inactivation according to an embodiment of the present invention.
FIG. 3 is a schematic diagram of a Monte Carlo deactivation process provided by an embodiment of the present invention.
Fig. 4 is a schematic diagram of single random forward propagation according to an embodiment of the present invention.
Fig. 5 is a schematic diagram of an uncertainty-guided learning framework according to an embodiment of the present invention.
Fig. 6 is a schematic diagram of a PointNet (++) network according to an embodiment of the present invention.
Fig. 7 is a schematic diagram illustrating the contribution of noise points to the loss function during model learning according to an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
The following describes a point cloud semantic uncertainty sensing method and device based on Neighborhood Aggregation Monte Carlo deactivation (NSA-MC-Dropout) according to an embodiment of the present invention with reference to the drawings.
Fig. 1 is a schematic flow chart of a point cloud semantic uncertainty sensing method based on neighborhood aggregation monte carlo inactivation provided by an embodiment of the present invention.
As shown in fig. 1, the point cloud semantic uncertainty perception method based on neighborhood aggregation monte carlo inactivation includes the following steps:
s1: acquiring an original point cloud of a scene to be processed, wherein the original point cloud is composed of unordered points describing the geometric structure of the scene to be processed;
unlike two-dimensional mesh data such as images, a point cloud is composed of a set of unordered points that describe the geometry of the object. On one hand, the mainstream point cloud segmentation model independently operates each point, and shares the model weight among the points, so that the arrangement disorder of the point cloud is ensured. Therefore, random reasoning of points can be achieved by adding randomness to the model weights. On the other hand, due to the geometric feature similarity of the local regions in the point cloud, the sum of the stochastic inference results of the target point and the field point can be used to approximate the sum of the stochastic inference results of the target point several times. Therefore, the predicted distribution of each point can realize the point cloud semantic segmentation of uncertainty perception by adding random reasoning results of each point in the field.
S2: the method comprises the steps that original point cloud is used as input, a PointNet (++) is used as a basic model to construct an NSA-MC-dropout framework, wherein the NSA-MC-dropout framework generates feature vectors with different granularities in a coding stage, the feature vectors are spliced to corresponding space points in a decoding stage, and random reasoning results of all disordered points are generated after reasoning is carried out by a multilayer perceptron with space sampling;
further, in one embodiment of the invention, the model sampling is accomplished by random inactivation factors after part of the trainable layer of the NSA-MC-dropout framework.
Further, in one embodiment of the present invention, the multi-layered perceptron includes a fully connected weight-sharing layer, and FC layers used to reason about different target points may exhibit different random weights by random inactivation factors at the weight-sharing layer, thereby achieving spatial sampling in a single random forward direction.
The mainstream point cloud semantic segmentation method independently processes each point, thereby keeping the ordering invariance of the input points. These segmentation models are for each
The shared weight at which the points reason for can be expressed as
。
Can be obtained by combining Bernoulli matrix
Placing in a shared weight matrix
Obtained by
Represents the dropout process. The model space sampling module and the neighborhood space aggregation module cooperate to realize the target point
The predictive distribution is established in a single forward propagation.
Specifically, as shown in fig. 4. We propose a spatial sampling module that generates random mask weights by randomly deactivating factors in the shared weights of the model for random inference of different spatial points. In addition, we propose a new Neighborhood Spatial Aggregation module (NSA) that builds the predicted distribution of each point by aggregating the probability output of each point in the Neighborhood. The novel prediction distribution establishment method based on single inference is called neighborhood space aggregation Monte Carlo inactivation (NSA-MC-dropout). On the basis, different types of uncertainty of each point are established and obtained by using different acquisition functions. In uncertainty assessment, we can achieve quantification of prediction uncertainty with only a single inference. In uncertainty guided learning, considering the relation between accidental uncertainty and data set inherent noise, the NSA-MC-dropout is used for quickly establishing the prediction distribution of each point, and the accidental uncertainty is used as weight to be introduced into a cross entropy loss function. The loss function prevents overfitting of the model due to noisy data, thereby improving the robustness and segmentation performance of the model.
In view of the great success of PointNet and PointNet + + in point cloud segmentation, we chose them as the base model. PointNet (++) is an encoder-decoder network, as shown in FIG. 6. The encoding stage learns global and local characteristics, the decoding stage splices the encoding information with different granularities with each point, and the establishment of the predictive distribution of each point is further realized.
NSA-MC-dropout with PointNet + + as the basic model: in the encoding phase, the model generates feature vectors of different granularities. In the decoding stage, the generated features are spliced to corresponding spatial points, and a point-by-point random reasoning result is generated after reasoning by using a multilayer perceptron with spatial sampling. And then, establishing the prediction distribution of each point through NSA.
A complete uncertainty-aware bayesian network should incorporate random deactivations after each trainable layer of the model. Tests show that the regularization is too strong, so that the convergence speed of model training is slow. Furthermore, we explored the addition of random deactivations after only a portion of the trainable layer, and found that this lightweight bayesian network produces similar uncertainty outputs as the full bayesian network. Therefore, the model sampling can be realized by adding random inactivation only after a plurality of training layers.
At the decoding layer, the multi-layer perceptron (MLP) for decoding contains some fully connected weight-sharing (FC) layers. By randomly inactivating factors in the MLP, FC layers used for reasoning different target points can present different random weights, thereby realizing spatial sampling in a single random forward direction. On the other hand, the coding information is shared in a certain spatial range, random inactivation is carried out on the training layer in the coding stage, and no contribution is made to random reasoning of a single point. Therefore, the dropout layer can only be used after MLP in the decoding stage.
S3: the prediction distribution of each disordered point in the original point cloud is established by fusing the random reasoning results to realize single random reasoning;
traditional MC-dropout aggregates T random inference probability outputs of each point
And represents the predicted distribution as an empirical sample, as shown in fig. 3. Probability output of each point
Obtained by random reasoning.
The output of the t-th probability is shown,
in order to predict the label(s),
representing model samplesRaw random deactivation weights. We design a neighborhood space aggregation method to eliminate the dependency of prediction distribution establishment on T forward propagation times.
Further, in an embodiment of the present invention, the establishing prediction distribution of each disordered point in the original point cloud by single stochastic reasoning through fusing stochastic reasoning results includes:
for a set of contained target points
And it
A point set of adjacent points
Can be written as:
wherein, the point set
Probability output of each point in
Probability output of T times which can be similar to target point established by MC dropout
The distribution mean obtained by summarizing the T probability outputs can be expressed as:
if used in
Stochastic inference weights of neighbors
And establishing the model in the random forward propagation of the primary model, and realizing the establishment of the predictive distribution of each point in the point cloud through one-time reasoning.
The noise points contribute more to the loss function during model learning, as shown in fig. 7. This can over-fit the model and result in degraded performance. To address this problem, the prediction output of the target is modeled as a prediction distribution to characterize all possible predicted values, so that noisy and non-noisy points can be quantitatively characterized in a uniform manner.
Further, in an embodiment of the present invention, after obtaining the prediction distribution of each chaotic point, the method further includes:
and (3) representing all possible predicted values by the prediction distribution, quantitatively representing noise points and non-noise points in a unified mode, and distributing lower weight to the noise points in an uncertainty guiding loss function to realize stronger robustness of the NSA-MC-dropout framework model to noise:
wherein N and M represent the number of spatial points and the number of categories, respectively,
the representation is based on a casual uncertainty,
a predicted variance of the characterization; when the predicted value is true
Otherwise, the value is 0; the first term of the uncertainty guiding loss function is used for suppressing noise points;
the predictor for the regularization term to prevent accidental uncertainty is infinite.
S4: point cloud semantic uncertainty is quantified by capturing the amount of information contained in a predicted distribution or by modeling the degree of dispersion of the predicted distribution.
The proposed predictive distribution building method can support different acquisition functions to assess and resolve uncertainties, such as Predictive Entropy (PE) and mean standard deviation (STD). PE function
Quantifying uncertainty by capturing the amount of information contained in the prediction distribution, the STD function
The uncertainty is quantified by modeling the degree of dispersion of the predicted distribution.
Taking an STD-based function as an example, occasional uncertainty
For quantifying noise floor, cognitive uncertainty in input data set
For quantifying the confidence level of the model itself.
And
the contingency and cognitive components of the mean standard deviation are represented separately. Combining these two uncertainties to generate a predicted uncertainty
And the overall credibility of the model to the prediction result of the data with the noise label.
The uncertainty in the test sample representation is shown in table 1. The prediction result with the dropout layer model can be expressed as
For convenience is shown as
. By using NSA-MC-dropout,
can approximate
Wherein
Sub-functions representing the acquisition function, e.g.
. This transformation allows the estimation and decomposition of uncertainty to be performed in one forward inference of the model.
TABLE 1 uncertainty evaluation and decomposition calculation function
The semantic segmentation framework of the uncertainty perception point cloud provided by the invention is shown in FIG. 5. The NSA-MC-Dropout mainly comprises a model space sampling module and a neighborhood space aggregation module (NSA), so that the establishment of prediction distribution of each point in single random inference is realized. The loss function integrates the contingent uncertainty obtained from the prediction distribution, guides the model to perform better convergence, and outputs the prediction uncertainty to quantify the credibility of the prediction result.
The point cloud semantic uncertainty perception method based on neighborhood aggregation Monte Carlo inactivation provided by the embodiment of the invention provides an efficient uncertainty perception point cloud semantic segmentation framework, improves segmentation performance by using uncertainty, and quantifies credibility of a prediction result, which is the first framework for realizing uncertainty perception point cloud semantic segmentation on the basis of not increasing model parameters and reasoning times. The NSA-MC-Dropout not only focuses on the efficient assessment of uncertainty, but also uses the brand-new method for uncertainty-guided learning, can realize efficient uncertainty quantification and punishment on inherent noise in the learning process, and can realize good model convergence.
In order to realize the embodiment, the invention further provides a point cloud semantic uncertainty sensing device based on neighborhood aggregation Monte Carlo inactivation.
Fig. 2 is a schematic structural diagram of a point cloud semantic uncertainty sensing device based on neighborhood aggregation monte carlo inactivation provided by an embodiment of the present invention.
As shown in fig. 2, the sensing apparatus for sensing semantic uncertainty of point cloud based on neighborhood aggregation monte carlo deactivation includes: the system comprises an acquisition module 10, a construction module 20, an output module 30 and a perception module 40, wherein the acquisition module is used for acquiring an original point cloud of a scene to be processed, and the original point cloud is composed of unordered points describing the geometric structure of the scene to be processed; the system comprises a construction module, a random inference module and a data processing module, wherein the construction module is used for constructing an NSA-MC-dropout framework by taking an original point cloud as input and taking PointNet (++) as a basic model, wherein the NSA-MC-dropout framework generates feature vectors with different granularities in an encoding stage, the feature vectors are spliced to corresponding space points in a decoding stage, and random inference results of all disordered points are generated after inference is carried out by using a multi-layer perceptron with space sampling; the output module is used for realizing single random reasoning and establishing prediction distribution of each disordered point in the original point cloud by fusing random reasoning results; a perception module to quantify a point cloud semantic uncertainty by capturing an amount of information contained in a predicted distribution or by modeling a degree of dispersion of the predicted distribution.
Further, in an embodiment of the present invention, the building module is further configured to implement model sampling by using random inactivation factors after part of the trainable layer of the NSA-MC-dropout framework.
Further, in an embodiment of the present invention, the building module is further configured to, the multilayer perceptron includes a weight sharing layer that is fully connected, and the FC layer for reasoning different target points may present different random weights by using a random inactivation factor at the weight sharing layer, so as to implement spatial sampling in a single random forward direction.
To achieve the above object, a third aspect of the present invention provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements a point cloud semantic uncertainty perception method based on neighborhood aggregation monte carlo deactivation when executing the computer program.
In order to achieve the above object, a fourth embodiment of the present invention provides a non-transitory computer-readable storage medium, where when the processor executes the computer program, the processor stores thereon the computer program to implement a point cloud semantic uncertainty sensing method based on neighborhood aggregation monte carlo deactivation.
In the description of the specification, reference to the description of "one embodiment," "some embodiments," "an example," "a specific example," or "some examples" or the like means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (9)
1. A point cloud semantic uncertainty perception method based on neighborhood polymerization Monte Carlo inactivation is characterized by comprising the following steps:
acquiring an original point cloud of a scene to be processed, wherein the original point cloud is composed of unordered points describing the geometric structure of the scene to be processed;
taking the original point cloud as input, and constructing an NSA-MC-dropout framework by taking PointNet (++) as a basic model, wherein in an encoding stage, the NSA-MC-dropout framework generates feature vectors with different granularities, in a decoding stage, the feature vectors are spliced to corresponding space points, and random reasoning results of all disordered points are generated after reasoning by using a multilayer perceptron with space sampling;
establishing the prediction distribution of each disordered point in the original point cloud by fusing the random reasoning results to realize single random reasoning;
quantifying a point cloud semantic uncertainty by capturing an amount of information contained in the predicted distribution or by modeling a degree of dispersion of the predicted distribution;
after obtaining the prediction distribution of each unordered point, the method further comprises the following steps:
and characterizing all possible predicted values by the prediction distribution, thereby quantitatively characterizing noise points and non-noise points in a uniform mode, and distributing lower weight to the noise points in an uncertainty guidance loss function to realize the stronger robustness of the NSA-MC-dropout framework to noise:
wherein the quantifying a point cloud semantic uncertainty by capturing an amount of information contained in the predicted distribution or the quantifying a point cloud semantic uncertainty by modeling a degree of dispersion of the predicted distribution further comprises:
the uncertainty is assessed and resolved by different acquisition functions, including a prediction entropy PE function and an average standard deviation STD function:
，
wherein, the prediction result with the dropout layer model is expressed as
For convenience is shown as
(ii) a By using NSA-MC-dropout,
approximation
，
Representing a sub-function of the acquisition function.
2. The method of claim 1, further comprising,
and adding random inactivation factors after part of the trainable layer of the NSA-MC-dropout framework to realize model sampling.
3. The method of claim 1 or 2, further comprising,
the multilayer perceptron comprises a weight sharing layer which is completely connected, and random inactivation factors are added into the weight sharing layer to infer that FC layers of different target points present different random weights, so that space model sampling is realized in a single random forward direction.
4. The method of claim 1, wherein the establishing a predictive distribution of each unordered point in the original point cloud by a single stochastic inference through fusing the stochastic inference results comprises:
for a set of contained target points
And it
Set of points of a neighborhood of points
Can be written as:
wherein, the point set
Probability output of each point in
Probability output of T times which can be similar to target point established by MC dropout
The distribution mean obtained by summarizing the T probability outputs can be expressed as:
if used in
Stochastic inference weights of neighbors
And establishing the model in the random forward propagation of the primary model, and realizing the establishment of the predictive distribution of each point in the point cloud through one-time reasoning.
5. A point cloud semantic uncertainty perception device based on neighborhood polymerization Monte Carlo inactivation is characterized by comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring an original point cloud of a scene to be processed, and the original point cloud is composed of unordered points describing the geometric structure of the scene to be processed;
the construction module is used for taking the original point cloud as input and constructing an NSA-MC-dropout framework by taking PointNet (++) as a basic model, wherein in the encoding stage, the NSA-MC-dropout framework generates feature vectors with different granularities, in the decoding stage, the feature vectors are spliced to corresponding space points, and random reasoning results of all the disordered points are generated after reasoning by using a multilayer perceptron with space sampling;
the output module is used for realizing single random reasoning and establishing the prediction distribution of each disordered point in the original point cloud by fusing the random reasoning results;
a perception module that quantifies point cloud semantic uncertainty by capturing an amount of information contained in the predicted distribution or by modeling a degree of dispersion of the predicted distribution;
after obtaining the prediction distribution of each unordered point, the method further comprises the following steps:
and characterizing all possible predicted values by the prediction distribution, thereby quantitatively characterizing noise points and non-noise points in a uniform mode, and distributing lower weight to the noise points in an uncertainty guidance loss function to realize the stronger robustness of the NSA-MC-dropout framework to noise:
wherein the quantifying a point cloud semantic uncertainty by capturing an amount of information contained in the predicted distribution or the quantifying a point cloud semantic uncertainty by modeling a degree of dispersion of the predicted distribution further comprises:
the uncertainty is assessed and resolved by different acquisition functions, including a prediction entropy PE function and an average standard deviation STD function:
，
wherein, the prediction result with dropout layer model is expressed as
For convenience is shown as
(ii) a By using NSA-MC-dropout,
approximation
，
Representing a sub-function of the acquisition function.
6. The apparatus of claim 5, wherein the build module is further configured to,
and adding random inactivation factors after part of the trainable layer of the NSA-MC-dropout framework to realize model sampling.
7. The apparatus of claim 5 or 6, wherein the building block is further configured to,
the multilayer perceptron comprises a weight sharing layer which is completely connected, and random inactivation factors are added into the weight sharing layer to infer that FC layers of different target points present different random weights, so that spatial sampling is realized in a single random forward direction.
8. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of any one of claims 1-4 when executing the computer program.
9. A non-transitory computer-readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the method of any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210165979.2A CN114241110B (en) | 2022-02-23 | 2022-02-23 | Point cloud semantic uncertainty sensing method based on neighborhood aggregation Monte Carlo inactivation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210165979.2A CN114241110B (en) | 2022-02-23 | 2022-02-23 | Point cloud semantic uncertainty sensing method based on neighborhood aggregation Monte Carlo inactivation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114241110A CN114241110A (en) | 2022-03-25 |
| CN114241110B true CN114241110B (en) | 2022-06-03 |
Family
ID=80748016
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210165979.2A Active CN114241110B (en) | 2022-02-23 | 2022-02-23 | Point cloud semantic uncertainty sensing method based on neighborhood aggregation Monte Carlo inactivation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114241110B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114637921B (en) * | 2022-05-12 | 2022-09-09 | 中国科学技术大学 | Item recommendation method, device and equipment based on modeling accidental uncertainty |
| CN114821074B (en) * | 2022-07-01 | 2022-10-25 | 湖南盛鼎科技发展有限责任公司 | Airborne LiDAR point cloud semantic segmentation method, electronic equipment and storage medium |
| CN117496309B (en) * | 2024-01-03 | 2024-03-26 | 华中科技大学 | Building scene point cloud segmentation uncertainty evaluation method and system and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020108311A1 (en) * | 2018-11-29 | 2020-06-04 | 北京市商汤科技开发有限公司 | 3d detection method and apparatus for target object, and medium and device |
| WO2020119619A1 (en) * | 2018-12-14 | 2020-06-18 | 中国科学院深圳先进技术研究院 | Network optimization structure employing 3d target classification and scene semantic segmentation |
| CN112819833A (en) * | 2021-02-05 | 2021-05-18 | 四川大学 | Large scene point cloud semantic segmentation method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111724478B (en) * | 2020-05-19 | 2021-05-18 | 华南理工大学 | Point cloud up-sampling method based on deep learning |
-
2022
- 2022-02-23 CN CN202210165979.2A patent/CN114241110B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020108311A1 (en) * | 2018-11-29 | 2020-06-04 | 北京市商汤科技开发有限公司 | 3d detection method and apparatus for target object, and medium and device |
| WO2020119619A1 (en) * | 2018-12-14 | 2020-06-18 | 中国科学院深圳先进技术研究院 | Network optimization structure employing 3d target classification and scene semantic segmentation |
| CN112819833A (en) * | 2021-02-05 | 2021-05-18 | 四川大学 | Large scene point cloud semantic segmentation method |
Non-Patent Citations (2)
| Title |
|---|
| Neighborhood Spatial Aggregation based Efficient Uncertainty Estimation for Point Cloud Semantic Segmentation;Chao Qi 等;《2021 IEEE International Conference on Robotics and Automation (ICRA 2021)》;20211018;第III、IV章 * |
| 采用深度级联卷积神经网络的三维点云识别与分割;杨军等;《光学精密工程》;20200513(第05期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114241110A (en) | 2022-03-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114241110B (en) | Point cloud semantic uncertainty sensing method based on neighborhood aggregation Monte Carlo inactivation | |
| Deb et al. | A taxonomy for metamodeling frameworks for evolutionary multiobjective optimization | |
| Mata et al. | Type-1 OWA methodology to consensus reaching processes in multi-granular linguistic contexts | |
| Liu et al. | Multiobjective reinforcement learning: A comprehensive overview | |
| KR102039397B1 (en) | Visual Question Answering Apparatus for Explaining Reasoning Process and Method Thereof | |
| CN114004328A (en) | AI model updating method, device, computing equipment and storage medium | |
| KR20190029083A (en) | Apparatus and Method for learning a neural network | |
| KR20200078214A (en) | Image processing apparatus and method for transfering style | |
| CN112200296A (en) | Network model quantification method and device, storage medium and electronic equipment | |
| Mourtzis et al. | An intelligent framework for modelling and simulation of artificial neural networks (ANNs) based on augmented reality | |
| KR20230051110A (en) | Method and apparatus for processing artificial intelligence learning model of on-board image processing system | |
| Saguil et al. | A layer-partitioning approach for faster execution of neural network-based embedded applications in edge networks | |
| Sarangpure et al. | Automating the machine learning process using PyCaret and Streamlit | |
| Zhang et al. | Generalizing universal adversarial perturbations for deep neural networks | |
| Gerrits | Soul of a new machine: Self-learning algorithms in public administration | |
| KR102559590B1 (en) | Method and apparatus for increasing knowledge based on uncertainty in neural networks | |
| CN112541556A (en) | Model construction optimization method, device, medium, and computer program product | |
| CN113516182B (en) | Visual question-answering model training and visual question-answering method and device | |
| CN112446385B (en) | Scene semantic segmentation method and device and electronic equipment | |
| Singh et al. | Explainability metrics and properties for counterfactual explanation methods | |
| Schöning et al. | AI-in-the-Loop--The impact of HMI in AI-based Application | |
| CN114359587A (en) | Class activation graph generation method, interpretable method, device, equipment and storage medium | |
| Chatterjee et al. | TorchEsegeta: Framework for Interpretability and Explainability of Image-based Deep Learning Models | |
| Khaleghi et al. | Human-centered multi-target tracking: A random set theoretic approach | |
| Radley et al. | IMPROVING THE PERFORMANCE AND FUNCTIONALITY OF AI AND MACHINE LEARNING IN ELECTRONIC DEVICES AND SYSTEMS. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |