CN117974815A - Progressive point cloud geometric information compression method based on curved surface feature constraint - Google Patents
Progressive point cloud geometric information compression method based on curved surface feature constraint Download PDFInfo
- Publication number
- CN117974815A CN117974815A CN202410117890.8A CN202410117890A CN117974815A CN 117974815 A CN117974815 A CN 117974815A CN 202410117890 A CN202410117890 A CN 202410117890A CN 117974815 A CN117974815 A CN 117974815A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- curved surface
- anchor
- point
- local
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000006835 compression Effects 0.000 title claims abstract description 62
- 238000007906 compression Methods 0.000 title claims abstract description 62
- 238000000034 method Methods 0.000 title claims abstract description 48
- 230000000750 progressive effect Effects 0.000 title claims abstract description 22
- 238000013528 artificial neural network Methods 0.000 claims abstract description 9
- 238000007781 pre-processing Methods 0.000 claims abstract description 7
- 238000013507 mapping Methods 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 14
- 238000012549 training Methods 0.000 claims description 8
- 230000006837 decompression Effects 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 5
- 238000000605 extraction Methods 0.000 claims description 4
- 238000012360 testing method Methods 0.000 claims description 4
- 238000013527 convolutional neural network Methods 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 2
- 238000013442 quality metrics Methods 0.000 claims description 2
- 241000509579 Draco Species 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Landscapes
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The invention belongs to the technical field of point cloud coding, and particularly relates to a progressive point cloud geometric information compression method based on curved surface feature constraint. The method comprises the following steps: and preprocessing the point cloud to obtain local point cloud patches with the same size, wherein the point cloud patches generally represent low-level local information of the point cloud, such as planes, paraboloids, cylindrical surfaces and the like. At the encoding end, extracting local curved surface characteristics and anchor point characteristics of the point cloud patch by using a neural network; and further compressing the local curved surface characteristics by utilizing the correlation between the anchor point characteristics and the local curved surface characteristics. At the decoding end, the point cloud surface patch can be regarded as a isomorphic body of a two-dimensional plane in consideration of the continuity of the point cloud surface patch in a three-dimensional point cloud space; and reconstructing the three-dimensional point cloud according to the curved surface characteristics by utilizing prior information of the two-dimensional plane, so that the point cloud is constrained in the curved surface manifold, and the efficient compression of the point cloud is realized.
Description
Technical Field
The invention belongs to the technical field of point cloud coding, and particularly relates to a progressive point cloud geometric information compression method based on curved surface feature constraint.
Background
In recent years, along with the rapid development and wide application of the three-dimensional perception technology, the point cloud becomes an indispensable important data form in the fields of three-dimensional reconstruction, virtual reality, automatic driving and the like. However, the high dimensionality and large scale of point clouds makes their storage and transmission very challenging.
At present, the traditional compression schemes widely applied, such as GPCC, draco and the like, generally perform voxel processing on point clouds, divide the point clouds by adopting a tree-shaped or block-shaped structure, and encode the structured point clouds. But the pre-processing of voxelization makes the reconstructed point cloud "mosaic" phenomenon serious, especially in high compression rate scenes. In recent years, deep learning methods have been introduced into the field of point cloud compression, such as DPCC. The deep learning method encodes the point cloud into hidden variables through nonlinear transformation, has the capability of abstracting high-level characteristics, and obtains good compression performance. However, due to the lack of constraints, at bit rates, the reconstructed point cloud has a large number of outliers that deviate from the point cloud surface.
Disclosure of Invention
The invention aims to overcome the technical defects in the prior art, and provides a progressive point cloud geometric information compression method constrained by curved surface features, which is assisted by prior information of a two-dimensional plane in a reconstruction stage and ensures the smoothness of the surface of a reconstructed point cloud.
The invention provides a progressive point cloud geometric information compression method based on curved surface feature constraint, which comprises the following steps: and preprocessing the point cloud to obtain local point cloud patches with the same size, wherein the point cloud patches generally represent low-level local information of the point cloud, such as planes, paraboloids, cylindrical surfaces and the like. At the encoding end, extracting local curved surface characteristics and anchor point characteristics of the point cloud patch by using a neural network; utilizing the correlation between the anchor point characteristic and the local curved surface characteristic to further compress the local curved surface characteristic; at the decoding end, the point cloud surface patch can be regarded as a isomorphic body of a two-dimensional plane in consideration of the continuity of the point cloud surface patch in a three-dimensional point cloud space; reconstructing a three-dimensional point cloud according to the curved surface characteristics by utilizing prior information of a two-dimensional plane, so that the point cloud is constrained in a curved surface manifold, and efficient compression of the point cloud is realized; the method comprises the following specific steps:
step 1, preprocessing point cloud data;
Step 2, constructing an encoder for extracting and compressing point cloud curved surface features, wherein the encoder comprises a local curved surface feature extraction module, an anchor point feature extraction module and a feature compression module;
step 3, compression coding is carried out on the curved surface characteristics and the anchor points by using an entropy coder;
step 4, constructing a decoder for reconstructing the three-dimensional point cloud, wherein the decoder comprises a 2D-3D mapping reconstruction module;
Step 5, constructing a loss function, wherein the loss function comprises the integral point cloud chamfering distance loss Loss of anchor reconstruction/>And a code rate loss R a,Rf;
Step 6, compressing a network model according to the progressive point cloud geometric information constrained by the surface features formed by the encoder and the decoder, and performing end-to-end training on the network model according to the loss function;
and 7, inputting the cloud data of the test points into the trained network model to realize the compression of the point cloud.
The specific operation of each step is as follows:
the specific operation of the step 1:
Step 1-1, adopting a furthest point sampling method (FPS), and downsampling an input point cloud to obtain a sparse anchor point;
and 1-2, constructing a local patch and an anchor patch by using a K Nearest Neighbor (KNN) algorithm for each anchor point, and normalizing patch coordinates.
The specific operation of the step 2:
step 2-1, extracting local curved surface features in a local patch by using a Dynamic Graph Convolutional Neural Network (DGCNN), wherein the local curved surface features contain point cloud local information;
Step 2-2, extracting anchor point characteristics in an anchor point panel by using a dynamic graph convolution neural network, wherein the anchor point characteristics comprise larger-scale point cloud information;
And 2-3, using a feature compression network, and further compressing the curved surface features in the step 2-1 by utilizing the anchor point features in the step 2-2. The feature compression network consists of fully connected networks sharing weights, with batch normalization (Batch Normalization) and linear rectification functions (ReLu) applied between each layer of networks, as shown in fig. 3.
In step 3, the curved surface features in step 2-3 are entropy coded by using an entropy coder, and the geometric coordinates of the sparse anchor points in step 1-1 are entropy coded by using the entropy coder.
The specific operation flow of the step4 is as follows:
Step 4-1, obtaining a reconstruction anchor point by entropy coding; establishing an anchor point face;
step 4-2, extracting anchor point characteristics from the reconstructed anchor point panel by using a dynamic graph convolution neural network;
step 4-3, using a characteristic decompression network, and using the anchor point characteristic in the step 4-2 to decompress the curved surface characteristic from the compressed curved surface characteristic; the characteristic decompression network adopts the same network structure as the characteristic compression network;
Step 4-4, the partial patches of the three-dimensional point cloud space have continuity, and each partial patch is regarded as a mapping of a two-dimensional plane; mapping the two-dimensional planar points to a three-dimensional local panel using a 2D-3D mapping reconstruction module; the reconstruction of the whole point cloud is completed by mapping all the local patches; the 2D-3D mapping reconstruction network is made up of a fully connected mesh, as shown in fig. 4.
In step 5, constructing a loss function according to rate distortion optimization; wherein the compression rate is the bit number R f of the characteristic bit stream after entropy coding and the bit number R a of the anchor bit stream; distortion uses reconstruction quality metrics, including reconstruction loss at anchor pointsAnd integral point cloud chamfer distance loss/>
In step 6, a plurality of point cloud compression models with different compression rates are obtained through training by controlling different rate distortion weights lambda.
The specific operation of the step 7: and inputting the point cloud to be compressed into a compression model after training is completed, and realizing the point cloud compression.
The method takes manifold characteristics of the point cloud local curved surface into consideration, and the reconstructed three-dimensional point cloud is constrained in the three-dimensional manifold by using the prior information assistance of the two-dimensional plane in the reconstruction stage. The method ensures the smoothness of the surface of the reconstructed point cloud, and effectively reduces the abnormal points of the reconstructed point cloud under the low code rate. According to the invention, the point cloud with better reconstruction quality is obtained by utilizing the correlation between the local characteristics of the point cloud and the characteristics of the large-scale anchor points to realize further compression.
Drawings
Fig. 1 is a main flow chart of the present invention.
Fig. 2 is a detailed structural design of the present invention.
Fig. 3 is a diagram of a feature compression network architecture of the present invention.
Fig. 4 is a 2D-3D mapping reconstruction network structure of the present invention.
Fig. 5 is a graph comparing compression results of the dataset at ShapeNet according to the present invention, the comparison methods being DPCC method, GPCC method, VPCC method and Draco method, respectively.
Detailed Description
The invention is further described below by way of examples with reference to the accompanying drawings.
Fig. 1 is a flowchart of a progressive point cloud compression method of curved surface feature constraint in an embodiment of the present invention. As shown in fig. 1, the progressive point cloud compression method of curved surface feature constraint includes the following steps:
step 1, preprocessing point cloud data;
The embodiment adopts a public dataset ShapeNetCoreV point cloud dataset comprising 55 types of point clouds. The present embodiment selects 50 point clouds from each class, from which the training set, the validation set, and the test set are partitioned. Each point cloud contains 200000 points and is normalized to be within [ -1,1 ]. The three-dimensional point cloud originally input in this embodiment is expressed as Where n represents the number of point clouds, i.e. 200000.
Step 1 comprises the following sub-steps:
step 1-1, sampling an original input point cloud X according to 21 multiplying power by adopting a furthest point sampling method to obtain a sparse anchor point Wherein the number of the anchor points is s;
Step 1-2, searching 9 nearest neighbors in the original input point cloud X by adopting a K nearest neighbor algorithm aiming at each anchor point a i, and constructing a local panel X i; for each anchor point a i, 4 nearest neighbors are searched in the anchor point cloud a, and an anchor point patch a i is constructed. For data unification scale, the coordinates of the patch are standardized with the anchor point as a center point. Specifically, subtracting the anchor point coordinates from the coordinates of each point of the patch to obtain a standardized point cloud patch.
Step 2, constructing an encoder for extracting and compressing point cloud curved surface characteristics;
Step 2 comprises the following sub-steps:
Step 2-1, extracting a local curved surface feature F i l from a local patch X i by adopting a dynamic graph convolutional neural network; the number of network channels is set to [64,128,256,50].
Step 2-2, extracting anchor point characteristics from the anchor point slice A i by adopting a dynamic graph convolution neural networkThe number of network channels is set to [64,128,256,50].
And 2-3, further compressing the features by using a feature compression network, inputting the features into a local curved surface feature and an anchor point feature, and outputting the features into a compressed curved surface feature F i. The characteristic compression network consists of 4 layers of fully-connected networks sharing weight, the channel number of each layer is [256,128,64,3], and batch normalization (Batch Normalization) and linear rectification functions (ReLu) are applied between the networks of each layer; as shown in fig. 3;
and 3, performing compression coding on the point cloud curved surface characteristic F i and the anchor point A by using an entropy coder with super prior to obtain a characteristic bit stream and an anchor point bit stream. The quantization process is converted into a process of superposing uniformly sampled random noise, and the superposed noise is restrained in an interval [ -0.5, 0.5), so that end-to-end training is performed.
Step 4, constructing a decoder for reconstructing the point cloud coordinates;
Step4 comprises the following sub-steps:
step 4-1, obtaining reconstructed anchor points from anchor point bit stream by using entropy coding Establishing anchor facet/>
Step 4-2, rolling neural network on reconstructed anchor pointExtracting anchor point characteristics from the tree
Step 4-3, using the characteristic decompression network, slaveAnd entropy coding/>In which the local curved surface features/>The characteristic decompression network and the characteristic compression network adopt the same network structure;
step 4-4, randomly sampling 84 two-dimensional points from the two-dimensional plane ranging from [ -1,1] As shown in fig. 4, a two-dimensional point/>, is input using a 2D-3D reconstruction mapping network ψAnd local surface features/>And outputting the reconstructed three-dimensional point cloud delta i. The 2D-3D reconstruction mapping network is composed of a 4-layer fully connected network, and the number of channels is [128,128,64,3]. Mapping and reconstructing all local patches to reconstruct the whole input point cloud/>Finally, sampling n points from the Y at the longest distance to obtain a final reconstruction point cloud/>
Step 5, constructing a loss function;
the specific operation of the step 5: constructing a loss function according to rate distortion optimization, wherein the rate distortion optimization is described as follows:
Where the compression rate R a represents the number of bits of the anchor bit stream, R f represents the number of bits of the feature bit stream, and λ is the langerhans multiplier. Distortion of Using the chamfer distance loss measure of the original point cloud X and the reconstructed point cloud Y *, the calculation is as follows:
Distortion of Using original anchor point cloud a and reconstructed anchor/>The mean square error loss measure of (2) is calculated as follows:
And 6, constructing a progressive point cloud compression model constrained by curved surface features, as shown in fig. 2. In order to realize the point cloud compression of different code rates, lambda is set as [20,10,1,0.1,0.001] in the embodiment, and 5 different compression models are trained on the constructed compression network. The present embodiment uses Pytorch frames to train 50 epochs on the NVIDIA A40 GPU with a learning rate set to 0.001.
And 7, inputting the point cloud to be compressed into a compression model after training is completed, and realizing the point cloud compression.
According to the progressive point cloud compression method based on the curved surface feature constraint, shapeNetCoreV test concentrated point clouds are taken as input, and compression performance of the progressive point cloud compression method under different code rate models in step 5 is tested. This example compares the performance differences of the present method with the DPCC method, GPCC method, VPCC method and Draco method simultaneously. The experimental results of the embodiment show that the progressive point cloud compression method constrained by the curved surface features is superior to other comparison methods under a plurality of code rates, the curved surface features of the point cloud can be well reserved, and the abnormal points of the reconstructed point cloud under the low code rate are effectively reduced.
Claims (8)
1. A progressive point cloud geometric information compression method of curved surface feature constraint; characterized by comprising the following steps: preprocessing point cloud to obtain local point cloud patches with the same size, wherein the point cloud patches represent low-level local information of point cloud; at the encoding end, extracting local curved surface characteristics and anchor point characteristics of the point cloud patch by using a neural network; utilizing the correlation between the anchor point characteristic and the local curved surface characteristic to further compress the local curved surface characteristic; at the decoding end, considering the continuity of the point cloud patch in the three-dimensional point cloud space, the point cloud patch is regarded as a isomorphic body of a two-dimensional plane; reconstructing a three-dimensional point cloud according to the curved surface characteristics by utilizing prior information of a two-dimensional plane, so that the point cloud is constrained in a curved surface manifold, and efficient compression of the point cloud is realized; the method comprises the following specific steps:
step 1, preprocessing point cloud data;
Step 2, constructing an encoder for extracting and compressing point cloud curved surface features, wherein the encoder comprises a local curved surface feature extraction module, an anchor point feature extraction module and a feature compression module;
step 3, compression coding is carried out on the curved surface characteristics and the anchor points by using an entropy coder;
step 4, constructing a decoder for reconstructing the three-dimensional point cloud, wherein the decoder comprises a 2D-3D mapping reconstruction module;
step 5, constructing a loss function, wherein the loss function comprises integral point cloud chamfer distance loss, anchor point reconstruction loss and code rate loss;
Step 6, compressing a network model according to the progressive point cloud geometric information constrained by the surface features formed by the encoder and the decoder, and performing end-to-end training on the network model according to the loss function;
and 7, inputting the cloud data of the test points into the trained network model to realize the compression of the point cloud.
2. The progressive point cloud geometrical information compression method of curved surface feature constraint according to claim 1; the method is characterized in that the specific operation flow of the step 1 is as follows:
Step 1-1, adopting a far-most point sampling method FPS, and downsampling an input point cloud to obtain a sparse anchor point;
And 1-2, constructing a local patch and an anchor point patch by using a K nearest neighbor algorithm KNN for each anchor point, and normalizing patch coordinates.
3. The progressive point cloud geometrical information compression method of curved surface feature constraint according to claim 2; the method is characterized in that the specific operation flow of the step 2 is as follows:
Step 2-1, extracting local curved surface features in a local patch by using a dynamic graph convolutional neural network DGCNN, wherein the local curved surface features contain point cloud local information;
Step 2-2, extracting anchor point characteristics in an anchor point panel by using a dynamic graph convolution neural network, wherein the anchor point characteristics comprise larger-scale point cloud information;
step 2-3, using a feature compression network, and further compressing the curved surface features in step 2-1 by using the anchor point features in step 2-2; the feature compression network consists of fully connected networks sharing weights, with batch normalization and linear rectification functions applied between each layer of networks.
4. A progressive point cloud geometrical information compression method of curved surface feature constraint according to claim 3; the method is characterized in that the specific operation flow of the step 3 is as follows: entropy encoding the curved surface features in the step 2-3 by using an entropy encoder; entropy encoding the geometric coordinates of the sparse anchor point in step 1-1 using an entropy encoder.
5. The progressive point cloud geometrical information compression method of curved surface feature constraint according to claim 4; the method is characterized in that the specific operation flow of the step 4 is as follows:
Step 4-1, obtaining a reconstruction anchor point by entropy coding; establishing an anchor point face;
step 4-2, extracting anchor point characteristics from the reconstructed anchor point panel by using a dynamic graph convolution neural network;
step 4-3, using a characteristic decompression network, and using the anchor point characteristic in the step 4-2 to decompress the curved surface characteristic from the compressed curved surface characteristic; the characteristic decompression network adopts the same network structure as the characteristic compression network;
Step 4-4, the partial patches of the three-dimensional point cloud space have continuity, and each partial patch is regarded as a mapping of a two-dimensional plane; mapping the two-dimensional planar points to a three-dimensional local panel using a 2D-3D mapping reconstruction module; the reconstruction of the whole point cloud is completed by mapping all the local patches; the 2D-3D mapping reconstruction network is composed of a fully connected mesh.
6. The progressive point cloud geometrical information compression method of curved surface feature constraint according to claim 5; the method is characterized in that in the step 5, a loss function is constructed according to rate distortion optimization; wherein the compression rate is the bit number R f of the characteristic bit stream after entropy coding and the bit number R a of the anchor bit stream; distortion uses reconstruction quality metrics, including reconstruction loss at anchor pointsAnd integral point cloud chamfer distance loss/>
7. The progressive point cloud geometrical information compression method of curved surface feature constraint according to claim 6; in step 6, a plurality of point cloud compression models with different compression rates are obtained through training by controlling different rate distortion weights lambda.
8. The progressive point cloud geometrical information compression method of curved surface feature constraint according to claim 6; the method is characterized in that in step 5, a loss function is constructed according to rate distortion optimization, and the rate distortion optimization is described as follows:
Wherein the compression rate R a represents the bit number of the anchor bit stream, R f represents the bit number of the characteristic bit stream, and lambda is the Langerhans multiplier; distortion of Using the chamfer distance loss measure of the original point cloud X and the reconstructed point cloud Y *, the calculation is as follows:
Distortion of Using original anchor point cloud a and reconstructed anchor/>The mean square error loss measure of (2) is calculated as follows:
s is the number of anchor points.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410117890.8A CN117974815A (en) | 2024-01-29 | 2024-01-29 | Progressive point cloud geometric information compression method based on curved surface feature constraint |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410117890.8A CN117974815A (en) | 2024-01-29 | 2024-01-29 | Progressive point cloud geometric information compression method based on curved surface feature constraint |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117974815A true CN117974815A (en) | 2024-05-03 |
Family
ID=90854815
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410117890.8A Pending CN117974815A (en) | 2024-01-29 | 2024-01-29 | Progressive point cloud geometric information compression method based on curved surface feature constraint |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117974815A (en) |
-
2024
- 2024-01-29 CN CN202410117890.8A patent/CN117974815A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11004240B2 (en) | Hierarchical division-based point cloud attribute compression method | |
| CN109996071B (en) | Variable code rate image coding and decoding system and method based on deep learning | |
| US11494948B2 (en) | Point cloud geometric compression method based on depth auto-encoder | |
| CN109889839B (en) | Region-of-interest image coding and decoding system and method based on deep learning | |
| WO2019210531A1 (en) | Point cloud attribute compression method based on deleting 0 elements in quantisation matrix | |
| CN110602494A (en) | Image coding and decoding system and method based on deep learning | |
| CN110691243A (en) | Point cloud geometric compression method based on deep convolutional network | |
| CN112734867B (en) | Multispectral image compression method and multispectral image compression system based on spatial spectrum feature separation and extraction | |
| CN102804230A (en) | Compression of 3D meshes with repeated patterns | |
| CN110166779A (en) | Video-frequency compression method based on super-resolution reconstruction | |
| CN108960333A (en) | Lossless compression method for high spectrum image based on deep learning | |
| CN112149652A (en) | Space-spectrum joint depth convolution network method for lossy compression of hyperspectral image | |
| CN111209972A (en) | Image classification method and system based on hybrid connectivity deep convolution neural network | |
| WO2022057091A1 (en) | Encoding method, decoding method, encoding device, and decoding device for point cloud attribute | |
| CN111754592A (en) | End-to-end multispectral remote sensing image compression method based on characteristic channel information | |
| Li et al. | Multiple description coding based on convolutional auto-encoder | |
| CN114386595A (en) | SAR image compression method based on super-prior-check architecture | |
| CN104299256B (en) | Almost-lossless compression domain volume rendering method for three-dimensional volume data | |
| CN111343458B (en) | Sparse gray image coding and decoding method and system based on reconstructed residual | |
| CN117974815A (en) | Progressive point cloud geometric information compression method based on curved surface feature constraint | |
| CN112488117A (en) | Point cloud analysis method based on direction-induced convolution | |
| CN116071441A (en) | Remote sensing image compression method based on end-to-end convolutional neural network | |
| CN110349228B (en) | Triangular mesh compression method for data-driven least square prediction | |
| CN113194312A (en) | Planetary science exploration image adaptive quantization coding system combined with visual saliency | |
| CN117915107B (en) | Image compression system, image compression method, storage medium and chip |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |