CN116192904A - Dynamic human-computer interaction point cloud compression method - Google Patents
Dynamic human-computer interaction point cloud compression method Download PDFInfo
- Publication number
- CN116192904A CN116192904A CN202310206858.2A CN202310206858A CN116192904A CN 116192904 A CN116192904 A CN 116192904A CN 202310206858 A CN202310206858 A CN 202310206858A CN 116192904 A CN116192904 A CN 116192904A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- projection
- cloud frame
- frame
- coordinate system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
- G06V10/16—Image acquisition using multiple overlapping images; Image stitching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/761—Proximity, similarity or dissimilarity measures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/08—Protocols specially adapted for terminal emulation, e.g. Telnet
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3179—Video signal processing therefor
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Networks & Wireless Communication (AREA)
- Medical Informatics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Image Processing (AREA)
Abstract
The invention relates to a dynamic man-machine interaction point cloud compression method, which comprises the following steps: calculating a maximum change coordinate system of adjacent point cloud frames in the human-computer interaction three-dimensional point cloud frame sequence based on a multi-layer projection method; segmenting a three-dimensional point cloud frame sequence; for each section of three-dimensional point cloud frame, calculating a fusion coordinate system according to the maximum change coordinate system of the adjacent point cloud frames; and projecting each point cloud frame of the three-dimensional point cloud frame on the projection direction of the fusion coordinate system to obtain a complete projection image of each point cloud frame on each projection direction, and compressing the three-dimensional point cloud frame according to the similarity of the complete projection images of two adjacent point cloud frames on each projection direction.
Description
Technical Field
The invention relates to the technical field of human-computer interaction point clouds, in particular to a dynamic human-computer interaction point cloud compression method.
Background
The man-machine interaction point cloud can be divided into two types of static point cloud and dynamic man-machine interaction point cloud. The static point cloud consists of a group of points with three-dimensional position information, and each point is provided with one or more attributes for storing additional information; the dynamic human-computer interaction point cloud is composed of a plurality of continuous static point clouds, wherein each static point cloud is called a frame of the dynamic human-computer interaction point cloud. The position information of points in the point cloud is typically (x, y, z) three coordinates in a cartesian coordinate system, which is used to characterize the spatial topology and reconstruct the shape of a three-dimensional object. A common attribute of point clouds is color information, typically using RGB values to store the red, green, blue three channels, or intensity values to record pulse return amplitudes to occupy the color field.
The point cloud occupies an important position in the future real-time holographic scene man-machine interaction, the equipment based on the laser radar and photogrammetry principle can rapidly acquire real-time man-machine interaction point cloud information with large scale and high frequency change, and the equipment is mainly required to be transmitted to a user side for display, a computing center for processing and a storage center for storage due to non-uniform network of computing and storage resources. However, the amount of the point cloud data is large, no structure and no metadata exist, and if the direct transmission occupies a huge amount of bandwidth, the problem is a main problem restricting the point cloud transmission. Compression of the point cloud is required to reduce the amount of data.
The current compression methods of dynamic man-machine interaction point clouds can be roughly divided into two categories: compression methods based on three-dimensional structures and compression methods based on two-dimensional mappings. The compression method based on the three-dimensional structure directly researches the compression algorithm on the point cloud, traverses and matches the compression algorithm through the structuring algorithm, most of the methods are based on octree data structures, and one problem that the calculation cost of the algorithm increases exponentially along with the increase of the octree depth is common. This problem is particularly acute for compression of high precision and high detail point cloud sequences. The basic idea of the compression method based on the two-dimensional mapping is to project the point cloud data onto a two-dimensional plane through a certain rule, and then to perform compression coding on the two-dimensional mapping sequence of the dynamic human-computer interaction point cloud by utilizing the existing mature video coding and decoding technology. Compared with a compression algorithm based on a three-dimensional structure, the compression efficiency of the algorithm in time and space is greatly improved, and a remarkable effect can be achieved. In 2017, MPEG (Moving Picture Expert Group ) under ISO/IEC JTC1 establishes related internationalization standards and issues TMC2 (Test Model Category, test model 2) for dynamic man-machine interaction point cloud compression algorithm, which compresses point cloud by adopting a compression method based on two-dimensional mapping, so that compression efficiency is higher, but time and space continuity in three-dimensional space is completely destroyed due to mapping three-dimensional point cloud into two-dimensional space, so that a part of information is lost by the compression method, and compression efficiency is low.
Disclosure of Invention
In view of the above analysis, the embodiment of the invention aims to provide a dynamic man-machine interaction point cloud compression method, which is used for solving the problems of complex calculation and low compression efficiency of the existing dynamic man-machine interaction point cloud compression method.
In one aspect, the embodiment of the invention provides a dynamic man-machine interaction point cloud compression method, which comprises the following steps:
calculating a maximum change coordinate system of adjacent point cloud frames in the human-computer interaction three-dimensional point cloud frame sequence based on a multi-layer projection method;
segmenting a three-dimensional point cloud frame sequence; for each section of three-dimensional point cloud frame, calculating a fusion coordinate system according to the maximum change coordinate system of the adjacent point cloud frames; and projecting each point cloud frame of the three-dimensional point cloud frame on the projection direction of the fusion coordinate system to obtain a complete projection image of each point cloud frame on each projection direction, and compressing the three-dimensional point cloud frame according to the similarity of the complete projection images of two adjacent point cloud frames on each projection direction.
Based on the further improvement of the technical scheme, calculating the maximum change coordinate system of the adjacent point cloud frames in the three-dimensional point cloud frame sequence based on a multi-layer projection algorithm comprises the following steps:
s11, for each point cloud frame in the three-dimensional point cloud frame sequence, establishing a plurality of projection coordinate systems by taking the mass center of each point cloud frame as an origin; respectively projecting each point cloud frame on the projection directions of a plurality of corresponding projection coordinate systems based on a layered projection method;
S12, taking a second point cloud frame in the point cloud frame sequence as a current point cloud frame;
s13, calculating the chamfering distance between the current point cloud frame and the previous frame of the current point cloud frame for each projection direction of each projection coordinate system corresponding to the current point cloud frame;
s14, the projection direction corresponding to the maximum chamfering distance is a first projection direction, and among other projection directions except the first projection direction in all projection coordinate systems in which the first projection direction is located, the projection direction with the maximum chamfering distance is a second projection direction, and the coordinate systems in which the first projection direction and the second projection direction are located are the largest change coordinate systems from the previous frame of the current point cloud frame to the current point cloud frame; and returning to the step S13 by taking the next point cloud frame as the current frame until all the point cloud frames are traversed.
Further, projecting each point cloud frame on the projection directions of a plurality of corresponding projection coordinate systems based on a hierarchical projection method respectively comprises the following steps:
s1121, for a j-th projection coordinate system of a current point cloud frame, forming a point set to be projected by all points of the current point cloud frame; let k=1, l=1;
s1122, taking the kth coordinate axis of the jth projection coordinate system as a reference axis;
s1123, sorting each point of the point set to be projected according to the sequence of the values on the reference coordinate axis from large to small, if a projection point exists at the corresponding position of the L layer in the projection direction corresponding to the reference axis, not projecting the point, otherwise, projecting the point at the corresponding position of the L layer in the projection direction corresponding to the reference axis, and deleting the point from the point set to be projected; if the point set to be projected is empty, the projection is finished; otherwise, go to step S1124;
If k=4, l=l+1, set k to 1, and return to step S1122; otherwise, the process returns to step S1122.
Further, for each projection direction of each projection coordinate system, the chamfering distance of the current point cloud frame and the frame preceding the current point cloud frame is calculated in the following manner:
for the current projection direction O i X j Searching the current point cloud frame to project in the current projection direction O i X j The points on the first set of points; wherein O is i X j O representing the jth projection coordinate system of the ith point cloud frame i X j The projection direction of the shaft;
searching for the projection of the previous frame of the current point cloud frame in the projection direction O i-1 X j The points on the first set of points form a second set of points; o (O) i-1 X j O representing the j-th projection coordinate system of the i-1-th point cloud frame i-1 X j The projection direction of the axis;
and calculating the chamfering distances of the first point set and the second point set to obtain the chamfering distances of the current point cloud frame and the previous frame of the current point cloud frame in the current projection direction.
Further, the chamfer distance d is calculated using the following formula (S 1 ,S 2 )：
Wherein S is 1 A first set of points is represented and,
represent S 1 The number of midpoints S 2 Representing a second set of points, +.>
Represent S 2 Quantity of midpoint->
Represents the distance from point x to point y, +.>
Representing the distance from point y to point x.
Further, for each point cloud frame in the three-dimensional point cloud frame sequence, establishing a plurality of projection coordinate systems with the centroid as the origin, including:
establishing a coordinate system which is parallel to a coordinate axis of a reference coordinate system and has the same direction by taking the centroid of the current point cloud frame as a coordinate origin as a first coordinate;
fixing each coordinate axis in the first coordinate system, and rotating two coordinate axes except the fixed coordinate axes around the fixed coordinate axes according to the rotation step length to obtain a plurality of coordinate systems which are the same as the X axis, the Y axis or the Z axis of the first projection coordinate system;
the first coordinate system and the same coordinate systems as the X-axis, Y-axis or Z-axis thereof form a plurality of projection coordinate systems of the current point cloud frame.
Further, projecting each point cloud frame of the three-dimensional point cloud frame on the projection direction of the fusion coordinate system to obtain a complete projection image of each point cloud frame on each projection direction, including:
projecting each point of each point cloud frame of the three-dimensional point cloud frame on a projection direction corresponding to the fusion coordinate system by adopting a layered projection method to obtain a multi-layer block projection diagram of each point cloud frame on each projection direction corresponding to the fusion coordinate system;
and splicing the multi-layer block projection images of each point cloud frame in each projection direction to obtain a complete projection image of each point cloud frame in each projection direction.
Further, compressing the segment of three-dimensional point cloud frame according to the similarity of the complete projection images of two adjacent point cloud frames in each projection direction, including:
the complete projection images of the first point cloud frame and the last point cloud frame in each projection direction in the three-dimensional point cloud frame of the current section are key frame images;
for other point cloud frames in the current three-dimensional point cloud frame, calculating the structural similarity of the complete projection image of the point cloud frame and the previous point cloud frame in each projection direction; if the structural similarity does not exceed a second threshold, taking the complete projection image as a key frame image; all key frame images of the three-dimensional point cloud frame form an image after the three-dimensional point cloud frame is compressed.
Further, the structural similarity of the point cloud frame and the complete projection image of the previous point cloud frame in each projection direction is calculated by adopting the following method:
for the current projection direction, according to the formula
Respectively calculating SSIM values of the complete projection images of the point cloud frame and the previous point cloud frame on a kth image channel;
wherein mu k,a Representing an element mean value of a kth channel of a complete projection image of the point cloud frame; mu (mu) k,b Representing an element mean value of a kth channel of a complete projection image of a previous point cloud frame of the point cloud frame; sigma (sigma) k,a Representing the element variance of the kth channel of the complete projection image of the point cloud frame; sigma (sigma) k,b Element variance, sigma, of the kth channel of the complete projection image representing the previous point cloud frame of the point cloud frame k,ab Element covariance of kth channel representing complete projection image of the point cloud frame and complete projection image of previous point cloud frame of the point cloud frame, c 1 And c 2 Is a constant;
and adding the SSIM values of all the image channels to obtain the structural similarity value of the complete projection image of the point cloud frame and the previous point cloud frame in the current projection direction.
Further, for each segment of three-dimensional point cloud frame, calculating a fusion coordinate system according to the largest change coordinate system of the adjacent point cloud frame, including:
for the current segment point cloud frame, translating all maximum change coordinate systems corresponding to the current segment point cloud frame to the origin point which coincides with the origin point of the reference coordinate system;
for each coordinate axis of each maximum change coordinate system, the coordinate axis is assigned to the type corresponding to the coordinate axis of the reference coordinate with the smallest included angle;
adding the unit vectors of the coordinate axes corresponding to each type to obtain the coordinate axes of the fusion coordinate system corresponding to the type; and obtaining a fusion coordinate system corresponding to the current three-dimensional point cloud frame.
Compared with the prior art, the method and the device have the advantages that the maximum change coordinate system of the adjacent point cloud frames in the human-computer interaction three-dimensional point cloud frame sequence is calculated, so that the change direction of the connected point cloud frames is obtained; and calculating a fusion coordinate system according to the maximum change coordinate system of the adjacent point cloud frames for each three-dimensional point cloud frame, and projecting the point cloud frames in the projection direction corresponding to the fusion coordinate system, so that the relevance of the point cloud frames in time and space is still reserved after projection, the time-space consistency characteristic change characteristics of the point cloud frames are reserved during compression, and the compression efficiency is improved.
In the invention, the technical schemes can be mutually combined to realize more preferable combination schemes. Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and drawings.
Drawings
The drawings are only for purposes of illustrating particular embodiments and are not to be construed as limiting the invention, like reference numerals being used to refer to like parts throughout the several views.
Fig. 1 is a flowchart of a dynamic man-machine interaction point cloud compression method according to an embodiment of the invention.
Detailed Description
Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form a part hereof, and together with the description serve to explain the principles of the invention, and are not intended to limit the scope of the invention.
The invention discloses a dynamic human-computer interaction point cloud compression method, which comprises the following steps as shown in fig. 1:
s1, calculating a maximum change coordinate system of adjacent point cloud frames in a human-computer interaction three-dimensional point cloud frame sequence based on a multi-layer projection method;
s2, segmenting a three-dimensional point cloud frame sequence; for each section of three-dimensional point cloud frame, calculating a fusion coordinate system according to the maximum change coordinate system of the adjacent point cloud frames; and projecting each point cloud frame of the three-dimensional point cloud frame on the projection direction of the fusion coordinate system to obtain a projection image of each point cloud frame on each projection direction, and compressing the three-dimensional point cloud frame according to the similarity of the projection images of two adjacent point cloud frames on each projection direction.
Compared with the prior art, the method and the device have the advantages that the maximum change coordinate system of the adjacent point cloud frames in the human-computer interaction three-dimensional point cloud frame sequence is calculated, so that the change direction of the connected point cloud frames is obtained; and calculating a fusion coordinate system according to the maximum change coordinate system of the adjacent point cloud frames for each three-dimensional point cloud frame, and projecting the point cloud frames in the projection direction corresponding to the fusion coordinate system, so that the relevance of the point cloud frames in time and space is still reserved after projection, the time-space consistency characteristic change characteristics of the point cloud frames are reserved during compression, and the compression efficiency is improved.
Specifically, step S1 calculates a maximum change coordinate system of adjacent point cloud frames in the three-dimensional point cloud frame sequence based on a multi-layer projection algorithm, including:
s11, for each point cloud frame in the three-dimensional point cloud frame sequence, establishing a plurality of projection coordinate systems by taking the mass center of each point cloud frame as an origin; each point cloud frame is projected on the projection directions of a plurality of corresponding projection coordinate systems based on a layered projection method
S12, taking a second point cloud frame in the point cloud frame sequence as a current point cloud frame;
s13, calculating the chamfering distance between the current point cloud frame and the previous frame of the current point cloud frame for each projection direction of each projection coordinate system corresponding to the current point cloud frame;
s14, the projection direction corresponding to the maximum chamfering distance is a first projection direction, and among other projection directions except the first projection direction in all projection coordinate systems in which the first projection direction is located, the projection direction with the maximum chamfering distance is a second projection direction, and the coordinate systems in which the first projection direction and the second projection direction are located are the largest change coordinate systems from the previous frame of the current point cloud frame to the current point cloud frame; and returning to the step S13 by taking the next point cloud frame as the current frame until all the point cloud frames are traversed.
Specifically, in step S11, for each point cloud frame in the three-dimensional point cloud frame sequence, a plurality of projection coordinate systems are established with the centroid as the origin, including:
S1111, establishing a coordinate system which is parallel to a coordinate axis of a reference coordinate system and has the same direction with the centroid of the current point cloud frame as a coordinate origin as a first coordinate;
s1112, sequentially fixing one coordinate axis in the first coordinate system, and rotating the two coordinate axes except the fixed coordinate axis around the fixed coordinate axis for a plurality of times according to the rotation step length to obtain one coordinate system each time, thereby obtaining a plurality of coordinate systems identical to the X axis, the Y axis or the Z axis of the first projection coordinate system;
s1113, the first coordinate system and the same coordinate systems as the X-axis, Y-axis or Z-axis thereof constitute a plurality of projection coordinate systems of the current point cloud frame.
The reference coordinate system may be, for example, a world coordinate system.
In implementation, for the ith point cloud frame, the point cloud centroid is calculated first, and a coordinate system which is parallel to the coordinate axis of the reference coordinate system and has the same direction is established as the first coordinate O i X 1 Y 1 Z 1 。
O is added with i X 1 Y 1 Z 1 The X axis in the coordinate system is fixed, and the Y axis and the Z axis are rotated around the X axis according to the rotation step length, so that a plurality of coordinate systems with the same X axis are obtained; o is added with i X 1 Y 1 Z 1 The Y axis of the coordinate system is fixed, and the X axis and the Z axis are fixed according to the rotation step lengthRotating around the Y axis to obtain a plurality of coordinate systems with the same Y axis; o is added with i X 1 Y 1 Z 1 And fixing the Z axis in the coordinate system, and rotating the X axis and the Y axis around the Z axis according to the rotation step length to obtain a plurality of coordinate systems with the same Z axis. The first projection coordinate system and the rotated plural coordinate systems which are the same as the X-axis, Y-axis or Z-axis thereof constitute plural projection coordinate systems of the i-th point cloud frame.
In practice, the rotation step may be determined according to the calculation speed and the accuracy of the direction of change, and may be set to a smaller number, for example, 10 degrees if the calculation accuracy is required to be high, and may be set to a larger number, for example, 30 degrees if the calculation speed is required to be high.
In implementation, the projection coordinate systems of each point cloud frame are ordered, and the ordering modes of the projection coordinate systems of all the point cloud frames are the same, so that three coordinate axes of the j-th coordinate system of any two point cloud frames are parallel and have the same direction. Namely, the j coordinate system of the i-th point cloud frame is translated to the centroid of the k-th point cloud frame and then completely coincides with the j coordinate system of the k-th point cloud frame.
Each projection coordinate system of each point cloud frame corresponds to a plurality of projection directions, and one projection coordinate system corresponds to 3 projection directions according to one embodiment of the invention. For example, for the jth coordinate system O of the ith point cloud frame i X j Y j Z j The three projection directions are O i X j Direction of axis, O i Z j Direction of axis and O i Y j The direction of the axis.
Specifically, in step S11, each point cloud frame is projected on a projection direction of a plurality of corresponding projection coordinate systems based on a hierarchical projection method, which includes:
s1121, for a j-th projection coordinate system of a current point cloud frame, forming a point set to be projected by all points of the current point cloud frame; let k=1, l=1;
it should be noted that k represents the kth coordinate axis of the jth projection coordinate system, and L represents the first projection layer of the jth projection coordinate system. Initially, k=1, l=1.
S1122, taking the kth coordinate axis of the jth projection coordinate system as a reference axis;
s1123, sorting each point of the point set to be projected according to the sequence of the values on the reference coordinate axis from large to small, if a projection point exists at the corresponding position of the L layer in the projection direction corresponding to the reference axis, not projecting the point, otherwise, projecting the point at the corresponding position of the L layer in the projection direction corresponding to the reference axis, and deleting the point from the point set to be projected; if the point set to be projected is empty, the projection is finished; otherwise, go to step S1124;
If k=4, l=l+1, set k to 1, and return to step S1122; otherwise, the process returns to step S1122.
In practice, for the j-th coordinate system O of the current point cloud frame (e.g., i-th point cloud frame) i X j Y j Z j And forming a point set to be projected by all points of the current point cloud frame.
Will be the j-th coordinate system O i X j Y j Z j For example, the X-axis is used as the reference axis, in which case the projection direction is O i X j And (3) ordering each point in the point set to be projected according to the sequence of the numerical values of the points on the X axis from large to small in the direction of the axis, so that the point with the large coordinate value is projected preferentially. Sequentially judging whether each point can be projected on O according to the ordered sequence of the points i X j Layer 1 of the projection direction of the axis.
The first ordered point must be projected on layer 1 in the axial direction, which is projected on O i X j Layer 1 in the axial direction, marking this point at O i X j The projection position of the 1 st layer in the axial direction is occupied, and the point is deleted from the point set to be projected. Starting from the second point, it is necessary to determine that this point is at O i X j If the projection position of the 1 st layer in the axial direction is occupied, if so, that is, the existing point is projected at the position, the point is not projected, and remains in the point set to be projected, otherwise, the point is projected at O i X j Layer 1 in the axial direction, marking this point at O i X j The projection position of the 1 st layer in the axial direction is occupied, and the point is deleted from the point set to be projected.
After traversing the point set to be projected once, if the point set to be projected is empty, finishing projection; if the point set to be projected is not empty, i.e. there is a point which is not projected, the j-th coordinate system O i X j Y j Z j For example, the Y-axis is used as the reference axis, and the projection direction is O i Y j And (3) ordering each point in the point set to be projected according to the sequence of the numerical values of the points on the Y axis from large to small in the direction of the axis, so that the point with the large coordinate value is projected preferentially. Sequentially judging whether each point can be projected on O according to the ordered sequence of the points i Y j The layer 1 in the axial direction is specifically determined by the above description. After the point set to be projected is traversed once, if the point set to be projected is empty, the projection is finished, and if the point set to be projected is still not empty, the j coordinate system O is adopted i X j Y j Z j Taking the Z axis of the third coordinate axis as a reference axis, and projecting the points in the point set to be projected in the third projection direction O according to the same process i Z j On layer 1 in the axial direction.
O i X j Y j Z j After all three coordinate axes of the set of points to be projected are used as reference axes for polling once, due to the existence of the shielding relation, points still can be not projected, namely, the set of points to be projected is still not empty, the number of layers is increased by one, namely, L is increased by 1, the points in the set of points to be projected are projected on the 2 nd layer of the three projection directions according to the process, if the points which are not projected still exist, the points in the set of points to be projected are projected on the 3 rd layer of the three projection directions, and the like until the set of points to be projected is empty.
Compared with the prior art, the method has the advantages that the point cloud is projected on a plurality of layers in the projection direction through the layered projection method, so that the shielding relation of the points is considered, and a basis is provided for accurately calculating the change direction of the point cloud frame.
Specifically, in step S13, for each projection direction of each projection coordinate system, the following manner is adopted to calculate the chamfer distance between the current point cloud frame and the previous frame of the current point cloud frame:
for the current projection direction O i X j Searching the current point cloud frame to project in the current projection direction O i X j The points on the first set of points; wherein O is i X j O representing the jth projection coordinate system of the ith point cloud frame i X j The projection direction in which the axis is located.
Searching for the projection of the previous frame of the current point cloud frame in the projection direction O i-1 X j The points on the first set of points form a second set of points; o (O) i-1 X j O representing the j-th projection coordinate system of the i-1-th point cloud frame i-1 X j The projection direction of the shaft;
and calculating the chamfering distances of the first point set and the second point set to obtain the chamfering distances of the current point cloud frame and the previous frame of the current point cloud frame in the current projection direction.
For example, the current point cloud frame is in the jth projection coordinate system O i X j Y j Z j The lower projection is performed by 8 layers, and then the current point cloud frame is taken to be projected on O i X j Points on all 8 layers of directions constitute a first set of points. Projected on the j-th projection coordinate system O in the previous point cloud frame of the current point cloud frame i-1 X j Y j Z j O of (2) i-1 X j Points on all layers of the direction (e.g., 10 layers) constitute a second set of points.
Specifically, the chamfering distance d (S) of the first point set and the second point set is calculated using the following formula 1 ,S 2 )：
Wherein S is 1 A first set of points is represented and,
represent S 1 The number of midpoints S 2 Representing a second set of points, +.>
Represent S 2 Quantity of midpoint->
Represents the distance from point x to point y, +.>
Representing the distance from point y to point x.
The calculated chamfer distance between the first point set and the second point set indicates that the current point cloud frame (i-th point cloud frame) and the previous point cloud frame are in O under the j-th coordinate system i X j The larger the difference in the axial direction, the larger the chamfer distance. The same process is adopted to calculate the current point cloud frame (i-th point cloud frame) and the previous point cloud frame at O i Y j Shaft and O i Z j Differences in the axial direction.
For the current point cloud frame, after the corresponding chamfer distances of each projection direction under all projection coordinate systems are calculated, the projection direction (e.g. O) in which the maximum chamfer distance is located is found i X j In which direction) is used as the first projection direction, since the first projection direction may correspond to the first projection direction (and O i X j Y j Z j O of coordinate system i X j A projection coordinate system with the axes on the same axis), the projection coordinate system in which the first projection direction is located constitutes a candidate coordinate system set in which the first projection direction O is divided i X j And in other projection directions, the projection direction with the largest corresponding chamfering distance is the second projection direction, and the coordinate system of the first projection direction and the second projection direction is the largest change coordinate system from the previous frame of the current point cloud frame to the current point cloud frame.
The first projection direction is the maximum change direction from the previous frame of the current point cloud frame to the current point cloud frame.
And after calculating the maximum change coordinate system of every two adjacent point cloud frames in the three-dimensional point cloud frame sequence, segmenting the three-dimensional point cloud frame sequence. In practice, the point cloud sequence may be segmented by a fixed number of frames, e.g., one segment every 10 frames.
Segmentation can also be performed according to the maximum change direction of the adjacent point cloud frames. The method specifically comprises the following steps:
s211, taking a first point cloud frame of the three-dimensional point cloud frame sequence as a starting point Yun Zhen of the first section; taking the first section as a current section; taking the second point cloud frame as the current point cloud frame;
s212, if the included angle between the maximum change direction of the current point cloud frame and the maximum change direction of the starting point cloud frame of the current section is smaller than a first threshold, adding the current point cloud frame into the current section, otherwise, taking the current point cloud frame as the starting point cloud frame of a new section, taking the new section as the current section, taking a next point cloud frame of the current point cloud frame as the current point cloud frame, and returning to the step S212; and if the next point cloud frame does not exist, ending the segmentation.
In practice, the first threshold may be determined based on computational accuracy requirements, and may be set to 15 degrees, for example.
That is, frames having substantially the same direction of change are one segment, thereby improving the compression rate and improving the accuracy of compressed data. And fusing the maximum change coordinate systems of adjacent point cloud frames of each three-dimensional point cloud frame, so as to facilitate the compression of the point cloud.
Specifically, in step S2, for each segment of three-dimensional point cloud frame, a fused coordinate system is calculated according to the largest change coordinate system of the adjacent point cloud frames, including:
s221, for the current segment point cloud frame, translating all maximum change coordinate systems corresponding to the current segment point cloud frame to the original point which coincides with the original point of the reference coordinate system.
In implementation, translating the maximum change coordinate system of all adjacent point cloud frames in the three-dimensional point cloud frame of the current segment to an origin and a reference coordinate system O 0 X 0 Y 0 Z 0 Origin O of (2) 0 And (5) overlapping.
S222, for each coordinate axis of each maximum change coordinate system, the coordinate axis is assigned to the type corresponding to the coordinate axis of the reference coordinate with the smallest included angle.
For each coordinate axis of each maximum change coordinate system, judging the included angles between each coordinate axis and three coordinate axes of the reference coordinate system, if the current coordinate isO of axis and reference coordinate system 0 X 0 If the shaft clamping angle is minimum, classifying the current coordinate axis into a first class; if O of the current coordinate axis and the reference coordinate system 0 Y 0 If the shaft clamping angle is minimum, classifying the current coordinate axis into a second class; if O of the current coordinate axis and the reference coordinate system 0 Z 0 And if the shaft clamping angle is minimum, classifying the current coordinate axis into a third class.
It should be noted that, at this time, it is only necessary to ensure that three coordinate axes of the current maximum change coordinate system belong to different classes, if there is a possibility that an included angle between a certain coordinate axis of the maximum change coordinate system and two coordinate axes of the reference coordinate system is the same and minimum.
S223, adding the unit vectors of the coordinate axes corresponding to each type to obtain the coordinate axes of the fusion coordinate system corresponding to the type; and obtaining a fusion coordinate system corresponding to the current three-dimensional point cloud frame.
And adding the unit vectors corresponding to the coordinate axes of the same class to obtain the coordinate axes corresponding to the fusion coordinate system. For a fused coordinate system corresponding to an s-th three-dimensional point cloud frame sequence
And (3) representing.
Specifically, in step S2, projecting each point cloud frame of the three-dimensional point cloud frame on the projection direction of the fusion coordinate system to obtain a complete projection image of each point cloud frame on each projection direction, including:
s231, projecting each point of each point cloud frame of the three-dimensional point cloud frame on a projection direction corresponding to the fusion coordinate system by adopting a layered projection method, and obtaining a multi-layer block projection diagram of each point cloud frame on each projection direction corresponding to the fusion coordinate system;
In implementation, for the ith point cloud frame in the s-th three-dimensional point cloud frame sequence, the coordinate system is fused first
Translating to the mass center of the point cloud frame to obtain a fusion projection coordinate system +.>
The corresponding projection direction is +.>
The direction of the axis, < >>
The direction of the axis and->
In the direction of the position.
And translating the fusion coordinate system corresponding to the current segment to obtain a fusion projection coordinate system of each point cloud frame, so that the projection direction of each point cloud frame is the same as the projection direction of the fusion coordinate system corresponding to the current segment.
When in implementation, the ith point cloud frame in the S-th three-dimensional point cloud frame sequence is projected on the fusion projection coordinate system according to the same method in the steps S1121-S1124
And in the corresponding projection directions, obtaining a multi-layer block projection diagram of the ith point cloud frame of the ith three-dimensional point cloud frame sequence in each projection direction. For example, the projection of the ith point cloud frame is divided into 7 layers, then at +.>
Projection direction, & lt & gt>
Projection direction and +.>
The projection directions correspond to 7 partitioned projection maps respectively.
Note that, each pixel of each layer of projection map records three-dimensional coordinate values and attribute indexes of points in the fused projection coordinate system projected on the point cloud frame of the position, and the three-dimensional coordinate values and attribute indexes are expressed as (x, y, z, a), and if no point is projected on the position, the corresponding pixel of the position is (0, 0). In practice, the attribute may be color.
S232, splicing the multi-layer block projection images of each point cloud frame in each projection direction to obtain a complete projection image of each point cloud frame in each projection direction.
After the multi-layer block projection images of each point cloud frame in the current segment three-dimensional point cloud frame sequence in each projection direction are obtained, respectively splicing the multi-layer block projection images in each projection direction for each point cloud frame, such as the ith point cloud frame in the s-th segment three-dimensional point cloud frame sequence, so as to obtain a complete projection image of the ith point cloud frame in the s-th segment three-dimensional point cloud frame sequence on each projection image plane.
In implementation, assuming that the s-th three-dimensional point cloud frame sequence includes 10 frames of point cloud frames, each of the segmented projection images of the 10 point cloud frames has an origin of a corresponding projection coordinate system, a corresponding point in the segmented projection image is taken as the origin, a two-dimensional coordinate system is established, the maximum value of absolute values of coordinate values in the two-dimensional coordinate system is the side length of the segmented projection image, and the maximum side length in all the segmented projection images is L max The dimension of the reference block diagram is 2L max ×2L max For example, if the maximum side length is 100, the dimension of the reference block diagram is 200×200. Expanding each segmented projection map to the size of a reference segmented map, namely, establishing a two-dimensional coordinate system by taking the point corresponding to the original point of a projection coordinate system corresponding to the segmented projection map in the segmented projection map as the original point, wherein the X-axis of the two-dimensional coordinate system is-2L max To 2L max 2L of Y-axis max To 2L max The position without projection point in the range is filled with 0, and the filled partitioned projection graph is obtained.
For the ith point cloud frame, will
The projection direction corresponding to the partitioned projection images are spliced together according to the sequence from the first layer to the last layer to obtain an ith point cloud frame in +.>
A complete projection image in the projection direction; will->
The projection direction corresponding to the partitioned projection images are spliced together according to the sequence from the first layer to the last layer to obtain an ith point cloud frame in +.>
A complete projection image in the projection direction; will->
The projection direction corresponding to the partitioned projection images are spliced together according to the sequence from the first layer to the last layer to obtain an ith point cloud frame in +.>
A complete projection image in the projection direction.
Specifically, in step S2, the compressing the three-dimensional point cloud frame according to the similarity of the complete projection images of the two adjacent point cloud frames in each projection direction includes:
s241, the complete projection images of the first point cloud frame and the last point cloud frame in the three-dimensional point cloud frame of the current section in each projection direction are key frame images;
s242, calculating the structural similarity of the complete projection image of the point cloud frame and the previous point cloud frame in each projection direction for other point cloud frames in the three-dimensional point cloud frame of the current section; if the structural similarity does not exceed a second threshold, taking the complete projection image as a key frame image; all key frame images of the three-dimensional point cloud frame form an image after the three-dimensional point cloud frame is compressed.
For example, for the ith point cloud frame, get it in
Projection direction, & lt & gt>
Projection direction and +.>
The complete projection image of the projection direction is respectively calculated that the ith point cloud frame is in +.>
The complete projection image of the projection direction and the i-1 th point cloud frame are in +.>
Structural similarity of complete projection image in projection direction, i-th point cloud frame +.>
The complete projection image of the projection direction and the i-1 th point cloud frame are in +.>
Structural similarity of complete projection image in projection direction, i-th point cloud frame +.>
The complete projection image of the projection direction and the i-1 th point cloud frame are in +.>
Structural similarity of complete projection images in projection direction.
It is to be noted that,
the fused projection coordinate system is a fused coordinate system +.>
A coordinate system obtained by translating the centroid of the ith point cloud frame to the s-th segment,>
the fused projection coordinate system is a fused coordinate system +.>
A coordinate system obtained by translating the centroid of the i-1 th point cloud frame to the s-th segment, thus, < ->
Projection direction and +.>
The projection directions are the same, namely the fusion coordinate system +.>
Is->
The direction of the axis; />
Projection direction and +.>
The projection directions are the same, namely the fusion coordinate system +.>
Is->
A shaft; />
Projection direction and +.>
The projection directions are the same, namely the fusion coordinate system +. >
Is->
A shaft.
If the ith point cloud frame is in
The complete projection image of the projection direction and the i-1 th point cloud frame are in +.>
The structural similarity of the complete projection image in the projection direction is smaller than a second threshold, namely, the ith point cloud frame and the (i-1) th point cloud frame are in the following
The difference in the projection direction is larger, the ith point cloud frame is at +.>
Taking the complete projection image in the projection direction as a key frame image, otherwise, discarding the ith point cloud frame in +.>
A complete projection image of the projection direction.
Specifically, step S242 calculates the structural similarity of the complete projection image of the point cloud frame and the previous point cloud frame in each projection direction by adopting the following method:
s2421, for the current projection direction, according to the formula
wherein mu k,a Representing an element mean value of a kth channel of a complete projection image of the point cloud frame; mu (mu) k,b Representing an element mean value of a kth channel of a complete projection image of a previous point cloud frame of the point cloud frame; sigma (sigma) k,a Representing the element variance of the kth channel of the complete projection image of the point cloud frame; sigma (sigma) k,b Element variance, sigma, of the kth channel of the complete projection image representing the previous point cloud frame of the point cloud frame k,ab Element covariance of kth channel representing complete projection image of the point cloud frame and complete projection image of previous point cloud frame of the point cloud frame, c 1 And c 2 Is constant.
It should be noted that, each image channel, i.e., each coordinate, and the color index, i.e., four channels, correspond.
S2422, adding the SSIM values of all the image channels to obtain the structural similarity value of the complete projection image of the point cloud frame and the previous point cloud frame in the current projection direction.
And adding the SSIM values obtained by calculating the 4 channels to obtain the structural similarity value of the complete projection image of the point cloud frame and the previous point cloud frame in the current projection direction.
And after obtaining the key frame image of the s-th three-dimensional point cloud, taking the fusion coordinate corresponding to the s-th three-dimensional point cloud frame, the key frame image, the point cloud frame serial number corresponding to each key frame image, the centroid and the projection method as the compressed data of the current point cloud. Because the complete projection image with higher similarity is not used as a key frame image, namely the complete projection image reserved in the direction with smaller change is fewer, the compressed data volume is greatly reduced, the compression rate is greatly improved, more complete projection images in the direction with larger change are reserved as key frames, the compressed distortion is reduced, the reduction degree is higher, the reserved information of the original point cloud frame is higher, and the compression accuracy is higher when the point cloud is restored according to the compressed data.
In practice, decompression from compressed data may be performed in the following manner:
s31, calculating non-key frame images and corresponding point cloud frame numbers among key frame images with the same section sequence number and the same projection direction by adopting a linear difference method;
s32, carrying out three-dimensional point reduction on each point cloud frame sequence number according to the corresponding fusion coordinate system and the corresponding key frame image and/or the non-key frame image to obtain a point cloud corresponding to the point cloud frame sequence number; and the point clouds corresponding to all the point cloud frame serial numbers form a three-dimensional point cloud frame sequence.
Specifically, in step S31, a linear difference method is used to calculate a non-key frame image and a corresponding point cloud frame number between key frame images with the same segment number and the same projection direction, including:
s311, sorting the key frame images with the same segment sequence number and the same projection direction according to the sequence from small to large of the point cloud frame sequence numbers to be used as a group of key frame images;
s312, for two adjacent key frame images in each group of key frame images, if the corresponding point cloud frame sequence numbers are discontinuous, calculating a non-key frame image between the two adjacent key frame images according to the following formula:
wherein si represents the point cloud frame number corresponding to the first key frame image in the two adjacent key frame images, sj represents the point cloud frame number corresponding to the second key frame image in the two adjacent key frame images, I si Image data representing a first key frame image, I sj Image data representing a second key frame image, |I sj -I si I represents the computed image I si And I sj Sk represents the point cloud frame number between si and sj, si < sk < sj, P sk Image data representing a non-key frame image corresponding to the point cloud frame number sk.
Since the fusion coordinate systems corresponding to the point cloud frames of the different segments are different, the non-key frame image (i.e., the image corresponding to the complete projection image discarded in step S332) needs to be obtained according to the key frame image of the same segment. In implementation, first, the key frame images of each projection direction of the s-th segment (the segment sequence number s) are ordered according to the sequence of the point cloud frame sequence numbers from small to small. For example, a fused coordinate system
Is->
The axial projection direction, the point cloud frame number is 12,15. 19 (i.e. 12 th, 15 th, 19 th) point cloud frames, the point cloud frames with the point cloud frame numbers 13, 14, 16, 17, 18 (i.e. 13 th, 14 th, 16 th, 17 th, 18 th) are required to be calculated ∈>
Non-key frame images in the axial projection direction. For the 13 th point cloud frame +.>
The non-key frame image in the axial projection direction needs to be calculated according to the projection images of the 12 th and 15 th point cloud frames.
According to the formula
Calculating 13 th point cloud frame to be +.>
Non-key frame image P in axial projection direction sk . Where si=12, sj=15, sk=13, i si For the 12 th point cloud frame +.>
Key frame image in axial projection direction, I sj For the 15 th point cloud frame +.>
Key frame image in axial projection direction, I sj -I si Indicating that the 12 th point cloud frame is +.>
The key frame image and the 15 th point cloud frame in the axial projection direction are in +.>
Differences in key frame image corresponding elements in the direction of the axis projection (calculated separately on each channel).
After calculating the non-key frame image, carrying out three-dimensional point reduction on the corresponding key frame image and/or the non-key frame image according to the corresponding fusion coordinate system for each point cloud frame sequence number to obtain a point cloud corresponding to the point cloud frame sequence number, wherein the method comprises the following steps:
and converting the point corresponding to each pixel in each key frame image and/or non-key frame image corresponding to the current point cloud frame number into the reference coordinate system according to the relation between the fusion projection coordinate system and the reference coordinate system to obtain the point cloud corresponding to the current point cloud frame number.
For each point cloud frame, the projected image in each projection direction may be a key frame image or a non-key frame image calculated according to the above steps, and the corresponding projection coordinate system is a coordinate system (i.e. a fused projection coordinate system) obtained after the fused coordinate system corresponding to the end translates to the centroid of the point cloud frame, and the coordinate information recorded in the key frame image and the non-key frame image is the coordinate information of the key frame image and the non-key frame image under the corresponding fused projection coordinate, so that the point cloud of the point cloud frame under the reference coordinate system can be obtained by restoring according to the relation between the fused projection coordinate system and the reference coordinate system after the pixel point information of the key frame image and/or the non-key frame in all projection directions corresponding to the point cloud frame is extracted. And forming a three-dimensional point cloud frame sequence by the point clouds corresponding to all the point cloud frame sequence numbers, and obtaining the decompressed point cloud sequence.
Those skilled in the art will appreciate that all or part of the flow of the methods of the embodiments described above may be accomplished by way of a computer program to instruct associated hardware, where the program may be stored on a computer readable storage medium. Wherein the computer readable storage medium is a magnetic disk, an optical disk, a read-only memory or a random access memory, etc.
The present invention is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present invention are intended to be included in the scope of the present invention.
Claims (10)
1. The dynamic human-computer interaction point cloud compression method is characterized by comprising the following steps of:
calculating a maximum change coordinate system of adjacent point cloud frames in the human-computer interaction three-dimensional point cloud frame sequence based on a multi-layer projection method;
segmenting a three-dimensional point cloud frame sequence; for each section of three-dimensional point cloud frame, calculating a fusion coordinate system according to the maximum change coordinate system of the adjacent point cloud frames; and projecting each point cloud frame of the three-dimensional point cloud frame on the projection direction of the fusion coordinate system to obtain a complete projection image of each point cloud frame on each projection direction, and compressing the three-dimensional point cloud frame according to the similarity of the complete projection images of two adjacent point cloud frames on each projection direction.
2. The method of claim 1, wherein calculating a maximum change coordinate system of adjacent point cloud frames in a three-dimensional point cloud frame sequence based on a multi-layer projection algorithm comprises:
s11, for each point cloud frame in the three-dimensional point cloud frame sequence, establishing a plurality of projection coordinate systems by taking the mass center of each point cloud frame as an origin; respectively projecting each point cloud frame on the projection directions of a plurality of corresponding projection coordinate systems based on a layered projection method;
s12, taking a second point cloud frame in the point cloud frame sequence as a current point cloud frame;
s13, calculating the chamfering distance between the current point cloud frame and the previous frame of the current point cloud frame for each projection direction of each projection coordinate system corresponding to the current point cloud frame;
s14, the projection direction corresponding to the maximum chamfering distance is a first projection direction, and among other projection directions except the first projection direction in all projection coordinate systems in which the first projection direction is located, the projection direction with the maximum chamfering distance is a second projection direction, and the coordinate systems in which the first projection direction and the second projection direction are located are the largest change coordinate systems from the previous frame of the current point cloud frame to the current point cloud frame; and returning to the step S13 by taking the next point cloud frame as the current frame until all the point cloud frames are traversed.
3. The method of dynamic man-machine interaction point cloud compression according to claim 2, wherein projecting each point cloud frame on a projection direction of a corresponding plurality of projection coordinate systems based on a hierarchical projection method, respectively, comprises:
s1121, for a j-th projection coordinate system of a current point cloud frame, forming a point set to be projected by all points of the current point cloud frame; let k=1, l=1;
s1122, taking the kth coordinate axis of the jth projection coordinate system as a reference axis;
s1123, sorting each point of the point set to be projected according to the sequence of the values on the reference coordinate axis from large to small, if a projection point exists at the corresponding position of the L layer in the projection direction corresponding to the reference axis, not projecting the point, otherwise, projecting the point at the corresponding position of the L layer in the projection direction corresponding to the reference axis, and deleting the point from the point set to be projected; if the point set to be projected is empty, the projection is finished; otherwise, go to step S1124;
if k=4, l=l+1, set k to 1, and return to step S1122; otherwise, the process returns to step S1122.
4. The dynamic man-machine interaction point cloud compression method according to claim 2, wherein for each projection direction of each projection coordinate system, a chamfering distance of a current point cloud frame and a frame previous to the current point cloud frame is calculated by:
For the current projection direction O i X j Searching the current point cloud frame to project in the current projection direction O i X j The points on the first set of points; wherein O is i X j O representing the jth projection coordinate system of the ith point cloud frame i X j The projection direction of the shaft;
searching for the projection of the previous frame of the current point cloud frame in the projection direction O i-1 X j The points on the first set of points form a second set of points; o (O) i-1 X j O representing the j-th projection coordinate system of the i-1-th point cloud frame i-1 X j The projection direction of the axis;
and calculating the chamfering distances of the first point set and the second point set to obtain the chamfering distances of the current point cloud frame and the previous frame of the current point cloud frame in the current projection direction.
5. The method of dynamic man-machine interaction point cloud compression according to claim 4, wherein the chamfer distance d (S 1 ,S 2 )：
Wherein S is 1 A first set of points is represented and,
represent S 1 The number of midpoints S 2 Representing a second set of points, +.>
Represent S 2 Quantity of midpoint->
Represents the distance from point x to point y, +.>
Representing the distance from point y to point x.
6. The method of dynamic human-computer interaction point cloud compression according to claim 2, wherein for each point cloud frame in the three-dimensional point cloud frame sequence, establishing a plurality of projection coordinate systems with the centroid as the origin comprises:
Establishing a coordinate system which is parallel to a coordinate axis of a reference coordinate system and has the same direction by taking the centroid of the current point cloud frame as a coordinate origin as a first coordinate;
fixing each coordinate axis in the first coordinate system, and rotating two coordinate axes except the fixed coordinate axes around the fixed coordinate axes according to the rotation step length to obtain a plurality of coordinate systems which are the same as the X axis, the Y axis or the Z axis of the first projection coordinate system;
the first coordinate system and the same coordinate systems as the X-axis, Y-axis or Z-axis thereof form a plurality of projection coordinate systems of the current point cloud frame.
7. The method of claim 1, wherein projecting each point cloud frame of the segment of three-dimensional point cloud frames in a projection direction of the fusion coordinate system to obtain a complete projection image of each point cloud frame in each projection direction, comprises:
projecting each point of each point cloud frame of the three-dimensional point cloud frame on a projection direction corresponding to the fusion coordinate system by adopting a layered projection method to obtain a multi-layer block projection diagram of each point cloud frame on each projection direction corresponding to the fusion coordinate system;
and splicing the multi-layer block projection images of each point cloud frame in each projection direction to obtain a complete projection image of each point cloud frame in each projection direction.
8. The method for compressing a point cloud frame of dynamic human-computer interaction according to claim 1, wherein compressing the segment of three-dimensional point cloud frame according to the similarity of the complete projection images of two adjacent point cloud frames in each projection direction comprises:
the complete projection images of the first point cloud frame and the last point cloud frame in each projection direction in the three-dimensional point cloud frame of the current section are key frame images;
for other point cloud frames in the current three-dimensional point cloud frame, calculating the structural similarity of the complete projection image of the point cloud frame and the previous point cloud frame in each projection direction; if the structural similarity does not exceed a second threshold, taking the complete projection image as a key frame image; all key frame images of the three-dimensional point cloud frame form an image after the three-dimensional point cloud frame is compressed.
9. The method of claim 8, wherein the structural similarity of the complete projected image of the point cloud frame and its previous point cloud frame in each projection direction is calculated by:
for the current projection direction, according to the formula
Respectively calculating SSIM values of the complete projection images of the point cloud frame and the previous point cloud frame on a kth image channel;
Wherein mu k,a Representing an element mean value of a kth channel of a complete projection image of the point cloud frame; mu (mu) k,b Representing an element mean value of a kth channel of a complete projection image of a previous point cloud frame of the point cloud frame; sigma (sigma) k,a Representing the element variance of the kth channel of the complete projection image of the point cloud frame; sigma (sigma) k,b Element variance, sigma, of the kth channel of the complete projection image representing the previous point cloud frame of the point cloud frame k,ab Element covariance of kth channel representing complete projection image of the point cloud frame and complete projection image of previous point cloud frame of the point cloud frame, c 1 And c 2 Is a constant;
and adding the SSIM values of all the image channels to obtain the structural similarity value of the complete projection image of the point cloud frame and the previous point cloud frame in the current projection direction.
10. The method of claim 1, wherein for each segment of three-dimensional point cloud frame, calculating a fusion coordinate system according to a maximum change coordinate system of an adjacent point cloud frame, comprising:
for the current segment point cloud frame, translating all maximum change coordinate systems corresponding to the current segment point cloud frame to the origin point which coincides with the origin point of the reference coordinate system;
for each coordinate axis of each maximum change coordinate system, the coordinate axis is assigned to the type corresponding to the coordinate axis of the reference coordinate with the smallest included angle;
Adding the unit vectors of the coordinate axes corresponding to each type to obtain the coordinate axes of the fusion coordinate system corresponding to the type; and obtaining a fusion coordinate system corresponding to the current three-dimensional point cloud frame.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310206858.2A CN116192904A (en) | 2023-03-03 | 2023-03-03 | Dynamic human-computer interaction point cloud compression method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310206858.2A CN116192904A (en) | 2023-03-03 | 2023-03-03 | Dynamic human-computer interaction point cloud compression method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116192904A true CN116192904A (en) | 2023-05-30 |
Family
ID=86448671
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310206858.2A Pending CN116192904A (en) | 2023-03-03 | 2023-03-03 | Dynamic human-computer interaction point cloud compression method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116192904A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117152164A (en) * | 2023-11-01 | 2023-12-01 | 武汉精一微仪器有限公司 | Point cloud data layering method and device for transparent multilayer material and electronic equipment |
-
2023
- 2023-03-03 CN CN202310206858.2A patent/CN116192904A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117152164A (en) * | 2023-11-01 | 2023-12-01 | 武汉精一微仪器有限公司 | Point cloud data layering method and device for transparent multilayer material and electronic equipment |
| CN117152164B (en) * | 2023-11-01 | 2024-02-09 | 武汉精一微仪器有限公司 | Point cloud data layering method and device for transparent multilayer material and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210021869A1 (en) | Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device | |
| EP3794552A1 (en) | Motion compensation of geometry information | |
| CN110688905B (en) | Three-dimensional object detection and tracking method based on key frame | |
| CN113096198B (en) | Bidirectional point cloud attribute prediction compression method, device, equipment and medium | |
| US20210004993A1 (en) | Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device | |
| CN114120361B (en) | Crowd counting and positioning method based on coding and decoding structure | |
| CN116192904A (en) | Dynamic human-computer interaction point cloud compression method | |
| CN112381813B (en) | Panoramic view visual saliency detection method based on graph convolution neural network | |
| Tohidi et al. | Dynamic point cloud compression with cross-sectional approach | |
| Ying et al. | Pushing point cloud compression to the edge | |
| CN116320441A (en) | Dynamic point cloud compression and decompression system for three-dimensional object | |
| CN116260952A (en) | Point cloud frame sequence compression method based on maximum change direction | |
| CN113096199B (en) | Point cloud attribute prediction method, device and medium based on Morton code | |
| JP2006527945A (en) | Representation method of picture sequence using 3D model, corresponding signal, and corresponding apparatus | |
| JP2002190020A (en) | Method and device for image effect | |
| KR100923946B1 (en) | Method and apparatus for patch-based texture image preprocessing for efficient texture image compression | |
| US11170533B1 (en) | Method for compressing image data having depth information | |
| WO2022257143A1 (en) | Intra-frame prediction method and apparatus, encoding method and apparatus, decoding method and apparatus, codec, device and medium | |
| Li et al. | Point Cloud Compression: Technologies and Standardization | |
| WO2023193534A1 (en) | Methods and apparatus for coding presence flag for point cloud, and data stream including presence flag | |
| Jenco | Virtual LiDAR error models in point cloud compression | |
| Song et al. | Real‐Time Terrain Storage Generation from Multiple Sensors towards Mobile Robot Operation Interface | |
| Shi et al. | Volumetric Video Compression Through Neural-based Representation | |
| WO2023193533A1 (en) | Apparatus for coding vertex position for point cloud, and data stream including vertex position | |
| Rasmuson | Techniques for Fast and High-Quality 3D Reconstruction of General Scenes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |