WO2024007253A1 - Point cloud rate-distortion optimization method, attribute compression method and apparatus, and storage medium - Google Patents

Point cloud rate-distortion optimization method, attribute compression method and apparatus, and storage medium Download PDF

Info

Publication number
WO2024007253A1
WO2024007253A1 PCT/CN2022/104391 CN2022104391W WO2024007253A1 WO 2024007253 A1 WO2024007253 A1 WO 2024007253A1 CN 2022104391 W CN2022104391 W CN 2022104391W WO 2024007253 A1 WO2024007253 A1 WO 2024007253A1
Authority
WO
WIPO (PCT)
Prior art keywords
quantized residual
residual value
encoded
attribute
value
Prior art date
Application number
PCT/CN2022/104391
Other languages
French (fr)
Chinese (zh)
Inventor
魏红莲
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Priority to PCT/CN2022/104391 priority Critical patent/WO2024007253A1/en
Publication of WO2024007253A1 publication Critical patent/WO2024007253A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation

Definitions

  • Embodiments of the present disclosure relate to, but are not limited to, video technology, and more specifically, to a point cloud rate distortion optimization method and attribute compression method, device and storage medium.
  • a point cloud is a set of discrete points randomly distributed in space that expresses the spatial structure and surface properties of a three-dimensional object or scene.
  • Figure 1A and Figure 1B respectively show an exemplary three-dimensional point cloud image and a local enlargement. You can See that the point cloud surface is composed of densely distributed points.
  • Point clouds can flexibly and conveniently express the spatial structure and surface properties of three-dimensional objects or scenes, and because point clouds can be obtained directly by sampling real objects, they can provide a strong sense of reality while ensuring accuracy, so they are widely used.
  • the scope includes virtual reality games, computer-aided design, geographical information systems, automatic navigation systems, digital cultural heritage, free-viewpoint broadcasting, three-dimensional immersive telepresence, three-dimensional reconstruction of biological tissues and organs, etc. Changes in the way of obtaining point cloud data have made it possible to obtain large amounts of point cloud data. However, with the growth of application requirements, the processing of massive 3D point cloud data encounters storage space and transmission bandwidth limitations, and more effective compression methods need to be provided. .
  • An embodiment of the present disclosure provides a point cloud rate distortion optimization method, including:
  • the calculated first quantized residual value or a specified quantized residual value is used as the final quantized residual value of the current point to be encoded;
  • Entropy coding is performed on the final quantized residual value of the current point to be encoded.
  • An embodiment of the present disclosure also provides a point cloud attribute compression method, including:
  • the point cloud rate distortion optimization method determines the final quantized residual value of the current point to be encoded and performs entropy coding on the final quantized residual value.
  • An embodiment of the present disclosure also provides a point cloud attribute code stream, wherein the point cloud attribute code stream is generated according to the point cloud attribute compression method described in any embodiment of the present disclosure.
  • An embodiment of the present disclosure also provides a point cloud rate distortion optimization device, which includes a processor and a memory storing a computer program.
  • a processor executes the computer program, it can implement the method described in any embodiment of the present disclosure.
  • An embodiment of the present disclosure also provides a point cloud compression device, which includes a processor and a memory storing a computer program, wherein when the processor executes the computer program, it can implement the method described in any embodiment of the present disclosure.
  • An embodiment of the present disclosure also provides a non-transitory computer-readable storage medium.
  • the computer-readable storage medium stores a computer program. When executed by a processor, the computer program can implement any implementation of the present disclosure.
  • the point cloud rate distortion optimization method described in the example, or the point cloud attribute compression method described in any embodiment of the present disclosure can be implemented.
  • Figures 1A and 1B are an exemplary three-dimensional point cloud image and a partial enlargement respectively;
  • Figure 2A is a schematic diagram of the encoding framework of the point cloud compression platform of the first embodiment
  • Figure 2B is a schematic diagram of the decoding framework of the point cloud compression platform of the first embodiment
  • Figure 3 is a flow chart of a point cloud rate distortion optimization method according to an embodiment of the present disclosure
  • Figure 4 is a flow chart of a point cloud attribute compression method according to an embodiment of the present disclosure
  • Figure 5 is a schematic diagram of a point cloud rate distortion optimization device according to an embodiment of the present disclosure.
  • the words “exemplary” or “such as” are used to mean an example, illustration, or explanation. Any embodiment described in this disclosure as “exemplary” or “such as” is not intended to be construed as preferred or advantageous over other embodiments.
  • "And/or” in this article is a description of the relationship between associated objects, indicating that there can be three relationships, for example, A and/or B, which can mean: A exists alone, A and B exist simultaneously, and they exist alone. B these three situations.
  • "Plural” means two or more than two.
  • words such as “first” and “second” are used to distinguish the same or similar items with basically the same functions and effects. Those skilled in the art can understand that words such as “first” and “second” do not limit the number and execution order, and words such as “first” and “second” do not limit the number and execution order.
  • Two-dimensional images have information expressed in each pixel and are distributed regularly, so there is no need to record additional position information; however, the distribution of points in the point cloud in the three-dimensional space is random and irregular, so each point needs to be recorded Only the position of the point in space can completely express a point cloud.
  • each point in the collection process has corresponding attribute information, usually an RGB color value, and the color value reflects the color of the object; for point clouds, in addition to color, the attribute information corresponding to each point The more common one is reflectance.
  • the reflectance value reflects the surface material of the object.
  • point cloud data usually includes geometric information (x, y, z) composed of three-dimensional position information, attribute information composed of three-dimensional color information (r, g, b), and one-dimensional reflectance information (r).
  • the corresponding point cloud data storage format consists of a file header information part and a data part.
  • the header information includes the data format, data representation type, total point cloud points, and the content represented by the point cloud.
  • the point cloud is in ".ply" format and is represented by ASCII code.
  • the total number of points is 207242, and each point has three-dimensional position information xyz and three-dimensional color information RGB.
  • Point cloud collection mainly includes computer generation, 3D laser scanning, 3D photogrammetry, etc.
  • Computers can generate point clouds of virtual three-dimensional objects and scenes; 3D laser scanning can obtain point clouds of static real-world three-dimensional objects or scenes, and can obtain millions of point clouds per second; 3D photogrammetry can obtain dynamic real-world three-dimensional objects or scenes Point clouds can obtain tens of millions of point clouds per second.
  • point cloud compression has become a key issue in the development of point cloud applications.
  • Point cloud compression algorithms include two solutions developed by the International Moving Picture Experts Group (MPEG), namely video-based point cloud compression (V-PCC) and geometry-based point cloud compression.
  • MPEG Motion Picture Experts Group
  • V-PCC video-based point cloud compression
  • G-PCC Geometry-based Point Cloud Compression
  • geometric compression in G-PCC is mainly realized through octree model and/or triangular surface model.
  • V-PCC is mainly implemented through three-dimensional to two-dimensional projection and video compression.
  • PCC of AVS Audio Video coding Standard
  • Point cloud compression generally adopts the method of separately compressing point cloud geometric information and attribute information.
  • the point cloud geometric information is first encoded in the geometry encoder, and then the reconstructed geometric information is input into the attribute encoder as additional information, and the auxiliary point Compression of cloud attributes;
  • the point cloud geometric information is first decoded in the geometry decoder, and then the decoded geometric information is input into the attribute decoder as additional information to assist in the decompression of point cloud attributes.
  • FIG. 2A shows a schematic diagram of an exemplary point cloud encoding framework
  • FIG. 2B shows a schematic diagram of an exemplary point cloud decoding framework.
  • the encoding end of an exemplary point cloud compression reference platform includes a geometry encoder and an attribute encoder.
  • the geometry encoder includes a coordinate translation module 11 , a coordinate quantization module 13 , an octree building module 15 , a geometry entropy encoder 17 and a geometry reconstruction module 19 .
  • the coordinate translation module 11 and the coordinate quantization module 13 preprocess the original geometric information of the point cloud.
  • the octree building module 15 and the geometric entropy encoder 17 encode the preprocessed geometric information to obtain a geometric code stream. in:
  • the coordinate translation module 11 is configured to normalize the geometric origin of the original geometric information to the minimum value position in the point cloud space through coordinate translation.
  • the coordinate quantization module 13 is configured to convert the geometric information output by the coordinate translation module 11 from floating point numbers into integers through coordinate quantization to facilitate subsequent regularization processing.
  • the regularized geometric information is output to the octree building module 15 and the attribute encoder. .
  • the octree building module 15 is configured to geometrically encode regularized geometric information, use the octree structure to recursively divide the point cloud space, divide the current node into eight sub-blocks of the same size each time, and judge each sub-block.
  • the occupied codeword status of the block When the sub-block does not contain point cloud points, it is recorded as empty, otherwise it is recorded as non-empty.
  • the non-empty sub-block is used as the current node to continue dividing, and all blocks are recorded in the last layer of recursive division.
  • the codeword information is occupied and encoded; the geometric information expressed through the octree structure is output to the geometric entropy encoder 17 and the geometric reconstruction module 19.
  • the geometric entropy encoder 17 is configured to entropy encode the geometric information expressed by the octree structure to form a geometric code stream.
  • the geometric reconstruction module 19 is configured to perform geometric reconstruction processing in the geometry encoder, and the reconstructed geometric information is output to the attribute encoder as additional information.
  • the attribute encoder includes a color space transformation module 21 , an attribute recoloring module 23 , an attribute prediction module 25 , a quantization module 27 and an attribute entropy encoding module 29 .
  • the color space transformation module 21 and the attribute recoloring module 23 preprocess the original attribute information.
  • Other modules perform attribute encoding on the preprocessed attribute information to obtain an attribute code stream. in:
  • the color space conversion module 21 is configured to perform color space conversion when the processed original attribute information is a color value in RGB format, and convert it into a YUV color space that is more in line with the visual characteristics of the human eye.
  • the attribute recoloring module 23 is set to reassign the attribute value to each point after geometric encoding based on the geometric information output by the geometry encoder. This process is called attribute recoloring to avoid the original geometric information and the geometric information after geometric encoding. The difference between properties.
  • the attribute prediction module 25 is configured to reorder the point cloud.
  • the reordering method can be Hilberton code, Morton code, etc., and the traversal order of the corresponding attribute encoding is Morton order, Hilbert order, etc., attribute prediction
  • the method can be multi-point prediction based on neighboring points or single-point prediction. During multi-point prediction, first find multiple points closest to the current point to be encoded as the prediction reference points of the current point to be encoded, and weight the attribute reconstruction values of the multiple prediction reference points as the attribute prediction values of the current point to be encoded.
  • one point can be traced back from the current point to be encoded as the prediction reference point of the current point to be encoded, and the attribute reconstruction value of the prediction reference point can be used as the attribute prediction value of the current point to be encoded.
  • the difference between the attribute value of the current point to be encoded and the predicted attribute value is the attribute residual value of the current point to be encoded.
  • the quantization module 27 is configured to perform quantization processing on the attribute residual value of the point to obtain the quantized residual value of the attribute of the point.
  • the attribute entropy encoding module 29 is configured to entropy encode the quantized residual value of the attribute of the point to form an attribute code stream.
  • the decoding end of the PCRM platform includes a geometry decoder and an attribute decoder.
  • the geometry decoder includes a geometric entropy decoder 31 , an octree reconstruction module 33 , a coordinate inverse quantization module 35 and a coordinate inverse translation module 37 . in,
  • the geometric entropy decoder 31 is configured to entropy decode the geometric code stream to obtain the geometric information of each node.
  • the octree reconstruction module 33 is configured to construct an octree structure based on the geometric information of each node in the same manner as geometric encoding, and reconstruct the coordinate-transformed geometric information expressed through the octree structure in combination with the decoded geometry.
  • the coordinate inverse quantization module 35 is configured to perform coordinate inverse quantization on the geometric information output by the octree reconstruction module 33.
  • the geometric information after coordinate inverse quantization is output to the coordinate inverse translation module 37, and is output to the attribute decoder as additional information.
  • the coordinate inverse translation module 37 is configured to perform coordinate inverse translation on the geometric information output by the coordinate inverse quantization module 35 to obtain decoded geometric information.
  • the attribute decoder includes an attribute entropy decoder 41, an inverse quantization module 43, an attribute prediction module 45 and a color space inverse transform module 47. in:
  • the attribute entropy decoder 41 is configured to perform entropy decoding on the attribute code stream to obtain the quantized residual value of the attribute of the point.
  • the inverse quantization module 43 is configured to perform inverse quantization processing on the quantized residual value to obtain the attribute residual value of the point.
  • the attribute prediction module 45 is configured to construct Morton order, Hilbert order, etc. in the same way as attribute encoding based on the geometric information output by the geometry decoder, obtain the attribute prediction value of the current point to be decoded, and then convert the current point to be decoded The attribute prediction value is added to the attribute residual value, and the attribute value of the current point to be decoded is restored to obtain the attribute reconstruction value, such as YUV color value, reflectance value, etc.
  • the color space inverse transformation module is configured to perform inverse color space transformation on the attribute information output by the attribute prediction module 45 to obtain decoded attribute information.
  • the prediction branch When compressing, encoding and decoding point clouds based on the above encoding framework and decoding framework, there are two technical routes to distinguish based on the algorithm used for attribute compression.
  • One is called the prediction branch, and the other is called the transformation branch.
  • the attribute encoding and decoding method used by the prediction branch is described above.
  • the difference between the attribute encoding and decoding method used in the transformation branch and the prediction branch is that after obtaining the attribute residual value of the point through attribute prediction, the attribute residual value of multiple points (such as 4 points) is first transformed (the transformation algorithm can be Wavelet transform or k-element discrete cosine transform (DCT: Discrete Cosine Transform), etc.), and then quantize the transformed attribute residual value to obtain the quantized residual value of the attribute.
  • DCT Discrete Cosine Transform
  • the prediction branch performs attribute prediction based on the attribute residual value obtained by inverse quantization, and the transformation branch first performs attribute prediction on the multiple attribute residual values obtained by inverse quantization.
  • the difference value is inversely transformed, and then attribute prediction is performed based on the attribute residual value obtained by the inverse transformation.
  • the attributes of each point are encoded and decoded according to Hilbert order, where the attribute encoding process includes:
  • Entropy coding is performed on the quantized residual value of the attribute of the current point to be encoded to generate an attribute code stream.
  • the encoding end performs inverse quantization processing on the quantized residual value of the current point to be encoded to obtain the reconstructed residual value of the current point to be encoded.
  • the reconstructed residual value of the current point to be encoded is added to the attribute prediction value to obtain The attribute reconstruction value of the current point to be encoded.
  • the attribute reconstruction value of the current point to be encoded can be used to calculate the attribute prediction value of subsequent points.
  • the attribute decoding process includes:
  • the decoding end obtains the attribute prediction value A′ j of the current point to be decoded, and decodes it from the code stream to obtain the quantized residual value (signed) of the current point to be decoded;
  • the quantized residual value used is obtained by quantizing the attribute residual value.
  • the quantized residual value is written into the code stream and affects the code rate.
  • it is processed through inverse quantization.
  • the attribute reconstruction value is then generated, thereby causing distortion between the attribute reconstruction value and the original attribute value.
  • directly encoding the quantized residual value does not necessarily bring the best performance.
  • a rate-distortion optimization algorithm can be introduced to balance the cost between code rate and distortion and improve coding performance.
  • An embodiment of the present disclosure provides a point cloud rate distortion optimization method, which can be used in the quantization module of the above-mentioned coding framework. As shown in Figure 3, the method includes:
  • Step 110 Calculate the first quantized residual value of the attribute of the current point to be encoded
  • the calculated quantized residual value is called the first quantized residual value to distinguish it from other quantized residual values such as the specified quantized residual value and the final quantized residual value.
  • Step 120 According to the rate distortion cost of encoding the first quantized residual value and the rate distortion cost of encoding the specified second quantized residual value, convert the first quantized residual value and the second quantized residual value into One value is determined as the final quantized residual value of the current point to be encoded;
  • the specified quantized residual value should be different from the first quantized residual value.
  • the rate distortion penalty may also be called rate distortion overhead.
  • Step 130 Perform entropy coding on the final quantized residual value of the current point to be encoded.
  • entropy coding is not performed directly. Instead, the first quantized residual value or a specified second quantized residual is selected based on the rate-distortion cost. The value is used as the final quantized residual value of the current point to be encoded, and then entropy coding is performed on the final quantized residual value of the current point to be encoded. For example, if the calculated first quantized residual value is 1, and the rate distortion cost of encoding a specified quantized residual value such as 0 is smaller, the specified quantized residual value is used instead of the first quantized residual value for entropy coding, This results in better encoding performance overall.
  • the specified second quantized residual value is 0. Use 0 to replace the non-zero first quantized residual value as the final quantized residual value.
  • the codeword is the smallest.
  • the reconstructed residual value is also 0, which is equivalent to Using the attribute prediction value of the current point to be encoded directly as the attribute reconstruction value will bring certain distortion.
  • the rate-distortion cost can weigh the gain and distortion in code rate to determine whether to use 0 instead of the calculated first quantized residual value for entropy coding.
  • the specified second quantized residual value includes 0 and a non-zero integer value, and the non-zero integer value includes any one or more of the following values: 1, -1, 2, -2.
  • the rate distortion cost of encoding the first quantized residual value can be compared with the rate distortion cost of encoding multiple specified second quantized residual values, and a quantized one can be selected from them.
  • the residual value is the final quantized residual value of the current point to be encoded.
  • the rate distortion cost of encoding the first quantized residual value 1 can be calculated
  • the rate distortion cost of encoding the specified quantized residual value 0 can be calculated
  • the encoding specified quantization residual can be calculated
  • the rate distortion cost of value 2 the value with the smallest corresponding rate distortion cost is used as the final quantized residual value of the current point to be encoded.
  • non-zero integer values can also include more values, such as 3, -3, 4, -4, and so on.
  • the method further includes: when the first quantized residual value is 0, using the first quantized residual value as the final quantized residual value of the current point to be encoded. .
  • the rate distortion cost of using the first quantized residual value is usually the smallest, so the first quantized residual value is directly used as the final quantized residual value.
  • the attribute residual value when compressing and encoding the point cloud, the attribute residual value is not transformed, but the attribute residual value of a current point to be encoded is quantized to obtain the attribute residual value of the current point to be encoded.
  • the first quantized residual value when compressing and encoding the point cloud, the attribute residual value is not transformed, but the attribute residual value of a current point to be encoded is quantized to obtain the attribute residual value of the current point to be encoded. The first quantized residual value.
  • the first quantized residual value and the second quantized residual value are One value is determined as the final quantized residual value of the current point to be encoded, including:
  • the quantized residual value encoded by the encoding mode with the smaller rate distortion cost among the first encoding mode and the second encoding mode is used as the final quantized residual value of the current point to be encoded.
  • the rate distortion cost of using two encoding modes can be calculated.
  • One is the rate distortion cost of encoding 0.
  • one is to encode the rate distortion cost of the first quantized residual value, and select the encoding mode with the smallest rate distortion cost to encode the quantized residual value as the final quantized residual value.
  • the specified second quantized residual value includes 0 and non-zero integer values
  • there may be multiple second encoding modes and one of the second encoding modes is a quantized residual specified for the current point to be encoded.
  • the difference value is 0, and the quantization residual values specified by other second encoding modes for the current point to be encoded are non-zero integer values.
  • an absolute value smaller than the absolute value of the first quantized residual value and a sign that is the same as the first quantized residual value may be selected from the value set of the second quantized residual value.
  • each selected non-zero integer value can be used as a quantized residual value specified by the second encoding mode.
  • N there can be N+1 second coding modes, including a second coding mode that specifies 0 as the quantized residual value, and N one non-zero value to be selected
  • the first quantized residual value is 2.
  • the rate distortion cost of the two second coding modes encoding 0 and 1 and the rate distortion cost of the first encoding mode encoding 2 can be calculated, and the encoding mode with the smallest rate distortion cost can be calculated.
  • the encoded value is used as the final quantized residual value of the current point to be encoded. Considering the distortion and bit rate, you do not need to try values above 3 or -1, -2 and other values at this time.
  • the quantized residual value encoded in the second encoding mode may also be specified as a value whose absolute value is 1 smaller than the first quantized residual value and whose sign is the same as the first optimized residual value to control distortion. , at this time it is only necessary to compare the rate distortion cost of a second coding mode and a first coding mode.
  • the rate distortion cost of each coding mode used by the current point to be encoded is calculated according to the following formula:
  • lambda is a set constant
  • R is the codeword size required to encode the quantized residual value in this encoding mode
  • D is the attribute reconstruction value and attribute value of the current point to be encoded when encoding in this encoding mode. difference.
  • the size of R is related to the specific entropy coding algorithm used.
  • D can use attributes to reconstruct the difference between the value and the attribute value, such as the absolute value of the difference but is not limited to this.
  • the value of lambda is between 0 and 1, for example, it can be set to 0.5.
  • the user can set it according to the specific requirements for bit rate and distortion. If the requirement for bit rate is higher, the value can be set to 0.5.
  • Lambda is set relatively large to increase the weight of codeword size. On the contrary, if the distortion is required to be small, lambda can be set relatively small to reduce the weight of codeword size.
  • the attribute residual values are transformed when compressing and encoding the point cloud. Because when transforming and inverse transforming the attribute residual values of the point cloud points, multiple points are composed into one Group, the attribute residual values of multiple points in the group are transformed together (such as wavelet transform or DCT transform) and inverse transform. In this case, after replacing the first quantized residual value calculated at a single point with a specified quantized residual value, during the inverse transformation, it will affect the reconstructed residual values obtained by inverse transformation of other points in the same group, and thus Affects the attribute reconstruction values and attribute distortion of other points. Therefore, the final quantized residual value determined based on the minimum usage rate distortion cost of a single point is not necessarily the overall optimal one.
  • This embodiment proposes a rate-distortion optimization method based on a set of current points to be decoded for the transform branch, where:
  • the calculation of the first quantized residual value of the attribute of the current point to be encoded includes: using a plurality of current points to be encoded D 1 , D 2 ,..., D N as a group to jointly transform the attribute residual value, and perform the transformation
  • the final attribute residual values are quantized respectively, and the first quantized residual values of D 1 , D 2 ,..., D N are obtained;
  • one of the first quantized residual value and the second quantized residual value is The value is determined as the final quantized residual value of the current point to be encoded, including:
  • a set of quantized residual values specified for D 1 , D 2 ,..., D N in a second coding mode are composed of the same values, that is, D 1 , D 2 ,... ,D N uniformly specifies a value.
  • a quantized residual value is assigned to D 1 , D 2 , ..., D N respectively within a second encoding mode, and a quantized residual value is assigned to D 1 , D 2 , ..., D N respectively.
  • the method of the previous embodiment can be used, such as specifying it as 0, or specifying a non-zero integer value.
  • determining the encoding mode with the optimal rate-distortion cost among the first encoding mode and the second encoding mode includes:
  • the quantized residual values of D 1 , D 2 ,..., D N in the first encoding mode are the first quantized residual values of D 1 , D 2 ,..., D N respectively, D 1 , D 2 ,... , the quantized residual value of D N in the second encoding mode is the quantized residual value specified uniformly or separately for D 1 , D 2 ,..., D N in this mode.
  • the second coding mode that satisfies the condition is determined to be the coding mode with the optimal rate-distortion cost.
  • the second coding mode with the smallest sum of Cost 1 , Cost 2 ,..., Cost N can be determined as the coding mode with the optimal rate-distortion cost.
  • the first coding mode may be determined to be the coding mode with the optimal rate-distortion cost. But in other examples, it can also be further determined whether there is a second encoding mode that meets other conditions: for example, the sum of Cost 1 , Cost 2 ,..., Cost N is less than the Cost 1 , Cost 2 ,..., of the first encoding mode. The sum of Cost N , and the Cost of more than half of the points in the second encoding mode is smaller than the Cost in the first encoding mode.
  • An embodiment of the present disclosure also provides a point cloud attribute compression method, as shown in Figure 4, including:
  • Step 210 Determine the attribute residual value of the current point to be encoded based on the attribute value and attribute prediction value of the current point to be encoded;
  • Step 220 Determine the final quantized residual value of the current point to be encoded according to the point cloud rate distortion optimization method described in any embodiment of the present disclosure, and perform entropy coding on the final quantized residual value.
  • the point cloud attribute compression method of this embodiment adopts the rate-distortion optimization method of the previous embodiment of the present disclosure to optimize the value of the quantized residual value, which can improve the performance of compression encoding.
  • An embodiment of the present disclosure also provides a point cloud attribute code stream, wherein the point cloud attribute code stream is generated according to the point cloud attribute compression method described in any embodiment of the present disclosure.
  • An embodiment of the present disclosure also provides a point cloud rate distortion optimization device. As shown in Figure 5, it includes a processor 5 and a memory 6 storing a computer program. When the processor 5 executes the computer program, it can Implement the point cloud rate distortion optimization method as described in any embodiment of the present disclosure.
  • An embodiment of the present disclosure also provides a point cloud compression device.
  • a point cloud compression device Referring to Figure 5, it includes a processor and a memory storing a computer program.
  • the processor executes the computer program, it can implement any implementation of the present disclosure.
  • An embodiment of the present disclosure also provides a non-transitory computer-readable storage medium.
  • the computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it can implement any of the aspects of the present disclosure.
  • the point cloud rate distortion optimization method described in the embodiment may be able to implement the point cloud attribute compression method described in any embodiment of the present disclosure.
  • the processor in the above embodiments of the present disclosure may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), a microprocessor, etc., or it may be other conventional processors. Processor, etc.; the processor may also be a digital signal processor (DSP), application specific integrated circuit (ASIC), off-the-shelf programmable gate array (FPGA), discrete logic or other programmable logic devices, discrete gates or transistor logic devices , discrete hardware components; it can also be a combination of the above devices. That is, the processor in the above embodiments can be any processing device or device combination that implements the methods, steps and logical block diagrams disclosed in the embodiments of the present invention.
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • FPGA off-the-shelf programmable gate array
  • instructions for the software may be stored in a suitable non-volatile computer-readable storage medium and may be executed in hardware using one or more processors. Instructions are provided to perform the methods of embodiments of the present disclosure.
  • An embodiment of the present disclosure provides a point cloud rate distortion optimization method, which involves a prediction branch and introduces a rate distortion optimization algorithm on the encoding side to improve encoding efficiency.
  • the encoding end performs the following point cloud rate distortion optimization processing:
  • the absolute value of the attribute residual value can be quantified, and the quantization result is multiplied by the sign of the attribute residual value to calculate the first quantized residual value.
  • the absolute value of the first quantized residual value is inversely quantized, and the inverse quantization result is multiplied by the sign of the first quantized residual value to calculate the reconstructed residual value.
  • lambda is a set constant, and the impact of code rate and distortion on overhead can be controlled by changing the size of lambda.
  • the first encoding mode is a mode for encoding the calculated first quantized residual value, which is the same as the traditional encoding mode.
  • R in the formula is the codeword size required to encode the first quantized residual value.
  • the first quantized residual value can be encoded with entropy (or unsigned The binary bit representation required for the first quantized residual value) is related to the entropy coding algorithm used; D is the difference between the attribute reconstruction value and the attribute value encoded according to the first encoding mode.
  • the current to-be-encoded point uses the first
  • the rate distortion cost of coding mode encoding is recorded as cost1
  • the second encoding mode is a mode for encoding a specified second quantized residual value.
  • the specified second quantized residual value is 0. Since the residual reconstruction value after inverse quantization is also equal to 0, it is equivalent to directly using the attribute prediction value of the current point to be encoded as the attribute reconstruction value.
  • R is the smallest (for example, 0);
  • D is the difference between the attribute prediction value and the attribute value. The difference is the absolute value of the difference; the rate-distortion overhead of encoding the current point to be encoded using the second encoding mode is recorded as cost2;
  • this embodiment only uses one second coding mode, the disclosure is not limited thereto.
  • the rate distortion cost of multiple second coding modes can also be calculated. For example, when the first quantization residual value is 2, you can add The second encoding mode with a specified quantization residual value of 1 is involved in the comparison.
  • entropy encoding can be performed directly on 0, without the need to calculate and compare the rate-distortion cost.
  • the second coding mode is used, and the specified quantized residual value 0 is used as the final quantized residual value of the current point to be encoded (equivalent to setting the quantized residual value to 0) for entropy coding to generate a code stream.
  • the attribute reconstruction value of the current point to be encoded is the attribute prediction value;
  • the first coding mode is used to perform entropy coding on the first quantized residual value to generate a code stream; at this time, the attribute reconstruction value of the current point to be encoded is equal to the predicted attribute value of the current point to be encoded plus the reconstructed residual value.
  • the difference value, the reconstructed residual value is obtained by performing inverse quantization processing on the first quantized residual value.
  • the quantized residual value of the current point to be encoded may change, but the decoding end still decodes in the original way and calculates the attribute reconstruction value of the point.
  • this embodiment sets lambda to 0.5 and conducts tests on the point cloud compression reference platform PCRM6.0 of the Audio Video Coding Standard (AVS: Audio Video coding Standard), involving the following two test conditions. : limit-lossy geometry, lossy attributes, that is, the geometric position is limited and lossy, and the attributes are lossy; lossless geometry, lossy attributes, that is, the geometric position is lossless and the attributes are lossy.
  • the test sequences used include three general test sequences: Cat1A, Cat1C, and Cat2. The experimental results of the reflectivity properties are tested:
  • class represents the category of the general test sequence
  • EtE BD-AttrRate represents the end-to-end code rate-distortion
  • refl represents the reflectance attribute.
  • a rate-distortion optimization algorithm is introduced to determine a better quantized residual value by calculating the comprehensive cost of the code rate and distortion, thereby improving the encoding efficiency.
  • the decoding end does not Additional operations are required.
  • An embodiment of the present disclosure provides a point cloud rate distortion optimization method, which involves a transformation branch and introduces a rate distortion optimization algorithm on the encoding side, thereby improving encoding efficiency.
  • the encoding end performs the following point cloud rate distortion optimization processing:
  • lambda is a set constant, and the impact of code rate and distortion on overhead can be controlled by changing the size of lambda.
  • the first coding mode is a coding mode that codes the first quantized residual value of each of D 1 , D 2 ,..., D N . For each point in the group of current points to be encoded, it is the first quantized residual value of encoding the point, which is the same as the traditional encoding mode.
  • R j is the codeword size required to encode the first quantized residual value of the j-th point; D j is the j-th point when encoding in the first encoding mode.
  • Point attribute reconstruction value The difference from the attribute value A j , cost1 j is the calculated rate distortion cost of encoding the j-th point using the first encoding mode.
  • the second coding mode is a coding mode that codes a set of quantized residual values specified for D 1 , D 2 ,..., D N .
  • a quantized residual value specified for the point is encoded.
  • a quantized residual value can be specified uniformly for different points, or the quantized residual values can be specified separately. When specified separately, the quantized residual values specified for different points can be different.
  • the quantized residual value specified for each point is 0, which is equivalent to directly using the attribute prediction value of each point as the attribute reconstruction value during attribute reconstruction.
  • R j is 0, and D j is the difference between the attribute prediction value A′ j and the attribute value A j of the j-th point.
  • the calculated rate-distortion cost of the j-th point using the second coding mode is recorded as cost2 j ;
  • the second coding mode is determined to be the optimal coding mode. All points in the group use 0 as the final quantized residual value, perform entropy coding on 0, and generate a code stream;
  • the first coding mode as the optimal coding mode, and use the first quantized residual value of each N point in the group as the final quantized residual value for entropy coding.
  • the encoding end also performs inverse quantization processing on the first quantized residual values of each N point in the group to obtain N reconstructed residual values, and then performs N-dimensional DCT inverse transformation on the N reconstructed residual values.
  • add the reconstruction residual value after inverse transformation of N points and the attribute prediction value to obtain the attribute reconstruction value of each of the N current points to be encoded.
  • different strategies are used to determine the optimal coding mode.
  • the N rate distortion costs calculated in the first coding mode are summed, and the N rate distortion costs calculated in the second coding mode are summed.
  • the costs are summed, and the encoding mode with the smallest sum is determined as the encoding mode with the optimal rate-distortion cost.
  • a rate-distortion optimization algorithm is introduced to form a group of multiple points that undergo attribute residual value transformation together, and comprehensively consider the code rate and distortion of the current group of points to be encoded.
  • a better quantized residual value is determined for the group of current points to be encoded, thereby improving coding efficiency. No additional operations are required on the decoding end.
  • Computer-readable media may include computer-readable storage media that corresponds to tangible media, such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another, such as according to a communications protocol.
  • Computer-readable media generally may correspond to non-transitory, tangible computer-readable storage media or communication media such as a signal or carrier wave.
  • Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and/or data structures for implementing the techniques described in this disclosure.
  • a computer program product may include computer-readable media.
  • such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory or may be used to store instructions or data. Any other medium that stores the desired program code in the form of a structure and that can be accessed by a computer.
  • any connection is also termed a computer-readable medium if, for example, any connection is transmitted from a website, server, or using or other remote source transmits instructions, then coaxial cable, fiber optic cable, double-stranded wire, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of medium.
  • disks and optical discs include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, or Blu-ray discs. Disks usually reproduce data magnetically, while optical discs use lasers to reproduce data. Regenerate data optically. Combinations of the above should also be included within the scope of computer-readable media.
  • processors such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits.
  • DSPs digital signal processors
  • ASICs application specific integrated circuits
  • FPGAs field programmable logic arrays
  • Execute instructions may refer to any of the structures described above or any other structure suitable for implementing the techniques described herein.
  • the functionality described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or incorporated in a combined codec.
  • the techniques may be implemented entirely in one or more circuits or logic elements.
  • inventions of the present disclosure may be implemented in a wide variety of devices or equipment, including wireless handsets, integrated circuits (ICs), or a set of ICs (eg, chipsets).
  • ICs integrated circuits
  • a set of ICs eg, chipsets.
  • Various components, modules or units are depicted in embodiments of the present disclosure to emphasize functional aspects of devices configured to perform the described techniques, but do not necessarily require implementation by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit or provided by a collection of interoperating hardware units (including one or more processors as described above) in conjunction with suitable software and/or firmware.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

A point cloud rate-distortion optimization method, an attribute compression method and apparatus, and a storage medium. The method comprises: performing rate-distortion optimization when a point cloud attribute is compressed; calculating a first quantized residual value of an attribute of a current point to be encoded; then, on the basis of a rate-distortion cost, using the calculated first quantized residual value or a specified quantized residual value as a final quantized residual value of the current point to be encoded; and performing entropy encoding. Embodiments of the present disclosure further provide a corresponding attribute compression method and apparatus, and a storage medium. In the embodiments of the present disclosure, encoding efficiency can be improved by, on the basis of a rate-distortion cost, optimizing a quantized residual value of encoding.

Description

点云率失真优化方法及属性压缩方法、装置和存储介质Point cloud rate distortion optimization method and attribute compression method, device and storage medium 技术领域Technical field
本公开实施例涉及但不限于视频技术，更具体地，涉及一种点云率失真优化方法及属性压缩方法、装置和存储介质。Embodiments of the present disclosure relate to, but are not limited to, video technology, and more specifically, to a point cloud rate distortion optimization method and attribute compression method, device and storage medium.
背景技术Background technique
点云是空间中一组无规则分布的、表达三维物体或场景的空间结构及表面属性的离散点集，图1A和图1B分别展示了一示例性的三维点云图像和局部放大图，可以看到点云表面是由分布稠密的点所组成的。A point cloud is a set of discrete points randomly distributed in space that expresses the spatial structure and surface properties of a three-dimensional object or scene. Figure 1A and Figure 1B respectively show an exemplary three-dimensional point cloud image and a local enlargement. You can See that the point cloud surface is composed of densely distributed points.
点云可以灵活方便地表达三维物体或场景的空间结构及表面属性，并且由于点云可以通过对真实物体采样直接获得，在保证精度的前提下能提供极强的真实感，因而应用广泛，其范围包括虚拟现实游戏、计算机辅助设计、地理信息***、自动导航***、数字文化遗产、自由视点广播、三维沉浸远程呈现、生物组织器官三维重建等。点云数据获取方式的变革，使大量点云数据的获取成为可能，但是，伴随着应用需求的增长，海量3D点云数据的处理遭遇存储空间和传输带宽限制，需要提供更为有效的压缩方法。Point clouds can flexibly and conveniently express the spatial structure and surface properties of three-dimensional objects or scenes, and because point clouds can be obtained directly by sampling real objects, they can provide a strong sense of reality while ensuring accuracy, so they are widely used. The scope includes virtual reality games, computer-aided design, geographical information systems, automatic navigation systems, digital cultural heritage, free-viewpoint broadcasting, three-dimensional immersive telepresence, three-dimensional reconstruction of biological tissues and organs, etc. Changes in the way of obtaining point cloud data have made it possible to obtain large amounts of point cloud data. However, with the growth of application requirements, the processing of massive 3D point cloud data encounters storage space and transmission bandwidth limitations, and more effective compression methods need to be provided. .
发明概述Summary of the invention
以下是对本文详细描述的主题的概述。本概述并非是为了限制权利要求的保护范围。The following is an overview of the topics described in detail in this article. This summary is not intended to limit the scope of the claims.
本公开一实施例提供了一种点云率失真优化方法，包括：An embodiment of the present disclosure provides a point cloud rate distortion optimization method, including:
计算当前待编码点的属性的第一量化残差值；Calculate the first quantized residual value of the attribute of the current point to be encoded;
基于率失真代价，将计算出的第一量化残差值或者指定的一个量化残差值作为当前待编码点最终的量化残差值；Based on the rate distortion cost, the calculated first quantized residual value or a specified quantized residual value is used as the final quantized residual value of the current point to be encoded;
对当前待编码点最终的量化残差值进行熵编码。Entropy coding is performed on the final quantized residual value of the current point to be encoded.
本公开一实施例还提供了一种点云属性压缩方法，包括：An embodiment of the present disclosure also provides a point cloud attribute compression method, including:
根据当前待编码点的属性值和属性预测值，确定当前待编码点的属性残差值；Determine the attribute residual value of the current point to be encoded based on the attribute value and attribute prediction value of the current point to be encoded;
根据本公开任一实施例所述的点云率失真优化方法确定当前待编码点最终的量化残差值并对所述最终的量化残差值进行熵编码。The point cloud rate distortion optimization method according to any embodiment of the present disclosure determines the final quantized residual value of the current point to be encoded and performs entropy coding on the final quantized residual value.
本公开一实施例还提供了一种点云属性码流，其中，所述点云属性码流按照如本公开任一实施例所述的点云属性压缩方法生成。An embodiment of the present disclosure also provides a point cloud attribute code stream, wherein the point cloud attribute code stream is generated according to the point cloud attribute compression method described in any embodiment of the present disclosure.
本公开一实施例还提供了一种点云率失真优化装置，包括处理器以及存储有计算机程序的存储器，其中，所述处理器执行所述计算机程序时能够实现如本公开任一实施例所述的点云率失真优化方法。An embodiment of the present disclosure also provides a point cloud rate distortion optimization device, which includes a processor and a memory storing a computer program. When the processor executes the computer program, it can implement the method described in any embodiment of the present disclosure. The point cloud rate distortion optimization method described above.
本公开一实施例还提供了一种点云压缩装置，其中，包括处理器以及存储有计算机程序的存储器，其中，所述处理器执行所述计算机程序时能够实现如本公开任一实施例所述的点云属性压缩方法。An embodiment of the present disclosure also provides a point cloud compression device, which includes a processor and a memory storing a computer program, wherein when the processor executes the computer program, it can implement the method described in any embodiment of the present disclosure. The point cloud attribute compression method described above.
本公开一实施例还提供了一种非瞬态计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其中，所述计算机程序被处理器执行时能够实现如本公开任一实施例所述的点云率失真优化方法，或能够实现如本公开任一实施例所述的点云属性压缩方法。An embodiment of the present disclosure also provides a non-transitory computer-readable storage medium. The computer-readable storage medium stores a computer program. When executed by a processor, the computer program can implement any implementation of the present disclosure. The point cloud rate distortion optimization method described in the example, or the point cloud attribute compression method described in any embodiment of the present disclosure can be implemented.
在阅读并理解了附图和详细描述后，可以明白其他方面。Other aspects will be apparent after reading and understanding the drawings and detailed description.
附图概述Figure overview
附图用来提供对本公开实施例的理解，并且构成说明书的一部分，与本公开实施例一起用于解释本公开的技术方案，并不构成对本公开技术方案的限制。The drawings are used to provide an understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the technical solutions of the present disclosure and do not constitute a limitation of the technical solutions of the present disclosure.
图1A和图1B分别是一示例性的三维点云图像和局部放大图；Figures 1A and 1B are an exemplary three-dimensional point cloud image and a partial enlargement respectively;
图2A是第一实施例的点云压缩平台的编码框架的示意图；Figure 2A is a schematic diagram of the encoding framework of the point cloud compression platform of the first embodiment;
图2B是第一实施例的点云压缩平台的解码框架的示意图；Figure 2B is a schematic diagram of the decoding framework of the point cloud compression platform of the first embodiment;
图3是本公开一实施例点云率失真优化方法的流程图；Figure 3 is a flow chart of a point cloud rate distortion optimization method according to an embodiment of the present disclosure;
图4是本公开一实施例点云属性压缩方法的流程图；Figure 4 is a flow chart of a point cloud attribute compression method according to an embodiment of the present disclosure;
图5是本公开一实施例点云率失真优化装置的示意图。Figure 5 is a schematic diagram of a point cloud rate distortion optimization device according to an embodiment of the present disclosure.
详述Elaborate
本公开描述了多个实施例，但是该描述是示例性的，而不是限制性的，并且对于本邻域的普通技术人员来说显而易见的是，在本公开所描述的实施例包含的范围内可以有更多的实施例和实现方案。The present disclosure describes multiple embodiments, but the description is illustrative rather than restrictive, and it will be obvious to a person of ordinary skill in the art that within the scope of the embodiments described in the present disclosure Many more embodiments and implementations are possible.
本公开的描述中，“示例性的”或者“例如”等词用于表示作例子、例证或说明。本公开中被描述为“示例性的”或者“例如”的任何实施例不应被解释为比其他实施例更优选或更具优势。本文中的“和/或”是对关联对象的关联关系的一种描述，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。“多个”是指两个或多于两个。另外，为了便于清楚描述本公开实施例的技术方案，采用了“第一”、“第二”等字样对功能和作用基本相同的相同项或相似项进行区分。本邻域技术人员可以理解“第一”、“第二”等字样并不对数量和执行次序进行限定，并且“第一”、“第二”等字样也并不限定一定不同。In the description of the present disclosure, the words "exemplary" or "such as" are used to mean an example, illustration, or explanation. Any embodiment described in this disclosure as "exemplary" or "such as" is not intended to be construed as preferred or advantageous over other embodiments. "And/or" in this article is a description of the relationship between associated objects, indicating that there can be three relationships, for example, A and/or B, which can mean: A exists alone, A and B exist simultaneously, and they exist alone. B these three situations. "Plural" means two or more than two. In addition, in order to facilitate a clear description of the technical solutions of the embodiments of the present disclosure, words such as “first” and “second” are used to distinguish the same or similar items with basically the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the number and execution order, and words such as "first" and "second" do not limit the number and execution order.
在描述具有代表性的示例性实施例时，说明书可能已经将方法和/或过程呈现为特定的步骤序列。然而，在该方法或过程不依赖于本文所述步骤的特定顺序的程度上，该方法或过程不应限于所述的特定顺序的步骤。如本邻域普通技术人员将理解的，其它的步骤顺序也是可能的。因此，说明书中阐述的步骤的特定顺序不应被解释为对权利要求的限制。此外，针对该方法和/或过程的权利要求不应限于按照所写顺序执行它们的步骤，本邻域技术人员可以容易地理解，这些顺序可以变化，并且仍然保持在本公开实施例的精神和范围内。In describing representative exemplary embodiments, the specification may have presented methods and/or processes as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of steps described herein, the method or process should not be limited to the specific order of steps described. As one of ordinary skill in the art will appreciate, other sequences of steps are possible. Therefore, the specific order of steps set forth in the specification should not be construed as limiting the claims. Furthermore, claims directed to the method and/or process should not be limited to steps performing them in the order written, as those skilled in the art will readily understand that such order may be varied and still remain within the spirit and spirit of the disclosed embodiments. within the range.
二维图像在每一个像素点均有信息表达，分布规则，因此不需要额外记录其位置信息；然而点云中的点在三维空间中的分布具有随机性和不规则性，因此需要记录每一个点在空间中的位置，才能完整地表达一幅点云。与二维图像类似，采集过程中每一个点均有对应的属性信息，通常为RGB颜色值，颜色值反映物体的色彩；对于点云来说，每一个点所对应的属性信息除了颜色以外，比较常见的还有反射率(reflectance)，反射率的值反映物体的表面材质。因此，点云数据通常包括三维位置信息所组成的几何信息(x,y,z)和三维颜色信息(r,g,b)、一维反射率信息(r)所组成的属性信息。相应的点云数据存储格式由文件头信息部分和数据部分组成，其中，头信息包含了数据格式、数据表示类型、点云总点数、以及点云所表示的内容。在一个示例中，点云为“.ply”格式，由ASCII码表示，总点数为207242，每个点具有三维位置信息xyz和三维颜色信息RGB。Two-dimensional images have information expressed in each pixel and are distributed regularly, so there is no need to record additional position information; however, the distribution of points in the point cloud in the three-dimensional space is random and irregular, so each point needs to be recorded Only the position of the point in space can completely express a point cloud. Similar to two-dimensional images, each point in the collection process has corresponding attribute information, usually an RGB color value, and the color value reflects the color of the object; for point clouds, in addition to color, the attribute information corresponding to each point The more common one is reflectance. The reflectance value reflects the surface material of the object. Therefore, point cloud data usually includes geometric information (x, y, z) composed of three-dimensional position information, attribute information composed of three-dimensional color information (r, g, b), and one-dimensional reflectance information (r). The corresponding point cloud data storage format consists of a file header information part and a data part. The header information includes the data format, data representation type, total point cloud points, and the content represented by the point cloud. In one example, the point cloud is in ".ply" format and is represented by ASCII code. The total number of points is 207242, and each point has three-dimensional position information xyz and three-dimensional color information RGB.
点云的采集主要有计算机生成、3D激光扫描、3D摄影测量等。计算机可以生成虚拟三维物体及场景的点云；3D激光扫描可以获得静态现实世界三维物体或场景的点云，每秒可以获取百万级点云；3D摄影测量可以获得动态现实世界三维物体或场景的点云，每秒可以获取千万级点云。这些技术降低了点云数据获取成本和时间周期，提高了数据的精度。伴随着应用需求的增长，海量3D点云数据的处理遭遇存储空间和传输带宽限制的瓶颈。Point cloud collection mainly includes computer generation, 3D laser scanning, 3D photogrammetry, etc. Computers can generate point clouds of virtual three-dimensional objects and scenes; 3D laser scanning can obtain point clouds of static real-world three-dimensional objects or scenes, and can obtain millions of point clouds per second; 3D photogrammetry can obtain dynamic real-world three-dimensional objects or scenes Point clouds can obtain tens of millions of point clouds per second. These technologies reduce the cost and time period of point cloud data acquisition and improve the accuracy of the data. As application demands grow, the processing of massive 3D point cloud data encounters bottlenecks limited by storage space and transmission bandwidth.
以帧率为30fps(帧每秒)的点云视频为例，每帧点云的点数为70万，每个点具有坐标信息xyz(float，浮点数)和颜色信息RGB(uchar，无符号字符型)，则10s点云视频的数据量大约为0.7million·(4Byte·3+1Byte·3)·30fps·10s＝3.15GB，而YUV采样格式为4:2:0，帧率为24fps的1280·720二维视频，其10s的数据量约为1280·720·12bit·24frames·10s≈0.33GB，10s的两视角3D视频的数据量约为0.33·2＝0.66GB。可见，点云视频的数据量远超过相同时长的二维视频和三维视频。为更好地实现数据管理，节省服务器存储空间，降低服务器与客户端之间的传输流量及传输时间，点云压缩成为点云应用发展的关键问题。Taking a point cloud video with a frame rate of 30fps (frames per second) as an example, the number of points in each frame of the point cloud is 700,000, and each point has coordinate information xyz (float, floating point number) and color information RGB (uchar, unsigned character type), the data volume of 10s point cloud video is approximately 0.7 million·(4Byte·3+1Byte·3)·30fps·10s=3.15GB, while the YUV sampling format is 4:2:0, and the frame rate is 1280 at 24fps ·720 2D video, the data volume of 10s is about 1280·720·12bit·24frames·10s≈0.33GB, and the data volume of 10s two-view 3D video is about 0.33·2=0.66GB. It can be seen that the data volume of point cloud video far exceeds that of 2D and 3D videos of the same length. In order to better realize data management, save server storage space, and reduce transmission traffic and transmission time between the server and the client, point cloud compression has become a key issue in the development of point cloud applications.
点云压缩算法包括国际动态图像专家组(Moving Picture Experts Group，MPEG)所开发的两种方案，分别是基于视频的点云压缩(Video-based Point Cloud Compression,V-PCC)和基于几何的点云压缩(Geometry-based Point Cloud Compression，G-PCC)。其中，G-PCC中的几何压缩主要通过八叉树模型和/或三角形表面模型实现。V-PCC主要通过三维到二维的投影和视频压缩实现。此外，还包括AVS(Audio Video coding Standard，音视频编码标准)的PCC等。Point cloud compression algorithms include two solutions developed by the International Moving Picture Experts Group (MPEG), namely video-based point cloud compression (V-PCC) and geometry-based point cloud compression. Cloud compression (Geometry-based Point Cloud Compression, G-PCC). Among them, geometric compression in G-PCC is mainly realized through octree model and/or triangular surface model. V-PCC is mainly implemented through three-dimensional to two-dimensional projection and video compression. In addition, it also includes PCC of AVS (Audio Video coding Standard).
点云压缩一般采用点云几何信息和属性信息分别压缩的方式，在编码端，首先在几何编码器中编码点云几何信息，然后将重建几何信息作为附加信息输入到属性编码器中，辅助点云属性的压缩；在解码端，首先在几何解码器中解码点云几何信息，然后将解码后的几何信息作为附加信息输入到属性解码器中，辅助点云属性的解压缩。图2A所示为一种示例性的点云编码框架的示意图，图2B所示为一种示例性的点云解码框架的示意图。Point cloud compression generally adopts the method of separately compressing point cloud geometric information and attribute information. On the encoding side, the point cloud geometric information is first encoded in the geometry encoder, and then the reconstructed geometric information is input into the attribute encoder as additional information, and the auxiliary point Compression of cloud attributes; on the decoding side, the point cloud geometric information is first decoded in the geometry decoder, and then the decoded geometric information is input into the attribute decoder as additional information to assist in the decompression of point cloud attributes. FIG. 2A shows a schematic diagram of an exemplary point cloud encoding framework, and FIG. 2B shows a schematic diagram of an exemplary point cloud decoding framework.
一种示例性的点云压缩参考平台(PCRM：Point Cloud Reference Model)的编码端包括几何编码器和属性编码器。如图2A所示，几何编码器包括坐标平移模块11、坐标量化模块13、八叉树构建模块15、几何熵编码器17和几何重建模块19。坐标平移模块11和坐标量化模块13对点云的原始几何信息进行预处理。八叉树构建模块15和几何熵编码器17对预处理后的几何信息进行编码，得到几何码流。其中：The encoding end of an exemplary point cloud compression reference platform (PCRM: Point Cloud Reference Model) includes a geometry encoder and an attribute encoder. As shown in FIG. 2A , the geometry encoder includes a coordinate translation module 11 , a coordinate quantization module 13 , an octree building module 15 , a geometry entropy encoder 17 and a geometry reconstruction module 19 . The coordinate translation module 11 and the coordinate quantization module 13 preprocess the original geometric information of the point cloud. The octree building module 15 and the geometric entropy encoder 17 encode the preprocessed geometric information to obtain a geometric code stream. in:
坐标平移模块11设置为通过坐标平移将原始几何信息的几何原点归一化到点云空间中的最小值位置。The coordinate translation module 11 is configured to normalize the geometric origin of the original geometric information to the minimum value position in the point cloud space through coordinate translation.
坐标量化模块13设置为通过坐标量化将坐标平移模块11输出的几何信息从浮点数转化为整形，便于后续的规则化处理，规则化的几何信息输出到八叉树构建模块15和属性编码器中。The coordinate quantization module 13 is configured to convert the geometric information output by the coordinate translation module 11 from floating point numbers into integers through coordinate quantization to facilitate subsequent regularization processing. The regularized geometric information is output to the octree building module 15 and the attribute encoder. .
八叉树构建模块15设置为对规则化的几何信息进行几何编码，采用八叉树结构对点云空间进行递归划分，每次将当前节点划分成八个相同大小的子块，并判断每个子块的占***字情况，当子块内不包含点云点时记为空，否则记为非空，将非空的子块作为当前节点继续进行划分，在递归划分的最后一层记录所有块的占***字信息并进行编码；将通过八叉树结构表达的几何信息输出到几何熵编码器17和几何重建模块19。The octree building module 15 is configured to geometrically encode regularized geometric information, use the octree structure to recursively divide the point cloud space, divide the current node into eight sub-blocks of the same size each time, and judge each sub-block. The occupied codeword status of the block. When the sub-block does not contain point cloud points, it is recorded as empty, otherwise it is recorded as non-empty. The non-empty sub-block is used as the current node to continue dividing, and all blocks are recorded in the last layer of recursive division. The codeword information is occupied and encoded; the geometric information expressed through the octree structure is output to the geometric entropy encoder 17 and the geometric reconstruction module 19.
几何熵编码器17设置为对八叉树结构表达的几何信息进行熵编码，形成几何码流。The geometric entropy encoder 17 is configured to entropy encode the geometric information expressed by the octree structure to form a geometric code stream.
几何重建模块19设置为在几何编码器内进行几何重建处理，重建后的几何信息作为附加信息输出到属性编码器中。The geometric reconstruction module 19 is configured to perform geometric reconstruction processing in the geometry encoder, and the reconstructed geometric information is output to the attribute encoder as additional information.
如图2A所示，属性编码器包括颜色空间变换模块21、属性重上色模块23、属性预测模块25、量化模块27和属性熵编码模块29。颜色空间变换模块21和属性重上色模块23对原始属性信息进行预处理。其他模块对预处理后的属性信息进行属性编码，得到属性码流。其中：As shown in FIG. 2A , the attribute encoder includes a color space transformation module 21 , an attribute recoloring module 23 , an attribute prediction module 25 , a quantization module 27 and an attribute entropy encoding module 29 . The color space transformation module 21 and the attribute recoloring module 23 preprocess the original attribute information. Other modules perform attribute encoding on the preprocessed attribute information to obtain an attribute code stream. in:
颜色空间变换模块21设置为在处理的原始属性信息为RGB格式的颜色值时进行颜色空间变换，将其转变到更符合人眼视觉特性的YUV色彩空间。The color space conversion module 21 is configured to perform color space conversion when the processed original attribute information is a color value in RGB format, and convert it into a YUV color space that is more in line with the visual characteristics of the human eye.
属性重上色模块23设置为基于几何编码器输出的几何信息，对几何编码后的每一个点重新分配属性值，这个过程叫做属性重上色，以避免原始几何信息和几何编码之后的几何信息之间的差异对属性的影响。The attribute recoloring module 23 is set to reassign the attribute value to each point after geometric encoding based on the geometric information output by the geometry encoder. This process is called attribute recoloring to avoid the original geometric information and the geometric information after geometric encoding. The difference between properties.
属性预测模块25设置为将点云进行重排序，重排序的方式可以是希尔伯顿码、莫顿码等，相应属性编码的遍历顺序为莫顿顺序、希尔伯特顺序等，属性预测方法可以是基于邻近点的多点预测，也可以是单点预测。多点预测时先找到距离当前待编码点最近的多个点作为当前待编码点的预测参考点，将多个预测参考点的属性重建值加权作为当前待编码点的属性预测值。如果是单点预测可以从当前待编码点向前回溯一个点作为当前待编码点的预测参考点，将该预测参考点的属性重建值作为当前待编码点的属性预测值。当前待编码点的属性值与属性预测值之差即当前待编码点的属性残差值。The attribute prediction module 25 is configured to reorder the point cloud. The reordering method can be Hilberton code, Morton code, etc., and the traversal order of the corresponding attribute encoding is Morton order, Hilbert order, etc., attribute prediction The method can be multi-point prediction based on neighboring points or single-point prediction. During multi-point prediction, first find multiple points closest to the current point to be encoded as the prediction reference points of the current point to be encoded, and weight the attribute reconstruction values of the multiple prediction reference points as the attribute prediction values of the current point to be encoded. If it is single-point prediction, one point can be traced back from the current point to be encoded as the prediction reference point of the current point to be encoded, and the attribute reconstruction value of the prediction reference point can be used as the attribute prediction value of the current point to be encoded. The difference between the attribute value of the current point to be encoded and the predicted attribute value is the attribute residual value of the current point to be encoded.
量化模块27设置为对点的属性残差值进行量化处理，得到点的属性的量化残差值。The quantization module 27 is configured to perform quantization processing on the attribute residual value of the point to obtain the quantized residual value of the attribute of the point.
属性熵编码模块29设置为对点的属性的量化残差值进行熵编码，形成属性码流。The attribute entropy encoding module 29 is configured to entropy encode the quantized residual value of the attribute of the point to form an attribute code stream.
PCRM平台的解码端包括几何解码器和属性解码器。如图2B所示，几何解码器包括几何熵解码器31、八叉树重建模块33、坐标反量化模块35和坐标反平移模块37。其中，The decoding end of the PCRM platform includes a geometry decoder and an attribute decoder. As shown in FIG. 2B , the geometry decoder includes a geometric entropy decoder 31 , an octree reconstruction module 33 , a coordinate inverse quantization module 35 and a coordinate inverse translation module 37 . in,
几何熵解码器31设置为对几何码流熵解码，得到每个节点的几何信息。The geometric entropy decoder 31 is configured to entropy decode the geometric code stream to obtain the geometric information of each node.
八叉树重建模块33设置为基于每个节点的几何信息，按照和几何编码相同的方式构建八叉树结构，结合解码几何重建出坐标变换后的、通过八叉树结构表达的几何信息。The octree reconstruction module 33 is configured to construct an octree structure based on the geometric information of each node in the same manner as geometric encoding, and reconstruct the coordinate-transformed geometric information expressed through the octree structure in combination with the decoded geometry.
坐标反量化模块35设置为对八叉树重建模块33输出的几何信息进行坐标反量化，坐标反量化后的几何信息输出到坐标反平移模块37，并作为附加信息输出到属性解码器。The coordinate inverse quantization module 35 is configured to perform coordinate inverse quantization on the geometric information output by the octree reconstruction module 33. The geometric information after coordinate inverse quantization is output to the coordinate inverse translation module 37, and is output to the attribute decoder as additional information.
坐标反平移模块37设置为对坐标反量化模块35输出的几何信息进行坐标反平移，得到解码后的几何信息。The coordinate inverse translation module 37 is configured to perform coordinate inverse translation on the geometric information output by the coordinate inverse quantization module 35 to obtain decoded geometric information.
如图所示，属性解码器包括属性熵解码器41、反量化模块43、属性预测模块45和颜色空间反变换模块47。其中：As shown in the figure, the attribute decoder includes an attribute entropy decoder 41, an inverse quantization module 43, an attribute prediction module 45 and a color space inverse transform module 47. in:
属性熵解码器41设置为对属性码流进行熵解码，得到点的属性的量化残差值。The attribute entropy decoder 41 is configured to perform entropy decoding on the attribute code stream to obtain the quantized residual value of the attribute of the point.
反量化模块43设置为对量化残差值进行反量化处理，得到点的属性残差值。The inverse quantization module 43 is configured to perform inverse quantization processing on the quantized residual value to obtain the attribute residual value of the point.
属性预测模块45设置为基于几何解码器输出的几何信息，按照与属性编码相同的方式构建莫顿顺序、希尔伯特顺序等，获得当前待解码点的属性预测值，然后将当前待解码点的属性预测值与属性残差值相加，恢复出当前待解码点的属性值即得到属性重建值，如YUV颜色值、反射率值等。The attribute prediction module 45 is configured to construct Morton order, Hilbert order, etc. in the same way as attribute encoding based on the geometric information output by the geometry decoder, obtain the attribute prediction value of the current point to be decoded, and then convert the current point to be decoded The attribute prediction value is added to the attribute residual value, and the attribute value of the current point to be decoded is restored to obtain the attribute reconstruction value, such as YUV color value, reflectance value, etc.
颜色空间反变换模块设置为对属性预测模块45输出的属性信息进行颜色空间反变换，得到解码后的属性信息。The color space inverse transformation module is configured to perform inverse color space transformation on the attribute information output by the attribute prediction module 45 to obtain decoded attribute information.
基于上述编码框架和解码框架对点云进行压缩编码和解码时，以属性压缩采用的算法进行区分，有两种技术路线。一种称为预测分支，另一种称为变换分支，预测分支使用的属性编解码方法见上文的说明。变换分支采用的属性编解码方法与预测分支的区别在于：通过属性预测得到点的属性残差值之后，先对多个点(如4个点)的属性残差值进行变换(变换算法可以是小波变换或者k元离散余弦变换(DCT：Discrete Cosine Transform)等)，再对变换后的属性残差值进行量化得到属性的量化残差值。相应的，解码端对属性码流进行熵解码、反量化得到属性残差值之后，预测分支基于反量化得到的属性残差值进行属性预测，而变换分支先对反量化得到的多个属性残差值进行反变换，再基于反变换得到的属性残差值进行属性预测。When compressing, encoding and decoding point clouds based on the above encoding framework and decoding framework, there are two technical routes to distinguish based on the algorithm used for attribute compression. One is called the prediction branch, and the other is called the transformation branch. The attribute encoding and decoding method used by the prediction branch is described above. The difference between the attribute encoding and decoding method used in the transformation branch and the prediction branch is that after obtaining the attribute residual value of the point through attribute prediction, the attribute residual value of multiple points (such as 4 points) is first transformed (the transformation algorithm can be Wavelet transform or k-element discrete cosine transform (DCT: Discrete Cosine Transform), etc.), and then quantize the transformed attribute residual value to obtain the quantized residual value of the attribute. Correspondingly, after the decoding end performs entropy decoding and inverse quantization on the attribute code stream to obtain the attribute residual value, the prediction branch performs attribute prediction based on the attribute residual value obtained by inverse quantization, and the transformation branch first performs attribute prediction on the multiple attribute residual values obtained by inverse quantization. The difference value is inversely transformed, and then attribute prediction is performed based on the attribute residual value obtained by the inverse transformation.
在一个实施例的点云压缩方法中，按照希尔伯特顺序编码和解码每一个点的属性，其中，属性编码过程包括：In the point cloud compression method of one embodiment, the attributes of each point are encoded and decoded according to Hilbert order, where the attribute encoding process includes:
编码端获取当前待编码点的属性预测值A′ j，根据当前待编码点的属性值A j和属性预测值计算当前待编码点的属性残差值delta，delta＝A j-A′ j； The encoding end obtains the attribute prediction value A′ j of the current point to be encoded, and calculates the attribute residual value delta of the current point to be encoded based on the attribute value A j and the attribute prediction value of the current point to be encoded, delta=A j -A′ j ;
对当前待编码点的属性残差值进行量化处理，得到属性的量化残差值(带符号)；其中，量化处理时对属性残差值的绝对值进行量化，量化的结果与属性残差值的符号相乘，即计算出量化残差值；Quantize the attribute residual value of the current point to be encoded to obtain the quantized residual value (signed) of the attribute; during the quantization process, the absolute value of the attribute residual value is quantized, and the quantization result is the same as the attribute residual value. Multiply the signs to calculate the quantized residual value;
对当前待编码点属性的量化残差值进行熵编码，生成属性码流。Entropy coding is performed on the quantized residual value of the attribute of the current point to be encoded to generate an attribute code stream.
另一方面，编码端将当前待编码点的量化残差值进行反量化处理，得到当前待编码点的重建残差值，将当前待编码点的重建残差值与属性预测值相加即得到当前待编码点的属性重建值。当前待编码点的属性重建值可以用于计算后续点的属性预测值。On the other hand, the encoding end performs inverse quantization processing on the quantized residual value of the current point to be encoded to obtain the reconstructed residual value of the current point to be encoded. The reconstructed residual value of the current point to be encoded is added to the attribute prediction value to obtain The attribute reconstruction value of the current point to be encoded. The attribute reconstruction value of the current point to be encoded can be used to calculate the attribute prediction value of subsequent points.
属性解码过程包括：The attribute decoding process includes:
解码端获取当前待解码点的属性预测值A′ j，从码流中解码得到当前待解码点的量化残差值(带符号)； The decoding end obtains the attribute prediction value A′ j of the current point to be decoded, and decodes it from the code stream to obtain the quantized residual value (signed) of the current point to be decoded;
对当前待解码点的量化残差值进行反量化处理，得到当前待解码点的重建残差值
Figure PCTCN2022104391-appb-000001
其中，反量化处理时对量化残差值的绝对值进行反量化，反量化的结果再与量化残差值的符号相乘即得到重建残差值
Figure PCTCN2022104391-appb-000002
Perform inverse quantization processing on the quantized residual value of the current point to be decoded to obtain the reconstructed residual value of the current point to be decoded.
Figure PCTCN2022104391-appb-000001
Among them, during the inverse quantization process, the absolute value of the quantized residual value is inversely quantized, and the inverse quantization result is multiplied by the sign of the quantized residual value to obtain the reconstructed residual value.
Figure PCTCN2022104391-appb-000002
最后将当前待解码点的重建残差值
Figure PCTCN2022104391-appb-000003
与属性预测值A ′ j相加，得到当前待解码点的属性重建值
Figure PCTCN2022104391-appb-000004
即：
Figure PCTCN2022104391-appb-000005
Finally, the reconstructed residual value of the current point to be decoded is
Figure PCTCN2022104391-appb-000003
Added to the attribute prediction value A ′ j , the attribute reconstruction value of the current point to be decoded is obtained.
Figure PCTCN2022104391-appb-000004
Right now:
Figure PCTCN2022104391-appb-000005
上述实施例编码属性时，使用的量化残差值是对属性残差值做量化处理得到的，该量化残差值一方面写进码流，影响码率大小，另一方面会通过反量化处理之后生成属性重建值，从而带来属性重建值与原始的属性值之间的失真。但是，直接编码该量化残差值并不一定能够带来最好的性能，可以引入率失真优化算法，平衡码率与失真之间的代价，提高编码性能。When encoding attributes in the above embodiment, the quantized residual value used is obtained by quantizing the attribute residual value. On the one hand, the quantized residual value is written into the code stream and affects the code rate. On the other hand, it is processed through inverse quantization. The attribute reconstruction value is then generated, thereby causing distortion between the attribute reconstruction value and the original attribute value. However, directly encoding the quantized residual value does not necessarily bring the best performance. A rate-distortion optimization algorithm can be introduced to balance the cost between code rate and distortion and improve coding performance.
本公开一实施例提供了一种点云率失真优化方法，可用于上述编码框架的量化模块，如图3所示，所述方法包括：An embodiment of the present disclosure provides a point cloud rate distortion optimization method, which can be used in the quantization module of the above-mentioned coding framework. As shown in Figure 3, the method includes:
步骤110，计算当前待编码点的属性的第一量化残差值；Step 110: Calculate the first quantized residual value of the attribute of the current point to be encoded;
本文中，将计算得到的量化残差值称为第一量化残差值，以区别于其他的量化残差值如指定的量化残差值和最终的量化残差值。In this article, the calculated quantized residual value is called the first quantized residual value to distinguish it from other quantized residual values such as the specified quantized residual value and the final quantized residual value.
步骤120，根据编码所述第一量化残差值的率失真代价和编码指定的第二量化残差值的率失真代价，将所述第一量化残差值和第二量化残差值中的一个值确定为当前待编码点最终的量化残差值；Step 120: According to the rate distortion cost of encoding the first quantized residual value and the rate distortion cost of encoding the specified second quantized residual value, convert the first quantized residual value and the second quantized residual value into One value is determined as the final quantized residual value of the current point to be encoded;
显然，指定的该量化残差值应不同于所述第一量化残差值。率失真代价也可称为率失真开销。Obviously, the specified quantized residual value should be different from the first quantized residual value. The rate distortion penalty may also be called rate distortion overhead.
步骤130，对当前待编码点最终的量化残差值进行熵编码。Step 130: Perform entropy coding on the final quantized residual value of the current point to be encoded.
本公开实施例在计算得到当前待编码点的第一量化残差值之后，并不直接进行熵编码，而是基于率失真代价，选择第一量化残差值或者指定的一个第二量化残差值作为当前待编码点最终的量化残差值，再对当前待编码点最终的量化残差值进行熵编码。例如，计算的第一量化残差值为1，而编码指定的量化残差值如0的率失真代价更小，就使用指定的该量化残差值代替第一量化残差值进行熵编码，从而在整体上取得更好的编码性能。In the embodiment of the present disclosure, after calculating the first quantized residual value of the current point to be encoded, entropy coding is not performed directly. Instead, the first quantized residual value or a specified second quantized residual is selected based on the rate-distortion cost. The value is used as the final quantized residual value of the current point to be encoded, and then entropy coding is performed on the final quantized residual value of the current point to be encoded. For example, if the calculated first quantized residual value is 1, and the rate distortion cost of encoding a specified quantized residual value such as 0 is smaller, the specified quantized residual value is used instead of the first quantized residual value for entropy coding, This results in better encoding performance overall.
在本公开一示例性的实施例中，所述指定的第二量化残差值为0。用0替代非0的第一量化残差值作为最终的量化残差值，在熵编码时码字最小，而在重建当前待编码点的属性值时，重建残差值也为0，相当于将当前待编码点的属性预测值直接作为属性重建值，会带来一定的失真。通过率失真代价可以权衡码率上的收益和失真，确定是否使用0替代计算出的第一量化残差值进行熵编码。In an exemplary embodiment of the present disclosure, the specified second quantized residual value is 0. Use 0 to replace the non-zero first quantized residual value as the final quantized residual value. During entropy coding, the codeword is the smallest. When reconstructing the attribute value of the current point to be encoded, the reconstructed residual value is also 0, which is equivalent to Using the attribute prediction value of the current point to be encoded directly as the attribute reconstruction value will bring certain distortion. The rate-distortion cost can weigh the gain and distortion in code rate to determine whether to use 0 instead of the calculated first quantized residual value for entropy coding.
在本公开一示例性的实施例中，所述指定的第二量化残差值包括0和非零整数值，所述非零整数值包括以下数值中的任意一种或更多种：1，-1，2，-2。本实施例在计算出第一量化残差值后，可以将编码第一量化残差值的率失真代价与编码多个指定的第二量化残差值的率失真代价比较，从中选出一个量化残差值为当前待编码点最终的量化残差值。例如，在第一量化残差值为1时，可以计算编码第一量化残差值1的率失真代价，计算编码指定的量化残差值0的率失真代价，及计算编码指定的量化残差值2的率失真代价，将对应的率失真代价最小的一个值作为当前待编码点最终的量化残差值。当然在其他示例中，非零整数值也可以包括更多的值，如3，-3，4，-4，等等。In an exemplary embodiment of the present disclosure, the specified second quantized residual value includes 0 and a non-zero integer value, and the non-zero integer value includes any one or more of the following values: 1, -1, 2, -2. In this embodiment, after calculating the first quantized residual value, the rate distortion cost of encoding the first quantized residual value can be compared with the rate distortion cost of encoding multiple specified second quantized residual values, and a quantized one can be selected from them. The residual value is the final quantized residual value of the current point to be encoded. For example, when the first quantized residual value is 1, the rate distortion cost of encoding the first quantized residual value 1 can be calculated, the rate distortion cost of encoding the specified quantized residual value 0 can be calculated, and the encoding specified quantization residual can be calculated For the rate distortion cost of value 2, the value with the smallest corresponding rate distortion cost is used as the final quantized residual value of the current point to be encoded. Of course, in other examples, non-zero integer values can also include more values, such as 3, -3, 4, -4, and so on.
在本公开一示例性的实施例中，所述方法还包括：在所述第一量化残差值为0时，将所述第一量化残差值作为当前待编码点最终的量化残差值。此时通常使用第一量化残差值的率失真代价最小，因此直接将第一量化残差值作为最终的量化残差值。In an exemplary embodiment of the present disclosure, the method further includes: when the first quantized residual value is 0, using the first quantized residual value as the final quantized residual value of the current point to be encoded. . At this time, the rate distortion cost of using the first quantized residual value is usually the smallest, so the first quantized residual value is directly used as the final quantized residual value.
在本公开一示例性的实施例中，对点云进行压缩编码时不对属性残差值进行变换，而是对一个当前待编码点的属性残差值做量化处理，得到该当前待编码点的第一量化残差值。In an exemplary embodiment of the present disclosure, when compressing and encoding the point cloud, the attribute residual value is not transformed, but the attribute residual value of a current point to be encoded is quantized to obtain the attribute residual value of the current point to be encoded. The first quantized residual value.
此时，根据编码所述第一量化残差值的率失真代价和编码指定的第二量化残差值的率失真代价，将所述第一量化残差值和第二量化残差值中的一个值确定为当前待编码点最终的量化残差值，包括：At this time, according to the rate distortion cost of encoding the first quantized residual value and the rate distortion cost of encoding the specified second quantized residual value, the first quantized residual value and the second quantized residual value are One value is determined as the final quantized residual value of the current point to be encoded, including:
计算该当前待编码点使用第一编码模式的率失真代价，所述第一编码模式下对所述第一量化残差值编码；Calculate the rate distortion cost of the current to-be-encoded point using a first encoding mode in which the first quantized residual value is encoded;
计算该当前待编码点使用第二编码模式的率失真代价，所述第二编码模式下对所述第二量化残差值编码；Calculate the rate distortion cost of the current to-be-encoded point using a second encoding mode in which the second quantized residual value is encoded;
将所述第一编码模式和第二编码模式中率失真代价值小的一种编码模式所编码的量化残差值作为该当前待编码点最终的量化残差值。The quantized residual value encoded by the encoding mode with the smaller rate distortion cost among the first encoding mode and the second encoding mode is used as the final quantized residual value of the current point to be encoded.
在本实施例中，所述第二编码模式有一种，该第二编码模式下对一个所述第二量化残差值编码；或者，所述第二编码模式有多种，每一种第二编码模式下对一个所述第二量化残差值编码，不同第二编码模式下编码的第二量化残差值不同。In this embodiment, there is one second encoding mode, and one second quantized residual value is encoded in the second encoding mode; or, there are multiple second encoding modes, and each second encoding mode One of the second quantized residual values is encoded in the encoding mode, and the second quantized residual values encoded in different second encoding modes are different.
例如，在指定的第二量化残差值只有0这一个值，而第一量化残差值不为0时，可以计算使用两种编码模式的率失真代价，一种是编码0的率失真代价，一种是编码第一量化残差值的率失真代价，选择率失真代价最小的编码模式所编码的量化残差值为最终的量化残差值。For example, when the specified second quantized residual value only has a value of 0, and the first quantized residual value is not 0, the rate distortion cost of using two encoding modes can be calculated. One is the rate distortion cost of encoding 0. , one is to encode the rate distortion cost of the first quantized residual value, and select the encoding mode with the smallest rate distortion cost to encode the quantized residual value as the final quantized residual value.
又如，在指定的第二量化残差值包括0和非零整数值的情况下，第二编码模式可能有多种，其中一种第二编码模式为该当前待编码点指定的一个量化残差值为0，其他第二编码模式为该当前待编码点指定的量化残差值为非零整数值。在确定第二编码模式的个数和指定的量化残差值时，可以从第二量化残差值的数值集中选出绝对值小于第一量化残差值的绝对值且符号和第一量化残差值的符号相同的非零整数值，选出的每一个非零整数值可以作为一种第二编码模式指定的量化残差值。如果选出的非零整数值有N个，则第二编码模式可以有N+1个，包括一个将0指定为量化残差值的第二编码模式，以及N个将选出的一个非零整数值作为指定的量化残差值的第二编码模式。假定第一量化残差值为2，此时可以计算编码0，1的两种第二编码模式的率失真代价以及编码2的第一编码模式的率失真代价，将率失真代价最小的编码模式所编码的值作为当前待编码点最终的量化残差值。结合对失真和码率的考虑，此时可以不尝试3以上的数值或-1，-2等数值。For another example, when the specified second quantized residual value includes 0 and non-zero integer values, there may be multiple second encoding modes, and one of the second encoding modes is a quantized residual specified for the current point to be encoded. The difference value is 0, and the quantization residual values specified by other second encoding modes for the current point to be encoded are non-zero integer values. When determining the number of second coding modes and the specified quantized residual value, an absolute value smaller than the absolute value of the first quantized residual value and a sign that is the same as the first quantized residual value may be selected from the value set of the second quantized residual value. The differences are non-zero integer values with the same sign, and each selected non-zero integer value can be used as a quantized residual value specified by the second encoding mode. If there are N non-zero integer values to be selected, there can be N+1 second coding modes, including a second coding mode that specifies 0 as the quantized residual value, and N one non-zero value to be selected An integer value used as the second encoding mode specifying the quantized residual value. Assume that the first quantized residual value is 2. At this time, the rate distortion cost of the two second coding modes encoding 0 and 1 and the rate distortion cost of the first encoding mode encoding 2 can be calculated, and the encoding mode with the smallest rate distortion cost can be calculated. The encoded value is used as the final quantized residual value of the current point to be encoded. Considering the distortion and bit rate, you do not need to try values above 3 or -1, -2 and other values at this time.
在其他的一些示例中，也可以将第二编码模式编码的量化残差值指定为绝对值比第一量化残差值小1且符号与第一最化残差值相同的值，以控制失真，此时只需要比较一个第二编码模式和一个第一编码模式的率失真代价。In some other examples, the quantized residual value encoded in the second encoding mode may also be specified as a value whose absolute value is 1 smaller than the first quantized residual value and whose sign is the same as the first optimized residual value to control distortion. , at this time it is only necessary to compare the rate distortion cost of a second coding mode and a first coding mode.
在本公开一示例性的实施例中，按以下公式计算该当前待编码点使用每一编码模式的率失真代价cost：In an exemplary embodiment of the present disclosure, the rate distortion cost of each coding mode used by the current point to be encoded is calculated according to the following formula:
cost＝lambda*R+Dcost＝lambda*R+D
其中，lambda为设定的常数，R是编码该编码模式下的量化残差值所需的码字大小，D是按照该编码模式编码的情况下该当前待编码点的属性重建值与属性值的差异。R的大小与具体采用的熵编码算法有关，D可以用属性重建值与属性值的差异，如差的绝对值但不局限于此。Among them, lambda is a set constant, R is the codeword size required to encode the quantized residual value in this encoding mode, and D is the attribute reconstruction value and attribute value of the current point to be encoded when encoding in this encoding mode. difference. The size of R is related to the specific entropy coding algorithm used. D can use attributes to reconstruct the difference between the value and the attribute value, such as the absolute value of the difference but is not limited to this.
在一个示例中，lambda的取值为0和1之间，例如可以设定为0.5，使用者可以根据对码率和失真的具体需求加以设定，在对码率的要求较高，可以将lambda设定得相对较大以提高码字大小的权重，反之如果要求失真较小，可以将lambda设定得相对较小以减小码字大小的权重。In one example, the value of lambda is between 0 and 1, for example, it can be set to 0.5. The user can set it according to the specific requirements for bit rate and distortion. If the requirement for bit rate is higher, the value can be set to 0.5. Lambda is set relatively large to increase the weight of codeword size. On the contrary, if the distortion is required to be small, lambda can be set relatively small to reduce the weight of codeword size.
在本公一示例性的实施例中，对点云进行压缩编码时对属性残差值进行变换，由于对点云点的属性残差值进行变换和反变换时，是将多个点组成一组，对该组多个点的属性残差值一起进行变换(如小波变换或DCT变换)和反变换。在此情况下，将单个点计算的第一量化残差值替换为指定的一个量化残差值后，在反变换时，会对同组其他点反变换得到的重建残差值造成影响，进而影响其他点的属性重建值和属性失真。因此根据单个点使用率失真代价最小确定的最终的量化残差值，不一定是整体最优的。In an exemplary embodiment of the present disclosure, the attribute residual values are transformed when compressing and encoding the point cloud. Because when transforming and inverse transforming the attribute residual values of the point cloud points, multiple points are composed into one Group, the attribute residual values of multiple points in the group are transformed together (such as wavelet transform or DCT transform) and inverse transform. In this case, after replacing the first quantized residual value calculated at a single point with a specified quantized residual value, during the inverse transformation, it will affect the reconstructed residual values obtained by inverse transformation of other points in the same group, and thus Affects the attribute reconstruction values and attribute distortion of other points. Therefore, the final quantized residual value determined based on the minimum usage rate distortion cost of a single point is not necessarily the overall optimal one.
本实施例针对变换分支，提出了一种基于一组当前待解码点的率失真优化方法，其中：This embodiment proposes a rate-distortion optimization method based on a set of current points to be decoded for the transform branch, where:
所述计算当前待编码点的属性的第一量化残差值，包括：以多个当前待编码点D 1,D 2,…,D N为一组共同进行属性残差值的变换，对变换后的属性残差值分别做量化处理，得到D 1,D 2,…,D N各自的第一量化残差值； The calculation of the first quantized residual value of the attribute of the current point to be encoded includes: using a plurality of current points to be encoded D 1 , D 2 ,..., D N as a group to jointly transform the attribute residual value, and perform the transformation The final attribute residual values are quantized respectively, and the first quantized residual values of D 1 , D 2 ,..., D N are obtained;
所述根据编码所述第一量化残差值的率失真代价和编码指定的第二量化残差值的率失真代价，将所述第一量化残差值和第二量化残差值中的一个值确定为当前待编码点最终的量化残差值，包括：According to the rate distortion cost of encoding the first quantized residual value and the rate distortion cost of encoding the specified second quantized residual value, one of the first quantized residual value and the second quantized residual value is The value is determined as the final quantized residual value of the current point to be encoded, including:
计算D 1,D 2,…,D N使用第一编码模式的率失真代价，所述第一编码模式下对D 1,D 2,…,D N各自的第一量化残差值编码； Calculate the rate distortion cost of D 1 , D 2 ,..., D N using a first coding mode in which the first quantized residual value of each of D 1 , D 2 ,..., D N is coded;
计算D 1,D 2,…,D N使用第二编码模式的率失真代价，所述第二编码模式下对为D 1,D 2,…,D N指定的一组量化残差值编码； Calculate the rate distortion cost of D 1 , D 2 ,..., D N using a second encoding mode in which a set of quantized residual values specified for D 1 , D 2 ,..., D N is encoded;
确定所述第一编码模式和第二编码模式中率失真代价最优的一种编码模式，将该编码模式下所编码的D 1,D 2,…,D N各自的量化残差值作为D 1,D 2,…,D N各自最终的量化残差值。 Determine a coding mode with the optimal rate distortion cost in the first coding mode and the second coding mode, and use the respective quantized residual values of D 1 , D 2 ,..., D N coded in this coding mode as D 1 , D 2 ,...,D N are the final quantized residual values of each.
在本实施例的一个示例中，一个第二编码模式内为D 1,D 2,…,D N指定的一组量化残差值是相同的值组成的，即为D 1,D 2,…,D N统一指定一个值。在另一示例中，在一个第二编码模式内为D 1,D 2,…,D N分别指定一个量化残差值，为D 1,D 2,…,D N分别指定的量化残差值可以相同也可以不同。为D 1,D 2,…,D N 分别指定一个量化残差值时，可以采用前述实施例的方式，如指定为0，或者指定一个非零整数值。 In an example of this embodiment, a set of quantized residual values specified for D 1 , D 2 ,..., D N in a second coding mode are composed of the same values, that is, D 1 , D 2 ,... ,D N uniformly specifies a value. In another example, a quantized residual value is assigned to D 1 , D 2 , ..., D N respectively within a second encoding mode, and a quantized residual value is assigned to D 1 , D 2 , ..., D N respectively. Can be the same or different. When specifying a quantized residual value for D 1 , D 2 ,..., D N respectively, the method of the previous embodiment can be used, such as specifying it as 0, or specifying a non-zero integer value.
在本实施例的一个示例中，所述确定所述第一编码模式和第二编码模式中率失真代价最优的一种编码模式，包括：In an example of this embodiment, determining the encoding mode with the optimal rate-distortion cost among the first encoding mode and the second encoding mode includes:
对每一编码模式，计算对D 1,D 2,…,D N在该编码模式下各自的量化残差值分别进行编码的率失真代价Cost 1,Cost 2,…,Cost N；如前所述，D 1,D 2,…,D N在第一编码模式下的量化残差值即D 1,D 2,…,D N各自的第一量化残差值，D 1,D 2,…,D N在第二编码模式下的量化残差值即该模式下为D 1,D 2,…,D N统一指定或分别指定的量化残差值。 For each coding mode, calculate the rate distortion costs Cost 1 , Cost 2 ,..., Cost N for encoding the respective quantized residual values of D 1 , D 2 ,..., D N in this coding mode; as before As described above, the quantized residual values of D 1 , D 2 ,..., D N in the first encoding mode are the first quantized residual values of D 1 , D 2 ,..., D N respectively, D 1 , D 2 ,... , the quantized residual value of D N in the second encoding mode is the quantized residual value specified uniformly or separately for D 1 , D 2 ,..., D N in this mode.
计算得到Cost 1,Cost 2,…,Cost N后，可以按照以下策略确定率失真代价最优的一种编码模式： After calculating Cost 1 , Cost 2 ,...,Cost N , you can determine the encoding mode with the optimal rate-distortion cost according to the following strategy:
对每一所述编码模式下计算的Cost 1,Cost 2,…,Cost N求和，将和最小的一种编码模式确定为率失真代价最优的编码模式；或者， Sum the Cost 1 , Cost 2 ,..., Cost N calculated in each coding mode, and determine the coding mode with the smallest sum as the coding mode with the optimal rate distortion cost; or,
判断是否有所述第二编码模式满足条件：该模式下计算的Cost i均小于所述第一编码模式下计算的Cost i，如果有，将满足该条件的一种第二编码模式确定为率失真代价最优的编码模式，i＝1,2,…,N。 Determine whether any of the second coding modes satisfies the condition: the Cost i calculated in this mode are all smaller than the Cost i calculated in the first coding mode. If so, determine a second coding mode that satisfies this condition as the rate. Coding mode with optimal distortion cost, i=1,2,…,N.
满足该条件的第二编码模式可能有一种或多种。在满足该条件的第二编码模式有一种时，确定满足该条件的该第二编码模式为率失真代价最优的编码模式。而在第二编码模式有多种时，可以将Cost 1,Cost 2,…,Cost N之和最小的第二编码模式确定为率失真代价最优的编码模式。 There may be one or more second encoding modes that satisfy this condition. When there is one second coding mode that satisfies the condition, the second coding mode that satisfies the condition is determined to be the coding mode with the optimal rate-distortion cost. When there are multiple second coding modes, the second coding mode with the smallest sum of Cost 1 , Cost 2 ,..., Cost N can be determined as the coding mode with the optimal rate-distortion cost.
如果没有第二编码模式满足该条件，在一示例中，可以确定所述第一编码模式为率失真代价最优的编码模式。但在其他示例中，也可以进一步判断是否有第二编码模式满足其他的条件：如Cost 1,Cost 2,…,Cost N之和小于所述第一编码模式的Cost 1,Cost 2,…,Cost N之和，且有一半以上的点在该第二编码模式下的Cost小于在所述第一编码模式下的Cost。 If no second coding mode satisfies this condition, in an example, the first coding mode may be determined to be the coding mode with the optimal rate-distortion cost. But in other examples, it can also be further determined whether there is a second encoding mode that meets other conditions: for example, the sum of Cost 1 , Cost 2 ,..., Cost N is less than the Cost 1 , Cost 2 ,..., of the first encoding mode. The sum of Cost N , and the Cost of more than half of the points in the second encoding mode is smaller than the Cost in the first encoding mode.
本公开一实施例还提供了一种点云属性压缩方法，如图4所示，包括：An embodiment of the present disclosure also provides a point cloud attribute compression method, as shown in Figure 4, including:
步骤210，根据当前待编码点的属性值和属性预测值，确定当前待编码点的属性残差值；Step 210: Determine the attribute residual value of the current point to be encoded based on the attribute value and attribute prediction value of the current point to be encoded;
步骤220，按照本公开任一实施例所述的点云率失真优化方法确定当前待编码点最终的量化残差值并对所述最终的量化残差值进行熵编码。Step 220: Determine the final quantized residual value of the current point to be encoded according to the point cloud rate distortion optimization method described in any embodiment of the present disclosure, and perform entropy coding on the final quantized residual value.
本实施例的点云属性压缩方法采用了本公开前述实施例的率失真优化方法对量化残差值的取值进行优化，可以提高压缩编码的性能。The point cloud attribute compression method of this embodiment adopts the rate-distortion optimization method of the previous embodiment of the present disclosure to optimize the value of the quantized residual value, which can improve the performance of compression encoding.
本公开一实施例还提供了一种点云属性码流，其中，所述点云属性码流按照本公开任一实施例所述的点云属性压缩方法生成。An embodiment of the present disclosure also provides a point cloud attribute code stream, wherein the point cloud attribute code stream is generated according to the point cloud attribute compression method described in any embodiment of the present disclosure.
本公开一实施例还提供了一种点云率失真优化装置，如图5所示，包括处理器5以及存储有计算机程序的存储器6，其中，所述处理器5执行所述计算机程序时能够实现如本公开任一实施例所述的点云率失真优化方法。An embodiment of the present disclosure also provides a point cloud rate distortion optimization device. As shown in Figure 5, it includes a processor 5 and a memory 6 storing a computer program. When the processor 5 executes the computer program, it can Implement the point cloud rate distortion optimization method as described in any embodiment of the present disclosure.
本公开一实施例还提供了一种点云压缩装置，参照图5，包括处理器以及存储有计算机程序的存储器，其中，所述处理器执行所述计算机程序时能够实现如本公开任一实施例所述的点云属性压缩方法。An embodiment of the present disclosure also provides a point cloud compression device. Referring to Figure 5, it includes a processor and a memory storing a computer program. When the processor executes the computer program, it can implement any implementation of the present disclosure. The point cloud attribute compression method described in the example.
本公开一实施例还提供了一种非瞬态计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其中，所述计算机程序时被处理器执行时能够实现如本公开任一实施例所述的点云率失真优化方法，或能够实现如本公开任一实施例所述的点云属性压缩方法。An embodiment of the present disclosure also provides a non-transitory computer-readable storage medium. The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it can implement any of the aspects of the present disclosure. The point cloud rate distortion optimization method described in the embodiment may be able to implement the point cloud attribute compression method described in any embodiment of the present disclosure.
本公开上述实施例的处理器可以是通用处理器，包括中央处理器(Central Processing Unit，简称CPU)、网络处理器(Network Processor，简称NP)、微处理器等等，也可以是其他常规的处理器等；所述处理器还可以是数字信号处理器(DSP)、专用集成电路(ASIC)、现成可编程门阵列(FPGA)、离散逻辑或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件；也可以是上述器件的组合。即上述实施例的处理器可以是实现本发明实施例中公开的各方法、步骤及逻辑框图的任何处理器件或器件组合。如果部分地以软件来实施本公开实施例，那么可将用于软件的指令存储在合适的非易失性计算机可读存储媒体中，且可使用一个或多个处理器在硬件中执行所述指令从而实施本公开实施例的方法。The processor in the above embodiments of the present disclosure may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), a microprocessor, etc., or it may be other conventional processors. Processor, etc.; the processor may also be a digital signal processor (DSP), application specific integrated circuit (ASIC), off-the-shelf programmable gate array (FPGA), discrete logic or other programmable logic devices, discrete gates or transistor logic devices , discrete hardware components; it can also be a combination of the above devices. That is, the processor in the above embodiments can be any processing device or device combination that implements the methods, steps and logical block diagrams disclosed in the embodiments of the present invention. If embodiments of the present disclosure are implemented in part in software, instructions for the software may be stored in a suitable non-volatile computer-readable storage medium and may be executed in hardware using one or more processors. Instructions are provided to perform the methods of embodiments of the present disclosure.
本公开一实施例提供了一种点云率失真优化方法，涉及预测分支，通过在编码端引入率失真优化算法，从而提升编码效率。An embodiment of the present disclosure provides a point cloud rate distortion optimization method, which involves a prediction branch and introduces a rate distortion optimization algorithm on the encoding side to improve encoding efficiency.
编码端执行以下的点云率失真优化处理：The encoding end performs the following point cloud rate distortion optimization processing:
a)获取当前待编码点的属性预测值A‘；a) Obtain the attribute prediction value A′ of the current point to be encoded;
b)根据当前待编码点的属性值和属性预测值计算当前待编码点的属性残差值：b) Calculate the attribute residual value of the current point to be encoded based on the attribute value and attribute prediction value of the current point to be encoded:
delta＝A-A‘delta＝A-A‘
c)对当前待编码点的属性残差值做量化处理得到当前待编码点属性的量化残差值，称为第一量化残差值；c) Perform quantization processing on the attribute residual value of the current point to be encoded to obtain the quantized residual value of the attribute of the current point to be encoded, which is called the first quantized residual value;
量化处理时可以对属性残差值的绝对值进行量化，量化结果再与属性残差值的符号相乘，计算出第一量化残差值。During the quantization process, the absolute value of the attribute residual value can be quantified, and the quantization result is multiplied by the sign of the attribute residual value to calculate the first quantized residual value.
d)对第一量化残差值做反量化处理，得到当前待编码点的重建残差值，将当前待编码点的重建残差值与属性预测值相加得到当前待编码点的属性重建值
Figure PCTCN2022104391-appb-000006
d) Perform inverse quantization processing on the first quantized residual value to obtain the reconstructed residual value of the current point to be encoded, and add the reconstructed residual value of the current point to be encoded and the attribute prediction value to obtain the attribute reconstruction value of the current point to be encoded.
Figure PCTCN2022104391-appb-000006
反量化处理时对第一量化残差值的绝对值进行反量化，反量化的结果再与第一量化残差值的符号相乘，计算出重建残差值。During the inverse quantization process, the absolute value of the first quantized residual value is inversely quantized, and the inverse quantization result is multiplied by the sign of the first quantized residual value to calculate the reconstructed residual value.
e)引入以下代价方程，计算两种编码模式下的率失真开销：e) Introduce the following cost equation to calculate the rate-distortion overhead in the two coding modes:
cost＝lambda*R+Dcost＝lambda*R+D
其中，lambda为设定的常数，可以通过改变lambda的大小控制码率和失真对开销的影响。Among them, lambda is a set constant, and the impact of code rate and distortion on overhead can be controlled by changing the size of lambda.
本实施例的两种编码模式包括第一编码模式和第二编码模式：The two coding modes in this embodiment include the first coding mode and the second coding mode:
第一编码模式即编码计算出的第一量化残差值的模式，与传统的编码模式相同。计算当前待编码点使用第一编码模式的率失真代价时，公式中的R为编码第一量化残差值所需的码字大小，可以用熵编码第一量化残差值(或无符号的第一量化残差值)所需的二进制位表示，与采用的熵编码算法有关；D为按第一编码模式编码得到的属性重建值与属性值之间的差异，当前待编码点使用第一编码模式编码的率失真代价记为cost1The first encoding mode is a mode for encoding the calculated first quantized residual value, which is the same as the traditional encoding mode. When calculating the rate distortion cost of using the first encoding mode for the current point to be encoded, R in the formula is the codeword size required to encode the first quantized residual value. The first quantized residual value can be encoded with entropy (or unsigned The binary bit representation required for the first quantized residual value) is related to the entropy coding algorithm used; D is the difference between the attribute reconstruction value and the attribute value encoded according to the first encoding mode. The current to-be-encoded point uses the first The rate distortion cost of coding mode encoding is recorded as cost1
第二编码模式即编码指定的第二量化残差值的模式，本实施例该指定的第二量化残差值为0。由于反量化后的残差重建值也等于0，相当于将当前待编码点的属性预测值直接作为属性重建值，此时R最小(如为0)；D为属性预测值与属性值之间的差异如差的绝对值；当前待编码点使用第二编码模式编码的率失真开销记为cost2；The second encoding mode is a mode for encoding a specified second quantized residual value. In this embodiment, the specified second quantized residual value is 0. Since the residual reconstruction value after inverse quantization is also equal to 0, it is equivalent to directly using the attribute prediction value of the current point to be encoded as the attribute reconstruction value. At this time, R is the smallest (for example, 0); D is the difference between the attribute prediction value and the attribute value. The difference is the absolute value of the difference; the rate-distortion overhead of encoding the current point to be encoded using the second encoding mode is recorded as cost2;
虽然本实施例只采用一种第二编码模式，但本公开不局限于此，也可以计算多种第二编码模式的率失真代价，例如，在第一量化残差值为2时，可以增加编码指定的量化残差值为1的第二编码模式参与比较。Although this embodiment only uses one second coding mode, the disclosure is not limited thereto. The rate distortion cost of multiple second coding modes can also be calculated. For example, when the first quantization residual value is 2, you can add The second encoding mode with a specified quantization residual value of 1 is involved in the comparison.
此外，如果第一量化残差值为0，可以直接对0进行熵编码，不需要再进行率失真代价的计算和比较。In addition, if the first quantization residual value is 0, entropy encoding can be performed directly on 0, without the need to calculate and compare the rate-distortion cost.
f)比较两种编码模式的率失真开销：f) Compare the rate-distortion overhead of the two encoding modes:
如果cost2<cost1，则采用第二编码模式，将指定的量化残差值0作为当前待编码点最终的量化残差值(相当于将量化残差值置0)进行熵编码，生成码流，当前待编码点的属性重建值即为属性预测值；If cost2<cost1, the second coding mode is used, and the specified quantized residual value 0 is used as the final quantized residual value of the current point to be encoded (equivalent to setting the quantized residual value to 0) for entropy coding to generate a code stream. The attribute reconstruction value of the current point to be encoded is the attribute prediction value;
如果cost2>cost1，否则采用第一编码模式，对第一量化残差值进行熵编码，生成码流；此时当前待编码点的属性重建值等于当前待编码点的属性预测值加上重建残差值，重建残差值通过对第一量化残差值进行反量化处理而得到。If cost2>cost1, otherwise the first coding mode is used to perform entropy coding on the first quantized residual value to generate a code stream; at this time, the attribute reconstruction value of the current point to be encoded is equal to the predicted attribute value of the current point to be encoded plus the reconstructed residual value. The difference value, the reconstructed residual value is obtained by performing inverse quantization processing on the first quantized residual value.
在cost2＝cost1的情况下，可以采用第一编码模式和第二编码模式中的任一种。When cost2=cost1, either the first encoding mode or the second encoding mode may be used.
编码端采用上述率失真优化方法之后，当前待编码点的量化残差值可能有变化，但解码端仍按照原有的方式进行解码，计算该点的属性重建值。After the encoding end adopts the above rate-distortion optimization method, the quantized residual value of the current point to be encoded may change, but the decoding end still decodes in the original way and calculates the attribute reconstruction value of the point.
为了验证本实施例的效果，本实施例将lambda设为0.5，在音视频编码标准(AVS：Audio Video coding Standard)的点云压缩参考平台PCRM6.0上进行了测试，涉及以下2种测试条件：limit-lossy geometry,lossy attributes，即几何位置有限度有损，属性有损；lossless geometry,lossy attributes，即几何位置无损、属性有损。使用的测试序列包括通用测试序列Cat1A，Cat1C，Cat2三类，测试了反射率属性的实验结果：In order to verify the effect of this embodiment, this embodiment sets lambda to 0.5 and conducts tests on the point cloud compression reference platform PCRM6.0 of the Audio Video Coding Standard (AVS: Audio Video coding Standard), involving the following two test conditions. : limit-lossy geometry, lossy attributes, that is, the geometric position is limited and lossy, and the attributes are lossy; lossless geometry, lossy attributes, that is, the geometric position is lossless and the attributes are lossy. The test sequences used include three general test sequences: Cat1A, Cat1C, and Cat2. The experimental results of the reflectivity properties are tested:
表1 C1-limit-lossy geometry,lossy attributesTable 1 C1-limit-lossy geometry,lossy attributes
Figure PCTCN2022104391-appb-000007
Figure PCTCN2022104391-appb-000007
表2 C2-lossless geometry,lossy attributesTable 2 C2-lossless geometry,lossy attributes
Figure PCTCN2022104391-appb-000008
Figure PCTCN2022104391-appb-000008
上表中，class表示通用测试序列的类别，EtE BD-AttrRate表示端到端码率-失真，refl表示反射率属性。实验结果表明采用本实施例的率失真优化方法，能够带来性能增益。In the above table, class represents the category of the general test sequence, EtE BD-AttrRate represents the end-to-end code rate-distortion, and refl represents the reflectance attribute. Experimental results show that the rate-distortion optimization method of this embodiment can bring performance gains.
本实施例编码端基于预测分支进行点云压缩编码时，引入率失真优化算法，通过计算码率与失真的综合代价，确定更优的量化残差值，从而提升编码效率，此外在解码端不需要额外增加操作。When the encoding end of this embodiment performs point cloud compression encoding based on the prediction branch, a rate-distortion optimization algorithm is introduced to determine a better quantized residual value by calculating the comprehensive cost of the code rate and distortion, thereby improving the encoding efficiency. In addition, the decoding end does not Additional operations are required.
本公开一实施例提供了一种点云率失真优化方法，涉及变换分支，通过在编码端引入率失真优化算法，从而提升编码效率。An embodiment of the present disclosure provides a point cloud rate distortion optimization method, which involves a transformation branch and introduces a rate distortion optimization algorithm on the encoding side, thereby improving encoding efficiency.
编码端执行以下的点云率失真优化处理：The encoding end performs the following point cloud rate distortion optimization processing:
a)将多个当前待编码点作为一组，计算该组当前待编码点各自的属性预测值A′ j，j＝1,2,…,N，N为该组当前待编码点的个数，一个示例中N＝4； a) Group multiple current points to be coded as a group, and calculate the predicted attribute values A' j of each of the current points to be coded in the group, j=1,2,...,N, where N is the number of points currently to be coded in the group. , N=4 in one example;
b)根据该组当前待编码点各自的属性值和属性预测值，计算该组当前待编码点各自的属性残差值：b) Based on the respective attribute values and attribute prediction values of the current group of points to be encoded, calculate the respective attribute residual values of the group of current points to be encoded:
delta j＝A j-A′ j delta j =A j -A′ j
c)对该组当前待编码点的属性残差值一起进行N维DCT变换，得到该组当前待编码点各自的变换后的属性残差值；c) Perform N-dimensional DCT transformation on the attribute residual values of the current group of points to be encoded together to obtain the transformed attribute residual values of the current group of points to be encoded;
d)对变换后的属性残差值分别做量化处理，得到该组当前待编码点各自的第一量化残差值；d) Perform quantization processing on the transformed attribute residual values respectively to obtain the first quantized residual values of the current group of points to be encoded;
e)对该组当前待编码点各自的第一量化残差值进行反量化，得到该组当前待编码点各自的重建残差值；e) Perform inverse quantization on the first quantized residual values of the current set of points to be coded, and obtain the respective reconstructed residual values of the current set of points to be coded;
f)对该组当前待编码点的重建残差值一起进行N维DCT反变换，得到该组当前待编码点各自的变换后的重建残差值；f) Perform N-dimensional DCT inverse transformation on the reconstructed residual values of the current group of points to be encoded together to obtain the transformed reconstruction residual values of the current group of points to be encoded;
g)将该组当前待编码点各自的变换后的重建残差值与属性预测值相加，得到该组当前待编码点各自的属性重建值
Figure PCTCN2022104391-appb-000009
g) Add the transformed reconstruction residual values and attribute prediction values of the current group of points to be encoded to obtain the respective attribute reconstruction values of the group of current points to be encoded.
Figure PCTCN2022104391-appb-000009
h)引入代价方程，计算两种编码模式下的率失真开销：h) Introduce the cost equation to calculate the rate distortion overhead in the two coding modes:
cost j＝lambda*R j+D j cost j ＝lambda*R j +D j
其中，lambda为设定的常数，可以通过改变lambda的大小控制码率和失真对开销的影响。Among them, lambda is a set constant, and the impact of code rate and distortion on overhead can be controlled by changing the size of lambda.
本实施例的两种编码模式包括第一编码模式和第二编码模式：The two coding modes in this embodiment include the first coding mode and the second coding mode:
第一编码模式first coding mode
第一编码模式是编码D 1,D 2,…,D N各自的第一量化残差值的编码模式。对该组当前待编码点中的每一个点，均是编码该点的第一量化残差值，与传统的编码模式相同。计算第j个点使用第一编码模式的率失真代价时，R j为编码第j个点的第一量化残差值所需的码字大小；D j为按第一编码模式编码时第j个点的属性重建值
Figure PCTCN2022104391-appb-000010
与属性值A j之间的差异，cost1 j为计算出的第j个点使用第一编码模式编码的率失真代价。 The first coding mode is a coding mode that codes the first quantized residual value of each of D 1 , D 2 ,..., D N . For each point in the group of current points to be encoded, it is the first quantized residual value of encoding the point, which is the same as the traditional encoding mode. When calculating the rate distortion cost of the j-th point using the first encoding mode, R j is the codeword size required to encode the first quantized residual value of the j-th point; D j is the j-th point when encoding in the first encoding mode. Point attribute reconstruction value
Figure PCTCN2022104391-appb-000010
The difference from the attribute value A j , cost1 j is the calculated rate distortion cost of encoding the j-th point using the first encoding mode.
第二编码模式Second coding mode
第二编码模式是编码为D 1,D 2,…,D N指定的一组量化残差值的编码模式。使用第二编码模式时，对该组当前待编码点中的每一个点，编码为该点指定的一个量化残差值。对不同的点可以统一指定一个量化残差值，也可以分别指定量化残差值，分别指定时为不同的点指定的量化残差值可以不同。 The second coding mode is a coding mode that codes a set of quantized residual values specified for D 1 , D 2 ,..., D N . When using the second encoding mode, for each point in the group of points currently to be encoded, a quantized residual value specified for the point is encoded. A quantized residual value can be specified uniformly for different points, or the quantized residual values can be specified separately. When specified separately, the quantized residual values specified for different points can be different.
本实施例为每一个点指定的量化残差值均为0，相当于在属性重建时将每个点的属性预测值直接作为属性重建值。此时R j为0，D j为第j个点的属性预测值A′ j与属性值A j之间的差异，计算出的第j个点使用第二编码模式的率失真代价记为cost2 j； In this embodiment, the quantized residual value specified for each point is 0, which is equivalent to directly using the attribute prediction value of each point as the attribute reconstruction value during attribute reconstruction. At this time, R j is 0, and D j is the difference between the attribute prediction value A′ j and the attribute value A j of the j-th point. The calculated rate-distortion cost of the j-th point using the second coding mode is recorded as cost2 j ;
i)比较两种编码模式的率失真开销，判断该组当前待编码点中的每一个点是否都有cost2 j<cost1 j成立，j＝1,2,…,N； i) Compare the rate-distortion overhead of the two encoding modes, and determine whether each point in the group of current points to be encoded has cost2 j <cost1 j , j=1,2,...,N;
如果是，确定第二编码模式为最优的编码模式，组中所有点都将0作为最终的量化残差值，对0进行熵编码，生成码流；If so, the second coding mode is determined to be the optimal coding mode. All points in the group use 0 as the final quantized residual value, perform entropy coding on 0, and generate a code stream;
如果否，即至少有一个cost2 j≥cost1 j，确定第一编码模式为最优的编码模式，将组中N个点各自的第一量化残差值作为各自最终的量化残差值进行熵编码，生成码流；编码端还对组内N个点各自的第一量化残差值做反量化处理，得到N个重建残差值，然后对N个重建残差值进行N维DCT反变换，得到N个点各自的反变换后的重建残差值，最后将N个点的反变换后的重建残差值与属性预测值相加，得到N个当前待编码点各自的属性重建值。 If not, that is, there is at least one cost2 j ≥ cost1 j , determine the first coding mode as the optimal coding mode, and use the first quantized residual value of each N point in the group as the final quantized residual value for entropy coding. , generate a code stream; the encoding end also performs inverse quantization processing on the first quantized residual values of each N point in the group to obtain N reconstructed residual values, and then performs N-dimensional DCT inverse transformation on the N reconstructed residual values. Obtain the reconstruction residual value after inverse transformation of each N point. Finally, add the reconstruction residual value after inverse transformation of N points and the attribute prediction value to obtain the attribute reconstruction value of each of the N current points to be encoded.
在另一实施例中，采用不同的策略来确定最优的编码模式，该实施例对第一编码模式下计算的N个率失真代价求和，对第二编码模式下计算的N个率失真代价求和，将和最小的一种编码模式确定为率失真代价最优的编码模式。In another embodiment, different strategies are used to determine the optimal coding mode. In this embodiment, the N rate distortion costs calculated in the first coding mode are summed, and the N rate distortion costs calculated in the second coding mode are summed. The costs are summed, and the encoding mode with the smallest sum is determined as the encoding mode with the optimal rate-distortion cost.
本实施例编码端基于变换分支进行点云压缩编码时，引入率失真优化算法，将一起做属性残差值变换的多个点组成一组，综合考虑该组当前待编码点的码率与失真的代价，为该组当前待编码点确定更优的量化残差值，从而提升编码效率。解码端也不需要额外增加操作。When the encoding end of this embodiment performs point cloud compression encoding based on the transformation branch, a rate-distortion optimization algorithm is introduced to form a group of multiple points that undergo attribute residual value transformation together, and comprehensively consider the code rate and distortion of the current group of points to be encoded. At a cost, a better quantized residual value is determined for the group of current points to be encoded, thereby improving coding efficiency. No additional operations are required on the decoding end.
在一个或多个示例性实施例中，所描述的功能可以硬件、软件、固件或其任一组合来实施。如果以软件实施，那么功能可作为一个或多个指令或代码存储在计算机可读介质上或经由计算机可读介质传输，且由基于硬件的处理单元执行。计算机可读介质可包含对应于例如数据存储介质等有形介质的计算机可读存储介质，或包含促进计算机程序例如根据通信协议从一处传送到另一处的任何介质的通信介质。以此方式，计算机可读介质通常可对应于非暂时性的有形计算机可读存储介质或例如信号或载波等通信介质。数据存储介质可为可由一个或多个计算机或者一个或多个处理器存取以检索用于实施本公开中描述的技术的指令、代码和/或数据结构的任何可用介质。计算机程序产品可包含计算机可读介质。In one or more exemplary embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media that corresponds to tangible media, such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another, such as according to a communications protocol. In this manner, computer-readable media generally may correspond to non-transitory, tangible computer-readable storage media or communication media such as a signal or carrier wave. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and/or data structures for implementing the techniques described in this disclosure. A computer program product may include computer-readable media.
举例来说且并非限制，此类计算机可读存储介质可包括RAM、ROM、EEPROM、CD-ROM或其它光盘存储装置、磁盘存储装置或其它磁性存储装置、快闪存储器或可用来以指令或数据结构的形 式存储所要程序代码且可由计算机存取的任何其它介质。而且，还可以将任何连接称作计算机可读介质举例来说，如果使用同轴电缆、光纤电缆、双绞线、数字订户线(DSL)或例如红外线、无线电及微波等无线技术从网站、服务器或其它远程源传输指令，则同轴电缆、光纤电缆、双纹线、DSL或例如红外线、无线电及微波等无线技术包含于介质的定义中。然而应了解，计算机可读存储介质和数据存储介质不包含连接、载波、信号或其它瞬时(瞬态)介质，而是针对非瞬时有形存储介质。如本文中所使用，磁盘及光盘包含压缩光盘(CD)、激光光盘、光学光盘、数字多功能光盘(DVD)、软磁盘或蓝光光盘等，其中磁盘通常以磁性方式再生数据，而光盘使用激光以光学方式再生数据。上文的组合也应包含在计算机可读介质的范围内。By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory or may be used to store instructions or data. Any other medium that stores the desired program code in the form of a structure and that can be accessed by a computer. Furthermore, any connection is also termed a computer-readable medium if, for example, any connection is transmitted from a website, server, or using or other remote source transmits instructions, then coaxial cable, fiber optic cable, double-stranded wire, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of medium. It should be understood, however, that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient (transitory) media, but are directed to non-transitory tangible storage media. As used herein, disks and optical discs include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, or Blu-ray discs. Disks usually reproduce data magnetically, while optical discs use lasers to reproduce data. Regenerate data optically. Combinations of the above should also be included within the scope of computer-readable media.
可由例如一个或多个数字信号理器(DSP)、通用微处理器、专用集成电路(ASIC)现场可编程逻辑阵列(FPGA)或其它等效集成或离散逻辑电路等一个或多个处理器来执行指令。因此，如本文中所使用的术语“处理器”可指上述结构或适合于实施本文中所描述的技术的任一其它结构中的任一者。另外，在一些方面中，本文描述的功能性可提供于经配置以用于编码和解码的专用硬件和/或软件模块内，或并入在组合式编解码器中。并且，可将所述技术完全实施于一个或多个电路或逻辑元件中。May be performed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Execute instructions. Accordingly, the term "processor" as used herein may refer to any of the structures described above or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or incorporated in a combined codec. Furthermore, the techniques may be implemented entirely in one or more circuits or logic elements.
本公开实施例的技术方案可在广泛多种装置或设备中实施，包含无线手机、集成电路(IC)或一组IC(例如，芯片组)。本公开实施例中描各种组件、模块或单元以强调经配置以执行所描述的技术的装置的功能方面，但不一定需要通过不同硬件单元来实现。而是，如上所述，各种单元可在编解码器硬件单元中组合或由互操作硬件单元(包含如上所述的一个或多个处理器)的集合结合合适软件和/或固件来提供。Technical solutions of embodiments of the present disclosure may be implemented in a wide variety of devices or equipment, including wireless handsets, integrated circuits (ICs), or a set of ICs (eg, chipsets). Various components, modules or units are depicted in embodiments of the present disclosure to emphasize functional aspects of devices configured to perform the described techniques, but do not necessarily require implementation by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit or provided by a collection of interoperating hardware units (including one or more processors as described above) in conjunction with suitable software and/or firmware.

Claims (16)

  1. 一种点云率失真优化方法，包括：A point cloud rate distortion optimization method, including:
    计算当前待编码点的属性的第一量化残差值；Calculate the first quantized residual value of the attribute of the current point to be encoded;
    根据编码所述第一量化残差值的率失真代价和编码指定的第二量化残差值的率失真代价，将所述第一量化残差值和第二量化残差值中的一个值确定为当前待编码点最终的量化残差值；One of the first quantized residual value and the second quantized residual value is determined based on the rate distortion cost of encoding the first quantized residual value and the rate distortion cost of encoding the specified second quantized residual value. is the final quantized residual value of the current point to be encoded;
    对当前待编码点最终的量化残差值进行熵编码。Entropy coding is performed on the final quantized residual value of the current point to be encoded.
  2. 如权利要求1所述的方法，其中：The method of claim 1, wherein:
    所述指定的第二量化残差值为0；或者，所述指定的第二量化残差值包括0和非零整数值。The specified second quantized residual value is 0; or, the specified second quantized residual value includes 0 and a non-zero integer value.
  3. 如权利要求2所述的方法，其中：The method of claim 2, wherein:
    所述非零整数值包括以下数值中的任意一种或更多种：1，-1，2，-2。The non-zero integer value includes any one or more of the following values: 1, -1, 2, -2.
  4. 如权利要求1所述的方法，其中：The method of claim 1, wherein:
    所述方法还包括：在所述第一量化残差值为0时，将所述第一量化残差值作为当前待编码点最终的量化残差值。The method further includes: when the first quantized residual value is 0, using the first quantized residual value as the final quantized residual value of the current point to be encoded.
  5. 如权利要求1所述的方法，其中：The method of claim 1, wherein:
    所述计算当前待编码点的属性的第一量化残差值，包括：对一个当前待编码点的属性残差值做量化处理，得到该当前待编码点的第一量化残差值；The calculation of the first quantized residual value of the attribute of the current point to be encoded includes: quantizing the attribute residual value of a current point to be encoded to obtain the first quantized residual value of the current point to be encoded;
    所述根据编码所述第一量化残差值的率失真代价和编码指定的第二量化残差值的率失真代价，将所述第一量化残差值和第二量化残差值中的一个值确定为当前待编码点最终的量化残差值，包括：According to the rate distortion cost of encoding the first quantized residual value and the rate distortion cost of encoding the specified second quantized residual value, one of the first quantized residual value and the second quantized residual value is The value is determined as the final quantized residual value of the current point to be encoded, including:
    计算该当前待编码点使用第一编码模式的率失真代价，所述第一编码模式下对所述第一量化残差值编码；Calculate the rate distortion cost of the current to-be-encoded point using a first encoding mode in which the first quantized residual value is encoded;
    计算该当前待编码点使用第二编码模式的率失真代价，所述第二编码模式下对所述第二量化残差值编码；Calculate the rate distortion cost of the current to-be-encoded point using a second encoding mode in which the second quantized residual value is encoded;
    将所述第一编码模式和第二编码模式中率失真代价值小的一种编码模式所编码的量化残差值作为该当前待编码点最终的量化残差值。The quantized residual value encoded by the encoding mode with the smaller rate distortion cost among the first encoding mode and the second encoding mode is used as the final quantized residual value of the current point to be encoded.
  6. 如权利要求5所述的方法，其中：The method of claim 5, wherein:
    所述第二编码模式有一种，该第二编码模式下对一个所述第二量化残差值编码；或者The second coding mode includes one of the second coding modes, and one of the second quantized residual values is coded in the second coding mode; or
    所述第二编码模式有多种，每一种第二编码模式下对一个所述第二量化残差值编码，不同第二编码模式下编码的第二量化残差值不同。There are multiple second encoding modes, and one second quantized residual value is encoded in each second encoding mode. The second quantized residual value encoded in different second encoding modes is different.
  7. 如权利要求5所述的方法，其中：The method of claim 5, wherein:
    按以下公式计算该当前待编码点使用每一种编码模式的率失真代价cost：Calculate the rate distortion cost of each encoding mode for the current point to be encoded according to the following formula:
    cost＝lambda*R+Dcost＝lambda*R+D
    其中，lambda为设定的常数，R是编码该编码模式下的量化残差值所需的码字大小，D是按照该编码模式编码的情况下该当前待编码点的属性重建值与属性值的差异。Among them, lambda is a set constant, R is the codeword size required to encode the quantized residual value in this encoding mode, and D is the attribute reconstruction value and attribute value of the current point to be encoded when encoding in this encoding mode. difference.
  8. 如权利要求1所述的方法，其中：The method of claim 1, wherein:
    所述计算当前待编码点的属性的第一量化残差值，包括：以多个当前待编码点D 1,D 2,…,D N为一组共同进行属性残差值的变换，对变换后的属性残差值分别做量化处理，得到D 1,D 2,…,D N各自的第一量化残差值； The calculation of the first quantized residual value of the attribute of the current point to be encoded includes: using a plurality of current points to be encoded D 1 , D 2 ,..., D N as a group to jointly transform the attribute residual value, and perform the transformation The final attribute residual values are quantized respectively, and the first quantized residual values of D 1 , D 2 ,..., D N are obtained;
    所述根据编码所述第一量化残差值的率失真代价和编码指定的第二量化残差值的率失真代价，将所述第一量化残差值和第二量化残差值中的一个值确定为当前待编码点最终的量化残差值，包括：According to the rate distortion cost of encoding the first quantized residual value and the rate distortion cost of encoding the specified second quantized residual value, one of the first quantized residual value and the second quantized residual value is The value is determined as the final quantized residual value of the current point to be encoded, including:
    计算D 1,D 2,…,D N使用第一编码模式的率失真代价，所述第一编码模式下对D 1,D 2,…,D N各自的第一量化残差值编码； Calculate the rate distortion cost of D 1 , D 2 ,..., D N using a first coding mode in which the first quantized residual value of each of D 1 , D 2 ,..., D N is coded;
    计算D 1,D 2,…,D N使用第二编码模式的率失真代价，所述第二编码模式下对为D 1,D 2,…,D N指定的一组量化残差值编码； Calculate the rate distortion cost of D 1 , D 2 ,..., D N using a second encoding mode in which a set of quantized residual values specified for D 1 , D 2 ,..., D N is encoded;
    确定所述第一编码模式和第二编码模式中率失真代价最优的一种编码模式，将该编码模式下所编码的D 1,D 2,…,D N各自的量化残差值作为D 1,D 2,…,D N各自最终的量化残差值。 Determine a coding mode with the optimal rate distortion cost in the first coding mode and the second coding mode, and use the respective quantized residual values of D 1 , D 2 ,..., D N coded in this coding mode as D 1 , D 2 ,...,D N are the final quantized residual values of each.
  9. 如权利要求8所述的方法，其中：The method of claim 8, wherein:
    所述确定所述第一编码模式和第二编码模式中率失真代价最优的一种编码模式，包括：Determining a coding mode with the optimal rate-distortion cost among the first coding mode and the second coding mode includes:
    对每一种编码模式，计算对D 1,D 2,…,D N在该编码模式下各自的量化残差值分别进行编码的率失真代价Cost 1,Cost 2,…,Cost N； For each coding mode, calculate the rate distortion costs Cost 1 , Cost 2 ,..., Cost N for encoding the respective quantized residual values of D 1 , D 2 ,..., D N in this coding mode;
    按照以下策略确定率失真代价最优的一种编码模式：Determine the encoding mode with the optimal rate-distortion cost according to the following strategy:
    对每一编码模式下计算的Cost 1,Cost 2,…,Cost N求和，将和最小的一种编码模式确定为率失真代价最优的编码模式；或者 Sum the Cost 1 , Cost 2 ,..., Cost N calculated in each coding mode, and determine the coding mode with the smallest sum as the coding mode with the optimal rate-distortion cost; or
    判断是否有所述第二编码模式满足条件：该模式下计算的Cost i均小于所述第一编码模式下计算的Cost i，如果有，将满足该条件的一种第二编码模式确定为率失真代价最优的编码模式，i＝1,2,…,N。 Determine whether any of the second coding modes satisfies the condition: the Cost i calculated in this mode are all smaller than the Cost i calculated in the first coding mode. If so, determine a second coding mode that satisfies this condition as the rate. Coding mode with optimal distortion cost, i=1,2,…,N.
  10. 如权利要求9所述的方法，其中：The method of claim 9, wherein:
    所述方法还包括：如果没有所述第二编码模式满足该条件，确定所述第一编码模式为率失真代价最优的编码模式。The method further includes: if no second coding mode satisfies the condition, determining the first coding mode to be the coding mode with the optimal rate-distortion cost.
  11. 如权利要求9所述的方法，其中：The method of claim 9, wherein:
    所述将满足该条件的一种第二编码模式确定为率失真代价最优的编码模式，包括：Determining a second coding mode that satisfies this condition as the coding mode with the optimal rate-distortion cost includes:
    在满足该条件的第二编码模式有一种时，确定满足该条件的该第二编码模式为率失真代价最优的编码模式；When there is one second coding mode that satisfies the condition, determine that the second coding mode that satisfies the condition is the coding mode with the optimal rate distortion cost;
    在满足该条件的第二编码模式有多种时，将Cost 1,Cost 2,…,Cost N之和最小的第二编码模式确定为率失真代价最优的编码模式。 When there are multiple second coding modes that satisfy this condition, the second coding mode with the smallest sum of Cost 1 , Cost 2 ,..., Cost N is determined as the coding mode with the optimal rate-distortion cost.
  12. 一种点云属性压缩方法，包括：A point cloud attribute compression method, including:
    根据当前待编码点的属性值和属性预测值，确定当前待编码点的属性残差值；Determine the attribute residual value of the current point to be encoded based on the attribute value and attribute prediction value of the current point to be encoded;
    按照权利要求1至11中任一所述的点云率失真优化方法确定当前待编码点最终的量化残差值并对所述最终的量化残差值进行熵编码。According to the point cloud rate distortion optimization method according to any one of claims 1 to 11, the final quantized residual value of the current point to be encoded is determined and the final quantized residual value is entropy encoded.
  13. 一种点云属性码流，其中，所述点云属性码流按照如权利要求12所述的点云属性压缩方法生成。A point cloud attribute code stream, wherein the point cloud attribute code stream is generated according to the point cloud attribute compression method as claimed in claim 12.
  14. 一种点云率失真优化装置，包括处理器以及存储有计算机程序的存储器，其中，所述处理器执行所述计算机程序时能够实现如权利要求1至11中任一所述的点云率失真优化方法。A point cloud rate distortion optimization device, including a processor and a memory storing a computer program, wherein when the processor executes the computer program, it can achieve the point cloud rate distortion as described in any one of claims 1 to 11 Optimization.
  15. 一种点云压缩装置，其中，包括处理器以及存储有计算机程序的存储器，其中，所述处理器执行所述计算机程序时能够实现如权利要求12所述的点云属性压缩方法。A point cloud compression device, which includes a processor and a memory storing a computer program, wherein when the processor executes the computer program, it can implement the point cloud attribute compression method as claimed in claim 12.
  16. 一种非瞬态计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其中，所述计算机程序被处理器执行时能够实现如权利要求1至11中任一所述的点云率失真优化方法，或能够实现如权利要求12所述的点云属性压缩方法。A non-transitory computer-readable storage medium, the computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it can realize the point cloud as described in any one of claims 1 to 11 rate distortion optimization method, or can implement the point cloud attribute compression method as claimed in claim 12.
PCT/CN2022/104391 2022-07-07 2022-07-07 Point cloud rate-distortion optimization method, attribute compression method and apparatus, and storage medium WO2024007253A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/104391 WO2024007253A1 (en) 2022-07-07 2022-07-07 Point cloud rate-distortion optimization method, attribute compression method and apparatus, and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/104391 WO2024007253A1 (en) 2022-07-07 2022-07-07 Point cloud rate-distortion optimization method, attribute compression method and apparatus, and storage medium

Publications (1)

Publication Number Publication Date
WO2024007253A1 true WO2024007253A1 (en) 2024-01-11

Family

ID=89454559

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/104391 WO2024007253A1 (en) 2022-07-07 2022-07-07 Point cloud rate-distortion optimization method, attribute compression method and apparatus, and storage medium

Country Status (1)

Country Link
WO (1) WO2024007253A1 (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109274968A (en) * 2018-10-26 2019-01-25 西安科锐盛创新科技有限公司 Video compression self-adaptive quantization and inverse quantization method
CN109361922A (en) * 2018-10-26 2019-02-19 西安科锐盛创新科技有限公司 Predict quantization coding method
CN112565734A (en) * 2020-12-03 2021-03-26 西安电子科技大学 Point cloud attribute coding and decoding method and device based on hybrid coding
CN113284248A (en) * 2021-06-10 2021-08-20 上海交通大学 Encoding and decoding method, device and system for point cloud lossy compression
US20210385303A1 (en) * 2020-06-09 2021-12-09 Qualcomm Incorporated Attribute residual coding in g-pcc
WO2022061786A1 (en) * 2020-09-25 2022-03-31 Oppo广东移动通信有限公司 Point cloud coding method and related device
WO2022062369A1 (en) * 2020-09-25 2022-03-31 Oppo广东移动通信有限公司 Point cloud encoding and decoding method and system, and point cloud encoder and point cloud decoder
WO2022133753A1 (en) * 2020-12-22 2022-06-30 Oppo广东移动通信有限公司 Point cloud encoding and decoding methods and systems, point cloud encoder, and point cloud decoder

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109274968A (en) * 2018-10-26 2019-01-25 西安科锐盛创新科技有限公司 Video compression self-adaptive quantization and inverse quantization method
CN109361922A (en) * 2018-10-26 2019-02-19 西安科锐盛创新科技有限公司 Predict quantization coding method
US20210385303A1 (en) * 2020-06-09 2021-12-09 Qualcomm Incorporated Attribute residual coding in g-pcc
WO2022061786A1 (en) * 2020-09-25 2022-03-31 Oppo广东移动通信有限公司 Point cloud coding method and related device
WO2022062369A1 (en) * 2020-09-25 2022-03-31 Oppo广东移动通信有限公司 Point cloud encoding and decoding method and system, and point cloud encoder and point cloud decoder
CN112565734A (en) * 2020-12-03 2021-03-26 西安电子科技大学 Point cloud attribute coding and decoding method and device based on hybrid coding
WO2022133753A1 (en) * 2020-12-22 2022-06-30 Oppo广东移动通信有限公司 Point cloud encoding and decoding methods and systems, point cloud encoder, and point cloud decoder
CN113284248A (en) * 2021-06-10 2021-08-20 上海交通大学 Encoding and decoding method, device and system for point cloud lossy compression

Similar Documents

Publication Publication Date Title
CN108028941B (en) Method and apparatus for encoding and decoding digital images by superpixel
WO2020072665A1 (en) Hierarchical tree attribute coding in point cloud coding
KR102645508B1 (en) Method and apparatus for HAAR-based point cloud coding
JP2016220216A (en) Encoder and encoding method
WO2022057091A1 (en) Encoding method, decoding method, encoding device, and decoding device for point cloud attribute
Chawla et al. Image compression techniques: a review
Patidar et al. A review on medical image data compression techniques
Fred et al. Bat optimization based vector quantization algorithm for medical image compression
JP2023543752A (en) Point cloud codec method and system, and point cloud encoder and point cloud decoder
US20230377208A1 (en) Geometry coordinate scaling for ai-based dynamic point cloud coding
JP2024505798A (en) Point cloud encoding/decoding method and system, point cloud encoder, and point cloud decoder
WO2024007253A1 (en) Point cloud rate-distortion optimization method, attribute compression method and apparatus, and storage medium
WO2023168712A1 (en) Zero run-length value encoding and decoding methods and video encoding and decoding methods, apparatuses and systems
JP2022551184A (en) Method, device and program for decoding neural network with block division
CN118075464A (en) Point cloud attribute prediction method and device and codec
WO2023184196A1 (en) Zero run value encoding and decoding methods, and video encoding and decoding methods, devices and systems
WO2024011427A1 (en) Point cloud inter-frame compensation method and apparatus, point cloud encoding method and apparatus, point cloud decoding method and apparatus, and system
WO2024065269A1 (en) Point cloud encoding and decoding method and apparatus, device, and storage medium
WO2024082152A1 (en) Encoding and decoding methods and apparatuses, encoder and decoder, code stream, device, and storage medium
WO2024007144A1 (en) Encoding method, decoding method, code stream, encoders, decoders and storage medium
WO2023103565A1 (en) Point cloud attribute information encoding and decoding method and apparatus, device, and storage medium
WO2024065272A1 (en) Point cloud coding method and apparatus, point cloud decoding method and apparatus, and device and storage medium
WO2024065270A1 (en) Point cloud encoding method and apparatus, point cloud decoding method and apparatus, devices, and storage medium
WO2023240662A1 (en) Encoding method, decoding method, encoder, decoder, and storage medium
WO2024065406A1 (en) Encoding and decoding methods, bit stream, encoder, decoder, and storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22949837

Country of ref document: EP

Kind code of ref document: A1