WO2023172704A1 - Attribute level coding for geometry point cloud coding - Google Patents
Attribute level coding for geometry point cloud coding Download PDFInfo
- Publication number
- WO2023172704A1 WO2023172704A1 PCT/US2023/014930 US2023014930W WO2023172704A1 WO 2023172704 A1 WO2023172704 A1 WO 2023172704A1 US 2023014930 W US2023014930 W US 2023014930W WO 2023172704 A1 WO2023172704 A1 WO 2023172704A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- level
- color
- point
- absolute
- syntax elements
- Prior art date
Links
- 238000000034 method Methods 0.000 claims abstract description 65
- 230000015654 memory Effects 0.000 claims description 33
- 238000013139 quantization Methods 0.000 description 18
- 230000006870 function Effects 0.000 description 16
- 238000012545 processing Methods 0.000 description 13
- 230000008569 process Effects 0.000 description 9
- 238000004458 analytical method Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 8
- 238000013459 approach Methods 0.000 description 7
- 230000008520 organization Effects 0.000 description 7
- 230000009466 transformation Effects 0.000 description 5
- 230000006835 compression Effects 0.000 description 4
- 238000007906 compression Methods 0.000 description 4
- 230000015572 biosynthetic process Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000003786 synthesis reaction Methods 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 241000023320 Luma <angiosperm> Species 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- OSWPMRLSEDHDFF-UHFFFAOYSA-N methyl salicylate Chemical compound COC(=O)C1=CC=CC=C1O OSWPMRLSEDHDFF-UHFFFAOYSA-N 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000002195 synergetic effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/001—Model-based coding, e.g. wire frame
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/005—Statistical coding, e.g. Huffman, run length coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/13—Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
Definitions
- Embodiments of the present disclosure relate to point cloud coding.
- Point clouds are one of the major three-dimensional (3D) data representations, which provide, in addition to spatial coordinates, attributes associated with the points in a 3D world. Point clouds in their raw format require a huge amount of memory for storage or bandwidth for transmission. Furthermore, the emergence of higher resolution point cloud capture technology imposes, in turn, an even higher requirement on the size of point clouds. In order to make point clouds usable, compression is necessary. Two compression technologies have been proposed for point cloud compression/coding (PCC) standardization activities: video-based PCC (V-PCC) and geometry-based PCC (G-PCC). V-PCC approach is based on 3D to two-dimensional (2D) projections, while G-PCC, on the contrary, encodes the content directly in 3D space. In order to achieve that, G-PCC utilizes data structures, such as an octree that describes the point locations in 3D space.
- V-PCC video-based PCC
- G-PCC geometry-based PCC
- a method for encoding a point cloud includes a point associated with a color having components.
- a color level of the point is determined by a processor.
- Syntax elements representing the color level of the point are determined by the processor based on the color level of the point.
- the syntax elements include a parity flag.
- the color level of the point is encoded by the processor into a bitstream based on the syntax elements.
- a system for encoding a point cloud includes a memory configured to store instructions and a processor coupled to the memory.
- the point cloud includes a point associated with a color having components.
- the processor is configured to, upon executing the instructions, determine a color level of the point.
- the processor is also configured to, upon executing the instructions, determine syntax elements representing the color level of the point based on the color level of the point.
- the syntax elements include a parity flag.
- the processor is further configured to, upon executing the instructions, encode the color level of the point into a bitstream based on the syntax elements.
- a method for decoding a point cloud includes a point associated with a color having components.
- a bitstream is parsed by a processor to obtain syntax elements representing a color level of the point.
- the syntax elements include a parity flag.
- the color level of the point is determined by the processor based on the syntax elements.
- a system for decoding a point cloud includes a memory configured to store instructions and a processor coupled to the memory.
- the point cloud includes a point associated with a color having components.
- the processor is configured to, upon executing the instructions, parse a bitstream to obtain syntax elements representing a color level of the point.
- the syntax elements include a parity flag.
- the processor is also configured to, upon executing the instructions, determine the color level of the point based on the syntax elements.
- FIG. 1 illustrates a block diagram of an encoding system, according to some embodiments of the present disclosure.
- FIG. 2 illustrates a block diagram of a decoding system, according to some embodiments of the present disclosure.
- FIG. 3 illustrates a detailed block diagram of an encoder in the encoding system in FIG. 1, according to some embodiments of the present disclosure.
- FIG. 4 illustrates a detailed block diagram of a decoder in the decoding system in FIG. 2, according to some embodiments of the present disclosure.
- FIGs. 5A and 5B illustrate an octree structure of G-PCC and the corresponding digital representation, respectively, according to some embodiments of the present disclosure.
- FIG. 6 illustrates a structure of cube and the relationship with neighboring cubes in an octree structure of G-PCC, according to some embodiments of the present disclosure.
- FIG. 7 illustrates a ID array of points representing a point cloud, a set of candidate points, and a set of prediction points, according to some embodiments of the present disclosure.
- FIG. 8 illustrates a hierarchy of parameter sets of G-PCC, according to some embodiments of the present disclosure.
- FIG. 9 illustrates a flow chart of a method for encoding a point cloud, according to some embodiments of the present disclosure.
- FIG. 10 illustrates a flow chart of a method for decoding a point cloud, according to some embodiments of the present disclosure.
- FIG. 11 illustrates a flow chart of another method for encoding a point cloud, according to some embodiments of the present disclosure.
- FIG. 12 illustrates a flow chart of another method for decoding a point cloud, according to some embodiments of the present disclosure.
- FIG. 13 illustrates a flow chart of still another method for encoding a point cloud, according to some embodiments of the present disclosure.
- FIG. 14 illustrates a flow chart of still another method for decoding a point cloud, according to some embodiments of the present disclosure.
- references in the specification to “one embodiment,” “an embodiment,” “an example embodiment,” “some embodiments,” “certain embodiments,” etc. indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it would be within the knowledge of a person skilled in the pertinent art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.
- terminology may be understood at least in part from usage in context.
- the term “one or more” as used herein, depending at least in part upon context may be used to describe any feature, structure, or characteristic in a singular sense or may be used to describe combinations of features, structures or characteristics in a plural sense.
- terms, such as “a,” “an,” or “the,” again, may be understood to convey a singular usage or to convey a plural usage, depending at least in part upon context.
- the term “based on” may be understood as not necessarily intended to convey an exclusive set of factors and may, instead, allow for existence of additional factors not necessarily expressly described, again, depending at least in part on context.
- point cloud coding includes both encoding and decoding a point cloud.
- a point cloud is composed of a collection of points in a 3D space. Each point in the 3D space is associated with a geometry position together with the associated attribute information (e.g., color, reflectance, intensity, classification, etc.).
- attribute information e.g., color, reflectance, intensity, classification, etc.
- the geometry of a point cloud can be compressed first, and then the corresponding attributes, including color or reflectance, can be compressed based upon the geometry information according to a point cloud coding technique, such as G-PCC.
- G-PCC has been widely used in virtual reality/augmented reality (VR/AR), telecommunication, autonomous vehicle, etc., for entertainment and industrial applications, e.g., light detection and ranging (LiDAR) sweep compression for automotive or robotics and high-definition (HD) map for navigation.
- VR/AR virtual reality/augmented reality
- LiDAR light detection and ranging
- HD high-definition
- MPEG Moving Picture Experts Group
- AVS Audio Video Coding Standard
- an attribute residual is defined as the difference in attribute values between the current point and its predictor.
- point cloud coding may be either lossless or lossy.
- the attribute residual may or may not be quantized by using a predefined quantization operation.
- the attribute residual without or with quantization is called an “attribute level,” such as a color level or a reflectance level, which can be, for example, a signed integer coded into the bitstream (known as “attribute level coding”).
- attribute level coding such as a color level or a reflectance level
- the present disclosure provides various novel schemes to optimize attribute level coding, which are compatible with any suitable G-PCC standards, including, but not limited to, AVS G- PCC standards and MPEG G-PCC standards.
- G-PCC Global System for Mobile Communications
- MPEG G-PCC MPEG G-PCC
- parity -based attribute level coding schemes are disclosed, which can save about half of the absolute attribute levels in point cloud coding by introducing a parity flag indicative of whether the absolute attribute level of a point is an odd or even number.
- the syntax elements representing the signs of the color levels of different components are grouped together and separated from the syntax elements representing the absolute color levels, such that the signs can be coded using bypass coding with higher efficiency.
- FIG. 1 illustrates a block diagram of an encoding system 100, according to some embodiments of the present disclosure.
- FIG. 2 illustrates a block diagram of a decoding system 200, according to some embodiments of the present disclosure.
- Each system 100 or 200 may be applied or integrated into various systems and apparatuses capable of data processing, such as computers and wireless communication devices.
- system 100 or 200 may be the entirety or part of a mobile phone, a desktop computer, a laptop computer, a tablet, a vehicle computer, a gaming console, a printer, a positioning device, a wearable electronic device, a smart sensor, a virtual reality (VR) device, an argument reality (AR) device, or any other suitable electronic devices having data processing capability.
- VR virtual reality
- AR argument reality
- system 100 or 200 may include a processor 102, a memory 104, and an interface 106. These components are shown as connected one to another by a bus, but other connection types are also permitted. It is understood that system 100 or 200 may include any other suitable components for performing functions described here.
- Processor 102 may include microprocessors, such as graphic processing unit (GPU), image signal processor (ISP), central processing unit (CPU), digital signal processor (DSP), tensor processing unit (TPU), vision processing unit (VPU), neural processing unit (NPU), synergistic processing unit (SPU), or physics processing unit (PPU), microcontroller units (MCUs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gated logic, discrete hardware circuits, and other suitable hardware configured to perform the various functions described throughout the present disclosure.
- GPU graphic processing unit
- ISP image signal processor
- CPU central processing unit
- DSP digital signal processor
- TPU tensor processing unit
- VPU vision processing unit
- NPU neural processing unit
- SPU synergistic processing unit
- PPU physics processing unit
- MCUs microcontroller units
- ASICs application-specific integrated circuits
- FPGAs field-programmable gate array
- Processor 102 may be a hardware device having one or more processing cores.
- Processor 102 may execute software.
- Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.
- Software can include computer instructions written in an interpreted language, a compiled language, or machine code. Other techniques for instructing hardware are also permitted under the broad category of software.
- Memory 104 can broadly include both memory (a.k.a, primary/system memory) and storage (a.k.a., secondary memory).
- memory 104 may include random-access memory (RAM), read-only memory (ROM), static RAM (SRAM), dynamic RAM (DRAM), ferro- electric RAM (FRAM), electrically erasable programmable ROM (EEPROM), compact disc readonly memory (CD-ROM) or other optical disk storage, hard disk drive (HDD), such as magnetic disk storage or other magnetic storage devices, Flash drive, solid-state drive (SSD), or any other medium that can be used to carry or store desired program code in the form of instructions that can be accessed and executed by processor 102.
- RAM random-access memory
- ROM read-only memory
- SRAM static RAM
- DRAM dynamic RAM
- FRAM ferro- electric RAM
- EEPROM electrically erasable programmable ROM
- CD-ROM compact disc readonly memory
- HDD hard disk drive
- flash drive such as magnetic disk storage or other magnetic storage devices
- SSD solid
- Interface 106 can broadly include a data interface and a communication interface that is configured to receive and transmit a signal in a process of receiving and transmitting information with other external network elements.
- interface 106 may include input/output (VO) devices and wired or wireless transceivers.
- VO input/output
- FIGs. 1 and 2 it is understood that multiple interfaces can be included.
- Processor 102, memory 104, and interface 106 may be implemented in various forms in system 100 or 200 for performing point cloud coding functions.
- processor 102, memory 104, and interface 106 of system 100 or 200 are implemented (e.g., integrated) on one or more system-on-chips (SoCs).
- SoCs system-on-chips
- processor 102, memory 104, and interface 106 may be integrated on an application processor (AP) SoC that handles application processing in an operating system (OS) environment, including running point cloud encoding and decoding applications.
- API application processor
- processor 102, memory 104, and interface 106 may be integrated on a specialized processor chip for point cloud coding, such as a GPU or ISP chip dedicated to graphic processing in a real-time operating system (RTOS).
- RTOS real-time operating system
- processor 102 may include one or more modules, such as an encoder 101.
- FIG. 1 shows that encoder 101 is within one processor 102, it is understood that encoder 101 may include one or more sub-modules that can be implemented on different processors located closely or remotely with each other.
- Encoder 101 (and any corresponding sub-modules or sub-units) can be hardware units (e.g., portions of an integrated circuit) of processor 102 designed for use with other components or software units implemented by processor 102 through executing at least part of a program, i.e., instructions.
- processor 102 may include one or more modules, such as a decoder 201.
- FIG. 2 shows that decoder 201 is within one processor 102, it is understood that decoder 201 may include one or more sub-modules that can be implemented on different processors located closely or remotely with each other.
- Decoder 201 (and any corresponding sub-modules or sub-units) can be hardware units (e.g., portions of an integrated circuit) of processor 102 designed for use with other components or software units implemented by processor 102 through executing at least part of a program, i.e., instructions.
- the instructions of the program may be stored on a computer-readable medium, such as memory 104, and when executed by processor 102, it may perform a process having one or more functions related to point cloud decoding, such as arithmetic decoding, dequantization, inverse transformation, reconstruction, synthesis, as described below in detail.
- FIG. 3 illustrates a detailed block diagram of encoder 101 in encoding system 100 in FIG. 1, according to some embodiments of the present disclosure.
- encoder 101 may include a coordinate transform module 302, a voxelization module 304, a geometry analysis module 306, and an arithmetic encoding module 308, together configured to encode positions associated with points of a point cloud into a geometry bitstream (i.e., geometry encoding).
- a geometry bitstream i.e., geometry encoding
- encoder 101 may also include a color transform module 310, an attribute transform module 312, a quantization module 314, and an arithmetic encoding module 316, together configured to encode attributes associated with points of a point cloud into an attribute bitstream (i.e., attribute encoding).
- attribute encoding i.e., attribute encoding
- each of the elements shown in FIG. 3 is independently shown to represent characteristic functions different from each other in a point cloud encoder, and it does not mean that each component is formed by the configuration unit of separate hardware or single software. That is, each element is included to be listed as an element for convenience of explanation, and at least two of the elements may be combined to form a single element, or one element may be divided into a plurality of elements to perform a function.
- attribute coding depends on decoded geometry. As a consequence, point cloud positions may be coded first.
- coordinate transform module 302 and a voxelization module 304 may be configured to perform a coordinate transformation followed by voxelization that quantizes and removes duplicate points. The process of position quantization, duplicate point removal, and assignment of attributes to the remaining points is called voxelization.
- the voxelized point cloud may be represented using, for example, an octree structure in a lossless manner.
- Geometry analysis module 306 may be configured to perform geometry analysis using, for example, the octree or trisoup scheme.
- Arithmetic encoding module 308 may be configured to arithmetically encode the resulting structure from geometry analysis module 306 into the geometry bitstream.
- geometry analysis module 306 is configured to perform geometry analysis using the octree scheme.
- a cubical axis-aligned bounding box may be defined by the two extreme points (0,0,0) and (2 d , 2 d , 2 d ) where d is the maximum size of the given point cloud along the x, y, or z direction. All point cloud points may be bounded by this defined cube - bounding box.
- a cube may be divided into eight sub-cubes, which creates the octree structure allowing one parent to have 8 children, and an octree structure may then be built by recursively subdividing sub-cubes, as shown in FIG. 5 A. Each sub-cube may be considered as a separate lower-level bounding box for the respective portion of all point cloud points bounded thereby.
- an 8-bit code may be generated by associating a 1 -bit value with each sub-cube to indicate whether it contains points (i.e., full and has value 1) or not (i.e., empty and has value 0). Only full sub-cubes with a size greater than 1 (i.e., non-voxels) may be further subdivided.
- the geometry information (x, y, z) for one position may be represented by this defined octree structure. Since points may be duplicated, multiple points may be mapped to the same sub-cube of size 1 (i.e., the same voxel).
- the number of points for each sub-cube of dimension 1 is also arithmetically encoded.
- a current cube associated with a current node may be surrounded by six cubes of the same depth sharing a face with it.
- one cube may have up to six same-sized cubes to share one face, as shown in FIG. 6.
- the current cube may also have some neighboring cubes which share lines or points with the current cube.
- color transform module 310 may be configured to convert red/green/blue (RGB) color components of each point to YCbCr color components if the attributes include color. That is, a color attribute may include multiple components such as RGB, YCbCr, or YUV, and color transform module 310 may convert color attributes between different sets of components.
- RGB red/green/blue
- Attribute transform module 312 may be configured to perform attribute transformation based on the results from geometry analysis module 306 (e.g., using the octree scheme), including but not limited to, the region adaptive hierarchical transform (RAHT), interpolation-based hierarchical nearest-neighbor prediction (predicting transform), and interpolation-based hierarchical nearest-neighbor prediction with an update/lifting step (lifting transform).
- RAHT region adaptive hierarchical transform
- predicting transform predicting transform
- quantization module 314 may be configured to quantize the transformed coefficients of attributes (e.g., attribute residuals) from attribute transform module 312 to generate quantization levels of the attributes associated with each point to reduce the dynamic range.
- Arithmetic encoding module 316 may be configured to arithmetically encode the resulting transformed coefficients of attributes associated with each point or the quantization levels thereof into the attribute bitstream.
- a prediction may be formed from neighboring coded attributes, for example, in predicting transform and lifting transform by attribute transform module 312. Then, the difference between the current attribute and the prediction (i.e., attribute residual) may be coded.
- attribute residual the difference between the current attribute and the prediction (i.e., attribute residual) may be coded.
- a Morton code or Hilbert code may be used to convert a point cloud in a 3D space (e.g., a point cloud cube) into a ID array, as shown in FIG. 7.
- Each position in the cube will have a corresponding Morton or Hilbert code, but some positions may not have any corresponding point cloud attribute. In other words, some positions may be empty.
- the attribute coding may follow the predefined coding order, such as the Morton order or Hilbert order.
- a predictor may be generated from the previous coded points in the ID array following the Morton order or Hilbert order.
- the attribute difference between the current point and its prediction points may be encoded into the bitstream.
- the point cloud in the 3D space e.g., a point cloud cube
- the attribute coding may follow the native input order of the point cloud, instead of the predefined Morton order or Hilbert order.
- the coding order followed by the points in the ID array may be either a Morton order, a Hilbert order, or the native input order.
- some predefined numbers have been specified to limit the number of neighboring points that can be used in generating the prediction. For example, only at most AT points among previous at most N consecutively coded points may be used for coding the current attribute. That is, a set of n candidate points may be used as the candidates to select a set of m prediction points (m ⁇ ri) for predicting the current point in attribute coding.
- the number n of candidate points in the set is equal to or smaller than the maximum number N of candidate points (n ⁇ N), and the number m of prediction points in the set is equal to or smaller than the maximum number AT of prediction points (m ⁇ M). As shown in FIG.
- the maximum number AT of prediction points is set to be 3, and a set of 3 prediction points (P, bolded and underlined) may be selected from the set of n candidate points, for example, based on the positions associated with the n candidate points and the current points (e.g., the distances between each candidate point and the current point).
- Aland A are set as a fixed number of 3 and 128, respectively.
- m prediction points e.g., the 3 closest candidate points
- a weighted attribute average from these m points may be formed as the predictor to code the attribute of the current point, according to some embodiments. It is understood that in some examples, the prediction points may be selected from the candidate points that are in the cubes sharing the same face/line/point with the current point cloud.
- an attribute residual is defined as the difference in attribute values between the current point and its predictor.
- point cloud coding may be either lossless or lossy.
- the attribute residual may or may not be quantized by using a predefined quantization operation.
- the attribute residual without or with quantization is called an “attribute level,” such as a color level or a reflectance level, which can be, for example, a signed integer coded into the bitstream.
- FIG. 4 illustrates a detailed block diagram of decoder 201 in decoding system 200 in FIG. 2, according to some embodiments of the present disclosure.
- decoder 201 may include an arithmetic decoding module 402, a geometry synthesis module 404, a reconstruction module 406, and a coordinate inverse transform module 408, together configured to decode positions associated with points of a point cloud from the geometry bitstream (i.e., geometry decoding).
- arithmetic decoding module 402 a geometry synthesis module 404
- reconstruction module 406 a coordinate inverse transform module 408
- FIG. 4 illustrates a detailed block diagram of decoder 201 in decoding system 200 in FIG. 2, according to some embodiments of the present disclosure.
- decoder 201 may include an arithmetic decoding module 402, a geometry synthesis module 404, a reconstruction module 406, and a coordinate inverse transform module 408, together configured to decode positions associated with points of a point cloud from the geometry bitstream (i.e., geometry decoding
- decoder 201 may also include an arithmetic decoding module 410, a dequantization module 412, an attribute inverse transform module 414, and a color inverse transform module 416, together configured to decode attributes associated with points of a point cloud from the attribute bitstream (i.e., attribute decoding).
- attribute decoding i.e., attribute decoding
- each of the elements shown in FIG. 4 is independently shown to represent characteristic functions different from each other in a point cloud decoder, and it does not mean that each component is formed by the configuration unit of separate hardware or single software. That is, each element is included to be listed as an element for convenience of explanation, and at least two of the elements may be combined to form a single element, or one element may be divided into a plurality of elements to perform a function.
- a point cloud bitstream (e.g., a geometry bitstream or an attribute bitstream) is inputted from a point cloud encoder (e.g., encoder 101)
- the input bitstream may be decoded by decoder 201 in a procedure opposite to that of the point cloud encoder.
- Arithmetic decoding modules 402 and 410 may be configured to decode the geometry bitstream and attribute bitstream, respectively, to obtain various information encoded into the bitstream.
- arithmetic decoding module 410 may decode the attribute bitstream to obtain the attribute information associated with each point, such as the quantization levels or the coefficients of the attributes associated with each point.
- dequantization module 412 may be configured to dequantize the quantization levels of attributes associated with each point (a.k.a., attribute levels) to obtain the coefficients of attributes (e.g., attribute residuals) associated with each point.
- arithmetic decoding module 410 may parse the bitstream to obtain various other information (e.g., in the form of syntax elements), such as the syntax element indicative of the coding order followed by the points in the ID array for attribute coding.
- Inverse attribute transform module 414 may be configured to perform inverse attribute transformation, such as inverse RAHT, inverse predicting transform, or inverse lifting transform, to transform the data from the transform domain (e.g., coefficients) back to the attribute domain (e.g., luma and/or chroma information for color attributes).
- color inverse transform module 416 may be configured to convert color attributes between different sets of components, such as from YCbCr color components to RGB color components.
- geometry synthesis module 404, reconstruction module 406, and coordinate inverse transform module 408 of decoder 201 may be configured to perform the inverse operations of geometry analysis module 306, voxelization module 304, and coordinate transform module 302 of encoder 101, respectively.
- FIG. 8 illustrates a hierarchy of parameter sets of G-PCC, according to some embodiments of the present disclosure.
- a point cloud may be represented in a ID array including a set of points each associated with a property, such as geometry and attributes (e.g., color and reflectance).
- the set of points associated with different time stamps may be viewed as a “sequence.”
- the syntax elements used for coding the headers of the point cloud may be organized in a hierarchy having various levels.
- the hierarchy may include a sequence header, for example, a header of a sequence parameter set (SPS) associated with the sequence representing the point cloud.
- SPS sequence parameter set
- the hierarchy may include one or more property headers belonging to the sequence header, such as a geometry parameter header (a.k.a., geometry header) and one or more attribute parameter headers (a.k.a., attribute headers).
- geometry parameter headers and attribute parameter headers can also be at the same level as SPS. For example, as shown in FIG.
- the next level under the SPS may include one header of a geometry parameter set belonging to the SPS and associated with the geometry, as well as one or more headers (1 to ri) of attribute parameter sets belonging to the SPS and each associate with a respective attribute (e.g., color or reflectance).
- the hierarchy may include one or more slice headers belonging to each property header, such as slice geometry parameter headers and slice attribute parameter headers. That is, the sequence representing the point cloud may be divided into one or more slices each including a slice of points, and each slice of points may be associated with one or more slice headers. For example, as shown in FIG.
- the next level under the geometry parameter set may include one or more headers of slice geometry parameter sets (a.k.a., geometry slice headers) each belonging to the geometry parameter set and associated with a respective slice of points.
- the next level under each attribute parameter set may include one or more headers of slice attribute parameter sets (a.k.a., attribute slice headers) each belonging to the respective attribute parameter set and associated with a respective slice of points.
- the hierarchy may include fewer or more levels, such as picture/frame level(s).
- encoder 101 and decoder 201 may be configured to adopt various novel schemes to optimize attribute level coding, as disclosed herein, to improve the performance and efficiency of point cloud coding.
- parity-based attribute level coding at different levels is disclosed in detail below, for example, by introducing a parity flag, to improve the coding performance and efficiency.
- the parity flag may be used to indicate whether the current absolute value of non-zero residual is an odd or even number so that only the approximation of half of the absolute level needs to be coded.
- several different level coding schemes may be employed.
- the number of consecutive zero level points is called zero run-length for attribute level coding.
- the zero runlength of level and non-zero level may be coded into a bitstream.
- the zero run-length counter may be set as zero. Starting from the first point along the predefined coding order, the residuals between the predictors and corresponding original points may be obtained. Then the corresponding levels may be obtained. If the current level is zero, the zero run-length may be increased by one, and the process proceeds to the next point. If the level is not zero, the zero run-length may be coded first, and the non-zero level may be coded after the zero run-length coding.
- the zero run-length counter may be reset to zero, and the process proceeds to the next point.
- the number of zero run-length may be first decoded, and the levels corresponding to the number of zero run-length points may be set as zero. Then non-zero level may be decoded, and the next number of zero run-length may be decoded. The process may continue until all points are decoded.
- attribute level coding the coding of reflectance (a.k.a., reflectance level coding) is disclosed below in detail. Each point of a point cloud may be associated with a reflectance.
- FIG. 9 illustrates a flow chart of a method 900 of point cloud encoding, according to some embodiments of the present disclosure.
- Method 900 may be performed by encoder 101 of encoding system 100 or any other suitable point cloud decoding systems.
- Method 900 may include operations 902, 904, 906, 908, 910, and 912, as described below. It is understood that some of the operations may be optional, and some of the operations may be performed simultaneously, or in a different order other than shown in FIG. 9.
- a reflectance level of the point is determined.
- encoder 101 is configured to determine a reflectance level of the point.
- the reflectance level may be a reflectance residual with or without quantization.
- the reflectance residual is determined by calculating the difference between the reflectance value of a point and its predictor.
- the reflectance residual may be used as the reflectance level of the point without quantization.
- the reflectance residual may be further quantized to generate a quantization reflectance level, which may be used as the reflectance level of the point. It is understood that the reflectance level may be either transformed reflectance residual or non-transformed reflectance residual.
- a non-zero reflectance level may be determined by encoder 101 for a non-zero point in reflectance level coding.
- the non-zero reflectance level may include a sign and an absolute level.
- Syntax elements representing the reflectance level of the point, which includes a parity flag are determined based on the reflectance level of the point.
- encoder 101 is also configured to determine syntax elements representing the reflectance level of the point, which includes a parity flag, based on the reflectance level of the point.
- the syntax elements may further include multiple absolute level flags, a remainder parameter, and a sign flag.
- the parity flag is determined based on an absolute level of the reflectance level.
- encoder 101 is configured to determine the parity flag (e.g., parity flag) based on the absolute level of the reflectance level, e.g., either an odd or an even number.
- the parity flag e.g., parity flag
- the parity flag may indicate that the absolute level of the reflectance level is an odd number
- the parity flag equal to 1 may indicate that the absolute level of the reflectance level is an even number.
- the parity flag is 0 by default. That is, when not present, the parity flag may be inferred to be equal to 0, indicating that the absolute level is an odd number for the reflectance level.
- the absolute level flags are determined based on the absolute level of the reflectance level and the parity flag.
- the remainder parameter is determined based on the absolute level of the reflectance level, the absolute level flags, and the parity flag.
- encoder 101 is configured to determine the absolute level flags based on the absolute level of the reflectance level and the parity flag, and then determine the remainder parameter based on the absolute level of the reflectance level, the absolute level flags, and the parity flag.
- the parity flag, each of the absolute level flags, and the remainder parameter may be determined by encoder 101 in such a sequence.
- the absolute level flags may include two absolute level flags, and the parity flag may be determined prior to the two absolute level flags.
- the sign flag is determined based on the sign of the reflectance level.
- encoder 101 is configured to determine the sign flag (e.g., residual sign) based on the sign of the reflectance level, e.g., either positive or negative. For example, the sign flag equal to 0 may indicate that the sign of the reflectance level is negative, and the sign flag equal to 1 may indicate that the sign of the reflectance level is positive.
- parity flag may be coded first, followed by two absolute level flags (abs_level_gt2_flag and abs_level_gt4 flag), and ended with the remainder parameter (abs level remainder).
- parity flag may be coded based on the absolute level of the reflectance level. In one example, parity flag equal to 0 may specify that the absolute level is odd; parity flag equal to 1 may specify that the absolute level is even; when not present, parity flag may be inferred to be equal to 0.
- abs_level_gt2 flag and abs_level_gt4 flag may be coded based on the absolute level and the parity flag of the reflectance level.
- abs_level_gt2 flag equal to 0 may specify that the absolute level is not greater than 2;
- abs_level_gt2_flag equal to 1 may specify that the absolute level may be greater than 2; when not present, abs_level_gt2_flag may be inferred to be equal to 0.
- abs_level_gt4 flag equal to 0 may specify that the absolute level is not greater than 4; abs_level_gt4_flag equal to 1 may specify that the absolute level may be greater than 4; when not present, abs_level_gt2 flag may be inferred to be equal to 0.
- abs level remainder may be coded based on the absolute level, the two absolute level flags, and the parity flag. In one example, abs level remainder may specify the remaining absolute level; when not present, abs level remainder may be inferred to be equal to 0.
- the reflectance level may be determined based on the set of reflectance level syntax elements shown in Table I above as follows:
- Level (2* residual_sign-l) * (1 + parity _flag + (abs_level_gt2_flag + abs_level_gt4_flag + abs level remainder) «1) (2), where an arithmetic shift operation (X « 1) is used for calculating the multiplication (X * 2).
- Table II below shows example values of the set of reflectance level syntax elements shown in Table
- the location of the parity flag in the process of determining the syntax elements may affect the number of absolute levels to be coded.
- the maximum number of absolute levels e.g. 50%
- other locations e.g., between or after the absolute level flags.
- Table III illustrates another set of reflectance level syntax elements and their representation and organization in which the parity flag (parity flag) is coded between the two absolute level flags (abs level gtl flag and abs_level_gt3_flag).
- one absolute level flag (abs level gtl flag) may be coded first, followed by the parity flag (parity _flag), then the other absolute level flags (abs_level_gt3_flag), and ended with the remainder parameter (abs level remainder).
- abs level gtl flag may be coded based on the absolute level of the reflectance level.
- abs level gtl flag equal to 0 may specify that the absolute level is not greater than 1; abs level gtl flag equal to 1 may specify that the absolute level may be greater than 1; when not present, abs level gtl flag may be inferred to be equal to 0.
- parity flag may be coded based on one absolute level flag (abs level gtl flag) and the absolute level of the reflectance level. In one example, parity flag equal to 0 may specify that the absolute level is even; parity flag equal to 1 may specify that the absolute level is odd; when not present, parity flag may be inferred to be equal to 0.
- abs_level_gt3_flag may be coded based on the absolute level and the parity flag of the reflectance level.
- abs_level_gt3_flag equal to 0 may specify that the absolute level is not greater than 3;
- abs_level_gt3_flag equal to 1 may specify that the absolute level may be greater than 3; when not present, abs_level_gt3_flag may be inferred to be equal to 0.
- abs level remainder may be coded based on the absolute level, the two absolute level flags, and the parity flag.
- abs level remainder may specify the remaining absolute level; when not present, abs level remainder may be inferred to be equal to 0.
- the reflectance level may be determined based on the set of reflectance level syntax elements shown in Table III above as follows:
- Table V illustrates still another set of reflectance level syntax elements and their representation and organization in which the parity flag (parity flag) is coded after the two absolute level flags (abs level gtl flag and abs_level_gt2_flag).
- two absolute level flags (abs level gtl flag and abs_level_gt2_flag) may be coded first, followed by the parity flag (parity _flag), and ended with the remainder parameter (abs level remainder).
- abs level gtl flag and abs_level_gt2_flag may be coded based on the absolute level of the reflectance level.
- abs level gtl flag equal to 0 may specify that the absolute level is not greater than 1; abs level gtl flag equal to 1 may specify that the absolute level may be greater than 1; when not present, abs level gtl flag may be inferred to be equal to 0.
- abs_level_gt2_flag equal to 0 may specify that the absolute level is not greater than 2; abs_level_gt2_flag equal to 1 may specify that the absolute level may be greater than 2; when not present, abs_level_gt2_flag may be inferred to be equal to 0.
- parity flag may be coded based on the two absolute level flags (abs level gtl flag and abs_level_gt2_flag) and the absolute level of the reflectance level.
- parity flag equal to 0 may specify that the absolute level is odd; parity flag equal to 1 may specify that the absolute level is even; when not present, parity flag may be inferred to be equal to 0.
- abs level remainder may be coded based on the absolute level, the two absolute level flags, and the parity flag.
- abs level remainder may specify the remaining absolute level; when not present, abs level remainder may be inferred to be equal to 0.
- the reflectance level may be determined based on the set of reflectance level syntax elements shown in Table V above as follows:
- the absolute level of the reflectance level used to determine the parity flag may be the absolute level itself or the absolute level minus one when it is non-zero (which may be viewed as the “real absolute level”). It is also understood that the absolute level flags and the remainder parameter shown in Tables I - VI above are for illustrative purposes only and may include any other suitable syntax elements that can represent the absolute level of the reflectance level with the parity flag. For example, the absolute level flags may include absolute residual equal one and absolute residual equal two, and the remainder parameter may include absolute residual minus three.
- parity enable flag may be introduced at different locations to enable or disable the usage of the parity flag in reflectance level coding, such as parityBasedRefLevelCodingEnable flag in Tables I, III, and V above.
- parityBasedRefLevelCodingEnable flag 1 may specify that parity flag is enabled as disclosed above;
- parityBasedRefLevelCodingEnable flag 0 may specify that parity flag is disabled such that reflectance level coding is performed without parity flag; when not present, parityBasedRefLevelCodingEnable flag may be inferred to be equal to 0.
- the parity enable flag may be omitted and the party -based reflectance level coding may be always enabled.
- the parity -based reflectance level coding disclosed herein may be signaled at any suitable levels, such as in the sequence header, reflectance header, and/or slice reflectance header, as shown in FIG. 8.
- the reflectance level of the point is encoded into a bitstream based on the syntax elements.
- encoder 101 is further configured to encode the reflectance level of the point into a bitstream based on the syntax elements.
- FIG. 10 illustrates a flow chart of a method 1000 of point cloud decoding, according to some embodiments of the present disclosure.
- Method 1000 may be performed by decoder 201 of decoding system 200 or any other suitable point cloud decoding systems.
- Method 1000 may include operations 1002 and 1004, as described below. It is understood that some of the operations may be optional, and some of the operations may be performed simultaneously, or in a different order other than shown in FIG. 10.
- a bitstream is parsed to obtain syntax elements representing a reflectance level of the point, including a parity flag.
- decoder 201 is configured to parse a bitstream to obtain syntax elements representing a reflectance level of the point, including a parity flag.
- the syntax elements may also include a sign flag, multiple absolute level flags, and a remainder parameter. As disclosed above in detail with respect to FIG.
- the parity flag may be determined based on an absolute level of the reflectance level; the absolute level flags may be determined based on the absolute level of the reflectance level and the parity flag; the remainder parameter may be determined based on the absolute level of the reflectance level, the absolute level flags, and the parity flag; the sign flag may be determined based on a sign of the reflectance level.
- the absolute level flags include two absolute level flags, and the parity flag is determined prior to the two absolute level flags. For example, the parity flag equal to 0 may indicate that the absolute level of the reflectance level is an odd number, and the parity flag equal to 1 may indicate that the absolute level of the reflectance level is an even number. For example, the parity flag is 0 by default.
- the reflectance level of the point is determined based on the syntax elements.
- decoder 201 is also configured to determine the reflectance level of the point based on the syntax elements, including the sign flag, parity flag, absolute level flags, and remainder parameter.
- the reflectance level of the point may be determined based on Equations (2), (3), or (4) above, as disclosed above in detail with respect to FIG. 9.
- each point of a point cloud may be associated with a color, which has multiple components, such as RGB components, YUV components, or YCbCr components. That is, there may be three color attributes for each point with three color components. If the levels for all three color components are zeros, this point is called a zero level point in color level coding. Otherwise, if there is at least one non-zero level for one color component with the point, this point is called a non-zero level point in color level coding.
- a point that has at least one non-zero color component value may be coded by encoder 101 after coding the zero run-length of color levels.
- a nonzero level point there is at least one non-zero level among the three color components.
- Several one-bit flags plus the remainder of the absolute level may be coded to represent levels of three color components.
- the absolute level or absolute level minus one may be coded and referred to herein as “coded level.” If the levels for both the first and the second components are zeros, the coded level for the third component may be the absolute level of the third component minus one; otherwise, the code level for the current color component may be the absolute level itself.
- FIG. 11 illustrates a flow chart of a method 1100 of point cloud encoding, according to some embodiments of the present disclosure.
- Method 1100 may be performed by encoder 101 of encoding system 100 or any other suitable point cloud decoding systems.
- Method 1100 may include operations 1102, 1104, 1106, 1108, 1110, and 1112, as described below. It is understood that some of the operations may be optional, and some of the operations may be performed simultaneously, or in a different order other than shown in FIG. 11.
- a color level of the point is determined.
- encoder 101 is configured to determine a color level of the point. The color level may be associated with one of the color components.
- the point may have multiple color levels (e.g., three color levels) for the multiple color components (e.g., three color components), respectively. It is understood that in some examples, if the color levels for both the first and the second color components are zeros, the coded level for the third color component is the absolute level of the third color component minus one; otherwise, the code level for the current color component is the absolute level itself.
- the color level may be a color component residual with or without quantization.
- the color component residual is determined by calculating the difference between the color component value of a point and its predictor.
- the color component residual may be used as the color level of the point without quantization.
- the color component residual may be further quantized to generate a quantization color level, which may be used as the color level of the point. It is understood that the color level may be either transformed color component residual or nontransformed color component residual.
- a non-zero color level may be determined by encoder 101 for a non-zero point in color level coding.
- the non-zero color level may include a sign and an absolute level.
- Syntax elements representing the color level of the point are determined based on the color level of the point.
- encoder 101 is also configured to determine syntax elements representing the color level of the point, which includes a parity flag, based on the color level of the point.
- the syntax elements may further include multiple absolute level flags, a remainder parameter, and a sign flag.
- the parity flag is determined based on an absolute level of the color level.
- encoder 101 is configured to determine the parity flag (e.g., parity flag) based on the absolute level of the color level, e.g., either an odd or an even number. For example, the parity flag equal to 0 may indicate that the absolute level of the reflectance level is an even number, and the parity flag equal to 1 may indicate that the absolute level of the color level is an odd number. In one example, the parity flag is 0 by default. That is, when not present, the parity flag may be inferred to be equal to 0, indicating that the absolute level is an even number for the color level.
- the parity flag e.g., parity flag
- At operation 1106 At least one of the absolute level flags is determined based on the absolute level of the color level and the parity flag.
- the remainder parameter is determined based on the absolute level of the color level, the absolute level flags, and the parity flag.
- encoder 101 is configured to determine at least one of the absolute level flags based on the absolute level of the color level and the parity flag, and then determine the remainder parameter based on the absolute level of the color level, the absolute level flags, and the parity flag. For example, at least one absolute level flag, the parity flag, at least one absolute level flag, and the remainder parameter may be determined by encoder 101 in such a sequence.
- the absolute level flags may include three absolute level flags, and the parity flag may be determined after two of the three absolute level flags but prior to one of the three absolute level flags.
- the first one of the absolute level flags is determined based on comparing the absolute level of the color level with zero or one. For example, one of the three absolute level flags determined prior to the parity flag may be determined based on whether the current level value is 0, and the other one of the three absolute level flags determined prior to the parity flag may be determined based on whether the current absolute level is 1.
- the sign flag is determined based on the sign of the color level.
- encoder 101 is configured to determine the sign flag (e.g., residual sign) based on the sign of the color level, e.g., either positive or negative. For example, the sign flag equal to 0 may indicate that the sign of the reflectance level is negative, and the sign flag equal to 1 may indicate that the sign of the reflectance level is positive.
- Table VII illustrates a set of color level syntax elements and their representation and organization.
- two absolute level flags (abs_codedlevel_flagO[idx] and abs_codedlevel_flagl[idx]) may be coded first, then the parity flag (parity_flag[idx]), followed by another absolute level flag (abs_codedlevel_flag2[idx]), and ended with the remainder parameter (abs_codedlevel_remainder[idx]).
- abs_codedlevel_flagO[idx] and abs_codedlevel_flagl[idx] may be coded based on the absolute level of the color level, e.g., comparing the current absolute level with 0 and 1, respectively.
- abs_codedlevel_flagO[idx] may specify the value of the first syntax element for level coding for color component idx; when not present, abs_codedlevel_flagO[idx] may be inferred to be equal to 0.
- abs_codedlevel_flagl[idx] may specify the value of the second syntax element for level coding for color component idx; when not present, abs_codedlevel_flagl[idx] may be inferred to be equal to 0.
- parity _flag[idx] may be coded based on the two absolute level flags (abs_codedlevel_flagO[idx] and abs_codedlevel_flagl [idx]) and the absolute level of the color level.
- parity _flag[idx] 0 may specify that the absolute level for color component idx is odd; parity flag equal to 1 may specify that the absolute level for color component idx is even.
- abs_codedlevel_flag2[idx] may be coded based on the absolute level and the parity flag of the color level.
- abs_codedlevel_flag2[idx] may specify the value of the third syntax element for level coding for color component idx; when not present, abs_codedlevel_flag2[idx] may be inferred to be equal to 0.
- abs level remainder may be coded based on the absolute level, the three absolute level flags, and the parity flag.
- abs_codedlevel_remainder[idx] may specify the remaining absolute level for component idx; when not present, abs codedlevel remainderfidx] may be inferred to be equal to 0.
- the color level may be determined based on the set of color level syntax elements shown in Table VII above as follows when the current color component is the third component and the absolute levels for both the first and the second color component are zero:
- the color level may be determined based on the set of color level syntax elements shown in Table VII above as follows when the current color component is the first or the second color component, or the current component is the third component, and there is at least one non-zero residual level for the first or the second color component:
- Table IX below shows example values of the set of color level syntax elements shown in Table VII above when the current color component is the first or the second color component, or the current component is the third component, and there is at least one non-zero residual level for the first or the second color component:
- Table X illustrates another set of color level syntax elements and their representation and organization in which the parity flag (parity flagfidx]) is coded at a different location.
- the parity flag parity flagfidx
- the parity flag parity flagfidx
- two other absolute level flags abs codedlevel flagl fidx] and abs_codedlevel_flag2[idx]
- the remainder parameter abs codedlevel remainderfidx]
- abs codedlevel flagOfidx may be coded based on the absolute level of the color level, e.g., comparing the current absolute level with 0.
- abs codedlevel flagOfidx] may specify the value of the first syntax element for level coding for color component idx; when not present, abs codedlevel flagOfidx] may be inferred to be equal to 0.
- parity flagfidx] may be coded based on the one absolute level flag (abs codedlevel flagOfidx]) and the absolute level of the color level.
- parity flagfidx] may specify that the absolute level for color component idx is even; parity flag equal to 1 may specify that the absolute level for color component idx is odd.
- abs codedlevel flaglfidx] and abs_codedlevel_flag2[idx] may be coded based on the absolute level and the parity flag of the color level.
- abs codedlevel flaglfidx] may specify the value of the second syntax element for level coding for color component idx; when not present, abs codedlevel flaglfidx] may be inferred to be equal to 0.
- abs_codedlevel_flag2[idx] may specify the value of the third syntax element for level coding for color component idx; when not present, abs_codedlevel_flag2[idx] may be inferred to be equal to 0.
- abs level remainder may be coded based on the absolute level, the three absolute level flags, and the parity flag.
- abs codedlevel remainderfidx] may specify the remaining absolute level for component idx; when not present, abs codedlevel remainderfidx] may be inferred to be equal to 0.
- the color level may be determined based on the set of color level syntax elements shown in Table X above as follows when the current color component is the third component and the absolute levels for both the first and the second color component are zero:
- Table XI shows example values of the set of color level syntax elements shown in Table X above when the current color component is the third component and the absolute levels for both the first and the second color component are zero:
- the color level may be determined based on the set of color level syntax elements shown in Table X above as follows when the current color component is the first or the second color component, or the current component is the third component and there is at least one non-zero residual level for the first or the second color component:
- Table XII shows example values of the set of color level syntax elements shown in Table X above when the current color component is the first or the second color component, or the current component is the third component, and there is at least one non-zero residual level for the first or the second color component:
- parity flagfidx a parity flag
- Table XIII illustrates another set of color level syntax elements and their representation and organization in which the parity flag (parity flagfidx]) is coded at another different location.
- the parity flag parity flagfidx]
- the parity flag may be coded first, followed by three absolute level flags (abs codedlevel flagOfidx], abs codedlevel flaglOfidx], and abs_codedlevel_flag2[idx]), and ended with the remainder parameter (abs codedlevel remainderfidx]).
- parity flagfidx may be coded based on the absolute level of the color level.
- parity flagfidx] equal to 0 may specify that the absolute level for color component idx is odd; parity flag equal to 1 may specify that the absolute level for color component idx is even.
- abs codedlevel flagOfidx] abs codedlevel flaglfidx]
- abs_codedlevel_flag2[idx] may be coded based on the absolute level and the parity flag of the color level.
- abs codedlevel flagOfidx may specify the value of the first syntax element for level coding for color component idx; when not present, abs codedlevel flagOfidx] may be inferred to be equal to 0.
- abs codedlevel flagl [idx] may specify the value of the second syntax element for level coding for color component idx; when not present, abs codedlevel flaglfidx] may be inferred to be equal to 0.
- abs_codedlevel_flag2[idx] may specify the value of the third syntax element for level coding for color component idx; when not present, abs_codedlevel_flag2[idx] may be inferred to be equal to 0.
- abs level remainder may be coded based on the absolute level, the three absolute level flags, and the parity flag.
- abs codedlevel remainderfidx] may specify the remaining absolute level for component idx; when not present, abs codedlevel remainderfidx] may be inferred to be equal to 0.
- the color level may be determined based on the set of color level syntax elements shown in Table XIII above as follows when the current color component is the third component and the absolute levels for both the first and the second color component are zero:
- the color level may be determined based on the set of color level syntax elements shown in Table XIII above as follows when the current color component is the first or the second color component, or the current component is the third component and there is at least one non-zero residual level for the first or the second color component:
- Table XV shows example values of the set of color level syntax elements shown in Table XIII above when the current color component is the first or the second color component, or the current component is the third component, and there is at least one non-zero residual level for the first or the second color component:
- the absolute level flags and the remainder parameter shown in Tables VII - XV above are for illustrative purposes only and may include any other suitable syntax elements that can represent the absolute level of the color level with the parity flag. It is also understood that a parity enable flag may be introduced at different locations to enable or disable the usage of the parity flag in color level coding, such as parityBasedColorLevelCodingEnable flag in Tables VII, X, and XIII above.
- parityBasedColorLevelCodingEnable flag 1 may specify that parity flag is enabled as disclosed above; parityBasedColorLevelCodingEnable flag equal to 0 may specify that parity flag is disabled such that color level coding is performed without parity flag; when not present, parityBasedColorLevelCodingEnable flag may be inferred to be equal to 0. It is further understood that the parity -based color level coding disclosed herein may be signaled at any suitable levels, such as in the sequence header, color header, and/or slice color header, as shown in FIG. 8. [0093] At operation 1112, the color level of the point is encoded into a bitstream based on the syntax elements. In some embodiments, encoder 101 is further configured to encode the color level of the point into a bitstream based on the syntax elements.
- FIG. 12 illustrates a flow chart of a method 1200 of point cloud decoding, according to some embodiments of the present disclosure.
- Method 1200 may be performed by decoder 201 of decoding system 200 or any other suitable point cloud decoding systems.
- Method 1200 may include operations 1202 and 1204, as described below. It is understood that some of the operations may be optional, and some of the operations may be performed simultaneously, or in a different order other than shown in FIG. 12.
- a bitstream is parsed to obtain syntax elements representing a color level of the point, including a parity flag.
- decoder 201 is configured to parse a bitstream to obtain syntax elements representing a color level of the point, including a parity flag.
- the syntax elements may also include a sign flag, multiple absolute level flags, and a remainder parameter. As disclosed above in detail with respect to FIG.
- the parity flag may be determined based on an absolute level of the color level; at least one of the absolute level flags may be determined based on the absolute level of the color level and the parity flag; the remainder parameter may be determined based on the absolute level of the color level, the absolute level flags, and the parity flag; the sign flag may be determined based on a sign of the color level.
- the absolute level flags include three absolute level flags, and the parity flag is determined after two of the three absolute level flags but prior to one of the three absolute level flags.
- the parity flag equal to 0 may indicate that a remainder of the absolute level of the color level is an even number, and the parity flag equal to 1 indicates that a remainder of the absolute level of the color level is an odd number.
- the first one of the absolute level flags is determined based on comparing the absolute level of the color level with zero or one.
- the color level of the point is determined based on the syntax elements.
- decoder 201 is also configured to determine the color level of the point based on the syntax elements, including the sign flag, parity flag, absolute level flags, and remainder parameter.
- the color level of the point may be determined based on Equation (5), (6), (7), (8), (9), or (10) above, as disclosed above in detail with respect to FIG. 11.
- Equation (5), (6), (7), (8), (9), or (10) above as disclosed above in detail with respect to FIG. 11.
- parity-based attribute level coding schemes disclosed herein may be used for multispectral and hyperspectral systems to handle more than three attribute components like color attribute. For example, assuming there are N components for a multispectral and hyperspectral attribute, where N is an integer larger than 1, if the current attribute is a non-zero attribute, the levels for the first A- 1 components may have zero value; if there is at least one non-zero level for the first A-l components, the last component may also have zero value; if all the first A-l components are zero value, the last component may only have a non-zero value.
- the syntax elements representing the signs of the color levels of different components are grouped together and separated from the syntax elements representing the absolute color levels, such that the signs can be coded using bypass coding with higher efficiency.
- Attribute level coding may be performed by context-based approaches (a.k.a., context coding) or bypass approaches (a.k.a., bypass coding).
- context-based approaches a.k.a., context coding
- bypass approaches a.k.a., bypass coding
- CABAC context-adaptive modeling based binary arithmetic coding
- the context-based approaches may update the context model adaptively according to the neighboring coded information; syntax elements coded by this approach are called context-coded syntax elements, for example, context- coded bins (CCBs).
- CABAC context-adaptive modeling based binary arithmetic coding
- bypass approaches may assume the probability of 1 or 0 is always 50% and, therefore always uses a fixed context modeling without adaptation; syntax elements coded by this approach are called bypass-coded syntax elements, for example, bypass-code bins (BCBs).
- context-coded syntax elements may include the parity flags, absolute level flags, and remainder parameters disclosed herein, and bypass-code syntax elements may include the sign flags disclosed herein.
- FIG. 13 illustrates a flow chart of a method 1300 of point cloud encoding, according to some embodiments of the present disclosure.
- Method 1300 may be performed by encoder 101 of encoding system 100 or any other suitable point cloud decoding systems.
- Method 1300 may include operations 1302, 1304, 1306, and 1308, as described below. It is understood that some of the operations may be optional, and some of the operations may be performed simultaneously, or in a different order other than shown in FIG. 13.
- color levels of the point each associated with a respective one of the components of a color are determined.
- encoder 101 may determine the color levels for each color component of the point.
- context-coded syntax elements representing each of the color levels of the point are determined based on an absolute level of a respective one of the color levels of the point.
- encoder 101 may determine context-coded syntax elements representing each color level based on the absolute level of a respective color level.
- the context- coded syntax elements may include a parity flag associated with each of the color levels, i.e., three parity flags for three color levels of three color components (e.g., RGB, YUV, or YCrCb).
- the parity flag is determined based on the absolute level of the respective color level.
- the context-coded syntax elements may also include absolute level flags and a remainder parameter associated with each of the color levels.
- bypass-coded syntax elements representing each of the color levels of the point are determined based on a sign of the respective one of the color levels of the point.
- encoder 101 may determine bypass-coded syntax elements representing each color level based on the sign of a respective color level.
- the bypass-coded syntax elements may include at least one of three sign flags associated with three color components
- bypass-coded syntax elements are determined after the context-coded syntax elements.
- Table XVI illustrates still another set of color level syntax elements and their representation and organization in which the sign flags of all color components (residual_sgin[idx]) are determined together after all context-coded, color level syntax elements of all color components (abs_codedlevel_flagN[idx], abs_codedlevel_remainder[idx], and parity _flag[idx]).
- parity flags are not shown in Table XVI, they may be added in Table XVI as shown in the examples of Table VII, X, and XIIII above, such that all context-coded, color level syntax elements of all color components (abs_codedlevel_flagN[idx], abs_codedlevel_remainder[idx], and parity _flag[idx]) may be determined before determining the sign flags of all color components (residual_sgin[idx]).
- the color level of the point is encoded into a bitstream based on the context-coded syntax elements using context coding and the bypass-coded syntax elements using bypass coding.
- encoder 101 is further configured to encode the color level of the point into the bitstream based on the context-coded syntax elements (e.g., the parity flags, absolute level flags, and remainder parameters) using context coding and the bypass-coded syntax elements (e.g., the sign flags) using bypass coding.
- FIG. 14 illustrates a flow chart of a method 1400 of point cloud decoding, according to some embodiments of the present disclosure.
- Method 1400 may be performed by decoder 201 of decoding system 200 or any other suitable point cloud decoding systems.
- Method 1400 may include operations 1402, 1404, 1406, and 1408, as described below. It is understood that some of the operations may be optional, and some of the operations may be performed simultaneously, or in a different order other than shown in FIG. 14.
- a bitstream is parsed to obtain context-coded syntax elements representing each of color levels of the point.
- Each of the color levels may be associated with a respective one of the components.
- decoder 201 is configured to parse the bitstream to obtain context-coded syntax elements representing each of color levels of the point.
- the context-coded syntax elements may include a parity flag associated with each of the color levels, i.e., three parity flags for three color levels of three color components (e.g., RGB, YUV, or YCrCb). As disclosed above in detail with respect to FIG. 13 on the encoding side, the parity flag may be determined based on the absolute level of the respective color level.
- the context-coded syntax elements may also include absolute level flags and a remainder parameter associated with each of the color levels.
- an absolute level of each of the color levels of the point is determined based on the respective context-coded syntax elements using context coding.
- decoder 201 is also configured to determine the absolute level of each of the color levels of the point based on the respective context-coded syntax elements using context coding. For example, the absolute level of each of the color levels of the point may be determined based on Equation (5), (6), (7), (8), (9), or (10) above.
- the bitstream is parsed to obtain bypass-coded syntax elements representing each of color levels of the point.
- decoder 201 is further configured to parse the bitstream to obtain bypass-coded syntax elements representing each of color levels of the point.
- the bypass-coded syntax elements may include at least one of three sign flags associated with three color components (e.g., RGB, YUV, or YCrCb), for example, depending on whether one or more of the color components are zero levels.
- the bypass-coded syntax elements are parsed after the context-coded syntax elements.
- a sign of each of the color levels of the point is determined based on the respective bypass-coded syntax elements using bypass coding.
- decoder 201 is further configured to determine the sign of each of the color levels of the point based on the respective bypass-coded syntax elements using bypass coding. For example, the sign of each of the color levels of the point may be determined based on Equation (5), (6), (7), (8), (9), or (10) above.
- the functions described herein may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as instructions on a non-transitory computer-readable medium.
- Computer-readable media includes computer storage media. Storage media may be any available media that can be accessed by a processor, such as processor 102 in FIGs. 1 and 2.
- such computer-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, HDD, such as magnetic disk storage or other magnetic storage devices, Flash drive, SSD, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a processing system, such as a mobile device or a computer.
- Disk and disc includes CD, laser disc, optical disc, digital video disc (DVD), and floppy disk where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
- a method for encoding a point cloud includes a point associated with a color having components.
- a color level of the point is determined by a processor.
- Syntax elements representing the color level of the point are determined by the processor based on the color level of the point.
- the syntax elements include a parity flag.
- the color level of the point is encoded by the processor into a bitstream based on the syntax elements.
- the parity flag is determined based on an absolute level of the color level.
- the syntax elements further include absolute level flags and a remainder parameter.
- At least one of the absolute level flags is determined based on the absolute level of the color level and the parity flag, and the remainder parameter is determined based on the absolute level of the color level, the absolute level flags, and the parity flag.
- the absolute level flags include three absolute level flags, and the parity flag is determined after two of the three absolute level flags but prior to one of the three absolute level flags.
- the first one of the three absolute level flags is determined based on comparing the absolute level of the color level with zero or one.
- the parity flag equal to 0 indicates that a remainder of the absolute level of the color level is an even number
- the parity flag equal to 1 indicates that a remainder of the absolute level of the color level is an odd number
- the syntax elements further include a sign flag.
- the sign flag is determined based on a sign of the color level.
- a system for encoding a point cloud includes a memory configured to store instructions and a processor coupled to the memory.
- the point cloud includes a point associated with a color having components.
- the processor is configured to, upon executing the instructions, determine a color level of the point.
- the processor is also configured to, upon executing the instructions, determine syntax elements representing the color level of the point based on the color level of the point.
- the syntax elements include a parity flag.
- the processor is further configured to, upon executing the instructions, encode the color level of the point into a bitstream based on the syntax elements.
- a method for decoding a point cloud includes a point associated with a color having components.
- a bitstream is parsed by a processor to obtain syntax elements representing a color level of the point.
- the syntax elements include a parity flag.
- the color level of the point is determined by the processor based on the syntax elements.
- the parity flag is determined based on an absolute level of the color level.
- the syntax elements further include absolute level flags and a remainder parameter.
- At least one of the absolute level flags is determined based on the absolute level of the color level and the parity flag, and the remainder parameter is determined based on the absolute level of the color level, the absolute level flags, and the parity flag.
- the absolute level flags include three absolute level flags, and the parity flag is determined after two of the three absolute level flags but prior to one of the three absolute level flags.
- the first one of the three absolute level flags is determined based on comparing the absolute level of the color level with zero or one.
- the parity flag equal to 0 indicates that a remainder of the absolute level of the color level is an even number
- the parity flag equal to 1 indicates that a remainder of the absolute level of the color level is an odd number
- the syntax elements further include a sign flag.
- the sign flag is determined based on a sign of the color level.
- a system for decoding a point cloud includes a memory configured to store instructions and a processor coupled to the memory.
- the point cloud includes a point associated with a color having components.
- the processor is configured to, upon executing the instructions, parse a bitstream to obtain syntax elements representing a color level of the point.
- the syntax elements include a parity flag.
- the processor is also configured to, upon executing the instructions, determine the color level of the point based on the syntax elements.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
In certain aspects, a method for encoding a point cloud is disclosed. The point cloud includes a point associated with a color having components. A color level of the point is determined by a processor. Syntax elements representing the color level of the point are determined by the processor based on the color level of the point. The syntax elements include a parity flag. The color level of the point is encoded by the processor into a bitstream based on the syntax elements.
Description
ATTRIBUTE LEVEL CODING FOR GEOMETRY POINT CLOUD CODING
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims the benefit of priorities to U.S. Provisional Application No.
63/269,098, entitled “PARITY-BASED COLOR LEVEL CODING FOR GEOMETRY POINT CLOUD CODING,” and filed on March 9, 2022, U.S. Provisional Application No. 63/269,099, entitled “PARITY-BASED REFLECTANCE LEVEL CODING FOR GEOMETRY POINT CLOUD CODING,” and filed on March 9, 2022, and U.S. Provisional Application No. 63/269,338, entitled “PARITY-BASED COLOR LEVEL CODING FOR GEOMETRY POINT CLOUD CODING,” and filed on March 14, 2022, all of which are incorporated by reference herein in their entireties.
BACKGROUND
[0002] Embodiments of the present disclosure relate to point cloud coding.
[0003] Point clouds are one of the major three-dimensional (3D) data representations, which provide, in addition to spatial coordinates, attributes associated with the points in a 3D world. Point clouds in their raw format require a huge amount of memory for storage or bandwidth for transmission. Furthermore, the emergence of higher resolution point cloud capture technology imposes, in turn, an even higher requirement on the size of point clouds. In order to make point clouds usable, compression is necessary. Two compression technologies have been proposed for point cloud compression/coding (PCC) standardization activities: video-based PCC (V-PCC) and geometry-based PCC (G-PCC). V-PCC approach is based on 3D to two-dimensional (2D) projections, while G-PCC, on the contrary, encodes the content directly in 3D space. In order to achieve that, G-PCC utilizes data structures, such as an octree that describes the point locations in 3D space.
SUMMARY
[0004] According to one aspect of the present disclosure, a method for encoding a point cloud is disclosed. The point cloud includes a point associated with a color having components. A color level of the point is determined by a processor. Syntax elements representing the color
level of the point are determined by the processor based on the color level of the point. The syntax elements include a parity flag. The color level of the point is encoded by the processor into a bitstream based on the syntax elements.
[0005] According to another aspect of the present disclosure, a system for encoding a point cloud includes a memory configured to store instructions and a processor coupled to the memory. The point cloud includes a point associated with a color having components. The processor is configured to, upon executing the instructions, determine a color level of the point. The processor is also configured to, upon executing the instructions, determine syntax elements representing the color level of the point based on the color level of the point. The syntax elements include a parity flag. The processor is further configured to, upon executing the instructions, encode the color level of the point into a bitstream based on the syntax elements.
[0006] According to still another aspect of the present disclosure, a method for decoding a point cloud is disclosed. The point cloud includes a point associated with a color having components. A bitstream is parsed by a processor to obtain syntax elements representing a color level of the point. The syntax elements include a parity flag. The color level of the point is determined by the processor based on the syntax elements.
[0007] According to yet another aspect of the present disclosure, a system for decoding a point cloud includes a memory configured to store instructions and a processor coupled to the memory. The point cloud includes a point associated with a color having components. The processor is configured to, upon executing the instructions, parse a bitstream to obtain syntax elements representing a color level of the point. The syntax elements include a parity flag. The processor is also configured to, upon executing the instructions, determine the color level of the point based on the syntax elements.
[0008] These illustrative embodiments are mentioned not to limit or define the present disclosure, but to provide examples to aid understanding thereof. Additional embodiments are described in the Detailed Description, and further description is provided there.
BRIEF DESCRIPTION OF THE DRAWINGS
[0009] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments of the present disclosure and, together with the description, further serve to explain the principles of the present disclosure and to enable a person skilled in the pertinent art to make and use the present disclosure.
[0010] FIG. 1 illustrates a block diagram of an encoding system, according to some embodiments of the present disclosure.
[0011] FIG. 2 illustrates a block diagram of a decoding system, according to some embodiments of the present disclosure.
[0012] FIG. 3 illustrates a detailed block diagram of an encoder in the encoding system in FIG. 1, according to some embodiments of the present disclosure.
[0013] FIG. 4 illustrates a detailed block diagram of a decoder in the decoding system in FIG. 2, according to some embodiments of the present disclosure.
[0014] FIGs. 5A and 5B illustrate an octree structure of G-PCC and the corresponding digital representation, respectively, according to some embodiments of the present disclosure.
[0015] FIG. 6 illustrates a structure of cube and the relationship with neighboring cubes in an octree structure of G-PCC, according to some embodiments of the present disclosure.
[0016] FIG. 7 illustrates a ID array of points representing a point cloud, a set of candidate points, and a set of prediction points, according to some embodiments of the present disclosure.
[0017] FIG. 8 illustrates a hierarchy of parameter sets of G-PCC, according to some embodiments of the present disclosure.
[0018] FIG. 9 illustrates a flow chart of a method for encoding a point cloud, according to some embodiments of the present disclosure.
[0019] FIG. 10 illustrates a flow chart of a method for decoding a point cloud, according to some embodiments of the present disclosure.
[0020] FIG. 11 illustrates a flow chart of another method for encoding a point cloud, according to some embodiments of the present disclosure.
[0021] FIG. 12 illustrates a flow chart of another method for decoding a point cloud, according to some embodiments of the present disclosure.
[0022] FIG. 13 illustrates a flow chart of still another method for encoding a point cloud, according to some embodiments of the present disclosure.
[0023] FIG. 14 illustrates a flow chart of still another method for decoding a point cloud, according to some embodiments of the present disclosure.
[0024] Embodiments of the present disclosure will be described with reference to the accompanying drawings.
DETAILED DESCRIPTION
[0025] Although some configurations and arrangements are discussed, it should be understood that this is done for illustrative purposes only. A person skilled in the pertinent art will recognize that other configurations and arrangements can be used without departing from the spirit and scope of the present disclosure. It will be apparent to a person skilled in the pertinent art that the present disclosure can also be employed in a variety of other applications.
[0026] It is noted that references in the specification to “one embodiment,” “an embodiment,” “an example embodiment,” “some embodiments,” “certain embodiments,” etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it would be within the knowledge of a person skilled in the pertinent art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.
[0027] In general, terminology may be understood at least in part from usage in context. For example, the term “one or more” as used herein, depending at least in part upon context, may be used to describe any feature, structure, or characteristic in a singular sense or may be used to describe combinations of features, structures or characteristics in a plural sense. Similarly, terms, such as “a,” “an,” or “the,” again, may be understood to convey a singular usage or to convey a plural usage, depending at least in part upon context. In addition, the term “based on” may be understood as not necessarily intended to convey an exclusive set of factors and may, instead, allow for existence of additional factors not necessarily expressly described, again, depending at least in part on context.
[0028] Various aspects of point cloud coding systems will now be described with reference to various apparatus and methods. These apparatus and methods will be described in the following detailed description and illustrated in the accompanying drawings by various modules, components, circuits, steps, operations, processes, algorithms, etc. (collectively referred to as “elements”). These elements may be implemented using electronic hardware, firmware, computer software, or any combination thereof. Whether such elements are implemented as hardware, firmware, or software depends upon the particular application and design constraints imposed on the overall system. The techniques described herein may be used for various point cloud coding applications. As described herein, point cloud coding includes both encoding and decoding a point
cloud.
[0029] A point cloud is composed of a collection of points in a 3D space. Each point in the 3D space is associated with a geometry position together with the associated attribute information (e.g., color, reflectance, intensity, classification, etc.). In order to compress the point cloud data efficiently, the geometry of a point cloud can be compressed first, and then the corresponding attributes, including color or reflectance, can be compressed based upon the geometry information according to a point cloud coding technique, such as G-PCC. G-PCC has been widely used in virtual reality/augmented reality (VR/AR), telecommunication, autonomous vehicle, etc., for entertainment and industrial applications, e.g., light detection and ranging (LiDAR) sweep compression for automotive or robotics and high-definition (HD) map for navigation. Moving Picture Experts Group (MPEG) released the first version G-PCC standard, and Audio Video Coding Standard (AVS) is also developing a G-PCC standard.
[0030] In point cloud coding, an attribute residual is defined as the difference in attribute values between the current point and its predictor. Depending on the application, point cloud coding may be either lossless or lossy. Hence, the attribute residual may or may not be quantized by using a predefined quantization operation. The attribute residual without or with quantization is called an “attribute level,” such as a color level or a reflectance level, which can be, for example, a signed integer coded into the bitstream (known as “attribute level coding”). However, the performance and efficiency of attribute level coding in existing G-PCC standards may not be optimal.
[0031] To improve the performance and efficiency of attribute level coding in point cloud coding, the present disclosure provides various novel schemes to optimize attribute level coding, which are compatible with any suitable G-PCC standards, including, but not limited to, AVS G- PCC standards and MPEG G-PCC standards. According to some aspects of the present disclosure, parity -based attribute level coding schemes are disclosed, which can save about half of the absolute attribute levels in point cloud coding by introducing a parity flag indicative of whether the absolute attribute level of a point is an odd or even number. According to some aspects of the present disclosure, for a color attribute that includes multiple components (e.g., RGB, YCbCr, or YUV), the syntax elements representing the signs of the color levels of different components are grouped together and separated from the syntax elements representing the absolute color levels, such that the signs can be coded using bypass coding with higher efficiency.
[0032] FIG. 1 illustrates a block diagram of an encoding system 100, according to some
embodiments of the present disclosure. FIG. 2 illustrates a block diagram of a decoding system 200, according to some embodiments of the present disclosure. Each system 100 or 200 may be applied or integrated into various systems and apparatuses capable of data processing, such as computers and wireless communication devices. For example, system 100 or 200 may be the entirety or part of a mobile phone, a desktop computer, a laptop computer, a tablet, a vehicle computer, a gaming console, a printer, a positioning device, a wearable electronic device, a smart sensor, a virtual reality (VR) device, an argument reality (AR) device, or any other suitable electronic devices having data processing capability. As shown in FIGs. 1 and 2, system 100 or 200 may include a processor 102, a memory 104, and an interface 106. These components are shown as connected one to another by a bus, but other connection types are also permitted. It is understood that system 100 or 200 may include any other suitable components for performing functions described here.
[0033] Processor 102 may include microprocessors, such as graphic processing unit (GPU), image signal processor (ISP), central processing unit (CPU), digital signal processor (DSP), tensor processing unit (TPU), vision processing unit (VPU), neural processing unit (NPU), synergistic processing unit (SPU), or physics processing unit (PPU), microcontroller units (MCUs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gated logic, discrete hardware circuits, and other suitable hardware configured to perform the various functions described throughout the present disclosure. Although only one processor is shown in FIGs. 1 and 2, it is understood that multiple processors can be included. Processor 102 may be a hardware device having one or more processing cores. Processor 102 may execute software. Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise. Software can include computer instructions written in an interpreted language, a compiled language, or machine code. Other techniques for instructing hardware are also permitted under the broad category of software.
[0034] Memory 104 can broadly include both memory (a.k.a, primary/system memory) and storage (a.k.a., secondary memory). For example, memory 104 may include random-access memory (RAM), read-only memory (ROM), static RAM (SRAM), dynamic RAM (DRAM), ferro-
electric RAM (FRAM), electrically erasable programmable ROM (EEPROM), compact disc readonly memory (CD-ROM) or other optical disk storage, hard disk drive (HDD), such as magnetic disk storage or other magnetic storage devices, Flash drive, solid-state drive (SSD), or any other medium that can be used to carry or store desired program code in the form of instructions that can be accessed and executed by processor 102. Broadly, memory 104 may be embodied by any computer-readable medium, such as a non-transitory computer-readable medium. Although only one memory is shown in FIGs. 1 and 2, it is understood that multiple memories can be included.
[0035] Interface 106 can broadly include a data interface and a communication interface that is configured to receive and transmit a signal in a process of receiving and transmitting information with other external network elements. For example, interface 106 may include input/output (VO) devices and wired or wireless transceivers. Although only one memory is shown in FIGs. 1 and 2, it is understood that multiple interfaces can be included.
[0036] Processor 102, memory 104, and interface 106 may be implemented in various forms in system 100 or 200 for performing point cloud coding functions. In some embodiments, processor 102, memory 104, and interface 106 of system 100 or 200 are implemented (e.g., integrated) on one or more system-on-chips (SoCs). In one example, processor 102, memory 104, and interface 106 may be integrated on an application processor (AP) SoC that handles application processing in an operating system (OS) environment, including running point cloud encoding and decoding applications. In another example, processor 102, memory 104, and interface 106 may be integrated on a specialized processor chip for point cloud coding, such as a GPU or ISP chip dedicated to graphic processing in a real-time operating system (RTOS).
[0037] As shown in FIG. 1, in encoding system 100, processor 102 may include one or more modules, such as an encoder 101. Although FIG. 1 shows that encoder 101 is within one processor 102, it is understood that encoder 101 may include one or more sub-modules that can be implemented on different processors located closely or remotely with each other. Encoder 101 (and any corresponding sub-modules or sub-units) can be hardware units (e.g., portions of an integrated circuit) of processor 102 designed for use with other components or software units implemented by processor 102 through executing at least part of a program, i.e., instructions. The instructions of the program may be stored on a computer-readable medium, such as memory 104, and when executed by processor 102, it may perform a process having one or more functions related to point cloud encoding, such as voxelization, transformation, quantization, arithmetic encoding, etc., as described below in detail.
[0038] Similarly, as shown in FIG. 2, in decoding system 200, processor 102 may include one or more modules, such as a decoder 201. Although FIG. 2 shows that decoder 201 is within one processor 102, it is understood that decoder 201 may include one or more sub-modules that can be implemented on different processors located closely or remotely with each other. Decoder 201 (and any corresponding sub-modules or sub-units) can be hardware units (e.g., portions of an integrated circuit) of processor 102 designed for use with other components or software units implemented by processor 102 through executing at least part of a program, i.e., instructions. The instructions of the program may be stored on a computer-readable medium, such as memory 104, and when executed by processor 102, it may perform a process having one or more functions related to point cloud decoding, such as arithmetic decoding, dequantization, inverse transformation, reconstruction, synthesis, as described below in detail.
[0039] FIG. 3 illustrates a detailed block diagram of encoder 101 in encoding system 100 in FIG. 1, according to some embodiments of the present disclosure. As shown in FIG. 3, encoder 101 may include a coordinate transform module 302, a voxelization module 304, a geometry analysis module 306, and an arithmetic encoding module 308, together configured to encode positions associated with points of a point cloud into a geometry bitstream (i.e., geometry encoding). As shown in FIG. 3, encoder 101 may also include a color transform module 310, an attribute transform module 312, a quantization module 314, and an arithmetic encoding module 316, together configured to encode attributes associated with points of a point cloud into an attribute bitstream (i.e., attribute encoding). It is understood that each of the elements shown in FIG. 3 is independently shown to represent characteristic functions different from each other in a point cloud encoder, and it does not mean that each component is formed by the configuration unit of separate hardware or single software. That is, each element is included to be listed as an element for convenience of explanation, and at least two of the elements may be combined to form a single element, or one element may be divided into a plurality of elements to perform a function. It is also understood that some of the elements are not necessary elements that perform functions described in the present disclosure but instead may be optional elements for improving performance. It is further understood that these elements may be implemented using electronic hardware, firmware, computer software, or any combination thereof. Whether such elements are implemented as hardware, firmware, or software depends upon the particular application and design constraints imposed on encoder 101. It is still further understood that the modules shown in FIG. 3 are for illustrative purposes only, and in some examples, different modules may be
included in encoder 101 for point cloud encoding.
[0040] As shown in FIG. 3, geometry positions and attributes associated with points may be encoded separately. A point cloud may be a collection of points with positions Xk = (xk, yk, zk), k = 1, ... , K, where K is the number of points in the point cloud, and attributes Ak = (Alk,A2k, ... , ADk), k = 1, ... , K, where D is the number of attributes for each point. In some embodiments, attribute coding depends on decoded geometry. As a consequence, point cloud positions may be coded first. Since geometry positions may be represented by floating-point numbers in an original coordinate system, coordinate transform module 302 and a voxelization module 304 may be configured to perform a coordinate transformation followed by voxelization that quantizes and removes duplicate points. The process of position quantization, duplicate point removal, and assignment of attributes to the remaining points is called voxelization. The voxelized point cloud may be represented using, for example, an octree structure in a lossless manner. Geometry analysis module 306 may be configured to perform geometry analysis using, for example, the octree or trisoup scheme. Arithmetic encoding module 308 may be configured to arithmetically encode the resulting structure from geometry analysis module 306 into the geometry bitstream.
[0041] In some embodiments, geometry analysis module 306 is configured to perform geometry analysis using the octree scheme. Under the octree scheme, a cubical axis-aligned bounding box may be defined by the two extreme points (0,0,0) and (2d, 2d, 2d) where d is the maximum size of the given point cloud along the x, y, or z direction. All point cloud points may be bounded by this defined cube - bounding box. A cube may be divided into eight sub-cubes, which creates the octree structure allowing one parent to have 8 children, and an octree structure may then be built by recursively subdividing sub-cubes, as shown in FIG. 5 A. Each sub-cube may be considered as a separate lower-level bounding box for the respective portion of all point cloud points bounded thereby.
[0042] As shown in FIG. 5B, an 8-bit code may be generated by associating a 1 -bit value with each sub-cube to indicate whether it contains points (i.e., full and has value 1) or not (i.e., empty and has value 0). Only full sub-cubes with a size greater than 1 (i.e., non-voxels) may be further subdivided. The geometry information (x, y, z) for one position may be represented by this defined octree structure. Since points may be duplicated, multiple points may be mapped to the same sub-cube of size 1 (i.e., the same voxel). In order to handle such a situation, the number of points for each sub-cube of dimension 1 is also arithmetically encoded. By construction of the
octree, a current cube associated with a current node may be surrounded by six cubes of the same depth sharing a face with it. Depending on the location of the current cube, one cube may have up to six same-sized cubes to share one face, as shown in FIG. 6. In addition, the current cube may also have some neighboring cubes which share lines or points with the current cube.
[0043] Referring back to FIG. 3, as to attribute encoding, optionally, color transform module 310 may be configured to convert red/green/blue (RGB) color components of each point to YCbCr color components if the attributes include color. That is, a color attribute may include multiple components such as RGB, YCbCr, or YUV, and color transform module 310 may convert color attributes between different sets of components. Attribute transform module 312 may be configured to perform attribute transformation based on the results from geometry analysis module 306 (e.g., using the octree scheme), including but not limited to, the region adaptive hierarchical transform (RAHT), interpolation-based hierarchical nearest-neighbor prediction (predicting transform), and interpolation-based hierarchical nearest-neighbor prediction with an update/lifting step (lifting transform). Optionally, quantization module 314 may be configured to quantize the transformed coefficients of attributes (e.g., attribute residuals) from attribute transform module 312 to generate quantization levels of the attributes associated with each point to reduce the dynamic range. Arithmetic encoding module 316 may be configured to arithmetically encode the resulting transformed coefficients of attributes associated with each point or the quantization levels thereof into the attribute bitstream.
[0044] In some embodiments, a prediction may be formed from neighboring coded attributes, for example, in predicting transform and lifting transform by attribute transform module 312. Then, the difference between the current attribute and the prediction (i.e., attribute residual) may be coded. For example, in the AVS G-PCC standard, after the geometry positions are coded, a Morton code or Hilbert code may be used to convert a point cloud in a 3D space (e.g., a point cloud cube) into a ID array, as shown in FIG. 7. Each position in the cube will have a corresponding Morton or Hilbert code, but some positions may not have any corresponding point cloud attribute. In other words, some positions may be empty. The attribute coding may follow the predefined coding order, such as the Morton order or Hilbert order. A predictor may be generated from the previous coded points in the ID array following the Morton order or Hilbert order. The attribute difference between the current point and its prediction points may be encoded into the bitstream. In some embodiments, the point cloud in the 3D space (e.g., a point cloud cube) is converted into a ID array without any pre-defined coding order, but instead, in its native input
order, for example, the order in which the point cloud data is collected. That is, in some examples, the attribute coding may follow the native input order of the point cloud, instead of the predefined Morton order or Hilbert order. In other words, the coding order followed by the points in the ID array may be either a Morton order, a Hilbert order, or the native input order.
[0045] As shown in FIG. 7, to reduce the memory usage, some predefined numbers have been specified to limit the number of neighboring points that can be used in generating the prediction. For example, only at most AT points among previous at most N consecutively coded points may be used for coding the current attribute. That is, a set of n candidate points may be used as the candidates to select a set of m prediction points (m < ri) for predicting the current point in attribute coding. The number n of candidate points in the set is equal to or smaller than the maximum number N of candidate points (n < N), and the number m of prediction points in the set is equal to or smaller than the maximum number AT of prediction points (m < M). As shown in FIG. 7, if the number of neighboring points prior to the current point in the ID array following the Morton order, the Hilbert order, or the native input order is larger than the maximum number N of candidate points, then the number n of candidate points in the set of candidate points for the current point (shaded in grey) is equal to the maximum number A; if the number of neighboring points prior to the current point in the ID array following the Morton order, the Hilbert order, or the native input order is larger than or equal to the maximum number A of candidate points, then the number n of candidate points in the set of candidate points for the current point (shaded in grey) is smaller to the maximum number A, then all the neighboring points prior to the current point are used as the set of candidate points for the current point. In FIG. 7, the maximum number AT of prediction points is set to be 3, and a set of 3 prediction points (P, bolded and underlined) may be selected from the set of n candidate points, for example, based on the positions associated with the n candidate points and the current points (e.g., the distances between each candidate point and the current point).
[0046] In some embodiments, Aland A are set as a fixed number of 3 and 128, respectively.
If more than 128 points before the current point are already coded, only 3 out of the previous 128 neighboring points could be used to form attribute predictors (prediction points) according to a predefined order. If there are fewer than 128 coded points before the current point, all coded points before the current point will be used as candidate points to find the prediction points. Among the previous up to 128 candidate points, up to 3 prediction points are selected, which have the closest “distance” (e.g., Euclidean distance) between these candidate points and the current point. The
Euclidean distance d as one example may be defined as follows, while other distance metrics can also be used in other examples: d = |xl — %2| + |yl — y2\ + |zl — z2| (1), where (xl, y 1, zl) and (x2, y2, z2) are the coordinates of the current point and the candidate point along the Morton order, the Hilbert order, or the native input order, respectively. Once m prediction points (e.g., the 3 closest candidate points) have been selected, a weighted attribute average from these m points may be formed as the predictor to code the attribute of the current point, according to some embodiments. It is understood that in some examples, the prediction points may be selected from the candidate points that are in the cubes sharing the same face/line/point with the current point cloud.
[0047] In point cloud coding, an attribute residual is defined as the difference in attribute values between the current point and its predictor. Depending on the application, point cloud coding may be either lossless or lossy. Hence, the attribute residual may or may not be quantized by using a predefined quantization operation. The attribute residual without or with quantization is called an “attribute level,” such as a color level or a reflectance level, which can be, for example, a signed integer coded into the bitstream.
[0048] FIG. 4 illustrates a detailed block diagram of decoder 201 in decoding system 200 in FIG. 2, according to some embodiments of the present disclosure. As shown in FIG. 4, decoder 201 may include an arithmetic decoding module 402, a geometry synthesis module 404, a reconstruction module 406, and a coordinate inverse transform module 408, together configured to decode positions associated with points of a point cloud from the geometry bitstream (i.e., geometry decoding). As shown in FIG. 4, decoder 201 may also include an arithmetic decoding module 410, a dequantization module 412, an attribute inverse transform module 414, and a color inverse transform module 416, together configured to decode attributes associated with points of a point cloud from the attribute bitstream (i.e., attribute decoding). It is understood that each of the elements shown in FIG. 4 is independently shown to represent characteristic functions different from each other in a point cloud decoder, and it does not mean that each component is formed by the configuration unit of separate hardware or single software. That is, each element is included to be listed as an element for convenience of explanation, and at least two of the elements may be combined to form a single element, or one element may be divided into a plurality of elements to perform a function. It is also understood that some of the elements are not necessary elements that perform functions described in the present disclosure but instead may be optional elements for
improving performance. It is further understood that these elements may be implemented using electronic hardware, firmware, computer software, or any combination thereof. Whether such elements are implemented as hardware, firmware, or software depends upon the particular application and design constraints imposed on decoder 201. It is still further understood that the modules shown in FIG. 4 are for illustrative purposes only, and in some examples, different modules may be included in decoder 201 for point cloud decoding.
[0049] When a point cloud bitstream (e.g., a geometry bitstream or an attribute bitstream) is inputted from a point cloud encoder (e.g., encoder 101), the input bitstream may be decoded by decoder 201 in a procedure opposite to that of the point cloud encoder. Thus, the details of decoding that are described above with respect to encoding may be skipped for ease of description. Arithmetic decoding modules 402 and 410 may be configured to decode the geometry bitstream and attribute bitstream, respectively, to obtain various information encoded into the bitstream. For example, arithmetic decoding module 410 may decode the attribute bitstream to obtain the attribute information associated with each point, such as the quantization levels or the coefficients of the attributes associated with each point. Optionally, dequantization module 412 may be configured to dequantize the quantization levels of attributes associated with each point (a.k.a., attribute levels) to obtain the coefficients of attributes (e.g., attribute residuals) associated with each point. Besides the attribute information, arithmetic decoding module 410 may parse the bitstream to obtain various other information (e.g., in the form of syntax elements), such as the syntax element indicative of the coding order followed by the points in the ID array for attribute coding.
[0050] Inverse attribute transform module 414 may be configured to perform inverse attribute transformation, such as inverse RAHT, inverse predicting transform, or inverse lifting transform, to transform the data from the transform domain (e.g., coefficients) back to the attribute domain (e.g., luma and/or chroma information for color attributes). Optionally, color inverse transform module 416 may be configured to convert color attributes between different sets of components, such as from YCbCr color components to RGB color components.
[0051] As to the geometry decoding, geometry synthesis module 404, reconstruction module 406, and coordinate inverse transform module 408 of decoder 201 may be configured to perform the inverse operations of geometry analysis module 306, voxelization module 304, and coordinate transform module 302 of encoder 101, respectively.
[0052] FIG. 8 illustrates a hierarchy of parameter sets of G-PCC, according to some embodiments of the present disclosure. A point cloud may be represented in a ID array including
a set of points each associated with a property, such as geometry and attributes (e.g., color and reflectance). The set of points associated with different time stamps may be viewed as a “sequence.”
[0053] As shown in FIG. 8, the syntax elements (e.g., parameters, flags, etc.) used for coding the headers of the point cloud may be organized in a hierarchy having various levels. At the top level, the hierarchy may include a sequence header, for example, a header of a sequence parameter set (SPS) associated with the sequence representing the point cloud. At the second level, the hierarchy may include one or more property headers belonging to the sequence header, such as a geometry parameter header (a.k.a., geometry header) and one or more attribute parameter headers (a.k.a., attribute headers). In some embodiments, geometry parameter headers and attribute parameter headers can also be at the same level as SPS. For example, as shown in FIG. 8, the next level under the SPS may include one header of a geometry parameter set belonging to the SPS and associated with the geometry, as well as one or more headers (1 to ri) of attribute parameter sets belonging to the SPS and each associate with a respective attribute (e.g., color or reflectance). At the third level, the hierarchy may include one or more slice headers belonging to each property header, such as slice geometry parameter headers and slice attribute parameter headers. That is, the sequence representing the point cloud may be divided into one or more slices each including a slice of points, and each slice of points may be associated with one or more slice headers. For example, as shown in FIG. 8, the next level under the geometry parameter set may include one or more headers of slice geometry parameter sets (a.k.a., geometry slice headers) each belonging to the geometry parameter set and associated with a respective slice of points. Similarly, the next level under each attribute parameter set may include one or more headers of slice attribute parameter sets (a.k.a., attribute slice headers) each belonging to the respective attribute parameter set and associated with a respective slice of points. It is understood that in some examples, the hierarchy may include fewer or more levels, such as picture/frame level(s).
[0054] Consistent with the scope of the present disclosure, encoder 101 and decoder 201 may be configured to adopt various novel schemes to optimize attribute level coding, as disclosed herein, to improve the performance and efficiency of point cloud coding.
[0055] According to some aspects of the present disclosure, parity-based attribute level coding at different levels is disclosed in detail below, for example, by introducing a parity flag, to improve the coding performance and efficiency. The parity flag may be used to indicate whether the current absolute value of non-zero residual is an odd or even number so that only the
approximation of half of the absolute level needs to be coded. Depending on the location of the parity flag in the bitstream, several different level coding schemes may be employed.
[0056] According to the attribute coding schemes of G-PCC standards, the number of consecutive zero level points is called zero run-length for attribute level coding. The zero runlength of level and non-zero level may be coded into a bitstream. For example, for encoder 101, before coding the first point, the zero run-length counter may be set as zero. Starting from the first point along the predefined coding order, the residuals between the predictors and corresponding original points may be obtained. Then the corresponding levels may be obtained. If the current level is zero, the zero run-length may be increased by one, and the process proceeds to the next point. If the level is not zero, the zero run-length may be coded first, and the non-zero level may be coded after the zero run-length coding. After coding a non-zero level, the zero run-length counter may be reset to zero, and the process proceeds to the next point. For decoder 201, the number of zero run-length may be first decoded, and the levels corresponding to the number of zero run-length points may be set as zero. Then non-zero level may be decoded, and the next number of zero run-length may be decoded. The process may continue until all points are decoded. [0057] As one example of attribute level coding, the coding of reflectance (a.k.a., reflectance level coding) is disclosed below in detail. Each point of a point cloud may be associated with a reflectance. For reflectance level coding, a point that has a non-zero reflectance value may be coded by encoder 101 after coding the zero run-length of the reflectance level. For example, FIG. 9 illustrates a flow chart of a method 900 of point cloud encoding, according to some embodiments of the present disclosure. Method 900 may be performed by encoder 101 of encoding system 100 or any other suitable point cloud decoding systems. Method 900 may include operations 902, 904, 906, 908, 910, and 912, as described below. It is understood that some of the operations may be optional, and some of the operations may be performed simultaneously, or in a different order other than shown in FIG. 9.
[0058] At operation 902, a reflectance level of the point is determined. In some embodiments, encoder 101 is configured to determine a reflectance level of the point. As disclosed above, the reflectance level may be a reflectance residual with or without quantization. In some embodiments, the reflectance residual is determined by calculating the difference between the reflectance value of a point and its predictor. In one example, the reflectance residual may be used as the reflectance level of the point without quantization. In another example, the reflectance residual may be further quantized to generate a quantization reflectance level, which may be used
as the reflectance level of the point. It is understood that the reflectance level may be either transformed reflectance residual or non-transformed reflectance residual. In any event, a non-zero reflectance level may be determined by encoder 101 for a non-zero point in reflectance level coding. The non-zero reflectance level may include a sign and an absolute level.
[0059] Syntax elements representing the reflectance level of the point, which includes a parity flag, are determined based on the reflectance level of the point. In some embodiments, encoder 101 is also configured to determine syntax elements representing the reflectance level of the point, which includes a parity flag, based on the reflectance level of the point. Besides the parity flag, the syntax elements may further include multiple absolute level flags, a remainder parameter, and a sign flag.
[0060] To determine the syntax elements representing the reflectance level of the point, at operation 904, the parity flag is determined based on an absolute level of the reflectance level. In some embodiments, encoder 101 is configured to determine the parity flag (e.g., parity flag) based on the absolute level of the reflectance level, e.g., either an odd or an even number. For example, the parity flag equal to 0 may indicate that the absolute level of the reflectance level is an odd number, and the parity flag equal to 1 may indicate that the absolute level of the reflectance level is an even number. In one example, the parity flag is 0 by default. That is, when not present, the parity flag may be inferred to be equal to 0, indicating that the absolute level is an odd number for the reflectance level.
[0061] To determine the syntax elements representing the reflectance level of the point, at operation 906, the absolute level flags are determined based on the absolute level of the reflectance level and the parity flag. At operation 908, the remainder parameter is determined based on the absolute level of the reflectance level, the absolute level flags, and the parity flag. In some embodiments, encoder 101 is configured to determine the absolute level flags based on the absolute level of the reflectance level and the parity flag, and then determine the remainder parameter based on the absolute level of the reflectance level, the absolute level flags, and the parity flag. For example, the parity flag, each of the absolute level flags, and the remainder parameter may be determined by encoder 101 in such a sequence. In one example, the absolute level flags may include two absolute level flags, and the parity flag may be determined prior to the two absolute level flags.
[0062] To determine the syntax elements representing the reflectance level of the point, at operation 910, the sign flag is determined based on the sign of the reflectance level. In some
embodiments, encoder 101 is configured to determine the sign flag (e.g., residual sign) based on the sign of the reflectance level, e.g., either positive or negative. For example, the sign flag equal to 0 may indicate that the sign of the reflectance level is negative, and the sign flag equal to 1 may indicate that the sign of the reflectance level is positive.
[0063] The example specification below (Table I) illustrates a set of reflectance level syntax elements and their representation and organization. For example, the parity flag (parity _flag) may be coded first, followed by two absolute level flags (abs_level_gt2_flag and abs_level_gt4 flag), and ended with the remainder parameter (abs level remainder). For example, parity flag may be coded based on the absolute level of the reflectance level. In one example, parity flag equal to 0 may specify that the absolute level is odd; parity flag equal to 1 may specify that the absolute level is even; when not present, parity flag may be inferred to be equal to 0. For example, abs_level_gt2 flag and abs_level_gt4 flag may be coded based on the absolute level and the parity flag of the reflectance level. In one example, abs_level_gt2 flag equal to 0 may specify that the absolute level is not greater than 2; abs_level_gt2_flag equal to 1 may specify that the absolute level may be greater than 2; when not present, abs_level_gt2_flag may be inferred to be equal to 0. In another example, abs_level_gt4 flag equal to 0 may specify that the absolute level is not greater than 4; abs_level_gt4_flag equal to 1 may specify that the absolute level may be greater than 4; when not present, abs_level_gt2 flag may be inferred to be equal to 0. For example, abs level remainder may be coded based on the absolute level, the two absolute level flags, and the parity flag. In one example, abs level remainder may specify the remaining absolute level; when not present, abs level remainder may be inferred to be equal to 0.
Table I
[0064] The reflectance level may be determined based on the set of reflectance level syntax elements shown in Table I above as follows:
Level = (2* residual_sign-l) * (1 + parity _flag + (abs_level_gt2_flag + abs_level_gt4_flag + abs level remainder) «1) (2), where an arithmetic shift operation (X « 1) is used for calculating the multiplication (X * 2).
Table II below shows example values of the set of reflectance level syntax elements shown in Table
Table II
[0065] It is understood that the location of the parity flag in the process of determining the syntax elements may affect the number of absolute levels to be coded. By putting the parity flag prior to the absolute level flags and the remainder parameter, the maximum number of absolute levels (e.g., 50%) can be saved in point cloud coding compared with other locations (e.g., between or after the absolute level flags).
[0066] The example specification below (Table III) illustrates another set of reflectance level syntax elements and their representation and organization in which the parity flag
(parity flag) is coded between the two absolute level flags (abs level gtl flag and abs_level_gt3_flag). For example, one absolute level flag (abs level gtl flag) may be coded first, followed by the parity flag (parity _flag), then the other absolute level flags (abs_level_gt3_flag), and ended with the remainder parameter (abs level remainder). For example, abs level gtl flag may be coded based on the absolute level of the reflectance level. In one example, abs level gtl flag equal to 0 may specify that the absolute level is not greater than 1; abs level gtl flag equal to 1 may specify that the absolute level may be greater than 1; when not present, abs level gtl flag may be inferred to be equal to 0. For example, parity flag may be coded based on one absolute level flag (abs level gtl flag) and the absolute level of the reflectance level. In one example, parity flag equal to 0 may specify that the absolute level is even; parity flag equal to 1 may specify that the absolute level is odd; when not present, parity flag may be inferred to be equal to 0. For example, abs_level_gt3_flag may be coded based on the absolute level and the parity flag of the reflectance level. In one example, abs_level_gt3_flag equal to 0 may specify that the absolute level is not greater than 3; abs_level_gt3_flag equal to 1 may specify that the absolute level may be greater than 3; when not present, abs_level_gt3_flag may be inferred to be equal to 0. For example, abs level remainder may be coded based on the absolute level, the two absolute level flags, and the parity flag. In one example, abs level remainder may specify the remaining absolute level; when not present, abs level remainder may be inferred to be equal to 0.
Table III
[0067] The reflectance level may be determined based on the set of reflectance level syntax elements shown in Table III above as follows:
Level = (2* residual_sign-l) * (1 + parity _flag + abs_level_gtl_flag + (abs_level_gt3_flag + abs level remainder) «1) (3), where an arithmetic shift operation (X « 1) is used for calculating the multiplication (X * 2). Table IV below shows example values of the set of reflectance level syntax elements shown in
Table IV
[0068] The example specification below (Table V) illustrates still another set of reflectance level syntax elements and their representation and organization in which the parity flag (parity flag) is coded after the two absolute level flags (abs level gtl flag and abs_level_gt2_flag). For example, two absolute level flags (abs level gtl flag and abs_level_gt2_flag) may be coded first, followed by the parity flag (parity _flag), and ended with the remainder parameter (abs level remainder). For example, abs level gtl flag and abs_level_gt2_flag may be coded based on the absolute level of the reflectance level. In one example, abs level gtl flag equal to 0 may specify that the absolute level is not greater than 1; abs level gtl flag equal to 1 may specify that the absolute level may be greater than 1; when not
present, abs level gtl flag may be inferred to be equal to 0. In another example, abs_level_gt2_flag equal to 0 may specify that the absolute level is not greater than 2; abs_level_gt2_flag equal to 1 may specify that the absolute level may be greater than 2; when not present, abs_level_gt2_flag may be inferred to be equal to 0. For example, parity flag may be coded based on the two absolute level flags (abs level gtl flag and abs_level_gt2_flag) and the absolute level of the reflectance level. In one example, parity flag equal to 0 may specify that the absolute level is odd; parity flag equal to 1 may specify that the absolute level is even; when not present, parity flag may be inferred to be equal to 0. For example, abs level remainder may be coded based on the absolute level, the two absolute level flags, and the parity flag. In one example, abs level remainder may specify the remaining absolute level; when not present, abs level remainder may be inferred to be equal to 0.
Table V
[0069] The reflectance level may be determined based on the set of reflectance level syntax elements shown in Table V above as follows:
Level = (2* residual_sign - !) * (! + parity _flag + abs_level_gtl_flag + abs_level_gt2_flag + abs level remainder «1) (4), where an arithmetic shift operation (X « 1) is used for calculating the multiplication (X * 2). Table VI below shows example values of the set of reflectance level syntax elements shown in
Table VI
[0070] It is understood that the absolute level of the reflectance level used to determine the parity flag may be the absolute level itself or the absolute level minus one when it is non-zero (which may be viewed as the “real absolute level”). It is also understood that the absolute level flags and the remainder parameter shown in Tables I - VI above are for illustrative purposes only and may include any other suitable syntax elements that can represent the absolute level of the reflectance level with the parity flag. For example, the absolute level flags may include absolute residual equal one and absolute residual equal two, and the remainder parameter may include absolute residual minus three. It is further understood that a parity enable flag may be introduced at different locations to enable or disable the usage of the parity flag in reflectance level coding, such as parityBasedRefLevelCodingEnable flag in Tables I, III, and V above. For example, parityBasedRefLevelCodingEnable flag equal to 1 may specify that parity flag is enabled as disclosed above; parityBasedRefLevelCodingEnable flag equal to 0 may specify that parity flag is disabled such that reflectance level coding is performed without parity flag; when not present, parityBasedRefLevelCodingEnable flag may be inferred to be equal to 0. In some examples, the parity enable flag may be omitted and the party -based reflectance level coding may be always enabled. It is further understood that the parity -based reflectance level coding disclosed
herein may be signaled at any suitable levels, such as in the sequence header, reflectance header, and/or slice reflectance header, as shown in FIG. 8.
[0071] At operation 912, the reflectance level of the point is encoded into a bitstream based on the syntax elements. In some embodiments, encoder 101 is further configured to encode the reflectance level of the point into a bitstream based on the syntax elements.
[0072] FIG. 10 illustrates a flow chart of a method 1000 of point cloud decoding, according to some embodiments of the present disclosure. Method 1000 may be performed by decoder 201 of decoding system 200 or any other suitable point cloud decoding systems. Method 1000 may include operations 1002 and 1004, as described below. It is understood that some of the operations may be optional, and some of the operations may be performed simultaneously, or in a different order other than shown in FIG. 10.
[0073] At operation 1002, a bitstream is parsed to obtain syntax elements representing a reflectance level of the point, including a parity flag. In some embodiments, decoder 201 is configured to parse a bitstream to obtain syntax elements representing a reflectance level of the point, including a parity flag. The syntax elements may also include a sign flag, multiple absolute level flags, and a remainder parameter. As disclosed above in detail with respect to FIG. 9 on the encoding side, the parity flag may be determined based on an absolute level of the reflectance level; the absolute level flags may be determined based on the absolute level of the reflectance level and the parity flag; the remainder parameter may be determined based on the absolute level of the reflectance level, the absolute level flags, and the parity flag; the sign flag may be determined based on a sign of the reflectance level. In some embodiments, as shown in Table I, the absolute level flags include two absolute level flags, and the parity flag is determined prior to the two absolute level flags. For example, the parity flag equal to 0 may indicate that the absolute level of the reflectance level is an odd number, and the parity flag equal to 1 may indicate that the absolute level of the reflectance level is an even number. For example, the parity flag is 0 by default.
[0074] At operation 1004, the reflectance level of the point is determined based on the syntax elements. In some embodiments, decoder 201 is also configured to determine the reflectance level of the point based on the syntax elements, including the sign flag, parity flag, absolute level flags, and remainder parameter. For example, the reflectance level of the point may be determined based on Equations (2), (3), or (4) above, as disclosed above in detail with respect to FIG. 9.
[0075] As another example of attribute level coding, the coding of color (a.k.a., color level
coding) is disclosed below in detail. Each point of a point cloud may be associated with a color, which has multiple components, such as RGB components, YUV components, or YCbCr components. That is, there may be three color attributes for each point with three color components. If the levels for all three color components are zeros, this point is called a zero level point in color level coding. Otherwise, if there is at least one non-zero level for one color component with the point, this point is called a non-zero level point in color level coding. For color level coding, a point that has at least one non-zero color component value (non-zero level point) may be coded by encoder 101 after coding the zero run-length of color levels. For a nonzero level point, there is at least one non-zero level among the three color components. Several one-bit flags plus the remainder of the absolute level may be coded to represent levels of three color components. The absolute level or absolute level minus one may be coded and referred to herein as “coded level.” If the levels for both the first and the second components are zeros, the coded level for the third component may be the absolute level of the third component minus one; otherwise, the code level for the current color component may be the absolute level itself.
[0076] For example, FIG. 11 illustrates a flow chart of a method 1100 of point cloud encoding, according to some embodiments of the present disclosure. Method 1100 may be performed by encoder 101 of encoding system 100 or any other suitable point cloud decoding systems. Method 1100 may include operations 1102, 1104, 1106, 1108, 1110, and 1112, as described below. It is understood that some of the operations may be optional, and some of the operations may be performed simultaneously, or in a different order other than shown in FIG. 11. [0077] At operation 1102, a color level of the point is determined. In some embodiments, encoder 101 is configured to determine a color level of the point. The color level may be associated with one of the color components. In other words, the point may have multiple color levels (e.g., three color levels) for the multiple color components (e.g., three color components), respectively. It is understood that in some examples, if the color levels for both the first and the second color components are zeros, the coded level for the third color component is the absolute level of the third color component minus one; otherwise, the code level for the current color component is the absolute level itself.
[0078] As disclosed above, the color level may be a color component residual with or without quantization. In some embodiments, the color component residual is determined by calculating the difference between the color component value of a point and its predictor. In one example, the color component residual may be used as the color level of the point without
quantization. In another example, the color component residual may be further quantized to generate a quantization color level, which may be used as the color level of the point. It is understood that the color level may be either transformed color component residual or nontransformed color component residual. In any event, a non-zero color level may be determined by encoder 101 for a non-zero point in color level coding. The non-zero color level may include a sign and an absolute level.
[0079] Syntax elements representing the color level of the point, which includes a parity flag, are determined based on the color level of the point. In some embodiments, encoder 101 is also configured to determine syntax elements representing the color level of the point, which includes a parity flag, based on the color level of the point. Besides the parity flag, the syntax elements may further include multiple absolute level flags, a remainder parameter, and a sign flag. [0080] To determine the syntax elements representing the color level of the point, at operation 1104, the parity flag is determined based on an absolute level of the color level. In some embodiments, encoder 101 is configured to determine the parity flag (e.g., parity flag) based on the absolute level of the color level, e.g., either an odd or an even number. For example, the parity flag equal to 0 may indicate that the absolute level of the reflectance level is an even number, and the parity flag equal to 1 may indicate that the absolute level of the color level is an odd number. In one example, the parity flag is 0 by default. That is, when not present, the parity flag may be inferred to be equal to 0, indicating that the absolute level is an even number for the color level.
[0081] To determine the syntax elements representing the color level of the point, at operation 1106, at least one of the absolute level flags is determined based on the absolute level of the color level and the parity flag. At operation 1108, the remainder parameter is determined based on the absolute level of the color level, the absolute level flags, and the parity flag. In some embodiments, encoder 101 is configured to determine at least one of the absolute level flags based on the absolute level of the color level and the parity flag, and then determine the remainder parameter based on the absolute level of the color level, the absolute level flags, and the parity flag. For example, at least one absolute level flag, the parity flag, at least one absolute level flag, and the remainder parameter may be determined by encoder 101 in such a sequence. In one example, the absolute level flags may include three absolute level flags, and the parity flag may be determined after two of the three absolute level flags but prior to one of the three absolute level flags. In some embodiments, the first one of the absolute level flags is determined based on comparing the absolute level of the color level with zero or one. For example, one of the three
absolute level flags determined prior to the parity flag may be determined based on whether the current level value is 0, and the other one of the three absolute level flags determined prior to the parity flag may be determined based on whether the current absolute level is 1.
[0082] To determine the syntax elements representing the color level of the point, at operation 1110, the sign flag is determined based on the sign of the color level. In some embodiments, encoder 101 is configured to determine the sign flag (e.g., residual sign) based on the sign of the color level, e.g., either positive or negative. For example, the sign flag equal to 0 may indicate that the sign of the reflectance level is negative, and the sign flag equal to 1 may indicate that the sign of the reflectance level is positive.
[0083] The example specification below (Table VII) illustrates a set of color level syntax elements and their representation and organization. For example, when the current color component is the third color component and the absolute levels for both the first and the second color component are zero, two absolute level flags (abs_codedlevel_flagO[idx] and abs_codedlevel_flagl[idx]) may be coded first, then the parity flag (parity_flag[idx]), followed by another absolute level flag (abs_codedlevel_flag2[idx]), and ended with the remainder parameter (abs_codedlevel_remainder[idx]). For example, abs_codedlevel_flagO[idx] and abs_codedlevel_flagl[idx] may be coded based on the absolute level of the color level, e.g., comparing the current absolute level with 0 and 1, respectively. In one example, abs_codedlevel_flagO[idx] may specify the value of the first syntax element for level coding for color component idx; when not present, abs_codedlevel_flagO[idx] may be inferred to be equal to 0. In another example, abs_codedlevel_flagl[idx] may specify the value of the second syntax element for level coding for color component idx; when not present, abs_codedlevel_flagl[idx] may be inferred to be equal to 0. For example, parity _flag[idx] may be coded based on the two absolute level flags (abs_codedlevel_flagO[idx] and abs_codedlevel_flagl [idx]) and the absolute level of the color level. In one example, parity _flag[idx] equal to 0 may specify that the absolute level for color component idx is odd; parity flag equal to 1 may specify that the absolute level for color component idx is even. For example, abs_codedlevel_flag2[idx] may be coded based on the absolute level and the parity flag of the color level. In one example, abs_codedlevel_flag2[idx] may specify the value of the third syntax element for level coding for color component idx; when not present, abs_codedlevel_flag2[idx] may be inferred to be equal to 0. For example, abs level remainder may be coded based on the absolute level, the three absolute level flags, and the parity flag. In one example, abs_codedlevel_remainder[idx] may specify the remaining
absolute level for component idx; when not present, abs codedlevel remainderfidx] may be inferred to be equal to 0.
Table VII
[0084] The color level may be determined based on the set of color level syntax elements shown in Table VII above as follows when the current color component is the third component and the absolute levels for both the first and the second color component are zero:
Level = (2* residual_sign[idx] -1) * (1 + parity flagfidx] + abs_codedlevel_flagO[idx] + abs codedlevel flaglfidx] + (abs_codedlevel_flag2[idx] + abs_codedlevel_remainder[idx])«l)(5), where an arithmetic shift operation (X « 1) is used for calculating the multiplication (X * 2). Table VIII below shows example values of the set of color level syntax elements shown in Table
VII above when the current color component is the third component and the absolute levels for both the first and the second color component are zero:
Table VIII
[0085] It is understood that the color level may be determined based on the set of color level syntax elements shown in Table VII above as follows when the current color component is the first or the second color component, or the current component is the third component, and there is at least one non-zero residual level for the first or the second color component:
Level = (2* residual_sign[idx] -1) * (parity_flag[idx] + abs_codedlevel_flagO[idx] + abs codedlevel flaglfidx] + abs_codedlevel_flag2[idx] + (abs_codedlevel_flag3[idx] + abs_codedlevel_remainder[idx])«l) (6), where an arithmetic shift operation (X « 1) is used for calculating the multiplication (X * 2), and abs_codedlevel_flag3[idx] may specify the value of the fourth syntax element for level coding for color component idx; when not present, abs_codedlevel_flag3[idx] may be inferred to be equal to 0. Table IX below shows example values of the set of color level syntax elements shown in Table VII above when the current color component is the first or the second color component, or the current component is the third component, and there is at least one non-zero residual level for the first or the second color component:
Table IX
[0086] The example specification below (Table X) illustrates another set of color level syntax elements and their representation and organization in which the parity flag (parity flagfidx]) is coded at a different location. For example, when the current color component is the third color component and the absolute levels for both the first and the second color component are zero, one absolute level flag (abs codedlevel flagOfidx]) may be coded first, then
the parity flag (parity flagfidx]), followed by two other absolute level flags (abs codedlevel flagl fidx] and abs_codedlevel_flag2[idx]), and ended with the remainder parameter (abs codedlevel remainderfidx]). For example, abs codedlevel flagOfidx] may be coded based on the absolute level of the color level, e.g., comparing the current absolute level with 0. In one example, abs codedlevel flagOfidx] may specify the value of the first syntax element for level coding for color component idx; when not present, abs codedlevel flagOfidx] may be inferred to be equal to 0. For example, parity flagfidx] may be coded based on the one absolute level flag (abs codedlevel flagOfidx]) and the absolute level of the color level. In one example, parity flagfidx] equal to 0 may specify that the absolute level for color component idx is even; parity flag equal to 1 may specify that the absolute level for color component idx is odd. For example, abs codedlevel flaglfidx] and abs_codedlevel_flag2[idx] may be coded based on the absolute level and the parity flag of the color level. In one example, abs codedlevel flaglfidx] may specify the value of the second syntax element for level coding for color component idx; when not present, abs codedlevel flaglfidx] may be inferred to be equal to 0. In another example, abs_codedlevel_flag2[idx] may specify the value of the third syntax element for level coding for color component idx; when not present, abs_codedlevel_flag2[idx] may be inferred to be equal to 0. For example, abs level remainder may be coded based on the absolute level, the three absolute level flags, and the parity flag. In one example, abs codedlevel remainderfidx] may specify the remaining absolute level for component idx; when not present, abs codedlevel remainderfidx] may be inferred to be equal to 0.
Table X
[0087] The color level may be determined based on the set of color level syntax elements shown in Table X above as follows when the current color component is the third component and the absolute levels for both the first and the second color component are zero:
Level = (2* residual_sign[idx] -1) * (1 + parity flagfidx] + abs_codedlevel_flagO[idx] + (abs codedlevel flagl fidx] + abs_codedlevel_flag2[idx] + abs_codedlevel_remainder[idx])«l).
(7), where an arithmetic shift operation (X « 1) is used for calculating the multiplication (X * 2). Table XI below shows example values of the set of color level syntax elements shown in Table X above when the current color component is the third component and the absolute levels for both the first and the second color component are zero:
Table XI
[0088] It is understood that the color level may be determined based on the set of color level syntax elements shown in Table X above as follows when the current color component is the first or the second color component, or the current component is the third component and there is at least one non-zero residual level for the first or the second color component:
Level = (2* residual_sign[idx] -1) * (parity_flag[idx] + abs_codedlevel_flagO[idx] + abs codedlevel flaglfidx] + (abs_codedlevel_flag2[idx] + abs_codedlevel_flag3[idx] + abs_codedlevel_remainder[idx])«l) (8),
where an arithmetic shift operation (X « 1) is used for calculating the multiplication (X * 2), and abs_codedlevel_flag3[idx] may specify the value of the fourth syntax element for level coding for color component idx; when not present, abs_codedlevel_flag3[idx] may be inferred to be equal to 0. Table XII below shows example values of the set of color level syntax elements shown in Table X above when the current color component is the first or the second color component, or the current component is the third component, and there is at least one non-zero residual level for the first or the second color component:
Table XII
[0089] The example specification below (Table XIII) illustrates another set of color level syntax elements and their representation and organization in which the parity flag (parity flagfidx]) is coded at another different location. For example, when the current color component is the third color component and the absolute levels for both the first and the second color component are zero, the parity flag (parity flagfidx]) may be coded first, followed by three absolute level flags (abs codedlevel flagOfidx], abs codedlevel flaglOfidx], and abs_codedlevel_flag2[idx]), and ended with the remainder parameter (abs codedlevel remainderfidx]). For example, parity flagfidx] may be coded based on the absolute level of the color level. In one example, parity flagfidx] equal to 0 may specify that the absolute level for color component idx is odd; parity flag equal to 1 may specify that the absolute level for color component idx is even. For example, abs codedlevel flagOfidx], abs codedlevel flaglfidx], and abs_codedlevel_flag2[idx] may be coded based on the absolute level and the parity flag of the color level. In one example, abs codedlevel flagOfidx] may specify the value of the first syntax element for level coding for color component idx; when not present, abs codedlevel flagOfidx] may be inferred to be equal to 0. In another example, abs codedlevel flagl [idx] may specify the value of the second syntax element for level coding for color component idx; when not present, abs codedlevel flaglfidx] may be inferred to be equal to 0. In another example, abs_codedlevel_flag2[idx] may specify the value of the third syntax element for level coding for color component idx; when not present, abs_codedlevel_flag2[idx] may be inferred to be equal to 0. For example, abs level remainder may be coded based on the
absolute level, the three absolute level flags, and the parity flag. In one example, abs codedlevel remainderfidx] may specify the remaining absolute level for component idx; when not present, abs codedlevel remainderfidx] may be inferred to be equal to 0.
Table XIII
[0090] The color level may be determined based on the set of color level syntax elements shown in Table XIII above as follows when the current color component is the third component and the absolute levels for both the first and the second color component are zero:
Level = (2* residual_sign[idx] -1) * (1 + parity flagfidx] + (abs_codedlevel_flagO[idx] + abs codedlevel flaglfidx] + abs_codedlevel_flag2[idx] + abs_codedlevel_remainder[idx])«l)(9),
where an arithmetic shift operation (X « 1) is used for calculating the multiplication (X * 2). Table XIV below shows example values of the set of color level syntax elements shown in Table XIII above when the current color component is the third component and the absolute levels for both the first and the second color component are zero:
Table XIV
[0091] It is understood that the color level may be determined based on the set of color level syntax elements shown in Table XIII above as follows when the current color component is the first or the second color component, or the current component is the third component and there is at least one non-zero residual level for the first or the second color component:
Level = (2* residual_sign[idx] -1) * (parity_flag[idx] + abs_codedlevel_flagO[idx] + (abs codedlevel flagl fidx] + abs codedlevel _flag2[idx] + abs codedlevel _flag3[idx] + abs_codedlevel_remainder[idx])«l) (10), where an arithmetic shift operation (X « 1) is used for calculating the multiplication (X * 2), and abs codedlevel _flag3[idx] may specify the value of the fourth syntax element for level coding for color component idx; when not present, abs_codedlevel_flag3[idx] may be inferred to be equal to 0. Table XV below shows example values of the set of color level syntax elements shown in Table XIII above when the current color component is the first or the second color component, or the current component is the third component, and there is at least one non-zero residual level for the first or the second color component:
'able XV
[0092] It is understood that the absolute level flags and the remainder parameter shown in Tables VII - XV above are for illustrative purposes only and may include any other suitable syntax elements that can represent the absolute level of the color level with the parity flag. It is also
understood that a parity enable flag may be introduced at different locations to enable or disable the usage of the parity flag in color level coding, such as parityBasedColorLevelCodingEnable flag in Tables VII, X, and XIII above. For example, parityBasedColorLevelCodingEnable flag equal to 1 may specify that parity flag is enabled as disclosed above; parityBasedColorLevelCodingEnable flag equal to 0 may specify that parity flag is disabled such that color level coding is performed without parity flag; when not present, parityBasedColorLevelCodingEnable flag may be inferred to be equal to 0. It is further understood that the parity -based color level coding disclosed herein may be signaled at any suitable levels, such as in the sequence header, color header, and/or slice color header, as shown in FIG. 8. [0093] At operation 1112, the color level of the point is encoded into a bitstream based on the syntax elements. In some embodiments, encoder 101 is further configured to encode the color level of the point into a bitstream based on the syntax elements.
[0094] FIG. 12 illustrates a flow chart of a method 1200 of point cloud decoding, according to some embodiments of the present disclosure. Method 1200 may be performed by decoder 201 of decoding system 200 or any other suitable point cloud decoding systems. Method 1200 may include operations 1202 and 1204, as described below. It is understood that some of the operations may be optional, and some of the operations may be performed simultaneously, or in a different order other than shown in FIG. 12.
[0095] At operation 1202, a bitstream is parsed to obtain syntax elements representing a color level of the point, including a parity flag. In some embodiments, decoder 201 is configured to parse a bitstream to obtain syntax elements representing a color level of the point, including a parity flag. The syntax elements may also include a sign flag, multiple absolute level flags, and a remainder parameter. As disclosed above in detail with respect to FIG. 11 on the encoding side, the parity flag may be determined based on an absolute level of the color level; at least one of the absolute level flags may be determined based on the absolute level of the color level and the parity flag; the remainder parameter may be determined based on the absolute level of the color level, the absolute level flags, and the parity flag; the sign flag may be determined based on a sign of the color level. In some embodiments, as shown in Table VII, the absolute level flags include three absolute level flags, and the parity flag is determined after two of the three absolute level flags but prior to one of the three absolute level flags. For example, the parity flag equal to 0 may indicate that a remainder of the absolute level of the color level is an even number, and the parity flag equal to 1 indicates that a remainder of the absolute level of the color level is an odd number. For
example, the first one of the absolute level flags is determined based on comparing the absolute level of the color level with zero or one.
[0096] At operation 1204, the color level of the point is determined based on the syntax elements. In some embodiments, decoder 201 is also configured to determine the color level of the point based on the syntax elements, including the sign flag, parity flag, absolute level flags, and remainder parameter. For example, the color level of the point may be determined based on Equation (5), (6), (7), (8), (9), or (10) above, as disclosed above in detail with respect to FIG. 11. [0097] It is understood that although reflectance and color are used as the examples above to disclose the parity -based attribute level coding schemes, any other suitable attributes associated with the points in point cloud coding may be similarly coded using the parity -based attribute level coding schemes disclosed herein. It is also understood that the parity-based attribute level coding schemes disclosed herein may be used for multispectral and hyperspectral systems to handle more than three attribute components like color attribute. For example, assuming there are N components for a multispectral and hyperspectral attribute, where N is an integer larger than 1, if the current attribute is a non-zero attribute, the levels for the first A- 1 components may have zero value; if there is at least one non-zero level for the first A-l components, the last component may also have zero value; if all the first A-l components are zero value, the last component may only have a non-zero value.
[0098] According to some aspects of the present disclosure, for a color attribute that includes multiple components (e.g., RGB, YCbCr, or YUV), the syntax elements representing the signs of the color levels of different components (e.g., sign flags) are grouped together and separated from the syntax elements representing the absolute color levels, such that the signs can be coded using bypass coding with higher efficiency.
[0099] Attribute level coding may be performed by context-based approaches (a.k.a., context coding) or bypass approaches (a.k.a., bypass coding). For example, context-adaptive modeling based binary arithmetic coding (CABAC), one example of context coding, may use syntax elements (e.g., bins) to code attribute levels into bits. The context-based approaches may update the context model adaptively according to the neighboring coded information; syntax elements coded by this approach are called context-coded syntax elements, for example, context- coded bins (CCBs). In contrast, bypass approaches may assume the probability of 1 or 0 is always 50% and, therefore always uses a fixed context modeling without adaptation; syntax elements coded by this approach are called bypass-coded syntax elements, for example, bypass-code bins
(BCBs). For example, context-coded syntax elements may include the parity flags, absolute level flags, and remainder parameters disclosed herein, and bypass-code syntax elements may include the sign flags disclosed herein.
[0100] For high bit-depth and high bit-rate coding, the throughput becomes a more serious issue. The coding using context-coded bins, however, requires relatively complex hardware implementations and reduces the throughput of coding in general, compared with coding bypass- coded bins and thus, has become a bottleneck for improving the throughput of high bit-depth and high bit-rate coding. Moreover, the frequent switch between context coding (e.g., for coding parity flags, absolute level flags, and remainder parameters) and bypass coding (e.g., for coding sign flags) may further affect the attribute coding efficiency and performance.
[0101] Thus, by grouping the sign flags of multiple color components and separating them from context-coded syntax elements (e.g., parity flags, absolute level flags, and remainder parameters), the bypass coding of all the sign flags of color components can be performed together after the context coding of all color components is done, thereby improving the coding efficiency and performance for color level coding. For example, FIG. 13 illustrates a flow chart of a method 1300 of point cloud encoding, according to some embodiments of the present disclosure. Method 1300 may be performed by encoder 101 of encoding system 100 or any other suitable point cloud decoding systems. Method 1300 may include operations 1302, 1304, 1306, and 1308, as described below. It is understood that some of the operations may be optional, and some of the operations may be performed simultaneously, or in a different order other than shown in FIG. 13.
[0102] At operation 1302, color levels of the point each associated with a respective one of the components of a color are determined. In some embodiments, encoder 101 may determine the color levels for each color component of the point.
[0103] At operation 1304, context-coded syntax elements representing each of the color levels of the point are determined based on an absolute level of a respective one of the color levels of the point. In some embodiments, encoder 101 may determine context-coded syntax elements representing each color level based on the absolute level of a respective color level. The context- coded syntax elements may include a parity flag associated with each of the color levels, i.e., three parity flags for three color levels of three color components (e.g., RGB, YUV, or YCrCb). In some embodiments, to determine the context-coded syntax elements, the parity flag is determined based on the absolute level of the respective color level. The context-coded syntax elements may also include absolute level flags and a remainder parameter associated with each of the color levels.
[0104] At operation 1306, bypass-coded syntax elements representing each of the color levels of the point are determined based on a sign of the respective one of the color levels of the point. In some embodiments, encoder 101 may determine bypass-coded syntax elements representing each color level based on the sign of a respective color level. The bypass-coded syntax elements may include at least one of three sign flags associated with three color components
(e.g., RGB, YUV, or YCrCb), for example, depending on whether one or more of the color components are zero levels. In some embodiments, the bypass-coded syntax elements are determined after the context-coded syntax elements.
[0105] The example specification below (Table XVI) illustrates still another set of color level syntax elements and their representation and organization in which the sign flags of all color components (residual_sgin[idx]) are determined together after all context-coded, color level syntax elements of all color components (abs_codedlevel_flagN[idx], abs_codedlevel_remainder[idx], and parity _flag[idx]). It is understood that although parity flags (parity _flag[idx]) are not shown in Table XVI, they may be added in Table XVI as shown in the examples of Table VII, X, and XIIII above, such that all context-coded, color level syntax elements of all color components (abs_codedlevel_flagN[idx], abs_codedlevel_remainder[idx], and parity _flag[idx]) may be determined before determining the sign flags of all color components (residual_sgin[idx]).
Table XVI
[0106] At operation 1308, the color level of the point is encoded into a bitstream based on the context-coded syntax elements using context coding and the bypass-coded syntax elements using bypass coding. In some embodiments, encoder 101 is further configured to encode the color level of the point into the bitstream based on the context-coded syntax elements (e.g., the parity flags, absolute level flags, and remainder parameters) using context coding and the bypass-coded syntax elements (e.g., the sign flags) using bypass coding.
[0107] FIG. 14 illustrates a flow chart of a method 1400 of point cloud decoding, according to some embodiments of the present disclosure. Method 1400 may be performed by decoder 201 of decoding system 200 or any other suitable point cloud decoding systems. Method 1400 may include operations 1402, 1404, 1406, and 1408, as described below. It is understood that some of the operations may be optional, and some of the operations may be performed simultaneously, or in a different order other than shown in FIG. 14.
[0108] At operation 1402, a bitstream is parsed to obtain context-coded syntax elements representing each of color levels of the point. Each of the color levels may be associated with a respective one of the components. In some embodiments, decoder 201 is configured to parse the bitstream to obtain context-coded syntax elements representing each of color levels of the point. The context-coded syntax elements may include a parity flag associated with each of the color levels, i.e., three parity flags for three color levels of three color components (e.g., RGB, YUV, or YCrCb). As disclosed above in detail with respect to FIG. 13 on the encoding side, the parity flag may be determined based on the absolute level of the respective color level. The context-coded syntax elements may also include absolute level flags and a remainder parameter associated with each of the color levels.
[0109] At operation 1404, an absolute level of each of the color levels of the point is determined based on the respective context-coded syntax elements using context coding. In some embodiments, decoder 201 is also configured to determine the absolute level of each of the color levels of the point based on the respective context-coded syntax elements using context coding. For example, the absolute level of each of the color levels of the point may be determined based on Equation (5), (6), (7), (8), (9), or (10) above.
[0110] At operation 1406, the bitstream is parsed to obtain bypass-coded syntax elements representing each of color levels of the point. In some embodiments, decoder 201 is further
configured to parse the bitstream to obtain bypass-coded syntax elements representing each of color levels of the point. The bypass-coded syntax elements may include at least one of three sign flags associated with three color components (e.g., RGB, YUV, or YCrCb), for example, depending on whether one or more of the color components are zero levels. In some embodiments, the bypass-coded syntax elements are parsed after the context-coded syntax elements.
[OHl] At operation 1408, a sign of each of the color levels of the point is determined based on the respective bypass-coded syntax elements using bypass coding. In some embodiments, decoder 201 is further configured to determine the sign of each of the color levels of the point based on the respective bypass-coded syntax elements using bypass coding. For example, the sign of each of the color levels of the point may be determined based on Equation (5), (6), (7), (8), (9), or (10) above.
[0112] Similar to the parity -based attribute level coding schemes, it is understood that the sign flag grouping and bypass coding schemes disclosed herein may be used for multispectral and hyperspectral systems to handle more than three attribute components like color attribute as well.
[0113] In various aspects of the present disclosure, the functions described herein may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as instructions on a non-transitory computer-readable medium. Computer-readable media includes computer storage media. Storage media may be any available media that can be accessed by a processor, such as processor 102 in FIGs. 1 and 2. By way of example, and not limitation, such computer-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, HDD, such as magnetic disk storage or other magnetic storage devices, Flash drive, SSD, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a processing system, such as a mobile device or a computer. Disk and disc, as used herein, includes CD, laser disc, optical disc, digital video disc (DVD), and floppy disk where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
[0114] According to one aspect of the present disclosure, a method for encoding a point cloud is disclosed. The point cloud includes a point associated with a color having components. A color level of the point is determined by a processor. Syntax elements representing the color level of the point are determined by the processor based on the color level of the point. The syntax elements include a parity flag. The color level of the point is encoded by the processor into a
bitstream based on the syntax elements.
[0115] In some embodiments, to determine the syntax elements, the parity flag is determined based on an absolute level of the color level.
[0116] In some embodiments, the syntax elements further include absolute level flags and a remainder parameter.
[0117] In some embodiments, to determine the syntax elements, at least one of the absolute level flags is determined based on the absolute level of the color level and the parity flag, and the remainder parameter is determined based on the absolute level of the color level, the absolute level flags, and the parity flag.
[0118] In some embodiments, the absolute level flags include three absolute level flags, and the parity flag is determined after two of the three absolute level flags but prior to one of the three absolute level flags.
[0119] In some embodiments, the first one of the three absolute level flags is determined based on comparing the absolute level of the color level with zero or one.
[0120] In some embodiments, the parity flag equal to 0 indicates that a remainder of the absolute level of the color level is an even number, and the parity flag equal to 1 indicates that a remainder of the absolute level of the color level is an odd number.
[0121] In some embodiments, the syntax elements further include a sign flag.
[0122] In some embodiments, to determine the syntax elements, the sign flag is determined based on a sign of the color level.
[0123] According to another aspect of the present disclosure, a system for encoding a point cloud includes a memory configured to store instructions and a processor coupled to the memory. The point cloud includes a point associated with a color having components. The processor is configured to, upon executing the instructions, determine a color level of the point. The processor is also configured to, upon executing the instructions, determine syntax elements representing the color level of the point based on the color level of the point. The syntax elements include a parity flag. The processor is further configured to, upon executing the instructions, encode the color level of the point into a bitstream based on the syntax elements.
[0124] According to still another aspect of the present disclosure, a method for decoding a point cloud is disclosed. The point cloud includes a point associated with a color having components. A bitstream is parsed by a processor to obtain syntax elements representing a color level of the point. The syntax elements include a parity flag. The color level of the point is
determined by the processor based on the syntax elements.
[0125] In some embodiments, the parity flag is determined based on an absolute level of the color level.
[0126] In some embodiments, the syntax elements further include absolute level flags and a remainder parameter.
[0127] In some embodiments, at least one of the absolute level flags is determined based on the absolute level of the color level and the parity flag, and the remainder parameter is determined based on the absolute level of the color level, the absolute level flags, and the parity flag.
[0128] In some embodiments, the absolute level flags include three absolute level flags, and the parity flag is determined after two of the three absolute level flags but prior to one of the three absolute level flags.
[0129] In some embodiments, the first one of the three absolute level flags is determined based on comparing the absolute level of the color level with zero or one.
[0130] In some embodiments, the parity flag equal to 0 indicates that a remainder of the absolute level of the color level is an even number, and the parity flag equal to 1 indicates that a remainder of the absolute level of the color level is an odd number.
[0131] In some embodiments, the syntax elements further include a sign flag.
[0132] In some embodiments, the sign flag is determined based on a sign of the color level.
[0133] According to yet another aspect of the present disclosure, a system for decoding a point cloud includes a memory configured to store instructions and a processor coupled to the memory. The point cloud includes a point associated with a color having components. The processor is configured to, upon executing the instructions, parse a bitstream to obtain syntax elements representing a color level of the point. The syntax elements include a parity flag. The processor is also configured to, upon executing the instructions, determine the color level of the point based on the syntax elements.
[0134] The foregoing description of the embodiments will so reveal the general nature of the present disclosure that others can, by applying knowledge within the skill of the art, readily modify and/or adapt for various applications such embodiments, without undue experimentation, without departing from the general concept of the present disclosure. Therefore, such adaptations and modifications are intended to be within the meaning and range of equivalents of the disclosed embodiments, based on the teaching and guidance presented herein. It is to be understood that the
phraseology or terminology herein is for the purpose of description and not of limitation, such that the terminology or phraseology of the present specification is to be interpreted by the skilled artisan in light of the teachings and guidance.
[0135] Embodiments of the present disclosure have been described above with the aid of functional building blocks illustrating the implementation of specified functions and relationships thereof. The boundaries of these functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternate boundaries can be defined so long as the specified functions and relationships thereof are appropriately performed.
[0136] The Summary and Abstract sections may set forth one or more but not all embodiments of the present disclosure as contemplated by the inventor(s), and thus, are not intended to limit the present disclosure and the appended claims in any way.
[0137] Various functional blocks, modules, and steps are disclosed above. The arrangements provided are illustrative and without limitation. Accordingly, the functional blocks, modules, and steps may be reordered or combined in different ways than in the examples provided above. Likewise, some embodiments include only a subset of the functional blocks, modules, and steps, and any such subset is permitted.
[0138] The breadth and scope of the present disclosure should not be limited by any of the above-described embodiments, but should be defined only in accordance with the following claims and their equivalents.
Claims
1. A method for encoding a point cloud, the point cloud comprising a point associated with a color having components, the method comprising: determining, by a processor, a color level of the point; determining, by the processor, syntax elements representing the color level of the point based on the color level of the point, wherein the syntax elements comprise a parity flag; and encoding, by the processor, the color level of the point into a bitstream based on the syntax elements.
2. The method of claim 1, wherein determining the syntax elements comprises determining the parity flag based on an absolute level of the color level.
3. The method of claim 2, wherein the syntax elements further comprise absolute level flags and a remainder parameter.
4. The method of claim 3, wherein determining the syntax elements further comprises: determining at least one of the absolute level flags based on the absolute level of the color level and the parity flag; and determining the remainder parameter based on the absolute level of the color level, the absolute level flags, and the parity flag.
5. The method of claim 4, wherein the absolute level flags comprise three absolute level flags; and the parity flag is determined after two of the three absolute level flags but prior to one of the three absolute level flags.
6. The method of claim 5, wherein the first one of the three absolute level flags is determined based on comparing the absolute level of the color level with zero or one.
7. The method of claim 2, wherein the parity flag equal to 0 indicates that a remainder of the absolute level of the color level is an even number, and the parity flag equal to 1 indicates that a remainder of the absolute level of the color level is an odd number.
8. The method of claim 1, wherein the syntax elements further comprise a sign flag.
9. The method of claim 8, wherein determining the syntax elements further comprises determining the sign flag based on a sign of the color level.
10. A system for encoding a point cloud, the point cloud comprising a point associated with a color having components, the system comprising: a memory configured to store instructions; and a processor coupled to the memory and configured to, upon executing the instructions: determine a color level of the point; determine syntax elements representing the color level of the point based on the color level of the point, wherein the syntax elements comprise a parity flag; and encode the color level of the point into a bitstream based on the syntax elements.
11. A method for decoding a point cloud, the point cloud comprising a point associated with a color having components, the method comprising: parsing, by a processor, a bitstream to obtain syntax elements representing a color level of the point, wherein the syntax elements comprise a parity flag; and determining, by the processor, the color level of the point based on the syntax elements.
12. The method of claim 11, wherein the parity flag is determined based on an absolute level of the color level.
13. The method of claim 12, wherein the syntax elements further comprise absolute level flags and a remainder parameter.
14. The method of claim 13, wherein the absolute level flags are determined based on the absolute level of the color level and the parity flag; and the remainder parameter is determined based on the absolute level of the color level, the absolute level flags, and the parity flag.
15. The method of claim 14, wherein the absolute level flags comprise three absolute level flags; and the parity flag is determined after two of the three absolute level flags but prior to one of the three absolute level flags.
16. The method of claim 15, wherein the first one of the three absolute level flags is determined based on comparing the absolute level of the color level with zero or one.
17. The method of claim 12, wherein the parity flag equal to 0 indicates that a remainder of the absolute level of the color level is an even number, and the parity flag equal to 1 indicates that a remainder of the absolute level of the color level is an odd number.
18. The method of claim 11, wherein the syntax elements further comprise a sign flag.
19. The method of claim 18, wherein the sign flag is determined based on a sign of the color level.
20. A system for decoding a point cloud, the point cloud comprising a point associated with a color having components, the system comprising: a memory configured to store instructions; and a processor coupled to the memory and configured to, upon executing the instructions: parse a bitstream to obtain syntax elements representing a color level of the point, wherein the syntax elements comprise a parity flag; and determine the color level of the point based on the syntax elements.
Applications Claiming Priority (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US202263269098P | 2022-03-09 | 2022-03-09 | |
US202263269099P | 2022-03-09 | 2022-03-09 | |
US63/269,099 | 2022-03-09 | ||
US63/269,098 | 2022-03-09 | ||
US202263269338P | 2022-03-14 | 2022-03-14 | |
US63/269,338 | 2022-03-14 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2023172704A1 true WO2023172704A1 (en) | 2023-09-14 |
Family
ID=87935761
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2023/014931 WO2023172705A1 (en) | 2022-03-09 | 2023-03-09 | Attribute level coding for geometry point cloud coding |
PCT/US2023/014932 WO2023172706A1 (en) | 2022-03-09 | 2023-03-09 | Attribute level coding for geometry point cloud coding |
PCT/US2023/014930 WO2023172704A1 (en) | 2022-03-09 | 2023-03-09 | Attribute level coding for geometry point cloud coding |
Family Applications Before (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2023/014931 WO2023172705A1 (en) | 2022-03-09 | 2023-03-09 | Attribute level coding for geometry point cloud coding |
PCT/US2023/014932 WO2023172706A1 (en) | 2022-03-09 | 2023-03-09 | Attribute level coding for geometry point cloud coding |
Country Status (1)
Country | Link |
---|---|
WO (3) | WO2023172705A1 (en) |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2021049758A1 (en) * | 2019-09-11 | 2021-03-18 | 엘지전자 주식회사 | Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, and point cloud data reception method |
WO2023278829A1 (en) * | 2021-07-02 | 2023-01-05 | Innopeak Technology, Inc. | Attribute coding in geometry point cloud coding |
WO2023028177A1 (en) * | 2021-08-25 | 2023-03-02 | Innopeak Technology, Inc. | Attribute coding in geometry point cloud coding |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11233998B2 (en) * | 2015-05-29 | 2022-01-25 | Qualcomm Incorporated | Coding data using an enhanced context-adaptive binary arithmetic coding (CABAC) design |
CN113711600B (en) * | 2019-04-26 | 2024-06-21 | 松下电器(美国)知识产权公司 | Encoding device, decoding device, encoding method, and decoding method |
CN117915098A (en) * | 2019-06-14 | 2024-04-19 | 松下电器(美国)知识产权公司 | Decoding device and non-transitory computer readable medium |
WO2020256244A1 (en) * | 2019-06-21 | 2020-12-24 | 엘지전자 주식회사 | Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, point cloud data reception method |
US11956470B2 (en) * | 2020-04-07 | 2024-04-09 | Qualcomm Incorporated | Predictor index signaling for predicting transform in geometry-based point cloud compression |
-
2023
- 2023-03-09 WO PCT/US2023/014931 patent/WO2023172705A1/en unknown
- 2023-03-09 WO PCT/US2023/014932 patent/WO2023172706A1/en unknown
- 2023-03-09 WO PCT/US2023/014930 patent/WO2023172704A1/en unknown
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2021049758A1 (en) * | 2019-09-11 | 2021-03-18 | 엘지전자 주식회사 | Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, and point cloud data reception method |
WO2023278829A1 (en) * | 2021-07-02 | 2023-01-05 | Innopeak Technology, Inc. | Attribute coding in geometry point cloud coding |
WO2023028177A1 (en) * | 2021-08-25 | 2023-03-02 | Innopeak Technology, Inc. | Attribute coding in geometry point cloud coding |
Also Published As
Publication number | Publication date |
---|---|
WO2023172706A1 (en) | 2023-09-14 |
WO2023172705A1 (en) | 2023-09-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN113615181B (en) | Method and device for point cloud encoding and decoding | |
US11750839B2 (en) | Method and apparatus for point cloud compression | |
CN110708560A (en) | Point cloud data processing method and device | |
US11469771B2 (en) | Method and apparatus for point cloud compression | |
US12020461B2 (en) | Method and apparatus for Haar-based point cloud coding | |
WO2020123469A1 (en) | Hierarchical tree attribute coding by median points in point cloud coding | |
US20220180567A1 (en) | Method and apparatus for point cloud coding | |
WO2023172703A1 (en) | Geometry point cloud coding | |
WO2023028177A1 (en) | Attribute coding in geometry point cloud coding | |
WO2023278829A1 (en) | Attribute coding in geometry point cloud coding | |
WO2023172704A1 (en) | Attribute level coding for geometry point cloud coding | |
WO2023249999A1 (en) | System and method for geometry point cloud coding | |
WO2024010919A1 (en) | System and method for geometry point cloud coding | |
WO2023096973A1 (en) | Geometry point cloud coding | |
WO2024085936A1 (en) | System and method for geometry point cloud coding | |
WO2023107868A1 (en) | Adaptive attribute coding for geometry point cloud coding | |
EP4364082A1 (en) | Attribute coding in geometry point cloud coding | |
CN113632142B (en) | Point cloud encoding and decoding method and device | |
CN115474041B (en) | Method and device for predicting point cloud attribute and related equipment | |
RU2778864C1 (en) | Implicit geometric division based on a quad-tree or binary tree for encoding a point cloud | |
WO2024065269A1 (en) | Point cloud encoding and decoding method and apparatus, device, and storage medium | |
US20240244242A1 (en) | Reduced Memory Coding | |
CN117813822A (en) | Attribute codec in geometric point cloud codec | |
CN115733990A (en) | Point cloud coding and decoding method, device and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 23767491 Country of ref document: EP Kind code of ref document: A1 |