WO2020187283A1 - Point cloud encoding method, point cloud decoding method, apparatus, and storage medium - Google Patents

Point cloud encoding method, point cloud decoding method, apparatus, and storage medium Download PDF

Info

Publication number
WO2020187283A1
WO2020187283A1 PCT/CN2020/080140 CN2020080140W WO2020187283A1 WO 2020187283 A1 WO2020187283 A1 WO 2020187283A1 CN 2020080140 W CN2020080140 W CN 2020080140W WO 2020187283 A1 WO2020187283 A1 WO 2020187283A1
Authority
WO
WIPO (PCT)
Prior art keywords
patch
information
decoded
encoded
point cloud
Prior art date
Application number
PCT/CN2020/080140
Other languages
French (fr)
Chinese (zh)
Inventor
蔡康颖
张德军
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN201910214557.8A external-priority patent/CN111726616B/en
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to JP2021556407A priority Critical patent/JP7453992B2/en
Priority to BR112021018435A priority patent/BR112021018435A2/en
Priority to SG11202110220RA priority patent/SG11202110220RA/en
Priority to EP20774250.3A priority patent/EP3934250A4/en
Priority to KR1020217033214A priority patent/KR20210134401A/en
Publication of WO2020187283A1 publication Critical patent/WO2020187283A1/en
Priority to US17/477,812 priority patent/US20220007037A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties

Definitions

  • This application relates to the field of data processing technology, and in particular to a point cloud encoding method, point cloud decoding method, device and storage medium.
  • This application provides a point cloud encoding method, point cloud decoding method, device and storage medium, which can solve the problem of how to effectively encode and decode point clouds in related technologies.
  • the technical solution is as follows:
  • a point cloud coding method comprising: obtaining auxiliary information of a patch to be coded (point cloud block); encoding the auxiliary information of the patch to be coded and a first identifier of the patch to be coded into a code stream;
  • the first identifier is a first value to indicate that the patch to be encoded is a patch of the first type, and the auxiliary information of the patch to be encoded includes three-dimensional information;
  • the first identifier is a second value to indicate that the patch to be encoded is a patch of the second type,
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded;
  • the first identifier is the third value to indicate that the patch to be encoded is a third-type patch, and the auxiliary information of the patch to be encoded Including two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
  • the value of the first identifier can be the first value, the second value, and the third value. Different values indicate different types of patches. Therefore, different types of patches can be distinguished by the first identifier, so that there is no need to write the number of the first type of patches in the code stream, which reduces the bit overhead of the code stream. Moreover, for different types of patches, the content of the auxiliary information compiled into the code stream may be different, and for the first type of patch and the second type of patch, only part of the auxiliary information is compiled into the code stream. In this way, the format of the code stream can be simplified, and the bit overhead of the code stream can be further reduced, thereby improving the coding efficiency.
  • the patch to be encoded may be any patch included in the current point cloud frame to be encoded.
  • the first identifier may indicate the type of patch to be encoded.
  • the first identified syntax element may be patch_mode or patch_type, etc. The value of the first identifier is different, and the type of the patch to be encoded is different.
  • the first identifier when the first identifier is the first value, it can indicate that the patch to be encoded is the first type of patch.
  • the auxiliary information of the patch to be encoded encoded into the code stream includes three-dimensional information, but does not include two-dimensional information, rotation transformation information, and normal direction. Coordinate axis information.
  • the first identifier is the second value, it can indicate that the patch to be encoded is the second type of patch.
  • the auxiliary information of the patch to be encoded encoded into the code stream includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded, and Does not include rotation transformation information and normal coordinate axis information.
  • the auxiliary information of the patch to be encoded encoded into the code stream includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the first-type patch has matching patches in all point clouds in the current point cloud group, and the two-dimensional information, rotation transformation information and normal coordinate axis between the first-type patches in the current point cloud group
  • the information can be the same, but the three-dimensional information can be different.
  • the second type of patch has a matching relationship in the reference frame of the current point cloud frame, and the rotation transformation information and normal coordinate axis between the second type of patch and its reference patch The information can be the same, while the two-dimensional information and the three-dimensional information can be different.
  • the third type of patch does not have a matching patch in the reference frame of the current point cloud frame. Therefore, in order to reduce the bit overhead of the bit stream, and then improve the coding Efficiency.
  • the auxiliary information of the patch to be encoded may include three-dimensional information, but not two-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded, but does not include rotation transformation information and normal coordinate axis information.
  • the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the first type of patch can be a global matching patch in the current group
  • the second type of patch can be a local matching patch in the current group
  • the third type of patch can be a non-matching patch in the current group.
  • the first value, the second value, and the third value are binary 10, 11, and 00 respectively, or the first value, the second value, and the third value are binary 11, 10, and 00, respectively.
  • the reference patch of the patch to be encoded may be included in the reference frame of the current point cloud frame.
  • the three-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the 3D space along the tangent direction (3d_shift_tangent_axis), and the offset of the patch to be encoded in the 3D space along the bitangent direction (3d_shift_bitangent_axis), And the offset of the patch to be encoded in the 3D space along the normal direction (3d_shift_normal_axis).
  • the two-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the current point cloud frame occupancy map along the u-axis direction (2d_shift_u), and the offset of the patch to be encoded in the current point cloud frame occupancy map along the v-axis direction ( 2d_shift_v), the width of the patch to be encoded in the current point cloud frame occupancy map (2d_size_u), and the height of the patch to be encoded in the current point cloud frame occupancy map (2d_size_v).
  • the syntax element of the reference patch of the patch to be encoded may be patchIndex, which is used to indicate the identification information of the reference patch; the syntax element may also include the identification information of the reference frame where the reference patch is located: frameIndex.
  • the syntax element of the rotation transformation information of the patch to be encoded may be rotation.
  • the identification information of the reference patch of the patch to be encoded may indicate the reference patch of the patch to be encoded.
  • the normal axis information may be the normal axis index of the patch to be encoded, for example, the normal axis index of the normalAxis may be 0, 1, or 2, for the X axis, Y axis, and Z axis, respectively.
  • the first identifiers of multiple patches in the current point cloud frame can be packaged together and compiled into a code stream.
  • the first identifiers of all patches in the current point cloud frame can be packaged together and compiled into the code stream.
  • the first identification of patch[i-1] to be encoded and the auxiliary information of patch[i-1] to be encoded, the first identification of patch[i] to be encoded and patch[ i] auxiliary information, as well as the first identifier of the patch[i+1] to be coded and the auxiliary information of the patch[i+1] to be coded are sequentially encoded into the bitstream, that is, the current point cloud frame
  • the first identifier of each patch is individually packaged and compiled into the code stream.
  • the three-dimensional information of the patch to be encoded can be encoded into the code stream according to the three-dimensional information of the reference patch of the patch to be encoded.
  • the differential encoding method can be used to encode the three-dimensional information of the patch to be encoded.
  • the 3D information is encoded into the code stream, or directly into the code stream, and the specific encoding method is not limited. Use fixed-length encoding to encode the first identifier of the patch to be encoded into the code stream.
  • the two-dimensional information and three-dimensional information of the patch to be encoded can be encoded into the code stream according to the two-dimensional information and three-dimensional information of the reference patch to be encoded.
  • the differential encoding method can be used to encode the two-dimensional information and three-dimensional information of the patch to be encoded.
  • the two-dimensional information and three-dimensional information of the encoded patch are encoded into the code stream, and the two-dimensional information and the three-dimensional information of the patch to be encoded can also be encoded into the code stream by using a more accurate predictive difference method.
  • the specific encoding method is not limited.
  • the identification information and the first identification of the reference patch of the patch to be encoded are encoded into the code stream using a fixed-length encoding method.
  • the fixed-length encoding method can be used to encode the two-dimensional information, three-dimensional information, rotation transformation information, normal coordinate axis information, and the first identifier of the patch to be encoded into the code stream.
  • the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.
  • the embodiment of the present application may divide the current point cloud group
  • the first identifier of all patches included in the point cloud of the first frame is set to the third value. In this way, when the current point cloud frame is the first point cloud in the current point cloud group, the first identifier of the first patch in the current point cloud frame is the third value.
  • the first identifier of the first patch in the current point cloud frame is the first value.
  • the first identifier of the patch to be encoded is the third value
  • the patch to be encoded is the first patch in the current point cloud frame
  • it can be determined that the current point cloud frame is the first point cloud in the current point cloud group .
  • the patches included in all point clouds in the current point cloud group can be arranged in the order of the first type of patch, the second type of patch, and the third type of patch.
  • the patch to be coded is the first type 3 patch, then it can be coded according to the fixed-length encoding method. If the patch to be coded is not the first type 3 patch, at this time, it can be based on the first type 3 patch
  • the two-dimensional information and the three-dimensional information of the patch to be encoded are encoded into the code stream using the differential encoding method, and other information is also encoded using the fixed-length encoding method described above.
  • the auxiliary information of the patch to be encoded may also include identification information of the reference patch of the patch to be encoded.
  • the identification information of the reference patch is used to uniquely identify the reference patch of the patch to be encoded in the reference frame of the patch to be encoded.
  • the identification information of the reference patch may be the position number of the reference patch in the reference frame.
  • a point cloud decoding method includes: parsing the first identifier of the patch to be decoded from the code stream; when the first identifier is the first value, parsing the auxiliary information of the patch to be decoded from the code stream Information, the auxiliary information includes the three-dimensional information of the patch to be decoded; when the first identifier is the second value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded, and the patch to be decoded
  • the patch refers to the identification information of the patch; when the first identification is the third value, the auxiliary information of the patch to be decoded is parsed from the code stream.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and normal direction of the patch to be decoded Coordinate axis information: reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded
  • the first identifiers of all patches in the current point cloud frame can be individually packaged and compiled into a code stream, or they can be packaged together and compiled into a code stream. Therefore, for the respective methods of packaging and encoding into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, then the first identifier of the patch to be decoded can be parsed from the code stream. Of course, for the non-first patch, the first identifier of the patch to be decoded can also be parsed from the code stream. That is, the first identifier of each patch to be decoded in the current point cloud frame can be analyzed sequentially.
  • the first identifier of the patch[i-1] to be decoded and the auxiliary information of the patch[i-1] to be decoded are sequentially parsed, the first identifier of the patch[i] to be decoded and the auxiliary information of the patch[i] to be decoded , And the first identifier of the patch[i+1] to be decoded and the auxiliary information of the patch[i+1] to be decoded.
  • the patch to be decoded is the first patch in the current point cloud frame
  • the first identification of multiple patches included in the current point cloud frame can be parsed from the code stream , wherein the multiple first identifiers parsed include the first identifier of the patch to be decoded. That is, the first identifiers of multiple patches included in the current point cloud frame can be obtained through one analysis, without the need for multiple analysis.
  • the first identifier when the first identifier is the first value, it indicates that the patch to be decoded is the first type of patch, that is, the patch to be decoded has a matching patch in all point clouds in the current point cloud group.
  • the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information includes the three-dimensional information of the patch to be decoded, but does not include the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.
  • the first identifier is the second value, it indicates that the patch to be decoded is of the second type, that is, there is a reference patch matching the patch to be decoded in the reference frame of the current point cloud frame.
  • the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be decoded, but does not include rotation transformation information and normal coordinate axis information.
  • the first identifier is the third value, it indicates that the patch to be decoded is a third-type patch, that is, there is no reference patch that matches the patch to be decoded in the reference frame of the current point cloud frame.
  • the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded can be parsed from the code stream.
  • a method that matches the encoding end can be used for decoding, for example, a patch whose position in the decoded point cloud is the same as that of the patch to be decoded in the current point cloud frame Use differential decoding to analyze the 3D information of the patch to be decoded from the code stream.
  • auxiliary information of the patch to be decoded for example, the two-dimensional information of the patch to be decoded, rotation transformation information and normal axis information
  • the position of the patch to be decoded can be obtained from the decoded point cloud and the position of the patch to be decoded in the current point cloud frame
  • the two-dimensional information, rotation transformation information and normal coordinate axis information of the same patch, and the obtained two-dimensional information, rotation transformation information and normal coordinate axis information are used as the two-dimensional information, rotation transformation information and normal direction of the patch to be decoded Coordinate axis information.
  • the two-dimensional information, rotation transformation information and normal coordinate axis information of the reference patch of the patch to be decoded can be obtained from the reference frame of the current point cloud frame, and the two-dimensional information, rotation transformation information and normal coordinate information of the reference patch can be obtained
  • the axis information is used as the two-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded. That is, for the first type of patch, three-dimensional information can be parsed from the code stream, while the two-dimensional information, rotation transformation information, and normal coordinate axis information are derived or obtained, thus avoiding the need to include in the code stream Two-dimensional information, rotation transformation information and normal coordinate axis information also reduce the bit overhead of the code stream and improve coding efficiency at the same time.
  • the auxiliary information of the patch to be decoded can also be obtained according to the auxiliary information of the reference patch of the to-be-decoded patch to assist
  • the information includes two-dimensional information, rotation transformation information, and normal coordinate axis information.
  • reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded includes: reconstructing the patch to be decoded according to the two-dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information of the decoded patch.
  • the reference patch of the patch to be decoded can be the position in the reference frame of the current point cloud frame and the patch to be decoded A patch with the same position in the current point cloud frame.
  • the first type of patch, the second type of patch and the third type of patch in each frame of point cloud in the current point cloud group are not arranged in sequence.
  • the auxiliary information of the patch to be decoded encoded into the code stream may also include the identification information of the reference patch of the patch to be decoded.
  • the auxiliary information parsed from the code stream may also include identification information of the reference patch of the patch to be decoded.
  • the reference patch of the patch to be decoded can be determined according to the identification information of the reference patch of the patch to be decoded.
  • the identification information of the reference patch of the patch to be decoded can be parsed from the code stream by a method matching the encoding end, for example, a fixed-length decoding method can be used for analysis.
  • the reference patch of the reference patch to be decoded can be determined through the identification information of the reference patch to be decoded, and then the rotation transformation information and normal coordinates of the reference patch to be decoded are obtained.
  • the two-dimensional information and three-dimensional information of the patch to be decoded can be parsed from the code stream in a differential decoding manner. That is, for the second type of patch, two-dimensional information and three-dimensional information can be parsed from the code stream, while the rotation transformation information and the normal coordinate axis information are derived or obtained, thus avoiding the need to include in the code stream
  • the rotation transformation information and the normal coordinate axis information also reduce the bit overhead of the code stream and improve the coding efficiency at the same time.
  • reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded includes: reconstructing the patch to be decoded according to the two-dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information of the patch to be decoded.
  • the first identifier is the third value
  • a method that matches the encoding end can be used for decoding, for example, fixed-length decoding is used to parse the two-dimensional information, three-dimensional information, and rotation of the patch to be decoded from the code stream. Transformation information and normal axis information.
  • the auxiliary information of the first third-type patch is coded according to the fixed-length coding method
  • the two-dimensional information and three-dimensional information of the non-first third-type patch are coded by the differential coding method
  • other auxiliary information is coded by the fixed-length coding method. coding.
  • the two-dimensional information and three-dimensional information of the patch to be decoded are parsed from the code stream by differential decoding, and fixed
  • the long decoding method parses the rotation transformation information and normal coordinate axis information of the patch to be decoded from the code stream.
  • the non-first third type of patch two-dimensional information and three-dimensional information can be parsed from the code stream, and the rotation transformation information and the normal coordinate axis information are derived or obtained, so as to avoid
  • the bit overhead of the code stream is also reduced, and the coding efficiency is improved at the same time.
  • the first value, the second value and the third value are binary 10, 11 and 00 respectively, or the first value, the second value and the third value are binary 11, 10 and 00 respectively.
  • the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.
  • the first type patch, the second type patch and the third type patch in each frame of point cloud in the current point cloud group can be arranged in sequence.
  • the patch to be decoded is the first one in the current point cloud frame patch
  • the first type of patch, the second type of patch, and the third type of patch in each frame of point cloud in the current point cloud group are not arranged in order, that is, any of the current point cloud group
  • the first identifier of the first patch in a frame of point cloud can be the first value, the second value or the third value.
  • the first identifiers of multiple patches included in the current point cloud frame can be determined.
  • the multiple patches are all the patches included in the patch to be decoded. If the first identifiers of the multiple patches are all the third value, it can be determined that the current point cloud frame is the first point cloud in the current point cloud group.
  • the auxiliary information of the patch to be decoded that is, after obtaining the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded
  • the auxiliary information can be reproduced according to but not limited to these auxiliary information.
  • this application does not limit the specific information needed to reconstruct the patch to be decoded.
  • a point cloud coding method includes: obtaining auxiliary information of the patch to be coded; encoding the auxiliary information of the patch to be coded and the first identifier of the patch to be coded into a code stream; wherein the first identifier is The first value indicates that the patch to be encoded is a first-type patch, and the information encoded into the code stream also includes a second identifier, where the second identifier is a fourth value to indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream ; Or, the second identifier is the fifth value to indicate that the three-dimensional information of the patch to be encoded has been encoded into the code stream, and the auxiliary information of the patch to be encoded includes three-dimensional information; wherein the first identifier is the second value to indicate that the patch to be encoded is For the second type of patch, the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the
  • the first identifier When the first identifier is the first value, it may indicate that the patch to be encoded is the first type of patch, and the information encoded into the code stream may also include the second identifier.
  • the second identifier is used to indicate whether the three-dimensional information of the patch to be encoded has been encoded into the code stream.
  • the syntax element of the second identification may be override_3d_shift_data_flag.
  • the second identifier is the fourth value, it can indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is the same as the three-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the three-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may not include the three-dimensional information. information.
  • the second identifier is the fifth value
  • the syntax element of the fourth value may be true, and the syntax element of the fifth value may be false.
  • the first identifier when the first identifier is the second value, it may indicate that the patch to be encoded is a second-type patch. That is, there is a reference patch that matches the patch to be encoded in the reference frame of the current point cloud frame.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded.
  • the first identifier when it is a third value, it may indicate that the patch to be encoded is a third-type patch. That is, there is no reference patch that matches the patch to be encoded in the reference frame of the current point cloud frame.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
  • a point cloud decoding method includes: parsing the first identifier of the patch to be decoded from the code stream; when the first identifier is the first value, parsing the first identifier of the patch to be decoded from the code stream Second identifier; when the second identifier is the fourth value, the auxiliary information of the reference patch of the patch to be decoded is used as the auxiliary information of the patch to be decoded; when the second identifier is the fifth value, the patch to be decoded is parsed from the code stream Auxiliary information.
  • the auxiliary information includes the three-dimensional information of the patch to be decoded; when the first identifier is the second value, the auxiliary information of the patch to be decoded is parsed from the code stream.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, and the patch to be decoded.
  • the identification information of the reference patch of the decoded patch; when the first identifier is the third value, the auxiliary information of the patch to be decoded is parsed from the code stream.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and method of the patch to be decoded. To coordinate axis information; according to the auxiliary information of the patch to be decoded, reconstruct the patch to be decoded.
  • the second identifier when the second identifier is the fourth value, it may indicate that the three-dimensional information of the patch to be decoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be decoded is the same as the three-dimensional information of the reference patch of the patch to be decoded. In this case, the auxiliary information of the reference patch of the patch to be decoded can be used as the auxiliary information of the patch to be decoded.
  • the second identifier is the fifth value, it can indicate that the 3D information of the patch to be decoded has been encoded into the bitstream, and the auxiliary information of the patch to be decoded includes 3D information.
  • the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded.
  • a differential decoding method can be used to parse the auxiliary information of the patch to be decoded from the code stream.
  • a point cloud encoding device includes: a patch information acquisition module for auxiliary information of the patch to be encoded; an auxiliary information encoding module for combining auxiliary information of the patch to be encoded and the auxiliary information of the patch to be encoded
  • the first identifier is encoded into the code stream; the first identifier is the first value to indicate that the patch to be encoded is a first type of patch, and the auxiliary information of the patch to be encoded includes three-dimensional information; the first identifier is the second value to indicate the The coded patch is a second type of patch, and the auxiliary information of the patch to be coded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be coded; the first identifier is the third value to indicate that the patch to be coded is the third type Patch, the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the first identifier of the patch to be encoded is the third value
  • the patch to be encoded is the first patch in the current point cloud frame to which it belongs
  • the current point cloud frame is the first point cloud frame.
  • the first value, the second value and the third value are binary 10, 11 and 00 respectively, or the first value, the second value and the third value are binary 11, 10 and 00 respectively.
  • the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.
  • the first type of patch is a global matching patch in the current group
  • the second type of patch is a local matching patch in the current group
  • the third type of patch is a non-matching patch in the current group.
  • a point cloud decoding device includes: an auxiliary information decoding module for parsing the first identifier of the patch to be decoded from the code stream; the auxiliary information decoding module is also used for when the first identifier is the first identifier When the value is one, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded; when the first identifier is the second value, the auxiliary information of the patch to be decoded is parsed from the code stream, auxiliary information Including the two-dimensional information, three-dimensional information of the patch to be decoded, and identification information of the reference patch of the patch to be decoded; when the first identifier is the third value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the patch to be decoded
  • the first value, the second value and the third value are binary 10, 11 and 00 respectively, or the first value, the second value and the third value are binary 11, 10 and 00 respectively.
  • the auxiliary information decoding module is also used for:
  • the patch to be decoded is the first patch and the first identifier is the third value, it is determined that the current point cloud frame to which the patch to be decoded belongs is the first point cloud frame.
  • the auxiliary information decoding module is also used for:
  • the first identifier of the patch to be decoded is parsed from the code stream; or, when the patch to be decoded is the first patch in the current point cloud frame, from the code stream Analyze multiple first identifiers of multiple patches included in the current point cloud frame, and the multiple first identifiers parsed include the first identifier of the patch to be decoded.
  • the auxiliary information decoding module is further configured to: obtain the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded;
  • the reconstruction module is also used to reconstruct the patch to be decoded according to the two-dimensional information, three-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded.
  • the auxiliary information decoding module is also used to: obtain the rotation transformation information and normal coordinate axis information of the patch to be decoded according to the identification information of the reference patch of the patch to be decoded; Used to reconstruct the patch to be decoded according to the two-dimensional information, three-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded.
  • the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.
  • a point cloud encoding device including: a patch information acquisition module for acquiring auxiliary information of the patch to be encoded;
  • Auxiliary information encoding module used to encode auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded into a code stream;
  • the first identifier is a first value to indicate that the patch to be encoded is a first-type patch
  • the information encoded into the code stream also includes a second identifier, where the second identifier is a fourth value to indicate the three-dimensional information of the patch to be encoded
  • the code stream is not coded; or, the second identifier is the fifth value to indicate that the three-dimensional information of the patch to be coded has been coded into the code stream, and the auxiliary information of the patch to be coded includes three-dimensional information;
  • the first identifier is the second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded;
  • the first identifier is a third value to indicate that the patch to be encoded is a third type of patch.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • a point cloud decoding device including: an auxiliary information decoding module, configured to parse the first identifier of the patch to be decoded from the code stream;
  • the auxiliary information decoding module is also used for parsing the second identification of the patch to be decoded from the code stream when the first identification is the first value; when the second identification is the fourth value, referencing the auxiliary information of the patch to be decoded As the auxiliary information of the patch to be decoded; when the second identifier is the fifth value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded; when the first identifier is the second value, The auxiliary information of the patch to be decoded is parsed from the code stream.
  • the auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded and the identification information of the reference patch of the patch to be decoded; when the first identifier is the third value, from the code stream Parse the auxiliary information of the patch to be decoded.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded;
  • the reconstruction module is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.
  • a point cloud encoding method comprising: obtaining auxiliary information of the patch to be encoded; encoding the auxiliary information of the patch to be encoded and the syntax elements of the patch to be encoded into a code stream, and the syntax elements include the first A grammatical element.
  • the syntax element encoded into the code stream also includes a second syntax element.
  • the value of the second syntax element is true to indicate that the two-dimensional information of the patch to be encoded is incorporated into the code stream, but the three-dimensional information of the patch to be encoded is not In the code stream, the auxiliary information of the patch to be coded includes the two-dimensional information; or
  • the syntax element encoded into the code stream also includes a second syntax element and a third syntax element.
  • the value of the second syntax element is false and the value of the third syntax element is true to indicate that the two-dimensional information of the patch to be encoded is not Code stream, and the three-dimensional information of the patch to be coded is coded into the code stream, and the auxiliary information of the patch to be coded includes the three-dimensional information; or
  • the syntax element encoded into the code stream also includes a second syntax element and a third syntax element.
  • the value of the second syntax element is false and the value of the third syntax element is false to indicate that the two-dimensional information of the patch to be encoded is not Code stream, and the 3D information of the patch to be coded is not coded into code stream.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the first syntax element indicates that the patch to be encoded does not have a reference patch
  • the patch to be encoded is the first patch in the current point cloud frame to which it belongs
  • the current point cloud frame is the first point cloud frame
  • the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.
  • the position of the patch to be encoded in the current point cloud frame is the same as the position of the reference patch of the patch to be encoded in the reference frame of the current point cloud frame.
  • a point cloud decoding method includes: parsing a first syntax element of a patch to be decoded from a code stream; when the first syntax element indicates that the patch to be decoded has a reference patch, read from the code stream Parse the second syntax element of the patch to be decoded, or parse the second syntax element and the third syntax element of the patch to be decoded from the code stream; when the value of the second syntax element is true, from the code Parse the two-dimensional information of the patch to be decoded in the stream, and use the three-dimensional information of the reference patch of the patch to be decoded as the three-dimensional information of the patch to be decoded; or, when the value of the second syntax element is false and the value of the third syntax element is false When the value is true, parse the three-dimensional information of the patch to be decoded from the code stream, and use the two-dimensional information of the reference patch of the patch to be decoded as the two-dimensional information of the code stream
  • the method further includes: when the first syntax element indicates that the patch to be decoded does not have a reference patch, parsing auxiliary information of the patch to be decoded from the code stream, and the auxiliary information of the patch to be decoded includes two-dimensional information, Three-dimensional information, rotation transformation information and normal coordinate axis information.
  • the method further includes: when the patch to be decoded is the first patch and the first syntax element indicates that the patch to be decoded does not have a reference patch, determining that the current point cloud frame to which the patch to be decoded belongs is the first point cloud frame .
  • parsing the first syntax element of the patch to be decoded from the code stream includes: when the patch to be decoded is the first patch in the current point cloud frame, parsing the first syntax element of the patch to be decoded from the code stream Element; or, when the patch to be decoded is the first patch in the current point cloud frame, parse multiple first syntax elements of multiple patches included in the current point cloud frame from the code stream, and multiple parsed ones
  • the first syntax element includes the first syntax element of the patch to be decoded.
  • the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.
  • the position of the patch to be decoded in the current point cloud frame is the same as the position of the reference patch of the patch to be decoded in the reference frame of the current point cloud frame.
  • a point cloud encoding device comprising: a point cloud block patch information acquisition module for acquiring auxiliary information of the patch to be encoded;
  • the auxiliary information encoding module is used to encode the auxiliary information of the patch to be encoded and the syntax element of the patch to be encoded into the code stream, and the syntax element includes the first syntax element, wherein:
  • the syntax element encoded into the code stream also includes a second syntax element.
  • the value of the second syntax element is true to indicate that the two-dimensional information of the patch to be encoded is incorporated into the code stream, but the three-dimensional information of the patch to be encoded is not encoded.
  • the auxiliary information of the patch to be encoded includes two-dimensional information; or
  • the syntax element encoded into the code stream also includes a second syntax element and a third syntax element.
  • the value of the second syntax element is false and the value of the third syntax element is true to indicate that the two-dimensional information of the patch to be encoded is not
  • the code stream is encoded, and the three-dimensional information of the patch to be encoded is encoded into the code stream, and the auxiliary information of the patch to be encoded includes the three-dimensional information; or
  • the syntax element encoded into the code stream also includes a second syntax element and a third syntax element.
  • the value of the second syntax element is false and the value of the third syntax element is false to indicate that the two-dimensional information of the patch to be encoded is not
  • the code stream is compiled, and the three-dimensional information of the patch to be coded is not compiled into the code stream.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the first syntax element indicates that the patch to be encoded does not have a reference patch
  • the patch to be encoded is the first patch in the current point cloud frame to which it belongs
  • the current point cloud frame is the first point cloud frame
  • the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.
  • the position of the patch to be encoded in the current point cloud frame is the same as the position of the reference patch of the patch to be encoded in the reference frame of the current point cloud frame.
  • a point cloud decoding device includes: an auxiliary information decoding module, configured to parse the first syntax element of the patch to be decoded from the code stream;
  • the auxiliary information decoding module is also used to parse the second syntax element of the patch to be decoded from the code stream when the first syntax element indicates that the patch to be decoded has a reference patch, or to parse the code stream to be decoded
  • the second syntax element and the third syntax element of the patch when the value of the second syntax element is true, the two-dimensional information of the patch to be decoded is parsed from the code stream, and the three-dimensional information of the patch to be decoded is referenced As the three-dimensional information of the patch to be decoded; or, when the value of the second syntax element is false and the value of the third syntax element is true, the three-dimensional information of the patch to be decoded is parsed from the code stream, and the The two-dimensional information of the reference patch of the decoded patch is used as the two-dimensional information of the patch to be decoded; or, when the value of the second syntax element is false and the value of the third syntax element is false, the reference of the patch to be decoded The two
  • the reconstruction module is used for reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.
  • the auxiliary information of the patch to be decoded includes the two-dimensional information of the patch to be decoded and the three-dimensional information of the patch to be decoded.
  • the auxiliary information decoding module is further configured to parse auxiliary information of the patch to be decoded from the code stream when the first syntax element indicates that the patch to be decoded does not have a reference patch.
  • the auxiliary information of the patch to be decoded includes two Dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
  • the auxiliary information decoding module is further configured to determine that the current point cloud frame to which the patch to be decoded belongs is the first point when the patch to be decoded is the first patch and the first syntax element indicates that the patch to be decoded does not have a reference patch Cloud frame.
  • the auxiliary information decoding module is further configured to parse the first syntax element of the patch to be decoded from the code stream when the patch to be decoded is the first patch in the current point cloud frame; or, when the patch to be decoded is the first patch in the current point cloud frame;
  • the decoded patch is the first patch in the current point cloud frame
  • the multiple first syntax elements of multiple patches included in the current point cloud frame are parsed from the code stream, and the multiple first syntax elements parsed include the The first syntax element of the patch to be decoded.
  • the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.
  • the position of the patch to be decoded in the current point cloud frame is the same as the position of the reference patch of the patch to be decoded in the reference frame of the current point cloud frame.
  • a device for encoding point cloud data may include a memory and an encoder.
  • the memory is used to store point cloud data.
  • the encoder is used to implement the point cloud coding method provided by the first aspect or any possible design of the first aspect, or the encoder is used to implement the points provided by the third aspect or any possible design of the third aspect.
  • the cloud coding method, or the encoder is used to execute the point cloud coding method provided by any one of the possible designs of the ninth aspect or the ninth aspect.
  • a device for decoding point cloud data may include a memory and a decoder.
  • the memory is used to store point cloud data in the form of code stream.
  • the decoder is used to implement the point cloud decoding method provided by the foregoing second aspect or any possible design of the second aspect, or the decoder is used to implement the points provided by any possible design of the foregoing fourth aspect or the fourth aspect
  • the cloud decoding method, or the decoder is used to execute the point cloud decoding method provided by any possible design of the tenth aspect or the tenth aspect.
  • an encoding device including: a memory and a processor; wherein the memory is used to store program code; the processor is used to call the program code to execute the first aspect or the third aspect or the ninth aspect.
  • Point cloud coding method provided by the aspect.
  • a decoding device including: a memory and a processor; wherein the memory is used to store program code; the processor is used to call the program code to execute the second aspect or the fourth aspect or the tenth aspect.
  • the point cloud decoding method provided by the aspect.
  • This application also provides a computer-readable storage medium, including program code, when the program code runs on a computer, the computer executes the first aspect and its possible designs, or the third aspect and its possible designs. Any point cloud coding method in the ninth aspect and its possible designs.
  • This application also provides a computer-readable storage medium, including program code, which when running on a computer, causes the computer to execute the second aspect and its possible designs, or the fourth aspect and its possible designs. Any point cloud decoding method, or any point cloud decoding method provided by the tenth aspect and its possible designs.
  • This application also provides a computer program product, which when the computer program product runs on a computer, causes the computer to execute any possible design of the first aspect or the first aspect, or execute the third aspect or the third aspect Part or all of the steps of the point cloud coding method provided by any possible design, or execute part or all of the steps of the point cloud coding method provided by any possible design of the ninth aspect or the ninth aspect.
  • This application also provides a computer program product, which when the computer program product runs on a computer, causes the computer to execute any possible design of the second aspect or the second aspect, or execute the fourth aspect or the fourth aspect Part or all of the steps of the point cloud decoding method provided by any possible design, or execute part or all of the steps of the point cloud decoding method provided by any possible design of the tenth aspect or the tenth aspect.
  • the value of the first identifier can be the first value, the second value, and the third value. Different values indicate different types of patches. Therefore, different types of patches can be distinguished by the first identifier, so that there is no need to write the number of the first type of patches in the code stream, which reduces the bit overhead of the code stream.
  • the content of the auxiliary information compiled into the code stream may be different, and for the first type of patch and the second type of patch, only part of the auxiliary information is compiled into the code stream. In this way, the format of the code stream can be simplified, and the bit overhead of the code stream can be further reduced, thereby improving the coding efficiency.
  • FIG. 1 is a schematic block diagram of a point cloud decoding system provided by an embodiment of this application;
  • Fig. 2 is a schematic block diagram of an encoder that can be used in an embodiment of the present application
  • Fig. 3 is a schematic diagram of a point cloud applicable to an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a point cloud patch applicable to an embodiment of the present application.
  • FIG. 5 is a schematic diagram of an occupancy map of a point cloud applicable to an embodiment of the present application.
  • Fig. 6 is a schematic block diagram of a decoder that can be used in an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a frame group that can be used in an embodiment of the present application.
  • FIG. 8 is a schematic diagram of the first current point cloud group provided by an embodiment of this application.
  • FIG. 9 is a flowchart of the first point cloud encoding method provided by an embodiment of the application.
  • FIG. 10 is a schematic diagram of a second current point cloud group provided by an embodiment of this application.
  • FIG. 11 is a flowchart of the first point cloud decoding method provided by an embodiment of this application.
  • FIG. 12 is a flowchart of a second point cloud encoding method provided by an embodiment of this application.
  • FIG. 13 is a flowchart of a second point cloud decoding method provided by an embodiment of this application.
  • FIG. 14 is a schematic block diagram of a first encoding device provided by an embodiment of this application.
  • FIG. 15 is a schematic block diagram of a first decoding device provided by an embodiment of this application.
  • FIG. 16 is a schematic block diagram of a second encoding device provided by an embodiment of this application.
  • FIG. 17 is a schematic block diagram of a second decoding device provided by an embodiment of this application.
  • FIG. 18 is a schematic block diagram of an implementation manner of a decoding device used in an embodiment of the present application.
  • FIG. 1 is a schematic block diagram of a point cloud decoding system provided by an embodiment of this application.
  • the term "point cloud decoding” or “decoding” may generally refer to point cloud encoding or point cloud decoding.
  • the point cloud decoding system includes a source device 10, a destination device 20, a link 30 and a storage device 40.
  • the source device 10 can generate coded point cloud data. Therefore, the source device 10 may also be referred to as a point cloud encoding device.
  • the destination device 20 may decode the encoded point cloud data generated by the source device 10. Therefore, the destination device 20 may also be referred to as a point cloud decoding device.
  • the link 30 can receive the coded point cloud data generated by the source device 10 and can transmit the coded point cloud data to the destination device 20.
  • the storage device 40 can receive the coded point cloud data generated by the source device 10, and can store the coded point cloud data, so that the destination device 20 can directly obtain the coded point cloud from the storage device 40 data.
  • the storage device 40 may correspond to a file server or another intermediate storage device that can save the encoded point cloud data generated by the source device 10, so that the destination device 20 may stream or download the stored data of the storage device 40 Coded point cloud data.
  • Both the source device 10 and the destination device 20 may include one or more processors and a memory coupled to the one or more processors.
  • the memory may include random access memory (RAM), read-only memory ( read-only memory, ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, which can be used to store the desired program in the form of instructions or data structures that can be accessed by a computer Any other media of the code, etc.
  • both the source device 10 and the destination device 20 may include desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, Televisions, cameras, display devices, digital media players, video game consoles, on-board computers, or the like.
  • the link 30 may include one or more media or devices capable of transmitting the encoded point cloud data from the source device 10 to the destination device 20.
  • the link 30 may include one or more communication media that enable the source device 10 to directly send the encoded point cloud data to the destination device 20 in real time.
  • the source device 10 may modulate the coded point cloud data according to a communication standard, which may be a wireless communication protocol or the like, and may send the modulated point cloud data to the destination device 20.
  • the one or more communication media may include wireless and/or wired communication media.
  • the one or more communication media may include a radio frequency (RF) spectrum or one or more physical transmission lines.
  • RF radio frequency
  • the one or more communication media may form part of a packet-based network, and the packet-based network may be a local area network, a wide area network, or a global network (for example, the Internet).
  • the one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from the source device 10 to the destination device 20, and the embodiment of the present application does not specifically limit this.
  • the storage device 40 may store the received encoded point cloud data sent by the source device 10, and the destination device 20 may directly obtain the encoded point cloud data from the storage device 40 .
  • the storage device 40 may include any of a variety of distributed or locally accessed data storage media.
  • any of the multiple distributed or locally accessed data storage media may be a hard disk drive, Blu-ray disc, digital versatile disc (DVD), compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or used to store Any other suitable digital storage media for encoding point cloud data, etc.
  • the storage device 40 can correspond to a file server or another intermediate storage device that can store the encoded point cloud data generated by the source device 10, and the destination device 20 can be stored via streaming or downloading.
  • the file server may be any type of server capable of storing the coded point cloud data and transmitting the coded point cloud data to the destination device 20.
  • the file server may include a network server, a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive.
  • FTP file transfer protocol
  • NAS network attached storage
  • the destination device 20 can obtain the coded point cloud data through any standard data connection (including an Internet connection).
  • Any standard data connection can include a wireless channel (for example, Wi-Fi connection), a wired connection (for example, digital subscriber line (DSL), cable modem, etc.), or is suitable for obtaining the encoded data stored on the file server The combination of the two point cloud data.
  • the transmission of the encoded point cloud data from the storage device 40 may be a streaming transmission, a download transmission, or a combination of both.
  • the point cloud decoding system shown in FIG. 1 is only one possible implementation, and the technology of the present application can not only be applied to the source device 10 that can encode the point cloud shown in FIG.
  • the destination device 20 for decoding point cloud data may also be applicable to other devices that can encode point clouds and decode encoded point cloud data, which is not specifically limited in the embodiment of the present application.
  • the source device 10 includes a data source 120, an encoder 100 and an output interface 140.
  • the output interface 140 may include a regulator/demodulator (modem) and/or a transmitter, where the transmitter may also be referred to as a transmitter.
  • the data source 120 may include a point cloud capture device (for example, a camera, etc.), a point cloud archive containing previously captured point cloud data, a point cloud feed interface for receiving point cloud data from a point cloud content provider, and/or A computer graphics system used to generate point cloud data, or a combination of these sources of point cloud data.
  • the data source may send a point cloud to the encoder 100, and the encoder 100 may encode the point cloud received from the data source 120 to obtain encoded point cloud data.
  • the encoder can send the encoded point cloud data to the output interface.
  • the source device 10 directly sends the encoded point cloud data to the destination device 20 via the output interface 140.
  • the encoded point cloud data may also be stored on the storage device 40 for the destination device 20 to obtain later and use for decoding and/or playback.
  • the destination device 20 includes an input interface 240, a decoder 200 and a display device 220.
  • the input interface 240 includes a receiver and/or a modem.
  • the input interface 240 can receive the encoded point cloud data via the link 30 and/or from the storage device 40, and then send it to the decoder 200.
  • the decoder 200 can decode the received encoded point cloud data to obtain the Decoded point cloud data.
  • the decoder may send the decoded point cloud data to the display device 220.
  • the display device 220 may be integrated with the destination device 20 or may be external to the destination device 20. Generally, the display device 220 displays the decoded point cloud data.
  • the display device 220 may be any one of multiple types of display devices.
  • the display device 220 may be a liquid crystal display (LCD), a plasma display, or an organic light-emitting diode (OLED). Display or other type of display device.
  • LCD liquid crystal display
  • OLED organic light-emitting
  • the encoder 100 and the decoder 200 may each be integrated with an audio encoder and decoder, and may include an appropriate multiplexer-demultiplexer (multiplexer- demultiplexer, MUX-DEMUX) unit or other hardware and software, used for encoding both audio and video in a common data stream or separate data streams.
  • MUX-DEMUX multiplexer-demultiplexer
  • the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol, or other protocols such as the user datagram protocol (UDP).
  • Each of the encoder 100 and the decoder 200 can be any of the following circuits: one or more microprocessors, digital signal processing (DSP), application specific integrated circuit, ASIC ), field-programmable gate array (FPGA), discrete logic, hardware, or any combination thereof.
  • DSP digital signal processing
  • ASIC application specific integrated circuit
  • FPGA field-programmable gate array
  • the device may store instructions for the software in a suitable non-volatile computer-readable storage medium, and may use one or more processors to execute the instructions in hardware So as to implement the technology of this application. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) can be regarded as one or more processors.
  • Each of the encoder 100 and the decoder 200 may be included in one or more encoders or decoders, and any of the encoders or the decoders may be integrated as a combined encoding in the corresponding device Part of the decoder/decoder (codec).
  • This application may generally refer to the encoder 100 as “signaling” or “sending” certain information to another device such as the decoder 200.
  • the term “signaling” or “sending” may generally refer to the transmission of syntax elements and/or other data used to decode compressed point cloud data. This transmission can occur in real time or almost in real time. Alternatively, this communication may occur after a period of time, for example, when the syntax element is stored in a computer-readable storage medium in the encoded bitstream during encoding, and the decoding device may then store the syntax element on this medium. retrieve the syntax element at any time.
  • FIG. 2 is a schematic block diagram of an encoder 100 provided by an embodiment of the application.
  • Fig. 2 illustrates the point cloud compression (PCC) coding framework of MPEG (moving pictrue expert group) as an example.
  • the encoder 100 may include a point cloud block information generation module 101, a packing module 102, a depth map generation module 103, a texture map generation module 104, a depth map filling module 105, a texture map filling module 106, image-based or video-based Encoding module 107, occupancy map encoding module 108, auxiliary information encoding module 109, multiplexing module 110, point cloud occupancy map down-sampling module 111, point cloud occupancy map filling module 112, point cloud reconstruction module 113, and point cloud filtering module 114 .
  • PCC point cloud compression
  • the point cloud block information generation module 101 may receive one or more point cloud groups sent by the data source 120, where each point cloud group may include one or more frames of point clouds.
  • each point cloud group may include one or more frames of point clouds.
  • the current point cloud group will be unified in the following Take the current point cloud frame in as an example.
  • the point cloud block information generation module 101 can determine the three-dimensional coordinates of each point included in the current point cloud frame in the three-dimensional space coordinate system, and the normal direction vector of each point in the three-dimensional space, and determine each point according to the determined
  • the normal direction vector in the three-dimensional space and the predefined projection plane divide the current point cloud frame into multiple patches (patch). Each patch includes one or more points in the current point cloud frame.
  • the point cloud block information generating module 101 can also determine the occupancy map of each patch and the depth map of each patch.
  • the point cloud block information generating module 101 can also determine the three-dimensional information of each patch and the normal coordinate axis information of each patch, which can be used as auxiliary information of each patch.
  • the point cloud block information generating module 101 can divide the patch into three types, namely the first type patch, the second type patch, and the third type patch.
  • the first identifier is used to indicate the type of each patch.
  • the point cloud block information generating module 101 may send the first identification of each patch and the auxiliary information of each patch in the current point cloud frame to the auxiliary information encoding module 109 for encoding, which may also be called compression encoding.
  • the three-dimensional information of each patch may include the offset of the patch to be encoded in the 3D space along the tangent direction, the offset of the patch to be encoded in the 3D space along the double tangent direction, and the offset of the patch to be encoded in the 3D space. Offset in the normal direction.
  • the point cloud block information generating module 101 may also send the occupancy map of each patch and the auxiliary information of each patch to the packaging module 102.
  • the point cloud block information generating module 101 can also send the depth map of each patch to the depth map generating module 103.
  • the packing module 102 may pack the received occupancy map of each patch and the auxiliary information of each patch sent by the point cloud block information generating module 101 to obtain the occupancy map of the current point cloud frame.
  • the packaging module 102 can arrange the occupancy map of each patch in a specific order, for example, in descending order (or ascending order) of the width/height of the occupancy map of each patch, and then arrange the occupancy map of each patch in descending order (or ascending order). In the order of the occupancy map, insert the occupancy map of each patch into the available area of the occupancy map of the current point cloud frame to obtain the occupancy map of the current point cloud frame and the packing information of the patch.
  • the packing module 102 may send the occupancy map of the current point cloud frame and the packing information of the patch to the auxiliary information encoding module 109 for compression encoding.
  • the packing information of the patch may include the two-dimensional information of each patch in the occupancy map of the current point cloud frame, the identification information of the reference patch of each patch, the rotation transformation information of each patch, and the reference of the current point cloud frame The index of the frame.
  • the two-dimensional information of each patch in the occupancy map of the current point cloud frame may be referred to as the two-dimensional information of each patch.
  • the two-dimensional information of each patch can include the offset of each patch along the u axis in the current point cloud frame occupancy map, the offset of each patch along the v axis in the current point cloud frame occupancy map, and each patch
  • the current point cloud frame occupies the width of the figure, and each patch occupies the height of the figure in the current point cloud frame.
  • the packing module 102 may also send the packing information of the patch to the depth map generating module 103 and the point cloud occupancy map down-sampling module 111.
  • FIG. 4 is a schematic diagram of the patch of the point cloud of the frame
  • Fig. 5 is the occupancy diagram of each patch of the point cloud of the frame shown in Fig. 4, the frame point obtained by packing by the packing module 102 Schematic diagram of cloud occupancy map.
  • the point cloud shown in FIG. 3 may be the current point cloud frame in the embodiment of the present application
  • the patch of the point cloud shown in FIG. 4 may be the patch of the current point cloud frame in the embodiment of the present application
  • the cloud occupancy map may be the occupancy map of the current point cloud frame in the embodiment of the present application.
  • the auxiliary information encoding module 109 receives the occupancy map of the current point cloud frame and the packing information of the patch sent by the packing module 102, as well as the first identification of each patch and the first identification of each patch sent by the point cloud block information generating module 101. After the auxiliary information, the current point cloud frame and each patch included in the current point cloud frame can be encoded to obtain a code stream including the encoded auxiliary information, and then the obtained code stream including the encoded auxiliary information is sent to the complex Use module 110. Among them, the two-dimensional information of each patch and the identification information of the reference patch of each patch in the packaging information of the patch may also be referred to as auxiliary information of each patch.
  • the depth map generation module 103 may receive the patch packaging information sent by the packaging module 102 and the depth map of each patch sent by the point cloud block information generation module 101, and then according to the packaging information of each patch and each The depth map of the patch generates the depth map of the current point cloud frame. Then the generated depth map of the current point cloud frame is sent to the depth map filling module 105 to fill the blank pixels in the depth map of the current point cloud frame to obtain the filled depth map of the current point cloud frame.
  • the depth map filling module 105 may send the obtained filled depth map of the current point cloud frame to the image or video-based encoding module 107 to perform image or video-based encoding on the filled depth map of the current point cloud frame, Obtain the reconstructed current point cloud depth map and the code stream including the encoded current point cloud frame depth map, and may send the obtained reconstructed current point cloud frame depth map to the point cloud reconstruction module 113 , And send the code stream including the encoded depth map of the current point cloud frame to the multiplexing module 110.
  • the point cloud occupancy map down-sampling module 111 may perform down-sampling processing on the received occupancy map of the current point cloud frame sent by the packing module 102 to obtain a low-resolution occupancy map of the current point cloud frame. After that, the point cloud occupancy map down-sampling module 111 may also send the occupancy map of the current low-resolution point cloud frame to the occupancy map encoding module 108 and the point cloud occupancy map filling module 112.
  • the occupancy map encoding module 108 can encode the received occupancy map of the current low-resolution point cloud frame to obtain a code stream including the occupancy map of the encoded low-resolution current point cloud frame, and the occupancy map encoding module 108 can also The code stream including the coded low-resolution occupancy map of the current point cloud frame may be sent to the multiplexing module 110.
  • the point cloud occupancy map filling module 112 fills the occupancy map of the current point cloud frame with the original resolution according to the received occupancy map of the low-resolution current point cloud frame to obtain the filled occupancy map of the current point cloud frame, The occupancy map of the filled current point cloud frame has the original resolution.
  • the point cloud reconstruction module 113 may be based on the received occupancy map of the filled current point cloud frame sent by the point cloud occupancy map filling module 112, and the reconstructed current point cloud frame sent by the image or video-based encoding module 107.
  • the depth map, patch packing information and patch auxiliary information reconstruct the geometry of the current point cloud frame to output the reconstructed point cloud.
  • the point cloud reconstruction module 113 can also output the reconstructed points in the reconstructed point cloud. Correspondence with patch, and the packaging position of the reconstructed point in the reconstructed point cloud.
  • the point cloud reconstruction module 113 can send the reconstructed point cloud, the corresponding relationship between the reconstructed point in the reconstructed point cloud and the patch to the point cloud filtering module 114, and the point cloud reconstruction module 113 can also reconstruct the reconstructed point cloud.
  • the packing positions of the points are sent to the texture generation module 104.
  • the point cloud filtering module 114 may filter the reconstructed point cloud after receiving the reconstructed point cloud sent by the point cloud reconstruction module 113 and the corresponding relationship between the reconstructed points in the reconstructed point cloud and the patch. Specifically, defects such as obvious noise points and gaps in the reconstructed point cloud can be removed to obtain a filtered reconstructed point cloud, which can also be referred to as a smooth reconstructed point cloud. Or it can be said that the point cloud filtering block 114 can perform smoothing processing on the reconstructed point cloud.
  • the texture map generating module 104 receives the smooth reconstructed point cloud sent by the point cloud filtering module 114, the packed position of the reconstructed point in the reconstructed point cloud sent by the point cloud reconstruction module 113, and is sent by the data source 120 After the current point cloud frame, the texture map of the current point cloud frame can be generated according to the smooth reconstructed point cloud, the packaging position of the reconstructed point in the reconstructed point cloud and the current point cloud frame, and the generated current point cloud The texture map of the frame is sent to the texture map filling module 106 to fill the blank pixels in the texture map of the current point cloud frame to obtain the filled texture map of the current point cloud frame.
  • the texture map filling module 106 may send the obtained filled texture map of the current point cloud frame to the image or video-based encoding module 107 to perform image or video-based encoding on the filled texture map of the current point cloud frame, Obtain the code stream including the texture map of the reconstructed current point cloud frame.
  • the image or video-based encoding module 107 may also send the obtained code stream including the reconstructed texture map of the current point cloud frame to the multiplexing module 110.
  • the image or video-based encoding module 107, the occupancy map encoding module 108, and the auxiliary information encoding module 109 can send the obtained code streams to the multiplexing module 110, and the multiplexing module 110 can combine the received code streams To form a combined code stream, and send the combined code stream to the output interface 140.
  • the output interface 140 may send the combined code stream to the decoder 200.
  • the encoder 100 shown in FIG. 2 is only an embodiment provided by the present application.
  • the encoder 100 may include more or less modules than those shown in FIG. 2 Module.
  • the embodiments of the present application do not specifically limit this.
  • FIG. 6 is a schematic block diagram of a decoder 200 according to an embodiment of the application.
  • Figure 6 illustrates the MPEG PCC decoding framework as an example.
  • the decoder 200 may include a demultiplexing module 201, an image or video-based decoding module 202, an occupancy map decoding module 203, an auxiliary information decoding module 204, a point cloud occupancy map filling module 205, and a point cloud reconstruction module 206 , Point cloud filtering module 207 and point cloud texture information reconstruction module 208.
  • the demultiplexing module 201 may receive the combined code stream sent by the output interface 140 of the encoder 100 through the input interface 204, and send the combined code stream to the corresponding decoding module. Specifically, the demultiplexing module 201 sends the code stream of the texture map of the current point cloud frame encoded and the code stream of the depth map of the encoded current point cloud frame to the image or video-based decoding module 202, and The code stream including the occupancy map of the current point cloud frame with the encoded low resolution is sent to the occupancy map decoding module 203, and the code stream including the encoded auxiliary information is sent to the auxiliary information decoding module 204.
  • the image or video-based decoding module 202 can decode the received bitstream including the texture map of the encoded current point cloud frame and the bitstream including the depth map of the encoded current point cloud frame to obtain the reconstructed current
  • the texture map information of the point cloud frame and the depth map information of the reconstructed current point cloud frame, and the reconstructed texture map information of the current point cloud frame can be sent to the point cloud texture information reconstruction module 208,
  • the constructed depth map information of the current point cloud frame is sent to the point cloud reconstruction module 206.
  • the occupancy map decoding module 203 can decode the received bitstream including the occupancy map of the current point cloud frame with encoded low resolution to obtain the occupancy map information of the reconstructed current point cloud frame with low resolution, and
  • the reconstructed low-resolution current point cloud frame occupancy map information is sent to the point cloud occupancy map filling module 205.
  • the point cloud occupancy map filling module 205 can obtain the reconstructed occupancy map information of the current point cloud frame with the original resolution according to the occupancy map information of the reconstructed low-resolution current point cloud frame, and then the reconstructed has The occupancy map information of the current point cloud frame at the original resolution is sent to the point cloud reconstruction module 206.
  • the reconstructed occupancy map information of the current point cloud frame with the original resolution will be uniformly referred to as the occupancy map information of the reconstructed current point cloud frame.
  • the auxiliary information decoding module 204 may decode the received code stream including the encoded auxiliary information to obtain auxiliary information, and may send the auxiliary information to the point cloud reconstruction module 206.
  • the specific decoding method will be described in subsequent embodiments.
  • the point cloud reconstruction module 206 can be based on the received depth map information of the reconstructed current point cloud frame sent by the image or video-based decoding module 202, and the reconstructed current point cloud sent by the point cloud occupancy map filling module 205.
  • the occupancy map information of the frame and the auxiliary information sent by the auxiliary information decoding module 204 reconstruct the geometry of the current point cloud frame to obtain the reconstructed point cloud.
  • the reconstructed point cloud is similar to the reconstructed point cloud obtained by the point cloud reconstruction module 112 in the encoder 100, and the specific reconstruction process can refer to the reconstruction process of the point cloud reconstruction module 112 in the encoder 100. I won't repeat it here.
  • the point cloud reconstruction module 206 may also send the reconstructed point cloud to the point cloud filtering module 207.
  • the point cloud filtering module 207 can filter the reconstructed point cloud according to the received reconstructed point cloud to obtain a smooth reconstructed point cloud.
  • the specific filtering process can refer to the filtering process of the point cloud filtering module 114 in the encoder 100 , I won’t repeat it here.
  • the point cloud filtering module 207 may send the smooth reconstructed point cloud to the texture information reconstruction module 208 of the point cloud.
  • the texture information reconstruction module 208 of the point cloud receives the smooth reconstructed point cloud sent by the point cloud filtering module 207 and the texture map information of the reconstructed current point cloud frame sent by the image or video-based decoding module 202. Afterwards, the texture information of the reconstructed point cloud can be reconstructed to obtain the reconstructed point cloud reconstructed by the texture information.
  • the decoder 200 shown in FIG. 6 is only an example, and in a specific implementation, the decoder 200 may include more or less modules than those shown in FIG. 6. The embodiment of the present application does not limit this.
  • the current point cloud group may be a group of frame (GOF), and a frame group may include 32 frames of point clouds, that is, the current point cloud group may include 32 frames of point clouds.
  • the number of frames of the point cloud included in the current point cloud group may also be other values, which is not limited in the embodiment of the present application.
  • a frame group may also be divided into multiple subgroups, each subgroup may include one or more frames of point clouds, and the current point cloud group may be any one of the multiple subgroups.
  • FIG. 7 is a schematic diagram of a frame group provided by an embodiment of the application. As shown in FIG.
  • a frame group can be divided into 10 subgroups, and the first subgroup includes the first, second, and third subgroups.
  • Frame point cloud the second subgroup includes the 4th frame point cloud
  • the third subgroup includes the 5th, 6, 7, and 8th frame point clouds
  • the 10th subgroup includes the 30th, 31st, and 32th frame point clouds
  • the current point cloud group can be any one of these 10 subgroups.
  • FIG. 7 only exemplarily shows the point clouds included in the first subgroup, the second subgroup, the third subgroup, and the tenth subgroup, and the point clouds included in other subgroups are not described in detail.
  • the current point cloud group can be a frame group, or the current point cloud group can be a subgroup of a frame group.
  • the current point cloud group can also be other forms of point cloud groups, such as custom ones.
  • the point cloud group is not limited in this embodiment of the application.
  • patches can be divided into three types, namely the first type of patch, the second type of patch and the third type of patch.
  • the patch can also be divided into a global matching patch (SKIP_patch), a local matching patch (local_patch), and a non-matching patch (I_INTRA, or P_INTRA). That is, the first type of patch can be a global matching patch, the second type of patch can be a local matching patch, and the third type of patch can be a non-matching patch.
  • the three types of patches are described.
  • the patch to be coded is any patch in the current point cloud frame, and the current point cloud frame is any of the current point cloud groups.
  • a frame of point cloud If the patch to be coded is the first type of patch, the patch to be coded has a matching relationship in all the point cloud frames in the current point cloud group, this can be called a global matching patch; if the patch to be coded is the second For the patch type, the patch to be encoded has a patch with a matching relationship in the reference frame of the current point cloud frame, but only a patch with a matching relationship exists in a part of the point cloud frame in the current point cloud group.
  • the reference frame of the current point cloud frame may be located in the current point cloud group.
  • the reference frame of the current point cloud frame may be the point cloud of the previous frame of the current point cloud frame.
  • the patches included in all point clouds in the current point cloud group can be arranged in the order of the first type patch, the second type patch, and the third type patch.
  • the first type patch in the current point cloud group The number of point clouds in the frame is the same, and when the number of the first-type patches is greater than 1, the first-type patches in each frame of the current point cloud group can also be arranged in the same order, that is, Yes, there is a matching relationship between the first-type patches at the same position in the point cloud of each frame.
  • the w-th first-type patch in each frame of point cloud has a matching relationship with each other, and w is less than or equal to the number of first-type patches.
  • the two-dimensional information, rotation transformation information, and normal coordinate axis information of the first-type patch in the occupancy map of the current point cloud frame are the same, but the three-dimensional information is different.
  • the two-dimensional information of the first-type patch can also be different, or the three-dimensional information can also be the same. That is, the rotation transformation information and the normal coordinate axis information of the first type patch may be the same, and the two-dimensional information and the three-dimensional information may be the same or different. In this way, for two-dimensional information and three-dimensional information, there may be three situations.
  • the following takes the patch to be coded as an example.
  • the two-dimensional information of the reference patch of the patch to be encoded is the same as the two-dimensional information of the patch to be encoded, and the three-dimensional information of the reference patch of the patch to be encoded Same as the three-dimensional information of the patch to be encoded.
  • the two-dimensional information of the reference patch of the patch to be encoded is the same as the two-dimensional information of the patch to be encoded
  • the three-dimensional information of the reference patch of the patch to be encoded is the same as The three-dimensional information of the patch to be encoded is different.
  • the two-dimensional information of the reference patch of the patch to be encoded is different from the two-dimensional information of the patch to be encoded, and the three-dimensional information of the reference patch of the patch to be encoded The information is different from the three-dimensional information of the patch to be encoded.
  • the reference patch of the patch to be decoded may be the same position in the reference frame of the current point cloud frame where the patch to be decoded is located. patch. For example, if the patch to be decoded is in the third position in the current point cloud frame, then the third patch in the reference frame of the current point cloud frame is the reference patch of the patch to be decoded.
  • the rotation transformation information may be rotation angle information, coordinate axis transformation information, or mirror transformation information.
  • the rotation angle information includes a specific rotation angle value or a rotation angle index. For example, if the rotation angle value of the patch to be coded is 0, it means that the patch to be coded does not rotate; if the rotation angle value of the patch to be coded is 20, it means that the patch to be coded is rotated by 20°; if the rotation angle of the patch to be coded is A value of 80 means that the patch to be coded is rotated by 80°.
  • the rotation angle index of the patch to be coded is 0, it means that the patch to be coded does not rotate; if the rotation angle index of the patch to be coded is 1, it means that the patch to be coded is rotated by 15°; if the rotation angle of the patch to be coded is If the index is 2, it means that the patch to be coded is rotated by 30°; if the rotation angle index of the patch to be coded is 3, it means that the patch to be coded is rotated by 45°.
  • the rotation angle information can also be characterized in other forms, which is not specifically limited here. For example, it can also be described in the form of a matrix, especially when acceleration processing is performed in a GPU (Graphic Processing Unit, image processor), the use of the matrix form will increase the processing speed.
  • rotation of the patch to be encoded can be clockwise or counterclockwise.
  • the coordinate axis exchange information can be expressed as the coordinate axis exchange flag.
  • the coordinate axis exchange flag when the coordinate axis exchange flag is false or 0, it indicates the tangent coordinate axis of the patch to be encoded and the double tangent coordinate of the patch to be encoded
  • the axes are not exchanged; for example, when the coordinate axis exchange flag is ture or 1, the tangent axis of the patch to be encoded and the double tangent axis of the patch to be encoded are exchanged.
  • the index of the coordinate axis of the patch to be encoded can be 0, 1, or 2.
  • the tangent axis index of the patch to be encoded and the double tangent axis index of the patch to be encoded can be 0, 1, and 2. Any two.
  • the tangent axis of the patch can be called the U axis
  • the double tangent axis of the patch can be called the V axis.
  • the obtained tangent axis index of the current patch and the double tangent axis index of the patch to be encoded are 0 and 2, respectively.
  • the tangent of the patch to be encoded is not exchanged
  • any of the following point cloud encoding methods may be the point cloud decoding system.
  • the execution by the encoder 100 may be executed by the auxiliary information encoding module 109 in the encoder 100.
  • FIG. 9 is a flowchart of a point cloud encoding method provided by an embodiment of the present application, and the method is applied to a point cloud decoding system. Referring to Figure 9, the method includes:
  • the patch to be encoded may be any patch included in the current point cloud frame to be encoded.
  • S902 Encode the auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded into the code stream.
  • the first identifier may indicate the type of patch to be encoded.
  • the first identified syntax element may be patch_mode or patch_type, etc.
  • the value of the first identifier is different, and the type of the patch to be encoded is different.
  • the first identifier when the first identifier is the first value, it can indicate that the patch to be encoded is the first type of patch.
  • the auxiliary information of the patch to be encoded encoded into the code stream includes three-dimensional information, but does not include two-dimensional information, rotation transformation information, and normal direction.
  • Coordinate axis information when the first identifier is the second value, it can indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded that is encoded into the code stream includes two-dimensional information, three-dimensional information, and the patch to be encoded
  • the identification information of the reference patch does not include the rotation transformation information and the normal coordinate axis information; and/or, when the first identification is the third value, it can indicate that the patch to be encoded is a third-type patch, and is encoded into the waiting stream of the code stream.
  • the auxiliary information of the encoded patch includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
  • the first-type patch has matching patches in all point clouds in the current point cloud group, and the two-dimensional information, rotation transformation information and method between the first-type patches in the current point cloud group
  • the direction axis information is the same, but the three-dimensional information may be different.
  • the second type of patch has a matching patch in the reference frame of the current point cloud frame, and the rotation transformation information and normal direction between the second type of patch and its reference patch
  • the coordinate axis information is the same, but the two-dimensional information and the three-dimensional information may not be the same.
  • the third type of patch does not have a matching patch in the reference frame of the current point cloud frame.
  • the auxiliary information of the patch to be encoded may include three-dimensional information, but not two-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded, but does not include rotation transformation information and normal coordinate axis information.
  • the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the three-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the 3D space along the tangent direction (3d_shift_tangent_axis), and the offset of the patch to be encoded in the 3D space along the bitangent direction (3d_shift_bitangent_axis), And the offset of the patch to be encoded in the 3D space along the normal direction (3d_shift_normal_axis).
  • the two-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the current point cloud frame occupancy map along the u-axis direction (2d_shift_u), and the offset of the patch to be encoded in the current point cloud frame occupancy map along the v-axis direction ( 2d_shift_v), the width of the patch to be encoded in the current point cloud frame occupancy map (2d_size_u), and the height of the patch to be encoded in the current point cloud frame occupancy map (2d_size_v).
  • the syntax element of the reference patch of the patch to be encoded may be patchIndex, which is used to indicate the identification information of the reference patch; the syntax element may also include the identification information of the reference frame where the reference patch is located: frameIndex.
  • the syntax element of the rotation transformation information of the patch to be encoded may be rotation.
  • the identification information of the reference patch of the patch to be encoded may be used to indicate the reference patch of the patch to be encoded.
  • the normal axis information may be the normal axis index of the patch to be encoded, for example, the normal axis index of the normalAxis may be 0, 1, or 2, for the X axis, Y axis, and Z axis, respectively.
  • the three-dimensional information of the patch to be encoded can be encoded into the code stream according to the three-dimensional information of the reference patch of the patch to be encoded.
  • the differential encoding method can be used to encode the three-dimensional information of the patch to be encoded.
  • the 3D information is encoded into the code stream, or directly into the code stream, and the specific encoding method is not limited.
  • the fixed-length encoding method is adopted to encode the first identifier of the patch to be encoded into the code stream.
  • the two-dimensional information and three-dimensional information of the patch to be encoded can be encoded into the code stream according to the two-dimensional information and three-dimensional information of the reference patch to be encoded.
  • the differential encoding method can be used to encode the two-dimensional information and three-dimensional information of the patch to be encoded.
  • the two-dimensional information and three-dimensional information of the encoded patch are encoded into the code stream, and the two-dimensional information and the three-dimensional information of the patch to be encoded can also be encoded into the code stream by using a more accurate predictive difference method.
  • the specific encoding method is not limited.
  • the identification information and the first identification of the reference patch of the patch to be encoded are encoded into the code stream using a fixed-length encoding method.
  • the fixed-length encoding method can be used to encode the two-dimensional information, three-dimensional information, rotation transformation information, normal coordinate axis information, and the first identifier of the patch to be encoded into the code stream.
  • encoding the three-dimensional information of the patch to be encoded into the code stream using the differential encoding method refers to encoding the difference between the three-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch of the patch to be decoded into the code stream.
  • Using differential encoding to encode the two-dimensional information and three-dimensional information of the patch to be encoded into the code stream refers to the difference between the two-dimensional information of the patch to be encoded and the two-dimensional information of the reference patch of the patch to be decoded, and the patch to be encoded
  • the difference between the 3D information of the reference patch and the 3D information of the patch to be decoded is encoded into the code stream.
  • the first identifiers of multiple patches in the current point cloud frame can be packaged together and compiled into a code stream.
  • the first identifiers of all patches in the current point cloud frame can be packaged together and compiled into the code stream.
  • the first identifier of each patch in the current point cloud frame can be individually packaged and compiled into the code stream.
  • the first identifier of the patch[i-1] to be encoded can be combined with The auxiliary information of the patch[i-1] to be encoded, the first identification of the patch[i] to be encoded and the auxiliary information of the patch[i] to be encoded, and the first identification of the patch[i+1] to be encoded and the patch to be encoded [i+1] Auxiliary information, etc. are sequentially encoded into the code stream. That is, for the current point cloud frame, the first identifiers of all patches in the current point cloud frame can be packaged separately and compiled into a code stream, or they can be packed together and compiled into a code stream.
  • the first value, the second value, and the third value may be binary 10, 11, and 00 respectively, or the first value, the second value, and the third value may be binary 11, 10, and 00, etc., respectively.
  • the reference patch of the coded patch is included in the reference frame of the current point cloud frame, which is not limited in the embodiment of the present application.
  • this article exemplifies a partial syntax structure when the first identifier of the patch to be encoded is the first value, as shown in Table 1 and Table 2.
  • skip_patch_data_unit in Table 1 is the information of the patch to be encoded that is encoded into the code stream when the first identifier of the patch to be encoded is the first value, and can also be referred to as the data unit of the patch to be encoded.
  • the embodiment of the present application may After receiving the first frame point cloud of the current point cloud group, the auxiliary information encoding module 109 sets the first identifiers of all patches included in the first frame point cloud of the current point cloud group to the third value. In this way, when the current point cloud frame is the first point cloud in the current point cloud group, the first identifier of the first patch in the current point cloud frame is the third value.
  • the first identifier of the first patch in the current point cloud frame is the first value. That is, when the first identifier of the patch to be encoded is the third value, and the patch to be encoded is the first patch in the current point cloud frame, it can be determined that the current point cloud frame is the first point cloud in the current point cloud group . In this way, there is no need to include the length of the point cloud frame of the current point cloud group in the code stream, thereby improving the coding efficiency and reducing the bit overhead of the code stream.
  • the first-type patch, the second-type patch, and the third-type patch in each frame of point cloud can be arranged in the manner shown in FIG. 8, and then encoded in the order shown in FIG. 8 when encoding.
  • the auxiliary information of the patch to be encoded encoded into the code stream may also include identification information of the reference patch of the patch to be encoded.
  • the identification information of the reference patch is used to uniquely identify the reference patch of the patch to be encoded in the reference frame of the patch to be encoded.
  • the identification information of the reference patch may be the position number of the reference patch in the reference frame.
  • the patch to be coded is the first type 3 patch, then it can be coded as described above. If the patch to be coded is not the first type 3 patch, at this time, it can be based on the two-dimensional information of the first type 3 patch and For the three-dimensional information, the two-dimensional information and the three-dimensional information of the patch to be encoded are encoded into the code stream using the differential encoding method, and other information is also encoded in the manner described above.
  • using differential encoding to encode the two-dimensional information and three-dimensional information of the patch to be encoded into the code stream means: the two-dimensional information and the first
  • the difference between the two-dimensional information of the third-type patch and the difference between the three-dimensional information of the patch to be encoded and the first three-dimensional patch of the third type are encoded into the code stream.
  • the value of the first identifier can be the first value, the second value, and the third value. Different values indicate different types of patches. Therefore, different types of patches can be distinguished by the first identifier, so that there is no need to write the number of the first type of patches in the code stream, which reduces the bit overhead of the code stream.
  • the content of the auxiliary information compiled into the code stream may be different, and for the first type of patch and the second type of patch, only part of the auxiliary information is compiled into the code stream. In this way, the format of the code stream can be simplified, and the bit overhead of the code stream can be further reduced, thereby improving the coding efficiency.
  • any of the following point cloud encoding methods may be the point cloud decoding system.
  • What is executed by the decoder 200 more specifically, may be executed by the auxiliary information decoding module 204 in the decoder 200.
  • the decoding method includes:
  • S1101 Parse the first identifier of the patch to be decoded from the code stream.
  • the first identifiers of all patches in the current point cloud frame can be individually packaged and compiled into a code stream, or they can be packaged together and compiled into a code stream. Therefore, for the respective methods of packaging and encoding into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, then the first identifier of the patch to be decoded can be parsed from the code stream. Of course, for the non-first patch, the first identifier of the patch to be decoded can also be parsed from the code stream. That is, the first identifier of each patch to be decoded in the current point cloud frame can be analyzed sequentially.
  • the first identifier of the patch[i-1] to be decoded and the auxiliary information of the patch[i-1] to be decoded are sequentially parsed, the first identifier of the patch[i] to be decoded and the auxiliary information of the patch[i] to be decoded , And the first identifier of the patch[i+1] to be decoded and the auxiliary information of the patch[i+1] to be decoded.
  • the patch to be decoded is the first patch in the current point cloud frame
  • the first identification of multiple patches included in the current point cloud frame can be parsed from the code stream , wherein the multiple first identifiers parsed include the first identifier of the patch to be decoded. That is, the first identifiers of multiple patches included in the current point cloud frame can be obtained through one analysis, without the need for multiple analysis.
  • a method of matching with the encoding end may be used to parse the first identifier of the patch to be decoded from the code stream.
  • a fixed-length decoding method can be used to parse the first identifier of the patch to be decoded from the code stream.
  • the first identifier is the first value, it indicates that the patch to be decoded is the first type of patch, that is, the patch to be decoded has a matching patch in all point clouds in the current point cloud group.
  • the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information includes the three-dimensional information of the patch to be decoded, but does not include the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.
  • the auxiliary information of the first type of patch that is encoded into the code stream during the encoding process includes three-dimensional information, but does not include two-dimensional information, rotation transformation information and normal coordinate axis information
  • the patch to be decoded is the first type of patch
  • the auxiliary information parsed from the code stream includes the three-dimensional information of the patch to be decoded, but does not include the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.
  • the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded can also be obtained according to the auxiliary information of the reference patch of the patch to be decoded.
  • a method matching the encoding end may be used for decoding, for example, a differential decoding method may be used to parse the three-dimensional information of the patch to be decoded from the code stream.
  • auxiliary information of the patch to be decoded for example, the two-dimensional information of the patch to be decoded, rotation transformation information and normal axis information
  • the position of the patch to be decoded can be obtained from the decoded point cloud and the position of the patch to be decoded in the current point cloud frame
  • the two-dimensional information, rotation transformation information and normal coordinate axis information of the same patch, and the obtained two-dimensional information, rotation transformation information and normal coordinate axis information are used as the two-dimensional information, rotation transformation information and normal direction of the patch to be decoded Coordinate axis information.
  • the two-dimensional information, rotation transformation information and normal coordinate axis information of the reference patch of the patch to be decoded can be obtained from the reference frame of the current point cloud frame, and the two-dimensional information, rotation transformation information and normal coordinate information of the reference patch can be obtained
  • the axis information is used as the two-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded.
  • the reference patch of the patch to be decoded can be the patch whose position in the reference frame of the current point cloud frame is the same as the position of the patch to be decoded in the current point cloud frame .
  • the first type of patch, the second type of patch, and the third type of patch in each frame of point cloud in the current point cloud group are not arranged in the manner shown in Figure 8.
  • the first identifier is the first value
  • the auxiliary information of the patch to be decoded encoded into the code stream may also include the identification information of the reference patch of the patch to be decoded.
  • the auxiliary information parsed from the code stream may also include identification information of the reference patch of the patch to be decoded.
  • the reference patch of the patch to be decoded can be determined according to the identification information of the reference patch of the patch to be decoded.
  • the method of using differential decoding to parse the 3D information of the patch to be decoded from the code stream may specifically be: obtaining the 3D information of the patch whose position is the same as the position of the patch to be decoded in the current point cloud frame from the decoded point cloud. Or, obtain the three-dimensional information of the reference patch of the patch to be decoded from the reference frame of the current point cloud frame. Then, the obtained three-dimensional information is added with the difference of the three-dimensional information parsed from the code stream to obtain the three-dimensional information of the patch to be decoded.
  • the auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded and identification information of the reference patch of the patch to be decoded.
  • the first identifier is the second value, it indicates that the patch to be decoded is a second type of patch, that is, there is a reference patch matching the patch to be decoded in the reference frame of the current point cloud frame.
  • the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be decoded, but does not include rotation transformation information and normal coordinate axis information.
  • the auxiliary information of the second type of patch that is encoded into the code stream during the encoding process includes two-dimensional information, three-dimensional information and identification information of the reference patch, but does not include the rotation transformation information and the normal coordinate axis information, so when the patch to be decoded When it is the second type of patch, the auxiliary information parsed from the code stream includes the two-dimensional information, three-dimensional information of the patch to be decoded and the identification information of the reference patch of the patch to be decoded, but does not include the rotation transformation information and normal direction of the patch to be decoded Coordinate axis information. At this time, the rotation transformation information and normal coordinate axis information of the patch to be decoded can also be obtained according to the identification information of the reference patch of the patch to be decoded.
  • the identification information of the reference patch of the patch to be decoded can be parsed from the code stream by a method matching the encoding end, for example, a fixed-length decoding method can be used for analysis.
  • the reference patch of the reference patch to be decoded can be determined through the identification information of the reference patch to be decoded, and then the rotation transformation information and normal coordinates of the reference patch to be decoded are obtained.
  • the two-dimensional information and three-dimensional information of the patch to be decoded can be parsed from the code stream in a differential decoding manner.
  • the reference patch of the patch to be decoded can be determined by the identification information of the reference patch of the patch to be decoded, and then the two-dimensional information and three-dimensional information of the reference patch of the patch to be decoded can be obtained. Then, the obtained two-dimensional information and three-dimensional information are respectively added with the two-dimensional information difference and the three-dimensional information difference parsed from the code stream, so as to obtain the two-dimensional information and the three-dimensional information of the patch to be decoded.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.
  • the first identifier is the third value, it indicates that the patch to be decoded is a third type of patch, that is, there is no reference patch matching the patch to be decoded in the current point cloud group.
  • the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded can be parsed from the code stream.
  • the first value, the second value, and the third value may be binary 10, 11, and 00 respectively, or the first value, the second value, and the third value may be binary 11, 10, and 00, etc., respectively.
  • the application embodiment does not limit this.
  • the first type patch, the second type patch, and the third type patch in each frame of point cloud in the current point cloud group can be arranged as shown in Figure 8.
  • the patch to be decoded is the current The first patch in the point cloud frame
  • the first type of patch, the second type of patch, and the third type of patch in each frame of point cloud in the current point cloud group are not arranged in the way shown in Figure 8, that is, the current point cloud group
  • the first identifier of the first patch in the point cloud of any frame in the point cloud group may be the first value, the second value, or the third value.
  • the first identifiers of multiple patches included in the current point cloud frame can be determined.
  • the multiple patches are all the patches included in the patch to be decoded. If the first identifiers of the multiple patches are all the third value, it can be determined that the current point cloud frame is the first point cloud in the current point cloud group.
  • the first identifier is the third value
  • the two-dimensional information and three-dimensional information of the non-first third-type patch are encoded by differential encoding.
  • Other auxiliary information is coded using fixed-length coding.
  • differential decoding can be used to parse the two-dimensional information and three-dimensional information of the patch to be decoded from the code stream
  • fixed-length decoding can be used to parse the to-be-decoded patch from the code stream. Decode the rotation transformation information and normal axis information of the patch.
  • the method of using differential decoding to parse the two-dimensional information and three-dimensional information of the patch to be decoded from the code stream is: parse the two-dimensional information and the first information of the patch to be decoded from the code stream.
  • the difference between the two-dimensional information of the third-type patch is added to the two-dimensional information of the first third-type patch to obtain the two-dimensional information of the patch to be decoded.
  • the auxiliary information of the patch to be decoded that is, after obtaining the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded
  • the auxiliary information can be reproduced according to but not limited to these auxiliary information.
  • this application does not limit the specific information needed to reconstruct the patch to be decoded.
  • the first identifier when the first identifier is the first value, it indicates that the patch to be decoded is the first type of patch. At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information includes the patch to be decoded. Three-dimensional information.
  • the first identifier is the second value, it indicates that the patch to be decoded is a second type of patch. At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information includes the two-dimensional information, three-dimensional information and information of the patch to be decoded. The identification information of the reference patch of the patch to be decoded.
  • the first identifier When the first identifier is the third value, it indicates that the patch to be decoded is a third type of patch.
  • the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, and Rotation transformation information and normal coordinate axis information. That is, for the first type of patch and the second type of patch, only part of the auxiliary information needs to be parsed from the code stream, and other auxiliary information can be derived or obtained. In this way, the format of the parsed code stream can be simplified and the code can be reduced. Stream bit overhead, thereby improving decoding efficiency.
  • the embodiment of the present application can indicate different types of patches through the value of the first identifier, that is, it can distinguish between the first type of patch, the second type of patch, and the third type of patch, so there is no need to parse in the code stream.
  • the number of the first type of patch further reduces the bit overhead of the code stream and improves the decoding efficiency.
  • any of the following point cloud encoding methods may be the point cloud decoding system.
  • the execution by the encoder 100 may be executed by the auxiliary information encoding module 109 in the encoder 100.
  • the method includes:
  • S1201 can refer to S1101 in the embodiment shown in FIG. 11, which will not be repeated in this embodiment of the present application.
  • S1202 Coding the auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded into the code stream.
  • the first-type patch has matching patches in all the point clouds in the current point cloud group, and the two-dimensional information, rotation transformation information, and information between the first-type patches in the current point cloud group
  • the normal coordinate axis information is the same, but the three-dimensional information may be different, but in some cases, the three-dimensional information of the first type of patch may also be the same.
  • the first identifier is the first value, it may indicate that the patch to be encoded is a first-type patch, and the information encoded into the code stream may also include the second identifier.
  • the second identifier is used to indicate whether the three-dimensional information of the patch to be encoded has been encoded into the code stream.
  • the syntax element of the second identifier may be override_3d_shift_data_flag, that is, override_3d_shift_data_flag may be referred to as the third syntax element.
  • the second identifier is the fourth value, it can indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream.
  • the three-dimensional information of the patch to be encoded is the same as the three-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the three-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may not include the three-dimensional information. information.
  • the second identifier is the fifth value, it can indicate that the three-dimensional information of the patch to be encoded has been encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is different from the three-dimensional information of the reference patch of the patch to be encoded.
  • the auxiliary information of the patch to be encoded may include three-dimensional information.
  • the specific encoding method is not limited here. For example, a differential encoding method may be used, or a fixed-length encoding method may be used.
  • the syntax element of the fourth value may be false, and the syntax element of the fifth value may be true.
  • this article exemplifies a partial syntax structure when the first identifier of the patch to be encoded is the first value, and the information encoded into the code stream includes the second identifier, see Table 1 and Table 3.
  • the information encoded into the code stream may also include a third identifier for indicating whether the two-dimensional information of the patch to be encoded has been encoded into the code stream.
  • the syntax element of the third identifier may be override_2d_shift_data_flag, that is, override_2d_shift_data_flag may be referred to as the second syntax element. Among them, the second identifier has already been described, so it will not be repeated here.
  • the third identifier is the sixth value, it may indicate that the two-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the two-dimensional information of the patch to be encoded is the same as the two-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the two-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded can be Does not include two-dimensional information.
  • the third identifier is the seventh value, it can indicate that the two-dimensional information of the patch to be encoded has been encoded into the code stream.
  • the two-dimensional information of the patch to be encoded is not the same as the two-dimensional information of the reference patch of the patch to be encoded. Therefore, it is necessary to encode the two-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may include Two-dimensional information.
  • the syntax element of the sixth value may be false, and the syntax element of the seventh value may be true.
  • this article illustrates a partial syntax structure when the first identifier of the patch to be encoded is the first value, and the information encoded into the code stream includes the second identifier and the third identifier, see Table 1 and Table 4 .
  • the two-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch, and the two-dimensional information of the patch to be encoded and the two-dimensional information of the reference patch are not the same, not only the above-mentioned second and third marks can be used to indicate the coding Whether the information of the stream includes 2D information and 3D information of the patch to be decoded. It can also be indicated in other ways.
  • the fourth identifier can also be used to indicate whether the two-dimensional information and three-dimensional information of the patch to be encoded have been encoded into the code stream, that is, when the first identifier is the first value , It can indicate that the patch to be encoded is the first type of patch, and the information encoded into the code stream can also include a fourth identifier.
  • the fourth identified syntax element may be override_patch_data_flag.
  • the fourth identifier is the eighth value, it may indicate that the two-dimensional information and three-dimensional information of the patch to be decoded are the same as the two-dimensional information and three-dimensional information of the reference patch of the patch to be decoded, and the code stream is not included.
  • the fourth identifier is a ninth value
  • it can indicate that the two-dimensional information and three-dimensional information of the patch to be decoded are different from the two-dimensional information and three-dimensional information of the reference patch of the patch to be decoded, and both have been encoded into the code stream.
  • the syntax element of the eighth value may be flase, and the syntax element of the ninth value may be true.
  • the three-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the 3D space along the tangent direction (3d_shift_tangent_axis), the offset of the patch to be encoded in the 3D space along the bitangent direction (3d_shift_bitangent_axis), and The offset of the encoded patch in the 3D space along the normal direction (3d_shift_normal_axis).
  • the two-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the current point cloud frame occupancy map along the u-axis direction (2d_shift_u), and the offset of the patch to be encoded in the current point cloud frame occupancy map along the v-axis direction ( 2d_shift_v), the width of the patch to be encoded in the current point cloud frame occupancy map (2d_size_u), and the height of the patch to be encoded in the current point cloud frame occupancy map (2d_size_v).
  • each of the above identifiers can have two values, and the syntax elements of the two values can be true and false.
  • the meaning of each value can refer to the meaning of different values of the second identifier, the third identifier or the fourth identifier, which will not be repeated here.
  • this article exemplifies when the first identifier of the patch to be encoded is the first value, and the information encoded into the code stream includes the fourth identifier, and the identifier used to indicate different two-dimensional information and three-dimensional information. Part of the grammatical structure, see Table 1 and Table 5.
  • the first identifier when the first identifier is the second value, it may indicate that the patch to be encoded is a second-type patch. That is, there is a reference patch that matches the patch to be encoded in the reference frame of the current point cloud frame.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded.
  • the first identifier when it is a third value, it may indicate that the patch to be encoded is a third-type patch. That is, there is no reference patch that matches the patch to be encoded in the reference frame of the current point cloud frame.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
  • the value of the first identifier may be the first value, the second value, and the third value. Different values indicate different types of patches. Therefore, different types of patches can be distinguished by the first identifier, so that there is no need to write the number of the first type of patches in the code stream, which reduces the bit overhead of the code stream.
  • the information encoded into the code stream may also include a second identifier to indicate whether the three-dimensional information of the patch to be encoded has been encoded into the code stream. When the second identifier is the fourth value, it indicates that the three-dimensional information of the patch to be encoded is not encoded into the code stream.
  • the three-dimensional information of the patch to be encoded is the same as the three-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the three-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may not include the three-dimensional information.
  • Information in this way, can further reduce the bit overhead of the code stream.
  • only part of the auxiliary information is encoded into the code stream. In this way, the format of the code stream can be simplified, and the bit overhead of the code stream can be further reduced, thereby improving the coding efficiency.
  • any of the following point cloud encoding methods may be the point cloud decoding system.
  • What is executed by the decoder 200 more specifically, may be executed by the auxiliary information decoding module 204 in the decoder 200.
  • the decoding method includes:
  • auxiliary information includes the three-dimensional information of the patch to be decoded.
  • the second identifier when the second identifier is the fourth value, it may indicate that the three-dimensional information of the patch to be decoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be decoded is the same as the three-dimensional information of the reference patch of the patch to be decoded. In this case, the auxiliary information of the reference patch of the patch to be decoded can be used as the auxiliary information of the patch to be decoded.
  • the second identifier is the fifth value, it may indicate that the three-dimensional information of the patch to be decoded has been encoded into the bitstream, and the auxiliary information of the patch to be decoded includes the three-dimensional information.
  • the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded.
  • a differential decoding method can be used to parse the auxiliary information of the patch to be decoded from the code stream. The specific method can refer to the previous article.
  • the information of the codestream may also include a third identifier for indicating whether the two-dimensional information of the patch to be encoded has been encoded into the codestream. That is, when the first identifier is the first value, in addition to parsing the second identifier of the patch to be decoded from the code stream, the third identifier of the patch to be decoded can also be parsed from the code stream.
  • the third identifier is the sixth value, it may indicate that the two-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the two-dimensional information of the patch to be encoded is the same as the two-dimensional information of the reference patch of the patch to be encoded, so the two-dimensional information of the reference patch of the patch to be encoded can be used as the two-dimensional information of the patch to be encoded.
  • the third identifier is the seventh value, it can indicate that the two-dimensional information of the patch to be encoded has been encoded into the code stream.
  • the specific decoding method may be a decoding method that matches the encoding method, which will not be repeated here.
  • step 1202 when the three-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch, and the two-dimensional information of the patch to be encoded and the two-dimensional information of the reference patch are different, not only
  • the above-mentioned second identifier and third identifier can be used to indicate whether the information compiled into the code stream includes the two-dimensional information and three-dimensional information of the patch to be decoded. It can also be indicated in other ways, for example, the fourth identifier can also be used to indicate whether the two-dimensional information and three-dimensional information of the patch to be encoded have been encoded into the code stream.
  • the fourth identifier of the patch to be decoded can be parsed from the code stream.
  • the fourth identifier is the eighth value, it can indicate that the two-dimensional information and three-dimensional information of the patch to be decoded are the same as the two-dimensional information and three-dimensional information of the reference patch of the patch to be decoded, and the code stream is not encoded.
  • the two-dimensional information and three-dimensional information of the reference patch of the patch are used as the two-dimensional information and three-dimensional information of the patch to be encoded.
  • the fourth identifier is the ninth value, it can indicate that the two-dimensional information and three-dimensional information of the patch to be decoded are different from the two-dimensional information and three-dimensional information of the reference patch of the patch to be decoded, and both have been encoded into the code stream. Parse the two-dimensional information and three-dimensional information of the patch to be decoded from the code stream.
  • the specific decoding method may be a decoding method that matches the encoding method, which will not be repeated here.
  • the identifier used to indicate whether 2d_shift_u has been encoded into the code stream (override_2d_shift_u)
  • the identifier used to indicate whether 2d_shift_v has been encoded into the code stream (override_2d_shift_v)
  • the auxiliary information includes the two-dimensional information, the three-dimensional information of the patch to be decoded, and the identification information of the reference patch of the patch to be decoded.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded.
  • the value of the first identifier may be the first value, the second value, and the third value. Different values indicate different types of patches. Therefore, different types of patches can be distinguished by the first identifier, so that there is no need to parse the number of the first type of patches in the code stream, and the bit overhead of the code stream is reduced. Moreover, when the first identifier is the first value, the second identifier of the patch to be decoded is parsed from the code stream. When the second identifier is the fourth value, it indicates that the three-dimensional information of the patch to be decoded is not encoded into the code stream.
  • the three-dimensional information of the patch to be decoded is the same as the three-dimensional information of the reference patch of the patch to be decoded.
  • the three-dimensional information of the reference patch of the patch to be decoded can be used as the three-dimensional information of the patch to be decoded, without the need to extract from the code stream. Parsing the three-dimensional information of the patch to be decoded can simplify the format of the parsing code stream, reduce the bit overhead of the code stream, and improve the efficiency of decoding.
  • the encoding device 1400 may include:
  • An auxiliary information encoding module 1402 configured to encode auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded into a code stream;
  • the first identifier is a first value to indicate that the patch to be encoded is a first type of patch, and the auxiliary information of the patch to be encoded includes three-dimensional information;
  • the first identifier is a second value to indicate that the patch to be encoded is a second type of patch
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded
  • the first identifier is a third value to indicate that the patch to be encoded is a third-type patch.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the first value, the second value and the third value are binary 10, 11 and 00 respectively, or the first value, the second value and the third value are binary 11, 10 and 00 respectively.
  • the first identifier of the patch to be encoded is the third value, and the patch to be encoded is the first patch in the current point cloud frame.
  • the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.
  • the first type of patch is a global matching patch in the current group
  • the second type of patch is a local matching patch in the current group
  • the third type of patch is a non-matching patch in the current group.
  • the patch information acquisition module 1401 is used to execute the related content of step S901 of the embodiment shown in FIG. 9, and the auxiliary information encoding module 1402 is used to execute the related content of step S902 of the embodiment shown in FIG. 9.
  • the patch information acquisition module 1401 may correspond to the combination of the point cloud block information generation module 101 and the packing module 102 in FIG. 2
  • the auxiliary information encoding module 1402 may correspond to the auxiliary information encoding module 109 in FIG. 2, in other words,
  • the function of the patch information acquisition module 1401 may be implemented by the point cloud block information generating module 101 and the packing module 102 in FIG.
  • the function of the auxiliary information generating module 1402 may be implemented by the auxiliary information encoding module 109 in FIG. 2.
  • the point cloud block information generation module 101 is used to obtain the three-dimensional information, normal coordinate axis information, and first identification of the patch to be encoded
  • the packaging module 102 is used to obtain the rotation transformation information of the patch to be encoded and the reference patch Identification information and two-dimensional information, etc.
  • this application is not limited to this.
  • the decoding apparatus 1500 may include:
  • the auxiliary information decoding module 1501 is used to parse the first identifier of the patch to be decoded from the code stream;
  • the auxiliary information decoding module 1501 is further configured to parse auxiliary information of the patch to be decoded from the code stream when the first identifier is the first value, and the auxiliary information includes the three-dimensional information of the patch to be decoded; when the first identifier is the second value, The auxiliary information of the patch to be decoded is parsed from the code stream.
  • the auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded and the identification information of the reference patch of the patch to be decoded; when the first identifier is the third value, from the code stream Parse the auxiliary information of the patch to be decoded.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded;
  • the reconstruction module 1502 is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.
  • the first value, the second value, and the third value are binary 10, 11, and 00 respectively, or the first value, the second value, and the third value are binary 11, 10, and 00, respectively.
  • auxiliary information decoding module 1501 is also used for:
  • the patch to be decoded is the first patch and the first identifier is the third value, it is determined that the current point cloud frame to which the patch to be decoded belongs is the first point cloud frame.
  • auxiliary information decoding module 1501 is also used for:
  • the first identifier of the patch to be decoded is parsed from the code stream; or, when the patch to be decoded is the first patch in the current point cloud frame, from the code stream Analyze multiple first identifiers of multiple patches included in the current point cloud frame, and the multiple first identifiers parsed include the first identifier of the patch to be decoded.
  • the auxiliary information decoding module 1501 is further configured to:
  • the reconstruction module 1502 is also used to:
  • the patch to be decoded is reconstructed.
  • the auxiliary information decoding module 1501 is further configured to:
  • the reconstruction module 1502 is also used to:
  • the patch to be decoded is reconstructed.
  • the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.
  • the auxiliary information decoding module 1501 is used to execute the related content of steps S1101-S1104 in the embodiment shown in FIG. 11, and the reconstruction module 1502 is used to execute the related content of step S1105 in the embodiment shown in FIG.
  • the auxiliary information decoding module 1501 in FIG. 15 corresponds to the auxiliary information decoding module 204 in FIG. 6, and the reconstruction module 1502 in FIG. 15 corresponds to the geometric information reconstruction module 206 of the point cloud in FIG. 5.
  • the function of the auxiliary information decoding module 1501 may be implemented by the auxiliary information decoding module 204 in FIG. 6, and the function of the reconstruction module 1502 may be implemented by the point cloud geometric information reconstruction module 206 in FIG. 6.
  • this application is not limited to this.
  • the encoding apparatus 1600 may include:
  • the point cloud block patch information acquisition module 1601 is used to acquire auxiliary information of the patch to be encoded
  • An auxiliary information encoding module 1602 configured to encode auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded into a code stream;
  • the first identifier is a first value to indicate that the patch to be encoded is a first-type patch
  • the information encoded into the code stream also includes a second identifier, where the second identifier is a fourth value to indicate the three-dimensional information of the patch to be encoded
  • the code stream is not coded; or, the second identifier is the fifth value to indicate that the three-dimensional information of the patch to be coded has been coded into the code stream, and the auxiliary information of the patch to be coded includes three-dimensional information;
  • the first identifier is the second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded;
  • the first identifier is a third value to indicate that the patch to be encoded is a third type of patch.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the patch information acquisition module 1601 is used to execute the related content of step S1201 in the embodiment shown in FIG. 12, and the auxiliary information encoding module 1602 is used to execute the related content of step S1202 in the embodiment shown in FIG.
  • the patch information acquisition module 1601 may correspond to the combination of the point cloud block information generation module 101 and the packing module 102 in FIG. 2
  • the auxiliary information encoding module 1602 may correspond to the auxiliary information encoding module 109 in FIG. 2
  • the function of the patch information acquisition module 1601 may be implemented by the point cloud block information generating module 101 and the packing module 102 in FIG. 2
  • the function of the auxiliary information generating module 1602 may be implemented by the auxiliary information encoding module 109 in FIG.
  • the point cloud block information generation module 101 is used to obtain three-dimensional information, normal coordinate axis information, first identification and second identification of the patch to be encoded, and the packaging module 102 is used to obtain rotation transformation information of the patch to be encoded , Refer to patch identification information and two-dimensional information, etc.
  • this application is not limited to this.
  • FIG. 17 is a schematic structural diagram of a second point cloud decoding apparatus provided by an embodiment of this application.
  • the decoding apparatus 1700 may include:
  • the auxiliary information decoding module 1701 is used to parse the first identifier of the patch to be decoded from the code stream;
  • the auxiliary information decoding module 1701 is also used to parse the second identifier of the patch to be decoded from the code stream when the first identifier is the first value; when the second identifier is the fourth value, to add the auxiliary information of the reference patch of the patch to be decoded
  • the information is used as auxiliary information of the patch to be decoded; when the second identifier is the fifth value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded; when the first identifier is the second value Analyze the auxiliary information of the patch to be decoded from the code stream.
  • the auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded and the identification information of the reference patch of the patch to be decoded; when the first identifier is the third value, the auxiliary information Parse the auxiliary information of the patch to be decoded.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded;
  • the reconstruction module 1702 is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.
  • the auxiliary information decoding module 1701 is used to execute related content of steps S1301 to S1305 of the embodiment shown in FIG. 13, and the reconstruction module 1702 is used to execute related content of step S1306 of the embodiment shown in FIG. 13.
  • the auxiliary information decoding module 1701 in FIG. 17 corresponds to the auxiliary information decoding module 204 in FIG. 6, and the reconstruction module 1702 in FIG. 17 corresponds to the geometric information reconstruction module 206 of the point cloud in FIG. 5.
  • the function of the auxiliary information decoding module 1701 may be implemented by the auxiliary information decoding module 204 in FIG. 6, and the function of the reconstruction module 1702 may be implemented by the geometric information reconstruction module 206 of the point cloud in FIG. 6.
  • this application is not limited to this.
  • FIG. 18 is a schematic block diagram of an implementation manner of a decoding device 1800 used in an embodiment of the present application.
  • the obtaining apparatus 1800 may include a processor 1801, a memory 1802, and a bus system 1803.
  • the processor 1801 and the memory 1802 are connected through a bus system 1803, the memory 1802 is used to store instructions, and the processor 1801 is used to execute instructions stored in the memory 1802 to execute various point cloud encoding or decoding methods described in this application , Especially the method of filtering the current image block based on the block size of the current image block. To avoid repetition, it will not be described in detail here.
  • the processor 1801 may be a central processing unit (CPU), and the processor 1801 may also be other general-purpose processors, DSP, ASIC, FPGA or other programmable logic devices, discrete gates. Or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the memory 1802 may include a ROM device or a RAM device. Any other suitable type of storage device can also be used as the memory 1802.
  • the memory 1802 may include code and data 18021 accessed by the processor 1801 using the bus 1803.
  • the memory 1802 may further include an operating system 18023 and an application program 18022.
  • the application program 18022 includes at least one of the point cloud encoding or decoding methods described in this application (especially the point cloud encoding and decoding methods described in this application) that allows the processor 1801 to execute program.
  • the application 18022 may include applications 1 to N, which further include a point cloud encoding or decoding application (referred to as a point cloud decoding application) that executes the point cloud encoding or decoding method described in this application.
  • the bus system 1803 may also include a power bus, a control bus, and a status signal bus. However, for clear description, various buses are marked as the bus system 1803 in the figure.
  • the encoding apparatus 1800 may further include one or more output devices, such as a display 1804.
  • the display 1804 may be a touch-sensitive display that merges the display with a touch-sensitive unit operable to sense touch input.
  • the display 1804 may be connected to the processor 1801 via a bus 1803.
  • the decoding device 1800 can execute the point cloud encoding method in this application, and can also execute the point cloud decoding method in this application.
  • the computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium, such as a data storage medium, or a communication medium that includes any medium that facilitates the transfer of a computer program from one place to another (for example, according to a communication protocol) .
  • computer-readable media may generally correspond to (1) non-transitory tangible computer-readable storage media, or (2) communication media, such as signals or carrier waves.
  • Data storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and/or data structures for implementing the techniques described in this application.
  • the computer program product may include a computer-readable medium.
  • such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or structures that can be used to store instructions or data Any other media that can be accessed by the computer in the form of desired program code. And, any connection is appropriately called a computer-readable medium.
  • any connection is appropriately called a computer-readable medium.
  • coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave are included in the definition of media.
  • the computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other temporary media, but are actually directed to non-transitory tangible storage media.
  • magnetic disks and optical disks include compact disks (CDs), laser disks, optical disks, DVDs, and Blu-ray disks, where disks usually reproduce data magnetically, and optical disks use lasers to reproduce data optically. Combinations of the above should also be included in the scope of computer-readable media.
  • DSP digital signal processors
  • ASIC application-specific integrated circuits
  • FPGA field programmable logic arrays
  • processor may refer to any of the foregoing structure or any other structure suitable for implementing the techniques described herein.
  • the functions described by the various illustrative logical blocks, modules, and steps described herein may be provided in dedicated hardware and/or software modules configured for encoding and decoding, or combined Into the combined codec.
  • the technology may be fully implemented in one or more circuits or logic elements.
  • various illustrative logical blocks, units, and modules in the encoder 100 and the decoder 200 can be understood as corresponding circuit devices or logical elements.
  • the technology of this application can be implemented in a variety of devices or devices, including wireless handsets, integrated circuits (ICs), or a set of ICs (for example, chipsets).
  • ICs integrated circuits
  • a set of ICs for example, chipsets.
  • Various components, modules, or units are described in this application to emphasize the functional aspects of the device for performing the disclosed technology, but they do not necessarily need to be implemented by different hardware units.
  • various units can be combined with appropriate software and/or firmware in the codec hardware unit, or by interoperating hardware units (including one or more processors as described above). provide.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Disclosed by the present application are a point cloud encoding method, a point cloud decoding method, an apparatus, and a storage medium, belonging to the field of data processing. The method comprises: first obtaining auxiliary information of a patch to be encoded, then, encoding into a code stream the auxiliary information and a first identifier of the patch to be encoded. The value of the first identifier may be a first value, a second value, or a third value. Different values indicate different types of patch. Therefore different types of patch can be distinguished by means of the first identifier. For different types of patch, the content comprised by the auxiliary information encoded into the code stream may be different. Thus the format of the encoding into the code stream can be simplified, reducing the bit overhead of the code stream and thereby improving the efficiency of encoding.

Description

点云编码方法、点云解码方法、装置及存储介质Point cloud coding method, point cloud decoding method, device and storage medium
本申请要求于2019年03月19日提交的申请号为201910210181.3、发明名称为“点云编码方法、点云解码方法、装置及存储介质”的中国专利申请的最早优先权，其全部内容通过引用结合在本申请中。本申请还要求于2019年03月20日提交的申请号为201910214557.8、发明名称为“点云编码方法、点云解码方法、装置及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the earliest priority of a Chinese patent application filed on March 19, 2019 with the application number 201910210181.3 and the title of the invention "point cloud encoding method, point cloud decoding method, device and storage medium", the entire content of which is incorporated by reference Incorporated in this application. This application also claims the priority of the Chinese patent application filed on March 20, 2019 with the application number 201910214557.8 and the invention title "point cloud encoding method, point cloud decoding method, device and storage medium", the entire content of which is incorporated by reference Incorporated in this application.
技术领域Technical field
本申请涉及数据处理技术领域，特别涉及一种点云编码方法、点云解码方法、装置及存储介质。This application relates to the field of data processing technology, and in particular to a point cloud encoding method, point cloud decoding method, device and storage medium.
背景技术Background technique
随着三维传感技术的不断发展，点云的采集越来越便捷，并且采集到的点云的质量也越来越高，规模也越来越大。因此，如何有效地对点云进行编码和解码，成为目前亟待解决的问题。With the continuous development of 3D sensing technology, the collection of point clouds has become more and more convenient, and the quality of the collected point clouds has become higher and higher, and the scale has also become larger. Therefore, how to effectively encode and decode the point cloud has become an urgent problem to be solved.
发明内容Summary of the invention
本申请提供了一种点云编码方法、点云解码方法、装置及存储介质，可以解决相关技术的如何有效地对点云进行编码和解码的问题。所述技术方案如下：This application provides a point cloud encoding method, point cloud decoding method, device and storage medium, which can solve the problem of how to effectively encode and decode point clouds in related technologies. The technical solution is as follows:
第一方面，提供了一种点云编码方法，该方法包括：获取待编码patch(点云块)的辅助信息；将待编码patch的辅助信息和待编码patch的第一标识编入码流；其中第一标识为第一值，以指示待编码patch为第一类patch，待编码patch的辅助信息包括三维信息；其中第一标识为第二值，以指示待编码patch为第二类patch，待编码patch的辅助信息包括二维信息、三维信息和待编码patch的参考patch的标识信息；其中第一标识为第三值，以指示待编码patch为第三类patch，待编码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。In a first aspect, a point cloud coding method is provided, the method comprising: obtaining auxiliary information of a patch to be coded (point cloud block); encoding the auxiliary information of the patch to be coded and a first identifier of the patch to be coded into a code stream; The first identifier is a first value to indicate that the patch to be encoded is a patch of the first type, and the auxiliary information of the patch to be encoded includes three-dimensional information; the first identifier is a second value to indicate that the patch to be encoded is a patch of the second type, The auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded; the first identifier is the third value to indicate that the patch to be encoded is a third-type patch, and the auxiliary information of the patch to be encoded Including two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
由于第一标识的取值可以为第一值、第二值和第三值。不同的取值指示不同类型的patch。因此，可以通过第一标识将不同类型的patch进行区分，这样就不需要在码流中写入第一类patch的数量，减少码流的比特开销。而且，针对不同类型的patch，编入码流中的辅助信息包括的内容可以不相同，且对于第一类patch和第二类patch而言，只有部分辅助信息编入码流。这样，可以简化编入码流的格式，进一步减少码流的比特开销，从而提高编码的效率。Because the value of the first identifier can be the first value, the second value, and the third value. Different values indicate different types of patches. Therefore, different types of patches can be distinguished by the first identifier, so that there is no need to write the number of the first type of patches in the code stream, which reduces the bit overhead of the code stream. Moreover, for different types of patches, the content of the auxiliary information compiled into the code stream may be different, and for the first type of patch and the second type of patch, only part of the auxiliary information is compiled into the code stream. In this way, the format of the code stream can be simplified, and the bit overhead of the code stream can be further reduced, thereby improving the coding efficiency.
需要说明的是，待编码patch可以是待编码的当前点云帧包括的patch中的任一patch。第一标识可以指示待编码patch的类型。第一标识的语法元素可以是patch_mode或patch_type等。第一标识的取值不同，待编码patch的类型不同。It should be noted that the patch to be encoded may be any patch included in the current point cloud frame to be encoded. The first identifier may indicate the type of patch to be encoded. The first identified syntax element may be patch_mode or patch_type, etc. The value of the first identifier is different, and the type of the patch to be encoded is different.
其中，第一标识为第一值时，可以指示待编码patch为第一类patch，编入码流的待编码patch的辅助信息包括三维信息，而不包括二维信息、旋转变换信息和法向坐标轴信息。第一 标识为第二值时，可以指示待编码patch为第二类patch，编入码流的待编码patch的辅助信息包括二维信息、三维信息和待编码patch的参考patch的标识信息，而不包括旋转变换信息和法向坐标轴信息。第一标识为第三值时，可以指示待编码patch为第三类patch，编入码流的待编码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。Wherein, when the first identifier is the first value, it can indicate that the patch to be encoded is the first type of patch. The auxiliary information of the patch to be encoded encoded into the code stream includes three-dimensional information, but does not include two-dimensional information, rotation transformation information, and normal direction. Coordinate axis information. When the first identifier is the second value, it can indicate that the patch to be encoded is the second type of patch. The auxiliary information of the patch to be encoded encoded into the code stream includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded, and Does not include rotation transformation information and normal coordinate axis information. When the first identifier is a third value, it can indicate that the patch to be encoded is a third type of patch. The auxiliary information of the patch to be encoded encoded into the code stream includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
由于第一类patch在当前点云组内的所有点云中都存在具有匹配关系的patch，且当前点云组内的第一类patch之间的二维信息、旋转变换信息和法向坐标轴信息可以相同，而三维信息可以不相同，第二类patch在当前点云帧的参考帧中存在具有匹配关系的patch，且第二类patch与其参考patch之间的旋转变换信息和法向坐标轴信息可以相同，而二维信息和三维信息可以不相同，第三类patch在当前点云帧的参考帧中不存在具有匹配关系的patch，因此，为了减少码流的比特开销，进而提高编码的效率，当第一标识为第一值时，待编码patch的辅助信息可以包括三维信息，而不包括二维信息、旋转变换信息和法向坐标轴信息。当第一标识为第二值时，待编码patch的辅助信息可以包括二维信息、三维信息和待编码patch的参考patch的标识信息，而不包括旋转变换信息和法向坐标轴信息。当第一标识为第三值时，待编码patch的辅助信息可以包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。Because the first-type patch has matching patches in all point clouds in the current point cloud group, and the two-dimensional information, rotation transformation information and normal coordinate axis between the first-type patches in the current point cloud group The information can be the same, but the three-dimensional information can be different. The second type of patch has a matching relationship in the reference frame of the current point cloud frame, and the rotation transformation information and normal coordinate axis between the second type of patch and its reference patch The information can be the same, while the two-dimensional information and the three-dimensional information can be different. The third type of patch does not have a matching patch in the reference frame of the current point cloud frame. Therefore, in order to reduce the bit overhead of the bit stream, and then improve the coding Efficiency. When the first identifier is the first value, the auxiliary information of the patch to be encoded may include three-dimensional information, but not two-dimensional information, rotation transformation information, and normal coordinate axis information. When the first identifier is the second value, the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded, but does not include rotation transformation information and normal coordinate axis information. When the first identifier is the third value, the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
其中，第一类patch可以为当前组中的全局匹配patch，第二类patch可以为当前组中的局部匹配patch，第三类patch可以为当前组中的非匹配patch。第一值、第二值和第三值分别是二进制的10、11和00，或者，第一值、第二值和第三值分别是二进制的11、10和00。待编码patch的参考patch可以包括于当前点云帧的参考帧中。Among them, the first type of patch can be a global matching patch in the current group, the second type of patch can be a local matching patch in the current group, and the third type of patch can be a non-matching patch in the current group. The first value, the second value, and the third value are binary 10, 11, and 00 respectively, or the first value, the second value, and the third value are binary 11, 10, and 00, respectively. The reference patch of the patch to be encoded may be included in the reference frame of the current point cloud frame.
需要说明的是，待编码patch的三维信息可以包括待编码patch在3D空间中沿着切线方向的偏移(3d_shift_tangent_axis)、待编码patch在3D空间中沿着双切线方向的偏移(3d_shift_bitangent_axis)，以及待编码patch在3D空间中沿着法线方向的偏移(3d_shift_normal_axis)。待编码patch的二维信息可以包括待编码patch在当前点云帧占用图中沿u轴方向的偏移(2d_shift_u)、待编码patch在当前点云帧占用图中沿v轴方向的偏移(2d_shift_v)、待编码patch在当前点云帧占用图中的宽度(2d_size_u)，以及待编码patch在当前点云帧占用图中的高度(2d_size_v)。另外，待编码patch的参考patch的语法元素可以为patchIndex，用于指示参考patch的标识信息；语法元素还可以包括参考patch所在参考帧的标识信息：frameIndex。待编码patch的旋转变换信息的语法元素可以是rotation。再者，待编码patch的参考patch的标识信息可以指示待编码patch的参考patch。法向坐标轴信息可以为待编码patch的法向轴索引，例如normalAxis法向轴索引可以为0,1或者2，分别对于X轴、Y轴和Z轴。It should be noted that the three-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the 3D space along the tangent direction (3d_shift_tangent_axis), and the offset of the patch to be encoded in the 3D space along the bitangent direction (3d_shift_bitangent_axis), And the offset of the patch to be encoded in the 3D space along the normal direction (3d_shift_normal_axis). The two-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the current point cloud frame occupancy map along the u-axis direction (2d_shift_u), and the offset of the patch to be encoded in the current point cloud frame occupancy map along the v-axis direction ( 2d_shift_v), the width of the patch to be encoded in the current point cloud frame occupancy map (2d_size_u), and the height of the patch to be encoded in the current point cloud frame occupancy map (2d_size_v). In addition, the syntax element of the reference patch of the patch to be encoded may be patchIndex, which is used to indicate the identification information of the reference patch; the syntax element may also include the identification information of the reference frame where the reference patch is located: frameIndex. The syntax element of the rotation transformation information of the patch to be encoded may be rotation. Furthermore, the identification information of the reference patch of the patch to be encoded may indicate the reference patch of the patch to be encoded. The normal axis information may be the normal axis index of the patch to be encoded, for example, the normal axis index of the normalAxis may be 0, 1, or 2, for the X axis, Y axis, and Z axis, respectively.
在一种实现方式下，可以将当前点云帧中的多个patch的第一标识一起打包并编入码流。例如，可以将当前点云帧中的所有patch的第一标识一起打包并编入码流。在另一种实现方式下，可以将待编码patch[i-1]的第一标识和待编码patch[i-1]的辅助信息、待编码patch[i]的第一标识和待编码patch[i]的辅助信息，以及待编码patch[i+1]的第一标识和待编码patch[i+1]的辅助信息等依序编入码流中，也即是，当前点云帧中的每个patch的第一标识是单独打包并编入码流的。In an implementation manner, the first identifiers of multiple patches in the current point cloud frame can be packaged together and compiled into a code stream. For example, the first identifiers of all patches in the current point cloud frame can be packaged together and compiled into the code stream. In another implementation manner, the first identification of patch[i-1] to be encoded and the auxiliary information of patch[i-1] to be encoded, the first identification of patch[i] to be encoded and patch[ i] auxiliary information, as well as the first identifier of the patch[i+1] to be coded and the auxiliary information of the patch[i+1] to be coded are sequentially encoded into the bitstream, that is, the current point cloud frame The first identifier of each patch is individually packaged and compiled into the code stream.
作为一种示例，当第一标识为第一值时，可以根据待编码patch的参考patch的三维信息，将待编码patch的三维信息编入码流，具体可以采用差分编码方式将待编码patch的三维信息编入码流，也可以直接编入码流，具体的编码方式不做限定。采用定长编码方式将待编码patch 的第一标识编入码流。当第一标识为第二值时，可以根据待编码patch的参考patch的二维信息和三维信息，将待编码patch的二维信息和三维信息编入码流，具体可以采用差分编码方式将待编码patch的二维信息和三维信息编入码流，也可以采用更加精确的预测差分方式将待编码patch的二维信息和三维信息编入码流，具体的编码方式不做限定。采用定长编码方式将待编码patch的参考patch的标识信息和第一标识编入码流。当第一标识为第三值时，可以采用定长编码方式将待编码patch的二维信息、三维信息、旋转变换信息、法向坐标轴信息和第一标识编入码流。其中，待编码patch的参考patch包括于当前点云帧的参考帧中。As an example, when the first identifier is the first value, the three-dimensional information of the patch to be encoded can be encoded into the code stream according to the three-dimensional information of the reference patch of the patch to be encoded. Specifically, the differential encoding method can be used to encode the three-dimensional information of the patch to be encoded. The 3D information is encoded into the code stream, or directly into the code stream, and the specific encoding method is not limited. Use fixed-length encoding to encode the first identifier of the patch to be encoded into the code stream. When the first identifier is the second value, the two-dimensional information and three-dimensional information of the patch to be encoded can be encoded into the code stream according to the two-dimensional information and three-dimensional information of the reference patch to be encoded. Specifically, the differential encoding method can be used to encode the two-dimensional information and three-dimensional information of the patch to be encoded. The two-dimensional information and three-dimensional information of the encoded patch are encoded into the code stream, and the two-dimensional information and the three-dimensional information of the patch to be encoded can also be encoded into the code stream by using a more accurate predictive difference method. The specific encoding method is not limited. The identification information and the first identification of the reference patch of the patch to be encoded are encoded into the code stream using a fixed-length encoding method. When the first identifier is the third value, the fixed-length encoding method can be used to encode the two-dimensional information, three-dimensional information, rotation transformation information, normal coordinate axis information, and the first identifier of the patch to be encoded into the code stream. Wherein, the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.
为了在解码时确定当前点云帧是否为当前点云组内的首帧点云，进而在解码过程中对当前点云组与其他点云组进行划分，本申请实施例可以将当前点云组的首帧点云包括的所有patch的第一标识设置为第三值。这样，当当前点云帧为当前点云组内的首帧点云时，当前点云帧中的首个patch的第一标识即为第三值。当当前点云帧为当前点云组内的非首帧点云时，当前点云帧中的首个patch的第一标识即为第一值。也即是，待编码patch的第一标识为第三值，且待编码patch为当前点云帧中的首个patch时，即可确定当前点云帧为当前点云组内的首帧点云。这样就无需在码流中编入当前点云组的点云帧的长度，进而提高了编码效率，同时减少了码流的比特开销。In order to determine whether the current point cloud frame is the first point cloud in the current point cloud group during decoding, and then divide the current point cloud group from other point cloud groups during the decoding process, the embodiment of the present application may divide the current point cloud group The first identifier of all patches included in the point cloud of the first frame is set to the third value. In this way, when the current point cloud frame is the first point cloud in the current point cloud group, the first identifier of the first patch in the current point cloud frame is the third value. When the current point cloud frame is a non-first frame point cloud in the current point cloud group, the first identifier of the first patch in the current point cloud frame is the first value. That is, when the first identifier of the patch to be encoded is the third value, and the patch to be encoded is the first patch in the current point cloud frame, it can be determined that the current point cloud frame is the first point cloud in the current point cloud group . In this way, there is no need to include the length of the point cloud frame of the current point cloud group in the code stream, thereby improving the coding efficiency and reducing the bit overhead of the code stream.
为了进一步减少码流的比特开销，当前点云组中的所有点云包括的patch可以按照第一类patch、第二类patch和第三类patch的顺序依次进行排列，此时，对于第三类patch而言，如果待编码patch为首个第三类patch，那么，可以按照定长编码方式进行编码，如果待编码patch不是首个第三类patch，此时，可以根据首个第三类patch的三维信息，采用差分编码方式将待编码patch的二维信息和三维信息编入码流，而其他的信息还采用前文所述定长编码方式进行编码。In order to further reduce the bit overhead of the code stream, the patches included in all point clouds in the current point cloud group can be arranged in the order of the first type of patch, the second type of patch, and the third type of patch. At this time, for the third type For patch, if the patch to be coded is the first type 3 patch, then it can be coded according to the fixed-length encoding method. If the patch to be coded is not the first type 3 patch, at this time, it can be based on the first type 3 patch For the three-dimensional information, the two-dimensional information and the three-dimensional information of the patch to be encoded are encoded into the code stream using the differential encoding method, and other information is also encoded using the fixed-length encoding method described above.
由于一些因素，对每帧点云包括的patch进行编码时可能需要打乱patch在点云帧中的顺序，此时，为了便于解码，当第一标识为第一值时，编入码流的待编码patch的辅助信息中还可以包括待编码patch的参考patch的标识信息。其中，参考patch的标识信息用于在待编码patch的参考帧中唯一标识待编码patch的参考patch。示例性地，参考patch的标识信息可以为参考patch在参考帧中的位置编号。Due to some factors, it may be necessary to disturb the order of the patch in the point cloud frame when encoding the patch included in the point cloud of each frame. At this time, in order to facilitate decoding, when the first identifier is the first value, the code stream is encoded The auxiliary information of the patch to be encoded may also include identification information of the reference patch of the patch to be encoded. The identification information of the reference patch is used to uniquely identify the reference patch of the patch to be encoded in the reference frame of the patch to be encoded. Exemplarily, the identification information of the reference patch may be the position number of the reference patch in the reference frame.
第二方面，提供了一种点云解码方法，该方法包括：从码流中解析待解码patch的第一标识；当第一标识为第一值时，从码流中解析待解码patch的辅助信息，辅助信息包括待解码patch的三维信息；当第一标识为第二值时，从码流中解析待解码patch的辅助信息，辅助信息包括待解码patch的二维信息、三维信息和待解码patch的参考patch的标识信息；当第一标识为第三值时，从码流中解析待解码patch的辅助信息，辅助信息包括待解码patch的二维信息、三维信息、旋转变换信息和法向坐标轴信息；根据待解码patch的辅助信息，重构待解码patch。In a second aspect, a point cloud decoding method is provided. The method includes: parsing the first identifier of the patch to be decoded from the code stream; when the first identifier is the first value, parsing the auxiliary information of the patch to be decoded from the code stream Information, the auxiliary information includes the three-dimensional information of the patch to be decoded; when the first identifier is the second value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded, and the patch to be decoded The patch refers to the identification information of the patch; when the first identification is the third value, the auxiliary information of the patch to be decoded is parsed from the code stream. The auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and normal direction of the patch to be decoded Coordinate axis information: reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.
由于当前点云帧中的所有patch的第一标识可以各自打包并编入码流，也可以打包在一起并编入码流。因此，对于各自打包并编入码流的方式而言，如果待解码patch为当前点云帧中的首个patch，那么，可以从码流中解析待解码patch的第一标识。当然，对于非首个patch，也可以从码流中解析待解码patch的第一标识。也即是，可以依次解析当前点云帧中待解码的每个patch的第一标识。例如，依次解析出待解码patch[i-1]的第一标识和待解码patch[i-1] 的辅助信息，待解码patch[i]的第一标识和待解码patch[i]的辅助信息，以及待解码patch[i+1]的第一标识和待解码patch[i+1]的辅助信息。对于打包在一起并编入码流的方式而言，如果待解码patch为当前点云帧中的首个patch，那么，可以从码流中解析当前点云帧包括的多个patch的第一标识，其中解析出的多个第一标识中包括待解码patch的第一标识。也即是，通过一次解析即可得到当前点云帧包括的多个patch的第一标识，而无需分多次进行解析。Because the first identifiers of all patches in the current point cloud frame can be individually packaged and compiled into a code stream, or they can be packaged together and compiled into a code stream. Therefore, for the respective methods of packaging and encoding into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, then the first identifier of the patch to be decoded can be parsed from the code stream. Of course, for the non-first patch, the first identifier of the patch to be decoded can also be parsed from the code stream. That is, the first identifier of each patch to be decoded in the current point cloud frame can be analyzed sequentially. For example, the first identifier of the patch[i-1] to be decoded and the auxiliary information of the patch[i-1] to be decoded are sequentially parsed, the first identifier of the patch[i] to be decoded and the auxiliary information of the patch[i] to be decoded , And the first identifier of the patch[i+1] to be decoded and the auxiliary information of the patch[i+1] to be decoded. For the method of packing together and compiling into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, then the first identification of multiple patches included in the current point cloud frame can be parsed from the code stream , Wherein the multiple first identifiers parsed include the first identifier of the patch to be decoded. That is, the first identifiers of multiple patches included in the current point cloud frame can be obtained through one analysis, without the need for multiple analysis.
需要说明的是，当第一标识为第一值时，表明待解码patch为第一类patch，也即待解码patch在当前点云组内的所有点云中都存在相匹配的patch。此时，可以从码流中解析待解码patch的辅助信息，该辅助信息包括待解码patch的三维信息，但不包括待解码patch的二维信息、旋转变换信息和法向坐标轴信息。当第一标识为第二值时，表明待解码patch为第二类patch，也即当前点云帧的参考帧中存在与待解码patch相匹配的参考patch。此时，可以从码流中解析待解码patch的辅助信息，该辅助信息包括二维信息、三维信息和待解码patch的参考patch的标识信息，但不包括旋转变换信息和法向坐标轴信息。当第一标识为第三值时，表明待解码patch为第三类patch，也即当前点云帧的参考帧中不存在与待解码patch相匹配的参考patch。此时，可以从码流中解析出待解码patch的二维信息、三维信息、旋转变换信息和法向坐标轴信息。It should be noted that when the first identifier is the first value, it indicates that the patch to be decoded is the first type of patch, that is, the patch to be decoded has a matching patch in all point clouds in the current point cloud group. At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream. The auxiliary information includes the three-dimensional information of the patch to be decoded, but does not include the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded. When the first identifier is the second value, it indicates that the patch to be decoded is of the second type, that is, there is a reference patch matching the patch to be decoded in the reference frame of the current point cloud frame. At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream. The auxiliary information includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be decoded, but does not include rotation transformation information and normal coordinate axis information. When the first identifier is the third value, it indicates that the patch to be decoded is a third-type patch, that is, there is no reference patch that matches the patch to be decoded in the reference frame of the current point cloud frame. At this point, the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded can be parsed from the code stream.
当第一标识为第一值时，作为一种示例，可以采用与编码端匹配的方法进行解码，例如可以根据已解码的点云中位置与待解码patch在当前点云帧中位置相同的patch的三维信息，采用差分解码方式从码流中解析待解码patch的三维信息。对于待解码patch的其他辅助信息，例如，待解码patch的二维信息、旋转变换信息和法向坐标轴信息，可以从已解码的点云中获取位置与待解码patch在当前点云帧中位置相同的patch的二维信息、旋转变换信息和法向坐标轴信息，并将获取的二维信息、旋转变换信息和法向坐标轴信息作为待解码patch的二维信息、旋转变换信息和法向坐标轴信息。或者，可以从当前点云帧的参考帧中获取待解码patch的参考patch的二维信息、旋转变换信息和法向坐标轴信息，并将参考patch的二维信息、旋转变换信息和法向坐标轴信息作为待解码patch的二维信息、旋转变换信息和法向坐标轴信息。也即是，对于第一类patch，可以从码流中解析三维信息，而二维信息、旋转变换信息和法向坐标轴信息是推导或者获取得到的，这样就避免了在码流中编入二维信息、旋转变换信息和法向坐标轴信息，从而也减少了码流的比特开销，同时提高了编码效率。When the first identifier is the first value, as an example, a method that matches the encoding end can be used for decoding, for example, a patch whose position in the decoded point cloud is the same as that of the patch to be decoded in the current point cloud frame Use differential decoding to analyze the 3D information of the patch to be decoded from the code stream. For other auxiliary information of the patch to be decoded, for example, the two-dimensional information of the patch to be decoded, rotation transformation information and normal axis information, the position of the patch to be decoded can be obtained from the decoded point cloud and the position of the patch to be decoded in the current point cloud frame The two-dimensional information, rotation transformation information and normal coordinate axis information of the same patch, and the obtained two-dimensional information, rotation transformation information and normal coordinate axis information are used as the two-dimensional information, rotation transformation information and normal direction of the patch to be decoded Coordinate axis information. Alternatively, the two-dimensional information, rotation transformation information and normal coordinate axis information of the reference patch of the patch to be decoded can be obtained from the reference frame of the current point cloud frame, and the two-dimensional information, rotation transformation information and normal coordinate information of the reference patch can be obtained The axis information is used as the two-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded. That is, for the first type of patch, three-dimensional information can be parsed from the code stream, while the two-dimensional information, rotation transformation information, and normal coordinate axis information are derived or obtained, thus avoiding the need to include in the code stream Two-dimensional information, rotation transformation information and normal coordinate axis information also reduce the bit overhead of the code stream and improve coding efficiency at the same time.
换句话说，当第一标识为第一值时，从码流中解析出待解码patch的三维信息之后，还可以根据待解码patch的参考patch的辅助信息，获取待解码patch的辅助信息，辅助信息包括二维信息、旋转变换信息和法向坐标轴信息。这样，根据待解码patch的辅助信息，重构待解码patch，包括：根据解码patch的二维信息、三维信息、法向坐标轴信息和旋转变换信息重构待解码patch。In other words, when the first identifier is the first value, after the three-dimensional information of the patch to be decoded is parsed from the code stream, the auxiliary information of the patch to be decoded can also be obtained according to the auxiliary information of the reference patch of the to-be-decoded patch to assist The information includes two-dimensional information, rotation transformation information, and normal coordinate axis information. In this way, reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded includes: reconstructing the patch to be decoded according to the two-dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information of the decoded patch.
当点云中的patch按照第一类patch、第二类patch和第三类patch依次排列的方式进行编码时，待解码patch的参考patch可以为当前点云帧的参考帧中位置与待解码patch在当前点云帧中的位置相同的patch。但是，在某些情况下，当前点云组内的每帧点云中的第一类patch、第二类patch和第三类patch并没有依次排列，此时，当第一标识为第一值时，编入码流的待解码patch的辅助信息中还可以包括待解码patch的参考patch的标识信息。这样，在进行解码时，当解析出的第一标识为第一值时，从码流中解析出的辅助信息中还可以包括待解码patch的参考patch的标识信息。之后，便可以根据待解码patch的参考patch的标识信息确定 待解码patch的参考patch。When the patch in the point cloud is coded in the order of the first type of patch, the second type of patch and the third type of patch, the reference patch of the patch to be decoded can be the position in the reference frame of the current point cloud frame and the patch to be decoded A patch with the same position in the current point cloud frame. However, in some cases, the first type of patch, the second type of patch and the third type of patch in each frame of point cloud in the current point cloud group are not arranged in sequence. At this time, when the first identifier is the first value At this time, the auxiliary information of the patch to be decoded encoded into the code stream may also include the identification information of the reference patch of the patch to be decoded. In this way, during decoding, when the parsed first identifier is the first value, the auxiliary information parsed from the code stream may also include identification information of the reference patch of the patch to be decoded. After that, the reference patch of the patch to be decoded can be determined according to the identification information of the reference patch of the patch to be decoded.
当第一标识为第二值时，对于待解码patch的参考patch的标识信息，可以采用与编码端匹配的方法从码流中解析，例如可以采用定长解码方式来解析。对于待解码patch的旋转变换信息和法向坐标轴信息，可以通过待解码patch的参考patch的标识信息确定待解码patch的参考patch，进而获取待解码patch的参考patch的旋转变换信息和法向坐标轴信息，并将参考patch的旋转变换信息和法向坐标轴信息作为待解码patch的旋转变换信息和法向坐标轴信息。对于待解码patch的二维信息和三维信息，作为一种示例，可以采用与编码端匹配的方法进行解码，例如可以差分解码方式从码流中解析待解码patch的二维信息和三维信息。也即是，对于第二类patch，可以从码流中解析二维信息和三维信息，而旋转变换信息和法向坐标轴信息是推导或者获取得到的，这样就避免了在码流中编入旋转变换信息和法向坐标轴信息，从而也减少了码流的比特开销，同时提高了编码效率。When the first identifier is the second value, the identification information of the reference patch of the patch to be decoded can be parsed from the code stream by a method matching the encoding end, for example, a fixed-length decoding method can be used for analysis. For the rotation transformation information and normal coordinate axis information of the patch to be decoded, the reference patch of the reference patch to be decoded can be determined through the identification information of the reference patch to be decoded, and then the rotation transformation information and normal coordinates of the reference patch to be decoded are obtained. Axis information, and use the rotation transformation information and normal coordinate axis information of the reference patch as the rotation transformation information and normal coordinate axis information of the patch to be decoded. For the two-dimensional information and three-dimensional information of the patch to be decoded, as an example, a method that matches the encoding end can be used for decoding. For example, the two-dimensional information and three-dimensional information of the patch to be decoded can be parsed from the code stream in a differential decoding manner. That is, for the second type of patch, two-dimensional information and three-dimensional information can be parsed from the code stream, while the rotation transformation information and the normal coordinate axis information are derived or obtained, thus avoiding the need to include in the code stream The rotation transformation information and the normal coordinate axis information also reduce the bit overhead of the code stream and improve the coding efficiency at the same time.
换句话说，当所述第一标识为第二值时，从码流中解析出待解码patch的二维信息、三维信息和待解码patch的参考patch的标识信息之后，还可以根据待解码patch的参考patch的标识信息，获取待解码patch的旋转变换信息和法向坐标轴信息。这样，根据待解码patch的辅助信息，重构待解码patch，包括：根据所述待解码patch的二维信息、三维信息、法向坐标轴信息和旋转变换信息重构所述待解码patch。In other words, when the first identifier is the second value, after the two-dimensional information, three-dimensional information of the patch to be decoded and the identification information of the reference patch of the patch to be decoded are parsed from the code stream, it can also be based on the patch to be decoded. Refer to the identification information of the patch to obtain the rotation transformation information and the normal coordinate axis information of the patch to be decoded. In this way, reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded includes: reconstructing the patch to be decoded according to the two-dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information of the patch to be decoded.
当第一标识为第三值时，作为一种示例，可以采用与编码端匹配的方法进行解码，例如，采用定长解码方式从码流中解析待解码patch的二维信息、三维信息、旋转变换信息和法向坐标轴信息。但是，如果首个第三类patch的辅助信息按照定长编码方式进行编码，非首个第三类patch的二维信息和三维信息采用差分编码方式进行编码，其他辅助信息采用定长编码方式进行编码。那么，对于非首个第三类patch来说，根据首个第三类patch的二维信息和三维信息，采用差分解码方式从码流中解析待解码patch的二维信息和三维信息，采用定长解码方式从码流中解析待解码patch的旋转变换信息和法向坐标轴信息。也即是，对于第三类patch，可以从码流中解析非首个第三类patch二维信息和三维信息，而旋转变换信息和法向坐标轴信息是推导或者获取得到的，这样就避免了在码流中编入旋转变换信息和法向坐标轴信息，从而也减少了码流的比特开销，同时提高了编码效率。When the first identifier is the third value, as an example, a method that matches the encoding end can be used for decoding, for example, fixed-length decoding is used to parse the two-dimensional information, three-dimensional information, and rotation of the patch to be decoded from the code stream. Transformation information and normal axis information. However, if the auxiliary information of the first third-type patch is coded according to the fixed-length coding method, the two-dimensional information and three-dimensional information of the non-first third-type patch are coded by the differential coding method, and other auxiliary information is coded by the fixed-length coding method. coding. Then, for the non-first third-type patch, according to the two-dimensional information and three-dimensional information of the first third-type patch, the two-dimensional information and three-dimensional information of the patch to be decoded are parsed from the code stream by differential decoding, and fixed The long decoding method parses the rotation transformation information and normal coordinate axis information of the patch to be decoded from the code stream. That is, for the third type of patch, the non-first third type of patch two-dimensional information and three-dimensional information can be parsed from the code stream, and the rotation transformation information and the normal coordinate axis information are derived or obtained, so as to avoid In order to encode the rotation transformation information and the normal coordinate axis information in the code stream, the bit overhead of the code stream is also reduced, and the coding efficiency is improved at the same time.
其中，第一值、第二值和第三值分别是二进制的10、11和00，或者，第一值、第二值和第三值分别是二进制的11、10和00。待解码patch的参考patch包括于当前点云帧的参考帧中。Wherein, the first value, the second value and the third value are binary 10, 11 and 00 respectively, or the first value, the second value and the third value are binary 11, 10 and 00 respectively. The reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.
基于上述描述，当前点云组内的每帧点云中的第一类patch、第二类patch和第三类patch可以依次排列，此时，如果待解码patch为当前点云帧中的首个patch，那么，从码流中解析待解码patch的第一标识之后，即可根据第一标识的取值确定当前点云帧是否为当前点云组内的首帧点云。也即是，当待解码patch为当前点云帧中的首个patch，且第一标识为第三值时，可以确定当前点云帧为当前点云组内的首帧点云。这样，就不需要在码流中编入一个点云组的长度。Based on the above description, the first type patch, the second type patch and the third type patch in each frame of point cloud in the current point cloud group can be arranged in sequence. At this time, if the patch to be decoded is the first one in the current point cloud frame patch, then, after parsing the first identifier of the patch to be decoded from the code stream, it can be determined whether the current point cloud frame is the first point cloud in the current point cloud group according to the value of the first identifier. That is, when the patch to be decoded is the first patch in the current point cloud frame, and the first identifier is the third value, it can be determined that the current point cloud frame is the first point cloud in the current point cloud group. In this way, there is no need to program the length of a point cloud group in the code stream.
但是，在某些情况下，当前点云组内的每帧点云中的第一类patch、第二类patch和第三类patch并没有依次排列，也即是，当前点云组中的任一帧点云中的首个patch的第一标识可以为第一值、第二值或第三值。这样，当待解码patch为当前点云帧中的首个patch时，无法根据从码流中解析的第一标识确定当前点云帧是否为当前点云组内的首帧点云。此时，可以 确定当前点云帧包括的多个patch的第一标识。在一种可能的实现方式中，该多个patch即是待解码patch包括的所有patch。如果该多个patch的第一标识均为第三值时，则可以确定当前点云帧为当前点云组内的首帧点云。However, in some cases, the first type of patch, the second type of patch, and the third type of patch in each frame of point cloud in the current point cloud group are not arranged in order, that is, any of the current point cloud group The first identifier of the first patch in a frame of point cloud can be the first value, the second value or the third value. In this way, when the patch to be decoded is the first patch in the current point cloud frame, it is impossible to determine whether the current point cloud frame is the first point cloud in the current point cloud group according to the first identifier parsed from the code stream. At this time, the first identifiers of multiple patches included in the current point cloud frame can be determined. In a possible implementation, the multiple patches are all the patches included in the patch to be decoded. If the first identifiers of the multiple patches are all the third value, it can be determined that the current point cloud frame is the first point cloud in the current point cloud group.
需要说明的是，在获取待解码patch的辅助信息之后，也即获取到了待解码patch的二维信息、三维信息、旋转变换信息和法向坐标轴信息之后，可以根据但不限于这些辅助信息重构待解码patch，本申请对重构待解码patch需要的具体信息不做限定。It should be noted that after obtaining the auxiliary information of the patch to be decoded, that is, after obtaining the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded, the auxiliary information can be reproduced according to but not limited to these auxiliary information. To construct the patch to be decoded, this application does not limit the specific information needed to reconstruct the patch to be decoded.
第三方面，提供了一种点云编码方法，该方法包括：获取待编码patch的辅助信息；将待编码patch的辅助信息和待编码patch的第一标识编入码流；其中第一标识为第一值，以指示待编码patch为第一类patch，且编入码流的信息还包括第二标识，其中第二标识为第四值，以指示待编码patch的三维信息未编入码流；或者，第二标识为第五值，以指示待编码patch的三维信息已编入码流，待编码patch的辅助信息包括三维信息；其中第一标识为第二值，以指示待编码patch为第二类patch，待编码patch的辅助信息包括二维信息、三维信息和待编码patch的参考patch的标识信息；其中第一标识为第三值，以指示待编码patch为第三类patch，待编码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。In a third aspect, a point cloud coding method is provided. The method includes: obtaining auxiliary information of the patch to be coded; encoding the auxiliary information of the patch to be coded and the first identifier of the patch to be coded into a code stream; wherein the first identifier is The first value indicates that the patch to be encoded is a first-type patch, and the information encoded into the code stream also includes a second identifier, where the second identifier is a fourth value to indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream ; Or, the second identifier is the fifth value to indicate that the three-dimensional information of the patch to be encoded has been encoded into the code stream, and the auxiliary information of the patch to be encoded includes three-dimensional information; wherein the first identifier is the second value to indicate that the patch to be encoded is For the second type of patch, the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded; the first identifier is the third value to indicate that the patch to be encoded is the third type of patch. The auxiliary information of the encoded patch includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
第一标识为第一值时，可以指示待编码patch为第一类patch，且编入码流的信息还可以包括第二标识。When the first identifier is the first value, it may indicate that the patch to be encoded is the first type of patch, and the information encoded into the code stream may also include the second identifier.
需要说明的是，第二标识用于指示待编码patch的三维信息是否已编入码流。第二标识的语法元素可以是override_3d_shift_data_flag。第二标识为第四值时，可以指示待编码patch的三维信息未编入码流。也即是，待编码patch的三维信息与待编码patch的参考patch的三维信息相同，因此可以不用将待编码patch的三维信息再次编入码流，也即待编码patch的辅助信息可以不包括三维信息。第二标识为第五值时，可以指示待编码patch的三维信息已编入码流。也即是，待编码patch的三维信息与待编码patch的参考patch的三维信息不相同，因此需要将待编码patch的三维信息编入码流，也即待编码patch的辅助信息可以包括三维信息。其中，第四值的语法元素可以是true，第五值的语法元素可以是false。It should be noted that the second identifier is used to indicate whether the three-dimensional information of the patch to be encoded has been encoded into the code stream. The syntax element of the second identification may be override_3d_shift_data_flag. When the second identifier is the fourth value, it can indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is the same as the three-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the three-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may not include the three-dimensional information. information. When the second identifier is the fifth value, it can indicate that the three-dimensional information of the patch to be encoded has been encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is different from the three-dimensional information of the reference patch of the patch to be encoded. Therefore, it is necessary to encode the three-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may include three-dimensional information. The syntax element of the fourth value may be true, and the syntax element of the fifth value may be false.
其中，第一标识为第二值时，可以指示待编码patch为第二类patch。也即当前点云帧的参考帧中存在与待编码patch相匹配的参考patch。待编码patch的辅助信息包括二维信息、三维信息和待编码patch的参考patch的标识信息。Wherein, when the first identifier is the second value, it may indicate that the patch to be encoded is a second-type patch. That is, there is a reference patch that matches the patch to be encoded in the reference frame of the current point cloud frame. The auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded.
其中，第一标识为第三值时，可以指示待编码patch为第三类patch。也即当前点云帧的参考帧中不存在与待编码patch相匹配的参考patch。待编码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。Wherein, when the first identifier is a third value, it may indicate that the patch to be encoded is a third-type patch. That is, there is no reference patch that matches the patch to be encoded in the reference frame of the current point cloud frame. The auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
第四方面，提供了一种点云解码方法，该方法包括：从码流中解析待解码patch的第一标识；当第一标识为第一值时，从码流中解析待解码patch的第二标识；当第二标识为第四值时，将待解码patch的参考patch的辅助信息作为待解码patch的辅助信息；当第二标识为第五值时，从码流中解析待解码patch的辅助信息，辅助信息包括待解码patch的三维信息；当第一标识为第二值时，从码流中解析待解码patch的辅助信息，辅助信息包括待解码patch的二维信息、三维信息和待解码patch的参考patch的标识信息；当第一标识为第三值时，从码流中解析待解码patch的辅助信息，辅助信息包括待解码patch的二维信息、三维信息、旋 转变换信息和法向坐标轴信息；根据待解码patch的辅助信息，重构待解码patch。In a fourth aspect, a point cloud decoding method is provided. The method includes: parsing the first identifier of the patch to be decoded from the code stream; when the first identifier is the first value, parsing the first identifier of the patch to be decoded from the code stream Second identifier; when the second identifier is the fourth value, the auxiliary information of the reference patch of the patch to be decoded is used as the auxiliary information of the patch to be decoded; when the second identifier is the fifth value, the patch to be decoded is parsed from the code stream Auxiliary information. The auxiliary information includes the three-dimensional information of the patch to be decoded; when the first identifier is the second value, the auxiliary information of the patch to be decoded is parsed from the code stream. The auxiliary information includes the two-dimensional information, three-dimensional information, and the patch to be decoded. The identification information of the reference patch of the decoded patch; when the first identifier is the third value, the auxiliary information of the patch to be decoded is parsed from the code stream. The auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and method of the patch to be decoded. To coordinate axis information; according to the auxiliary information of the patch to be decoded, reconstruct the patch to be decoded.
需要说明的是，第二标识为第四值时，可以指示待解码patch的三维信息未编入码流。也即是，待解码patch的三维信息与待解码patch的参考patch的三维信息相同，此时可以将待解码patch的参考patch的辅助信息作为待解码patch的辅助信息。第二标识为第五值时，可以指示待解码patch的三维信息已编入码流，待解码patch的辅助信息包括三维信息。此时，从码流中解析待解码patch的辅助信息，辅助信息包括待解码patch的三维信息。作为一种示例，可以采用差分解码方式从码流中解析待解码patch的辅助信息。It should be noted that when the second identifier is the fourth value, it may indicate that the three-dimensional information of the patch to be decoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be decoded is the same as the three-dimensional information of the reference patch of the patch to be decoded. In this case, the auxiliary information of the reference patch of the patch to be decoded can be used as the auxiliary information of the patch to be decoded. When the second identifier is the fifth value, it can indicate that the 3D information of the patch to be decoded has been encoded into the bitstream, and the auxiliary information of the patch to be decoded includes 3D information. At this time, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded. As an example, a differential decoding method can be used to parse the auxiliary information of the patch to be decoded from the code stream.
第五方面，提供了一种点云编码装置，该装置包括：patch信息获取模块，用于待编码patch的辅助信息；辅助信息编码模块，用于将待编码patch的辅助信息和待编码patch的第一标识编入码流；其中第一标识为第一值，以指示待编码patch为第一类patch，待编码patch的辅助信息包括三维信息；其中第一标识为第二值，以指示待编码patch为第二类patch，待编码patch的辅助信息包括二维信息、三维信息和待编码patch的参考patch的标识信息；其中第一标识为第三值，以指示待编码patch为第三类patch，待编码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。In a fifth aspect, a point cloud encoding device is provided. The device includes: a patch information acquisition module for auxiliary information of the patch to be encoded; an auxiliary information encoding module for combining auxiliary information of the patch to be encoded and the auxiliary information of the patch to be encoded The first identifier is encoded into the code stream; the first identifier is the first value to indicate that the patch to be encoded is a first type of patch, and the auxiliary information of the patch to be encoded includes three-dimensional information; the first identifier is the second value to indicate the The coded patch is a second type of patch, and the auxiliary information of the patch to be coded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be coded; the first identifier is the third value to indicate that the patch to be coded is the third type Patch, the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
其中，待编码patch的第一标识为第三值，待编码patch为所属的当前点云帧中的首个patch，当前点云帧为首个点云帧。The first identifier of the patch to be encoded is the third value, the patch to be encoded is the first patch in the current point cloud frame to which it belongs, and the current point cloud frame is the first point cloud frame.
其中，第一值、第二值和第三值分别是二进制的10、11和00，或者，第一值、第二值和第三值分别是二进制的11、10和00。Wherein, the first value, the second value and the third value are binary 10, 11 and 00 respectively, or the first value, the second value and the third value are binary 11, 10 and 00 respectively.
其中，待编码patch的参考patch包括于当前点云帧的参考帧中。Wherein, the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.
其中，第一类patch为当前组中的全局匹配patch，第二类patch为当前组中的局部匹配patch，第三类patch为当前组中的非匹配patch。Among them, the first type of patch is a global matching patch in the current group, the second type of patch is a local matching patch in the current group, and the third type of patch is a non-matching patch in the current group.
第六方面，提供了一种点云解码装置，该装置包括：辅助信息解码模块，用于从码流中解析待解码patch的第一标识；辅助信息解码模块还用于当第一标识为第一值时，从码流中解析待解码patch的辅助信息，辅助信息包括待解码patch的三维信息；当第一标识为第二值时，从码流中解析待解码patch的辅助信息，辅助信息包括待解码patch的二维信息、三维信息和待解码patch的参考patch的标识信息；当第一标识为第三值时，从码流中解析待解码patch的辅助信息，辅助信息包括待解码patch的二维信息、三维信息、旋转变换信息和法向坐标轴信息；重构模块，用于根据待解码patch的辅助信息，重构待解码patch。In a sixth aspect, a point cloud decoding device is provided. The device includes: an auxiliary information decoding module for parsing the first identifier of the patch to be decoded from the code stream; the auxiliary information decoding module is also used for when the first identifier is the first identifier When the value is one, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded; when the first identifier is the second value, the auxiliary information of the patch to be decoded is parsed from the code stream, auxiliary information Including the two-dimensional information, three-dimensional information of the patch to be decoded, and identification information of the reference patch of the patch to be decoded; when the first identifier is the third value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the patch to be decoded The two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information; the reconstruction module is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.
其中，第一值、第二值和第三值分别是二进制的10、11和00，或者，第一值、第二值和第三值分别是二进制的11、10和00。Wherein, the first value, the second value and the third value are binary 10, 11 and 00 respectively, or the first value, the second value and the third value are binary 11, 10 and 00 respectively.
其中，辅助信息解码模块还用于：Among them, the auxiliary information decoding module is also used for:
当待解码patch为首个patch，且第一标识为第三值时，确定待解码patch所属的当前点云帧为首个点云帧。When the patch to be decoded is the first patch and the first identifier is the third value, it is determined that the current point cloud frame to which the patch to be decoded belongs is the first point cloud frame.
其中，辅助信息解码模块还用于：Among them, the auxiliary information decoding module is also used for:
当待解码patch为当前点云帧中的首个patch时，从码流中解析待解码patch的第一标识；或者，当待解码patch为当前点云帧中的首个patch时，从码流中解析当前点云帧包括的多个patch的多个第一标识，其中解析出的多个第一标识中包括待解码patch的第一标识。When the patch to be decoded is the first patch in the current point cloud frame, the first identifier of the patch to be decoded is parsed from the code stream; or, when the patch to be decoded is the first patch in the current point cloud frame, from the code stream Analyze multiple first identifiers of multiple patches included in the current point cloud frame, and the multiple first identifiers parsed include the first identifier of the patch to be decoded.
其中，当第一标识为第一值时，辅助信息解码模块还用于：根据待解码patch的参考patch的辅助信息，获取待解码patch的二维信息、旋转变换信息和法向坐标轴信息；重构模块还用于：根据待解码patch的二维信息、三维信息、法向坐标轴信息和旋转变换信息重构待解码patch。Wherein, when the first identifier is the first value, the auxiliary information decoding module is further configured to: obtain the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded; The reconstruction module is also used to reconstruct the patch to be decoded according to the two-dimensional information, three-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded.
其中，当第一标识为第二值时，辅助信息解码模块还用于：根据待解码patch的参考patch的标识信息，获得待解码patch的旋转变换信息和法向坐标轴信息；重构模块还用于：根据待解码patch的二维信息、三维信息、法向坐标轴信息和旋转变换信息重构待解码patch。Wherein, when the first identifier is the second value, the auxiliary information decoding module is also used to: obtain the rotation transformation information and normal coordinate axis information of the patch to be decoded according to the identification information of the reference patch of the patch to be decoded; Used to reconstruct the patch to be decoded according to the two-dimensional information, three-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded.
其中，待解码patch的参考patch包括于当前点云帧的参考帧中。Wherein, the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.
第七方面，提供了一种点云编码装置，该装置包括：patch信息获取模块，用于获取待编码patch的辅助信息；In a seventh aspect, a point cloud encoding device is provided, the device including: a patch information acquisition module for acquiring auxiliary information of the patch to be encoded;
辅助信息编码模块，用于将待编码patch的辅助信息和待编码patch的第一标识编入码流；Auxiliary information encoding module, used to encode auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded into a code stream;
其中第一标识为第一值，以指示待编码patch为第一类patch，且编入码流的信息还包括第二标识，其中第二标识为第四值，以指示待编码patch的三维信息未编入码流；或者，第二标识为第五值，以指示待编码patch的三维信息已编入码流，待编码patch的辅助信息包括三维信息；The first identifier is a first value to indicate that the patch to be encoded is a first-type patch, and the information encoded into the code stream also includes a second identifier, where the second identifier is a fourth value to indicate the three-dimensional information of the patch to be encoded The code stream is not coded; or, the second identifier is the fifth value to indicate that the three-dimensional information of the patch to be coded has been coded into the code stream, and the auxiliary information of the patch to be coded includes three-dimensional information;
其中第一标识为第二值，以指示待编码patch为第二类patch，待编码patch的辅助信息包括二维信息、三维信息和待编码patch的参考patch的标识信息；The first identifier is the second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded;
其中第一标识为第三值，以指示待编码patch为第三类patch，待编码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。The first identifier is a third value to indicate that the patch to be encoded is a third type of patch. The auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
第八方面，提供了一种点云解码装置，该装置包括：辅助信息解码模块，用于从码流中解析待解码patch的第一标识；In an eighth aspect, a point cloud decoding device is provided, the device including: an auxiliary information decoding module, configured to parse the first identifier of the patch to be decoded from the code stream;
辅助信息解码模块还用于当第一标识为第一值时，从码流中解析待解码patch的第二标识；当第二标识为第四值时，将待解码patch的参考patch的辅助信息作为待解码patch的辅助信息；当第二标识为第五值时，从码流中解析待解码patch的辅助信息，辅助信息包括待解码patch的三维信息；当第一标识为第二值时，从码流中解析待解码patch的辅助信息，辅助信息包括待解码patch的二维信息、三维信息和待解码patch的参考patch的标识信息；当第一标识为第三值时，从码流中解析待解码patch的辅助信息，辅助信息包括待解码patch的二维信息、三维信息、旋转变换信息和法向坐标轴信息；The auxiliary information decoding module is also used for parsing the second identification of the patch to be decoded from the code stream when the first identification is the first value; when the second identification is the fourth value, referencing the auxiliary information of the patch to be decoded As the auxiliary information of the patch to be decoded; when the second identifier is the fifth value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded; when the first identifier is the second value, The auxiliary information of the patch to be decoded is parsed from the code stream. The auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded and the identification information of the reference patch of the patch to be decoded; when the first identifier is the third value, from the code stream Parse the auxiliary information of the patch to be decoded. The auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded;
重构模块，用于根据待解码patch的辅助信息，重构待解码patch。The reconstruction module is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.
第九方面，提供了一种点云编码方法，该方法包括：获取待编码patch的辅助信息；将该待编码patch的辅助信息和该待编码patch的语法元素编入码流，语法元素包括第一语法元素。In a ninth aspect, a point cloud encoding method is provided, the method comprising: obtaining auxiliary information of the patch to be encoded; encoding the auxiliary information of the patch to be encoded and the syntax elements of the patch to be encoded into a code stream, and the syntax elements include the first A grammatical element.
其中：当第一语法元素指示该待编码patch具有参考patch时，Wherein: when the first syntax element indicates that the patch to be encoded has a reference patch,
编入码流的语法元素还包括第二语法元素，第二语法元素的取值为真，以指示该待编码patch的二维信息编入所述码流，但该待编码patch的三维信息未编入码流，该待编码patch的辅助信息包括所述二维信息；或者The syntax element encoded into the code stream also includes a second syntax element. The value of the second syntax element is true to indicate that the two-dimensional information of the patch to be encoded is incorporated into the code stream, but the three-dimensional information of the patch to be encoded is not In the code stream, the auxiliary information of the patch to be coded includes the two-dimensional information; or
编入码流的语法元素还包括第二语法元素和第三语法元素，第二语法元素的取值为假且第三语法元素的取值为真，以指示该待编码patch的二维信息未编入码流，且该待编码patch的三维信息编入码流，该待编码patch的辅助信息包括所述三维信息；或者The syntax element encoded into the code stream also includes a second syntax element and a third syntax element. The value of the second syntax element is false and the value of the third syntax element is true to indicate that the two-dimensional information of the patch to be encoded is not Code stream, and the three-dimensional information of the patch to be coded is coded into the code stream, and the auxiliary information of the patch to be coded includes the three-dimensional information; or
编入码流的语法元素还包括第二语法元素和第三语法元素，第二语法元素的取值为假且第三语法元素的取值为假，以指示该待编码patch的二维信息未编入码流，且该待编码patch的三维信息未编入码流。The syntax element encoded into the code stream also includes a second syntax element and a third syntax element. The value of the second syntax element is false and the value of the third syntax element is false to indicate that the two-dimensional information of the patch to be encoded is not Code stream, and the 3D information of the patch to be coded is not coded into code stream.
可选地，当第一语法元素指示待编码patch不具有参考patch时，该待编码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。Optionally, when the first syntax element indicates that the patch to be encoded does not have a reference patch, the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
可选地，第一语法元素指示待编码patch不具有参考patch，该待编码patch为所属的当前点云帧中的首个patch，当前点云帧为首个点云帧。Optionally, the first syntax element indicates that the patch to be encoded does not have a reference patch, the patch to be encoded is the first patch in the current point cloud frame to which it belongs, and the current point cloud frame is the first point cloud frame.
可选地，该待编码patch的参考patch包括于当前点云帧的参考帧中。Optionally, the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.
可选地，该待编码patch在当前点云帧中的位置与待编码patch的参考patch在当前点云帧的参考帧的位置相同。Optionally, the position of the patch to be encoded in the current point cloud frame is the same as the position of the reference patch of the patch to be encoded in the reference frame of the current point cloud frame.
第十方面，提供了一种点云解码方法，该方法包括：从码流中解析待解码patch的第一语法元素；当第一语法元素指示该待解码patch具有参考patch时，从该码流中解析该待解码patch的第二语法元素，或者，从该码流中解析该待解码patch的第二语法元素和第三语法元素；当第二语法元素的取值为真时，从该码流中解析该待解码patch的二维信息，将该待解码patch的参考patch的三维信息作为该待解码patch的三维信息；或者，当第二语法元素的取值为假且第三语法元素的取值为真时，从该码流中解析该待解码patch的三维信息，将该待解码patch的参考patch的二维信息作为该待解码patch的二维信息；或者，当第二语法元素的取值为假且第三语法元素的取值为假时，将该待解码patch的参考patch的二维信息作为该待解码patch的二维信息，将该待解码patch的参考patch的三维信息作为该待解码patch的三维信息；根据该待解码patch的辅助信息重构待解码patch，该待解码patch的辅助信息包括该待解码patch的二维信息和该待解码patch的三维信息。In a tenth aspect, a point cloud decoding method is provided. The method includes: parsing a first syntax element of a patch to be decoded from a code stream; when the first syntax element indicates that the patch to be decoded has a reference patch, read from the code stream Parse the second syntax element of the patch to be decoded, or parse the second syntax element and the third syntax element of the patch to be decoded from the code stream; when the value of the second syntax element is true, from the code Parse the two-dimensional information of the patch to be decoded in the stream, and use the three-dimensional information of the reference patch of the patch to be decoded as the three-dimensional information of the patch to be decoded; or, when the value of the second syntax element is false and the value of the third syntax element is false When the value is true, parse the three-dimensional information of the patch to be decoded from the code stream, and use the two-dimensional information of the reference patch of the patch to be decoded as the two-dimensional information of the patch to be decoded; or, when the second syntax element is When the value is false and the value of the third syntax element is false, the two-dimensional information of the reference patch of the patch to be decoded is used as the two-dimensional information of the patch to be decoded, and the three-dimensional information of the reference patch of the patch to be decoded is used as The three-dimensional information of the patch to be decoded; reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded, and the auxiliary information of the patch to be decoded includes the two-dimensional information of the patch to be decoded and the three-dimensional information of the patch to be decoded.
可选地，该方法还包括：当第一语法元素指示该待解码patch不具有参考patch时，从该码流中解析待解码patch的辅助信息，该待解码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。Optionally, the method further includes: when the first syntax element indicates that the patch to be decoded does not have a reference patch, parsing auxiliary information of the patch to be decoded from the code stream, and the auxiliary information of the patch to be decoded includes two-dimensional information, Three-dimensional information, rotation transformation information and normal coordinate axis information.
可选地，该方法还包括：当该待解码patch为首个patch，且第一语法元素指示该待解码patch不具有参考patch时，确定该待解码patch所属的当前点云帧为首个点云帧。Optionally, the method further includes: when the patch to be decoded is the first patch and the first syntax element indicates that the patch to be decoded does not have a reference patch, determining that the current point cloud frame to which the patch to be decoded belongs is the first point cloud frame .
可选地，从码流中解析待解码patch的第一语法元素，包括：当该待解码patch为当前点云帧中的首个patch时，从该码流中解析待解码patch的第一语法元素；或者，当该待解码patch为当前点云帧中的首个patch时，从该码流中解析当前点云帧包括的多个patch的多个第一语法元素，其中解析出的多个第一语法元素中包括待解码patch的第一语法元素。Optionally, parsing the first syntax element of the patch to be decoded from the code stream includes: when the patch to be decoded is the first patch in the current point cloud frame, parsing the first syntax element of the patch to be decoded from the code stream Element; or, when the patch to be decoded is the first patch in the current point cloud frame, parse multiple first syntax elements of multiple patches included in the current point cloud frame from the code stream, and multiple parsed ones The first syntax element includes the first syntax element of the patch to be decoded.
可选地，该待解码patch的参考patch包括于当前点云帧的参考帧中。Optionally, the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.
可选地，该待解码patch在当前点云帧中的位置与待解码patch的参考patch在当前点云帧的参考帧的位置相同。Optionally, the position of the patch to be decoded in the current point cloud frame is the same as the position of the reference patch of the patch to be decoded in the reference frame of the current point cloud frame.
第十一方面，提供了一种点云编码装置，该装置包括：点云块patch信息获取模块，用 于获取待编码patch的辅助信息；In an eleventh aspect, there is provided a point cloud encoding device, the device comprising: a point cloud block patch information acquisition module for acquiring auxiliary information of the patch to be encoded;
辅助信息编码模块，用于将该待编码patch的辅助信息和该待编码patch的语法元素编入码流，语法元素包括第一语法元素，其中：The auxiliary information encoding module is used to encode the auxiliary information of the patch to be encoded and the syntax element of the patch to be encoded into the code stream, and the syntax element includes the first syntax element, wherein:
当第一语法元素指示该待编码patch具有参考patch时，When the first syntax element indicates that the patch to be encoded has a reference patch,
编入码流的语法元素还包括第二语法元素，第二语法元素的取值为真，以指示该待编码patch的二维信息编入该码流，但该待编码patch的三维信息未编入该码流，该待编码patch的辅助信息包括二维信息；或者The syntax element encoded into the code stream also includes a second syntax element. The value of the second syntax element is true to indicate that the two-dimensional information of the patch to be encoded is incorporated into the code stream, but the three-dimensional information of the patch to be encoded is not encoded. Into the code stream, the auxiliary information of the patch to be encoded includes two-dimensional information; or
编入码流的语法元素还包括第二语法元素和第三语法元素，第二语法元素的取值为假且第三语法元素的取值为真，以指示该待编码patch的二维信息未编入该码流，且该待编码patch的三维信息编入该码流，该待编码patch的辅助信息包括三维信息；或者The syntax element encoded into the code stream also includes a second syntax element and a third syntax element. The value of the second syntax element is false and the value of the third syntax element is true to indicate that the two-dimensional information of the patch to be encoded is not The code stream is encoded, and the three-dimensional information of the patch to be encoded is encoded into the code stream, and the auxiliary information of the patch to be encoded includes the three-dimensional information; or
编入码流的语法元素还包括第二语法元素和第三语法元素，第二语法元素的取值为假且第三语法元素的取值为假，以指示该待编码patch的二维信息未编入该码流，且该待编码patch的三维信息未编入该码流。The syntax element encoded into the code stream also includes a second syntax element and a third syntax element. The value of the second syntax element is false and the value of the third syntax element is false to indicate that the two-dimensional information of the patch to be encoded is not The code stream is compiled, and the three-dimensional information of the patch to be coded is not compiled into the code stream.
可选地，当第一语法元素指示该待编码patch不具有参考patch时，该待编码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。Optionally, when the first syntax element indicates that the patch to be encoded does not have a reference patch, the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
可选地，第一语法元素指示该待编码patch不具有参考patch，该待编码patch为所属的当前点云帧中的首个patch，当前点云帧为首个点云帧。Optionally, the first syntax element indicates that the patch to be encoded does not have a reference patch, the patch to be encoded is the first patch in the current point cloud frame to which it belongs, and the current point cloud frame is the first point cloud frame.
可选地，该待编码patch的参考patch包括于当前点云帧的参考帧中。Optionally, the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.
可选地，该待编码patch在当前点云帧中的位置与待编码patch的参考patch在当前点云帧的参考帧的位置相同。Optionally, the position of the patch to be encoded in the current point cloud frame is the same as the position of the reference patch of the patch to be encoded in the reference frame of the current point cloud frame.
第十二方面，提供了一种点云解码装置，该装置包括：辅助信息解码模块，用于从码流中解析待解码patch的第一语法元素；In a twelfth aspect, a point cloud decoding device is provided, the device includes: an auxiliary information decoding module, configured to parse the first syntax element of the patch to be decoded from the code stream;
辅助信息解码模块，还用于当第一语法元素指示该待解码patch具有参考patch时，从该码流中解析该待解码patch的第二语法元素，或者，从该码流中解析该待解码patch的第二语法元素和第三语法元素；当第二语法元素的取值为真时，从该码流中解析该待解码patch的二维信息，将该待解码patch的参考patch的三维信息作为该待解码patch的三维信息；或者，当第二语法元素的取值为假且第三语法元素的取值为真时，从该码流中解析该待解码patch的三维信息，将该待解码patch的参考patch的二维信息作为该待解码patch的二维信息；或者，当第二语法元素的取值为假且第三语法元素的取值为假时，将该待解码patch的参考patch的二维信息作为该待解码patch的二维信息，将该待解码patch的参考patch的三维信息作为该待解码patch的三维信息；The auxiliary information decoding module is also used to parse the second syntax element of the patch to be decoded from the code stream when the first syntax element indicates that the patch to be decoded has a reference patch, or to parse the code stream to be decoded The second syntax element and the third syntax element of the patch; when the value of the second syntax element is true, the two-dimensional information of the patch to be decoded is parsed from the code stream, and the three-dimensional information of the patch to be decoded is referenced As the three-dimensional information of the patch to be decoded; or, when the value of the second syntax element is false and the value of the third syntax element is true, the three-dimensional information of the patch to be decoded is parsed from the code stream, and the The two-dimensional information of the reference patch of the decoded patch is used as the two-dimensional information of the patch to be decoded; or, when the value of the second syntax element is false and the value of the third syntax element is false, the reference of the patch to be decoded The two-dimensional information of the patch is used as the two-dimensional information of the patch to be decoded, and the three-dimensional information of the reference patch of the patch to be decoded is used as the three-dimensional information of the patch to be decoded;
重构模块，用于根据该待解码patch的辅助信息重构待解码patch，该待解码patch的辅助信息包括该待解码patch的二维信息和该待解码patch的三维信息。The reconstruction module is used for reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded. The auxiliary information of the patch to be decoded includes the two-dimensional information of the patch to be decoded and the three-dimensional information of the patch to be decoded.
可选地，辅助信息解码模块还用于当第一语法元素指示该待解码patch不具有参考patch时，从该码流中解析该待解码patch的辅助信息，该待解码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。Optionally, the auxiliary information decoding module is further configured to parse auxiliary information of the patch to be decoded from the code stream when the first syntax element indicates that the patch to be decoded does not have a reference patch. The auxiliary information of the patch to be decoded includes two Dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
可选地，辅助信息解码模块还用于当该待解码patch为首个patch，且第一语法元素指示该待解码patch不具有参考patch时，确定该待解码patch所属的当前点云帧为首个点云帧。Optionally, the auxiliary information decoding module is further configured to determine that the current point cloud frame to which the patch to be decoded belongs is the first point when the patch to be decoded is the first patch and the first syntax element indicates that the patch to be decoded does not have a reference patch Cloud frame.
可选地，辅助信息解码模块，还用于当该待解码patch为当前点云帧中的首个patch时，从该码流中解析该待解码patch的第一语法元素；或者，当该待解码patch为当前点云帧中的首个patch时，从该码流中解析当前点云帧包括的多个patch的多个第一语法元素，其中解析出的多个第一语法元素中包括该待解码patch的第一语法元素。Optionally, the auxiliary information decoding module is further configured to parse the first syntax element of the patch to be decoded from the code stream when the patch to be decoded is the first patch in the current point cloud frame; or, when the patch to be decoded is the first patch in the current point cloud frame; When the decoded patch is the first patch in the current point cloud frame, the multiple first syntax elements of multiple patches included in the current point cloud frame are parsed from the code stream, and the multiple first syntax elements parsed include the The first syntax element of the patch to be decoded.
可选地，该待解码patch的参考patch包括于当前点云帧的参考帧中。Optionally, the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.
可选地，该待解码patch在当前点云帧中的位置与该待解码patch的参考patch在当前点云帧的参考帧的位置相同。Optionally, the position of the patch to be decoded in the current point cloud frame is the same as the position of the reference patch of the patch to be decoded in the reference frame of the current point cloud frame.
第十三方面，提供一种用于编码点云数据的设备，该设备可以包括：存储器和编码器。存储器用于存储点云数据。编码器用于执行上述第一方面或第一方面的任一种可能的设计提供的点云编码方法，或者，编码器用于执行上述第三方面或第三方面的任一种可能的设计提供的点云编码方法，或者，编码器用于执行上述第九方面或第九方面的任一种可能的设计提供的点云编码方法。In a thirteenth aspect, a device for encoding point cloud data is provided. The device may include a memory and an encoder. The memory is used to store point cloud data. The encoder is used to implement the point cloud coding method provided by the first aspect or any possible design of the first aspect, or the encoder is used to implement the points provided by the third aspect or any possible design of the third aspect. The cloud coding method, or the encoder is used to execute the point cloud coding method provided by any one of the possible designs of the ninth aspect or the ninth aspect.
第十四方面，提供一种用于解码点云数据的设备，该设备可以包括：存储器和解码器。存储器用于存储码流形式的点云数据。解码器用于执行上述第二方面或第二方面的任一种可能的设计提供的点云解码方法，或者，解码器用于执行上述第四方面或第四方面的任一种可能的设计提供的点云解码方法，或者，解码器用于执行上述第十方面或第十方面的任一种可能的设计提供的点云解码方法。In a fourteenth aspect, a device for decoding point cloud data is provided. The device may include a memory and a decoder. The memory is used to store point cloud data in the form of code stream. The decoder is used to implement the point cloud decoding method provided by the foregoing second aspect or any possible design of the second aspect, or the decoder is used to implement the points provided by any possible design of the foregoing fourth aspect or the fourth aspect The cloud decoding method, or the decoder is used to execute the point cloud decoding method provided by any possible design of the tenth aspect or the tenth aspect.
第十五方面，提供一种编码装置，包括：存储器和处理器；其中，该存储器用于存储程序代码；该处理器用于调用该程序代码，以执行上述第一方面或第三方面或第九方面提供的点云编码方法。In a fifteenth aspect, there is provided an encoding device, including: a memory and a processor; wherein the memory is used to store program code; the processor is used to call the program code to execute the first aspect or the third aspect or the ninth aspect. Point cloud coding method provided by the aspect.
第十六方面，提供一种解码装置，包括：存储器和处理器；其中，该存储器用于存储程序代码；该处理器用于调用该程序代码，以执行上述第二方面或第四方面或第十方面提供的点云解码方法。In a sixteenth aspect, a decoding device is provided, including: a memory and a processor; wherein the memory is used to store program code; the processor is used to call the program code to execute the second aspect or the fourth aspect or the tenth aspect. The point cloud decoding method provided by the aspect.
本申请还提供一种计算机可读存储介质，包括程序代码，该程序代码在计算机上运行时，使得该计算机执行如上述第一方面及其可能的设计，或第三方面及其可能的设计提供的任一种点云编码方法，或第九方面及其可能的设计提供的任一种点云编码方法。This application also provides a computer-readable storage medium, including program code, when the program code runs on a computer, the computer executes the first aspect and its possible designs, or the third aspect and its possible designs. Any point cloud coding method in the ninth aspect and its possible designs.
本申请还提供一种计算机可读存储介质，包括程序代码，该程序代码在计算机上运行时，使得该计算机执行上述第二方面及其可能的设计，或第四方面及其可能的设计提供的任一种点云解码方法，或第十方面及其可能的设计提供的任一种点云解码方法。This application also provides a computer-readable storage medium, including program code, which when running on a computer, causes the computer to execute the second aspect and its possible designs, or the fourth aspect and its possible designs. Any point cloud decoding method, or any point cloud decoding method provided by the tenth aspect and its possible designs.
本申请还提供一种计算机程序产品，当该计算机程序产品在计算机上运行时使得该计算机执行第一方面或第一方面的任一种可能的设计，或，执行第三方面或第三方面的任一种可能的设计提供的点云编码方法的部分或全部步骤，或，执行第九方面或第九方面的任一种可能的设计提供的点云编码方法的部分或全部步骤。This application also provides a computer program product, which when the computer program product runs on a computer, causes the computer to execute any possible design of the first aspect or the first aspect, or execute the third aspect or the third aspect Part or all of the steps of the point cloud coding method provided by any possible design, or execute part or all of the steps of the point cloud coding method provided by any possible design of the ninth aspect or the ninth aspect.
本申请还提供一种计算机程序产品，当该计算机程序产品在计算机上运行时使得该计算 机执行第二方面或第二方面的任一种可能的设计，或，执行第四方面或第四方面的任一种可能的设计提供的点云解码方法的部分或全部步骤，或，执行第十方面或第十方面的任一种可能的设计提供的点云解码方法的部分或全部步骤。This application also provides a computer program product, which when the computer program product runs on a computer, causes the computer to execute any possible design of the second aspect or the second aspect, or execute the fourth aspect or the fourth aspect Part or all of the steps of the point cloud decoding method provided by any possible design, or execute part or all of the steps of the point cloud decoding method provided by any possible design of the tenth aspect or the tenth aspect.
应当理解的是，上述提供的任一种编解码装置、计算机可读存储介质和计算机程序产品的有益效果均可以对应参考上文对应方面提供的方法实施例的有益效果，不再赘述。It should be understood that the beneficial effects of any codec device, computer-readable storage medium, and computer program product provided above can be referred to the beneficial effects of the method embodiments provided in the corresponding aspects above, and details are not repeated.
本申请提供的技术方案至少可以带来以下有益效果：The technical solution provided in this application can at least bring about the following beneficial effects:
在本申请实施例中，先获取待编码patch的辅助信息，然后将待编码patch的辅助信息和第一标识编入码流。由于第一标识的取值可以为第一值、第二值和第三值。不同的取值指示不同类型的patch。因此，可以通过第一标识将不同类型的patch进行区分，这样就不需要在码流中写入第一类patch的数量，减少码流的比特开销。而且，针对不同类型的patch，编入码流中的辅助信息包括的内容可以不相同，且对于第一类patch和第二类patch而言，只有部分辅助信息编入码流。这样，可以简化编入码流的格式，进一步减少码流的比特开销，从而提高编码的效率。In the embodiment of the present application, first obtain the auxiliary information of the patch to be encoded, and then encode the auxiliary information and the first identifier of the patch to be encoded into the code stream. Because the value of the first identifier can be the first value, the second value, and the third value. Different values indicate different types of patches. Therefore, different types of patches can be distinguished by the first identifier, so that there is no need to write the number of the first type of patches in the code stream, which reduces the bit overhead of the code stream. Moreover, for different types of patches, the content of the auxiliary information compiled into the code stream may be different, and for the first type of patch and the second type of patch, only part of the auxiliary information is compiled into the code stream. In this way, the format of the code stream can be simplified, and the bit overhead of the code stream can be further reduced, thereby improving the coding efficiency.
附图说明Description of the drawings
图1为本申请实施例提供的一种点云译码***的示意性框图；FIG. 1 is a schematic block diagram of a point cloud decoding system provided by an embodiment of this application;
图2为可用于本申请实施例的一种编码器的示意性框图；Fig. 2 is a schematic block diagram of an encoder that can be used in an embodiment of the present application;
图3为可适用于本申请实施例的一种点云的示意图；Fig. 3 is a schematic diagram of a point cloud applicable to an embodiment of the present application;
图4为可适用于本申请实施例的一种点云的patch的示意图；FIG. 4 is a schematic diagram of a point cloud patch applicable to an embodiment of the present application;
图5为可适用于本申请实施例的一种点云的占用图的示意图；FIG. 5 is a schematic diagram of an occupancy map of a point cloud applicable to an embodiment of the present application;
图6为可用于本申请实施例的一种解码器的示意性框图；Fig. 6 is a schematic block diagram of a decoder that can be used in an embodiment of the present application;
图7为可用于本申请实施例的一种帧组的示意图；FIG. 7 is a schematic diagram of a frame group that can be used in an embodiment of the present application;
图8为本申请实施例提供的第一种当前点云组的示意图；FIG. 8 is a schematic diagram of the first current point cloud group provided by an embodiment of this application;
图9为本申请实施例提供的第一种点云编码方法的流程图；FIG. 9 is a flowchart of the first point cloud encoding method provided by an embodiment of the application;
图10为本申请实施例提供的第二种当前点云组的示意图；FIG. 10 is a schematic diagram of a second current point cloud group provided by an embodiment of this application;
图11为本申请实施例提供的第一种点云解码方法的流程图；FIG. 11 is a flowchart of the first point cloud decoding method provided by an embodiment of this application;
图12为本申请实施例提供的第二种点云编码方法的流程图；FIG. 12 is a flowchart of a second point cloud encoding method provided by an embodiment of this application;
图13为本申请实施例提供的第二种点云解码方法的流程图；FIG. 13 is a flowchart of a second point cloud decoding method provided by an embodiment of this application;
图14为本申请实施例提供的第一种编码装置的示意性框图；FIG. 14 is a schematic block diagram of a first encoding device provided by an embodiment of this application;
图15为本申请实施例提供的第一种解码装置的示意性框图；FIG. 15 is a schematic block diagram of a first decoding device provided by an embodiment of this application;
图16为本申请实施例提供的第二种编码装置的示意性框图；FIG. 16 is a schematic block diagram of a second encoding device provided by an embodiment of this application;
图17为本申请实施例提供的第二种解码装置的示意性框图；FIG. 17 is a schematic block diagram of a second decoding device provided by an embodiment of this application;
图18为用于本申请实施例的译码设备的一种实现方式的示意性框图。FIG. 18 is a schematic block diagram of an implementation manner of a decoding device used in an embodiment of the present application.
具体实施方式detailed description
为使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。In order to make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.
本申请实施例中的术语“多个”是指两个或两个以上。在本申请的描述中，除非另有说明， “/”表示或的意思，例如，A/B可以表示A或B；本文中的“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，为了便于清楚描述本申请实施例的技术方案，在本申请的实施例中，采用了“第一”、“第二”等字样对功能和作用基本相同的相同项或相似项进行区分。本领域技术人员可以理解“第一”、“第二”等字样并不对数量和执行次序进行限定，并且“第一”、“第二”等字样也并不限定一定不同。The term "plurality" in the embodiments of the present application refers to two or more. In the description of this application, unless otherwise specified, "/" means or, for example, A/B can mean A or B; "and/or" in this document is only an association relationship describing associated objects, It means that there can be three kinds of relationships, for example, A and/or B, which can mean: A alone exists, A and B exist at the same time, and B exists alone. In addition, in order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish the same items or similar items with substantially the same function and effect. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and order of execution, and words such as "first" and "second" do not limit the difference.
在对本申请实施例进行详细的解释说明之前，先对本申请实施例的实施环境进行介绍：Before explaining the embodiments of the present application in detail, first introduce the implementation environment of the embodiments of the present application:
本申请实施例提供的一种实施环境包括点云译码***。图1为本申请实施例提供的一种点云译码***的示意性框图。其中，术语“点云译码”或“译码”可一般地指代点云编码或点云解码。参见图1，点云译码***包括源装置10、目的地装置20、链路30和存储装置40。其中，源装置10可以产生经编码的点云数据。因此，源装置10也可以被称为点云编码装置。目的地装置20可以对由源装置10所产生的经编码的点云数据进行解码。因此，目的地装置20也可以被称为点云解码装置。链路30可以接收源装置10所产生的经编码的点云数据，并可以将该经编码的点云数据传输给目的地装置20。存储装置40可以接收源装置10所产生的经编码的点云数据，并可以将该经编码的点云数据进行存储，这样，目的地装置20可以直接从存储装置40中获取经编码的点云数据。或者，存储装置40可以对应于文件服务器或可以保存由源装置10产生的经编码的点云数据的另一中间存储装置，这样，目的地装置20可以经由流式传输或下载存储装置40存储的经编码的点云数据。An implementation environment provided by an embodiment of the application includes a point cloud decoding system. FIG. 1 is a schematic block diagram of a point cloud decoding system provided by an embodiment of this application. Among them, the term "point cloud decoding" or "decoding" may generally refer to point cloud encoding or point cloud decoding. Referring to FIG. 1, the point cloud decoding system includes a source device 10, a destination device 20, a link 30 and a storage device 40. Among them, the source device 10 can generate coded point cloud data. Therefore, the source device 10 may also be referred to as a point cloud encoding device. The destination device 20 may decode the encoded point cloud data generated by the source device 10. Therefore, the destination device 20 may also be referred to as a point cloud decoding device. The link 30 can receive the coded point cloud data generated by the source device 10 and can transmit the coded point cloud data to the destination device 20. The storage device 40 can receive the coded point cloud data generated by the source device 10, and can store the coded point cloud data, so that the destination device 20 can directly obtain the coded point cloud from the storage device 40 data. Alternatively, the storage device 40 may correspond to a file server or another intermediate storage device that can save the encoded point cloud data generated by the source device 10, so that the destination device 20 may stream or download the stored data of the storage device 40 Coded point cloud data.
源装置10和目的地装置20均可以包括一个或多个处理器以及耦合到该一个或多个处理器的存储器，该存储器可以包括随机存取存储器(random access memory，RAM)、只读存储器(read-only memory，ROM)、带电可擦可编程只读存储器(electrically erasable programmable read-only memory，EEPROM)、快闪存储器、可用于以可由计算机存取的指令或数据结构的形式存储所要的程序代码的任何其它媒体等。例如，源装置10和目的地装置20均可以包括桌上型计算机、移动计算装置、笔记型(例如，膝上型)计算机、平板计算机、机顶盒、例如所谓的“智能”电话等电话手持机、电视机、相机、显示装置、数字媒体播放器、视频游戏控制台、车载计算机或其类似者。Both the source device 10 and the destination device 20 may include one or more processors and a memory coupled to the one or more processors. The memory may include random access memory (RAM), read-only memory ( read-only memory, ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, which can be used to store the desired program in the form of instructions or data structures that can be accessed by a computer Any other media of the code, etc. For example, both the source device 10 and the destination device 20 may include desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, Televisions, cameras, display devices, digital media players, video game consoles, on-board computers, or the like.
链路30可以包括能够将经编码的点云数据从源装置10传输到目的地装置20的一个或多个媒体或装置。在一种可能的实现方式中，链路30可以包括能够使源装置10实时地将经编码的点云数据直接发送到目的地装置20的一个或多个通信媒体。在本申请实施例中，源装置10可以根据通信标准来调制经编码的点云数据，该通信标准可以为无线通信协议等，并且可以将经调制的点云数据发送给目的地装置20。该一个或多个通信媒体可以包括无线和/或有线通信媒体，例如该一个或多个通信媒体可以包括射频(radio frequency，RF)频谱或一个或多个物理传输线。该一个或多个通信媒体可以形成基于分组的网络的一部分，基于分组的网络可以为局域网、广域网或全球网络(例如，因特网)等。该一个或多个通信媒体可以包括路由器、交换器、基站或促进从源装置10到目的地装置20的通信的其它设备等，本申请实施例对此不做具体限定。The link 30 may include one or more media or devices capable of transmitting the encoded point cloud data from the source device 10 to the destination device 20. In one possible implementation, the link 30 may include one or more communication media that enable the source device 10 to directly send the encoded point cloud data to the destination device 20 in real time. In the embodiment of the present application, the source device 10 may modulate the coded point cloud data according to a communication standard, which may be a wireless communication protocol or the like, and may send the modulated point cloud data to the destination device 20. The one or more communication media may include wireless and/or wired communication media. For example, the one or more communication media may include a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, and the packet-based network may be a local area network, a wide area network, or a global network (for example, the Internet). The one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from the source device 10 to the destination device 20, and the embodiment of the present application does not specifically limit this.
在一种可能的实现方式中，存储装置40可以将接收到的由源装置10发送的经编码的点云数据进行存储，目的地装置20可以直接从存储装置40中获取经编码的点云数据。这样，存储装置40可以包括多种分布式或本地存取的数据存储媒体中的任一者，例如，该多种分布 式或本地存取的数据存储媒体中的任一者可以为硬盘驱动器、蓝光光盘、数字多功能光盘(digital versatile disc，DVD)、只读光盘(compact disc read-only memory，CD-ROM)、快闪存储器、易失性或非易失性存储器，或用于存储经编码点云数据的任何其它合适的数字存储媒体等。In a possible implementation manner, the storage device 40 may store the received encoded point cloud data sent by the source device 10, and the destination device 20 may directly obtain the encoded point cloud data from the storage device 40 . In this way, the storage device 40 may include any of a variety of distributed or locally accessed data storage media. For example, any of the multiple distributed or locally accessed data storage media may be a hard disk drive, Blu-ray disc, digital versatile disc (DVD), compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or used to store Any other suitable digital storage media for encoding point cloud data, etc.
在一种可能的实现方式中，存储装置40可以对应于文件服务器或可以保存由源装置10产生的经编码点云数据的另一中间存储装置，目的地装置20可经由流式传输或下载存储装置40存储的点云数据。文件服务器可以为能够存储经编码的点云数据并且将经编码的点云数据发送给目的地装置20的任意类型的服务器。在一种可能的实现方式中，文件服务器可以包括网络服务器、文件传输协议(file transfer protocol，FTP)服务器、网络附属存储(network attached storage，NAS)装置或本地磁盘驱动器等。目的地装置20可以通过任意标准数据连接(包括因特网连接)来获取经编码点云数据。任意标准数据连接可以包括无线信道(例如，Wi-Fi连接)、有线连接(例如，数字用户线路(digital subscriber line，DSL)、电缆调制解调器等)，或适合于获取存储在文件服务器上的经编码的点云数据的两者的组合。经编码的点云数据从存储装置40的传输可为流式传输、下载传输或两者的组合。In a possible implementation, the storage device 40 can correspond to a file server or another intermediate storage device that can store the encoded point cloud data generated by the source device 10, and the destination device 20 can be stored via streaming or downloading. Point cloud data stored by the device 40. The file server may be any type of server capable of storing the coded point cloud data and transmitting the coded point cloud data to the destination device 20. In a possible implementation, the file server may include a network server, a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive. The destination device 20 can obtain the coded point cloud data through any standard data connection (including an Internet connection). Any standard data connection can include a wireless channel (for example, Wi-Fi connection), a wired connection (for example, digital subscriber line (DSL), cable modem, etc.), or is suitable for obtaining the encoded data stored on the file server The combination of the two point cloud data. The transmission of the encoded point cloud data from the storage device 40 may be a streaming transmission, a download transmission, or a combination of both.
图1所示的点云译码***仅为一种可能的实现方式，并且本申请的技术不仅可以适用于图1所示的可以对点云进行编码的源装置10，以及可以对经编码的点云数据进行解码的目的地装置20，还可以适用于其他可以对点云进行编码和对经编码的点云数据进行解码的装置，本申请实施例对此不做具体限定。The point cloud decoding system shown in FIG. 1 is only one possible implementation, and the technology of the present application can not only be applied to the source device 10 that can encode the point cloud shown in FIG. The destination device 20 for decoding point cloud data may also be applicable to other devices that can encode point clouds and decode encoded point cloud data, which is not specifically limited in the embodiment of the present application.
在图1所示的点云译码***中，源装置10包括数据源120、编码器100和输出接口140。在一些实施例中，输出接口140可以包括调节器/解调器(调制解调器)和/或发送器，其中发送器也可以称为发射器。数据源120可以包括点云捕获装置(例如，摄像机等)、含有先前捕获的点云数据的点云存档、用于从点云内容提供者接收点云数据的点云馈入接口，和/或用于产生点云数据的计算机图形***，或点云数据的这些来源的组合。In the point cloud decoding system shown in FIG. 1, the source device 10 includes a data source 120, an encoder 100 and an output interface 140. In some embodiments, the output interface 140 may include a regulator/demodulator (modem) and/or a transmitter, where the transmitter may also be referred to as a transmitter. The data source 120 may include a point cloud capture device (for example, a camera, etc.), a point cloud archive containing previously captured point cloud data, a point cloud feed interface for receiving point cloud data from a point cloud content provider, and/or A computer graphics system used to generate point cloud data, or a combination of these sources of point cloud data.
数据源可以向编码器100发送点云，编码器100可以对接收到由数据源120发送的点云进行编码，得到经编码的点云数据。编码器可以将经编码的点云数据发送给输出接口。在一些实施例中，源装置10经由输出接口140将经编码的点云数据直接发送到目的地装置20。在其它实施例中，经编码的点云数据还可存储到存储装置40上，供目的地装置20以后获取并用于解码和/或播放。The data source may send a point cloud to the encoder 100, and the encoder 100 may encode the point cloud received from the data source 120 to obtain encoded point cloud data. The encoder can send the encoded point cloud data to the output interface. In some embodiments, the source device 10 directly sends the encoded point cloud data to the destination device 20 via the output interface 140. In other embodiments, the encoded point cloud data may also be stored on the storage device 40 for the destination device 20 to obtain later and use for decoding and/or playback.
在图1的实施例中，目的地装置20包括输入接口240、解码器200和显示装置220。在一些实施例中，输入接口240包括接收器和/或调制解调器。输入接口240可经由链路30和/或从存储装置40接收经编码的点云数据，然后再发送给解码器200，解码器200可以对接收到的经编码的点云数据进行解码，得到经解码的点云数据。解码器可以将经解码的点云数据发送给显示装置220。显示装置220可与目的地装置20集成或可在目的地装置20外部。一般来说，显示装置220显示经解码的点云数据。显示装置220可以为多种类型中的任一种类型的显示装置，例如，显示装置220可以为液晶显示器(liquid crystal display，LCD)、等离子显示器、有机发光二极管(organic light-emitting diode，OLED)显示器或其它类型的显示装置。In the embodiment of FIG. 1, the destination device 20 includes an input interface 240, a decoder 200 and a display device 220. In some embodiments, the input interface 240 includes a receiver and/or a modem. The input interface 240 can receive the encoded point cloud data via the link 30 and/or from the storage device 40, and then send it to the decoder 200. The decoder 200 can decode the received encoded point cloud data to obtain the Decoded point cloud data. The decoder may send the decoded point cloud data to the display device 220. The display device 220 may be integrated with the destination device 20 or may be external to the destination device 20. Generally, the display device 220 displays the decoded point cloud data. The display device 220 may be any one of multiple types of display devices. For example, the display device 220 may be a liquid crystal display (LCD), a plasma display, or an organic light-emitting diode (OLED). Display or other type of display device.
尽管图1中未示出，但在一些方面，编码器100和解码器200可各自与音频编码器和解码器集成，且可以包括适当的多路复用器-多路分用器(multiplexer-demultiplexer， MUX-DEMUX)单元或其它硬件和软件，用于共同数据流或单独数据流中的音频和视频两者的编码。在一些实施例中，如果适用的话，那么MUX-DEMUX单元可符合ITU H.223多路复用器协议，或例如用户数据报协议(user datagram protocol，UDP)等其它协议。Although not shown in FIG. 1, in some aspects, the encoder 100 and the decoder 200 may each be integrated with an audio encoder and decoder, and may include an appropriate multiplexer-demultiplexer (multiplexer- demultiplexer, MUX-DEMUX) unit or other hardware and software, used for encoding both audio and video in a common data stream or separate data streams. In some embodiments, if applicable, the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol, or other protocols such as the user datagram protocol (UDP).
编码器100和解码器200各自可为以下各项电路中的任一者：一个或多个微处理器、数字信号处理器(digital signal processing，DSP)、专用集成电路(application specific integrated circuit，ASIC)、现场可编程门阵列(field-programmable gate array，FPGA)、离散逻辑、硬件或其任何组合。如果部分地以软件来实施本申请，那么装置可将用于软件的指令存储在合适的非易失性计算机可读存储媒体中，且可使用一个或多个处理器在硬件中执行所述指令从而实施本申请技术。前述内容(包括硬件、软件、硬件与软件的组合等)中的任一者可被视为一个或多个处理器。编码器100和解码器200中的每一者都可以包括在一个或多个编码器或解码器中，所述编码器或所述解码器中的任一者可以集成为相应装置中的组合编码器/解码器(编码解码器)的一部分。Each of the encoder 100 and the decoder 200 can be any of the following circuits: one or more microprocessors, digital signal processing (DSP), application specific integrated circuit, ASIC ), field-programmable gate array (FPGA), discrete logic, hardware, or any combination thereof. If the application is partially implemented in software, the device may store instructions for the software in a suitable non-volatile computer-readable storage medium, and may use one or more processors to execute the instructions in hardware So as to implement the technology of this application. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) can be regarded as one or more processors. Each of the encoder 100 and the decoder 200 may be included in one or more encoders or decoders, and any of the encoders or the decoders may be integrated as a combined encoding in the corresponding device Part of the decoder/decoder (codec).
本申请可大体上将编码器100称为将某些信息“发信号通知”或“发送”到例如解码器200的另一装置。术语“发信号通知”或“发送”可大体上指代用于对经压缩点云数据进行解码的语法元素和/或其它数据的传送。此传送可实时或几乎实时地发生。替代地，此通信可经过一段时间后发生，例如可在编码时在经编码位流中将语法元素存储到计算机可读存储媒体时发生，解码装置接着可在所述语法元素存储到此媒体之后的任何时间检索所述语法元素。This application may generally refer to the encoder 100 as "signaling" or "sending" certain information to another device such as the decoder 200. The term "signaling" or "sending" may generally refer to the transmission of syntax elements and/or other data used to decode compressed point cloud data. This transmission can occur in real time or almost in real time. Alternatively, this communication may occur after a period of time, for example, when the syntax element is stored in a computer-readable storage medium in the encoded bitstream during encoding, and the decoding device may then store the syntax element on this medium. Retrieve the syntax element at any time.
图2为本申请实施例提供的一种编码器100的示意性框图。图2是以MPEG(moving pictrue expert group)点云压缩(point cloud compression，PCC)编码框架为例进行说明的。参见图2，编码器100可以包括点云块信息生成模块101、打包模块102、深度图生成模块103、纹理图生成模块104、深度图填充模块105、纹理图填充模块106、基于图像或视频的编码模块107、占用图编码模块108、辅助信息编码模块109、复用模块110、点云占用图下采样模块111、点云占用图填充模块112、点云重构模块113和点云滤波模块114。FIG. 2 is a schematic block diagram of an encoder 100 provided by an embodiment of the application. Fig. 2 illustrates the point cloud compression (PCC) coding framework of MPEG (moving pictrue expert group) as an example. 2, the encoder 100 may include a point cloud block information generation module 101, a packing module 102, a depth map generation module 103, a texture map generation module 104, a depth map filling module 105, a texture map filling module 106, image-based or video-based Encoding module 107, occupancy map encoding module 108, auxiliary information encoding module 109, multiplexing module 110, point cloud occupancy map down-sampling module 111, point cloud occupancy map filling module 112, point cloud reconstruction module 113, and point cloud filtering module 114 .
点云块信息生成模块101可以接收由数据源120发送的一个或多个点云组，其中每个点云组可以包括一帧或多帧点云，为了方便描述，后续统一以当前点云组中的当前点云帧为例进行说明。点云块信息生成模块101可以确定当前点云帧包括的每个点在三维空间坐标系中的三维坐标，以及每个点在三维空间中的法线方向矢量，并根据确定出的每个点在三维空间中的法线方向矢量和预定义的投影平面将当前点云帧分割为多个patch(patch)。每个patch包括当前点云帧中的一个或多个点。点云块信息生成模块101还可以确定每个patch的占用图和每个patch的深度图。点云块信息生成模块101还可以确定出每个patch的三维信息以及每个patch的法向坐标轴信息，这些信息可以作为每个patch的辅助信息。另外，点云块信息生成模块101可以将patch分为三种类型，分别为第一类patch、第二类patch和第三类patch。并通过第一标识来指示每个patch的类型。点云块信息生成模块101可以将当前点云帧中的每个patch的第一标识和每个patch的辅助信息发送给辅助信息编码模块109，以进行编码，也可以称为进行压缩编码。其中，每个patch的三维信息可以包括待编码patch在3D空间中沿着切线方向的偏移、待编码patch在3D空间中沿着双切线方向的偏移，以及待编码patch在3D空间中沿着法线方向的偏移。点云块信息生成模块101还可以将每个patch的占用图和每个patch的辅助信息发送给打包模块102。另外，点云块信息生成模块101还可以将每个patch的深度图发送给深度图生成模块103。The point cloud block information generation module 101 may receive one or more point cloud groups sent by the data source 120, where each point cloud group may include one or more frames of point clouds. For ease of description, the current point cloud group will be unified in the following Take the current point cloud frame in as an example. The point cloud block information generation module 101 can determine the three-dimensional coordinates of each point included in the current point cloud frame in the three-dimensional space coordinate system, and the normal direction vector of each point in the three-dimensional space, and determine each point according to the determined The normal direction vector in the three-dimensional space and the predefined projection plane divide the current point cloud frame into multiple patches (patch). Each patch includes one or more points in the current point cloud frame. The point cloud block information generating module 101 can also determine the occupancy map of each patch and the depth map of each patch. The point cloud block information generating module 101 can also determine the three-dimensional information of each patch and the normal coordinate axis information of each patch, which can be used as auxiliary information of each patch. In addition, the point cloud block information generating module 101 can divide the patch into three types, namely the first type patch, the second type patch, and the third type patch. The first identifier is used to indicate the type of each patch. The point cloud block information generating module 101 may send the first identification of each patch and the auxiliary information of each patch in the current point cloud frame to the auxiliary information encoding module 109 for encoding, which may also be called compression encoding. Among them, the three-dimensional information of each patch may include the offset of the patch to be encoded in the 3D space along the tangent direction, the offset of the patch to be encoded in the 3D space along the double tangent direction, and the offset of the patch to be encoded in the 3D space. Offset in the normal direction. The point cloud block information generating module 101 may also send the occupancy map of each patch and the auxiliary information of each patch to the packaging module 102. In addition, the point cloud block information generating module 101 can also send the depth map of each patch to the depth map generating module 103.
打包模块102可以将接收到的由点云块信息生成模块101发送的每个patch的占用图和每个patch的辅助信息进行打包，得到当前点云帧的占用图。具体地，打包模块102可以将每个patch的占用图按照特定的顺序进行排列，例如按照每个patch的占用图的宽/高降序(或升序)排列，然后，按照排列后的每个patch的占用图的顺序，依次将每个patch的占用图***当前点云帧的占用图的可用区域中，得到当前点云帧的占用图和patch的打包信息。打包模块102可以将当前点云帧的占用图和patch的打包信息发送给辅助信息编码模块109，进行压缩编码。其中，patch的打包信息可以包括每个patch在当前点云帧的占用图中的二维信息、每个patch的参考patch的标识信息、每个patch的旋转变换信息，以及当前点云帧的参考帧的索引。其中，每个patch在当前点云帧的占用图中的二维信息可以简称为每个patch的二维信息。每个patch的二维信息可以包括每个patch在当前点云帧占用图中沿u轴方向的偏移、每个patch在当前点云帧占用图中沿v轴方向的偏移、每个patch在当前点云帧占用图中的宽度，以及每个patch在当前点云帧占用图中的高度。另外，打包模块102还可以将patch的打包信息发送给深度图生成模块103和点云占用图下采样模块111。The packing module 102 may pack the received occupancy map of each patch and the auxiliary information of each patch sent by the point cloud block information generating module 101 to obtain the occupancy map of the current point cloud frame. Specifically, the packaging module 102 can arrange the occupancy map of each patch in a specific order, for example, in descending order (or ascending order) of the width/height of the occupancy map of each patch, and then arrange the occupancy map of each patch in descending order (or ascending order). In the order of the occupancy map, insert the occupancy map of each patch into the available area of the occupancy map of the current point cloud frame to obtain the occupancy map of the current point cloud frame and the packing information of the patch. The packing module 102 may send the occupancy map of the current point cloud frame and the packing information of the patch to the auxiliary information encoding module 109 for compression encoding. Among them, the packing information of the patch may include the two-dimensional information of each patch in the occupancy map of the current point cloud frame, the identification information of the reference patch of each patch, the rotation transformation information of each patch, and the reference of the current point cloud frame The index of the frame. Among them, the two-dimensional information of each patch in the occupancy map of the current point cloud frame may be referred to as the two-dimensional information of each patch. The two-dimensional information of each patch can include the offset of each patch along the u axis in the current point cloud frame occupancy map, the offset of each patch along the v axis in the current point cloud frame occupancy map, and each patch The current point cloud frame occupies the width of the figure, and each patch occupies the height of the figure in the current point cloud frame. In addition, the packing module 102 may also send the packing information of the patch to the depth map generating module 103 and the point cloud occupancy map down-sampling module 111.
需要说明的是，为了更加直观地了解本申请技术中涉及的点云、点云的patch以及点云的占用图，参见图3、图4和图5，图3为可适用于本申请实施例的一帧点云的示意图，图4为该帧点云的patch的示意图，图5为图4所示的该帧点云的每个patch的占用图，经打包模块102打包得到的该帧点云的占用图的示意图。图3所示的点云可以是本申请实施例中的当前点云帧，图4所示的点云的patch可以是本申请实施例中的当前点云帧的patch，图5所示的点云的占用图可以是本申请实施例中的当前点云帧的占用图。It should be noted that, in order to more intuitively understand the point cloud, the patch of the point cloud, and the occupancy map of the point cloud involved in the technology of this application, see Figures 3, 4 and 5, and Figure 3 is an embodiment applicable to the application. Fig. 4 is a schematic diagram of the patch of the point cloud of the frame, and Fig. 5 is the occupancy diagram of each patch of the point cloud of the frame shown in Fig. 4, the frame point obtained by packing by the packing module 102 Schematic diagram of cloud occupancy map. The point cloud shown in FIG. 3 may be the current point cloud frame in the embodiment of the present application, the patch of the point cloud shown in FIG. 4 may be the patch of the current point cloud frame in the embodiment of the present application, and the point shown in FIG. 5 The cloud occupancy map may be the occupancy map of the current point cloud frame in the embodiment of the present application.
辅助信息编码模块109在接收到由打包模块102发送的当前点云帧的占用图和patch的打包信息，以及由点云块信息生成模块101发送的每个patch的第一标识和每个patch的辅助信息之后，可以将当前点云帧以及当前点云帧包括的每个patch进行编码，得到包括经编码的辅助信息的码流，然后将得到的包括经编码的辅助信息的码流发送给复用模块110。其中，patch的打包信息中的每个patch的二维信息和每个patch的参考patch的标识信息也可以称作每个patch的辅助信息。The auxiliary information encoding module 109 receives the occupancy map of the current point cloud frame and the packing information of the patch sent by the packing module 102, as well as the first identification of each patch and the first identification of each patch sent by the point cloud block information generating module 101. After the auxiliary information, the current point cloud frame and each patch included in the current point cloud frame can be encoded to obtain a code stream including the encoded auxiliary information, and then the obtained code stream including the encoded auxiliary information is sent to the complex Use module 110. Among them, the two-dimensional information of each patch and the identification information of the reference patch of each patch in the packaging information of the patch may also be referred to as auxiliary information of each patch.
深度图生成模块103可以在接收到由打包模块102发送的patch的打包信息，以及由点云块信息生成模块101发送的每个patch的深度图打包之后，根据每个patch的打包信息和每个patch的深度图生成当前点云帧的深度图。然后将所生成的当前点云帧的深度图发送给深度图填充模块105，以对当前点云帧的深度图中的空白像素点进行填充，得到经填充的当前点云帧的深度图。深度图填充模块105可以将得到的经填充的当前点云帧的深度图发送给基于图像或视频的编码模块107，以对经填充的当前点云帧的深度图进行基于图像或视频的编码，得到重构的当前点云的深度图，以及包括经编码的当前点云帧的深度图的码流，并可以将得到的重构的当前点云帧的深度图发送给点云重构模块113，将包括经编码的当前点云帧的深度图的码流发送给复用模块110。The depth map generation module 103 may receive the patch packaging information sent by the packaging module 102 and the depth map of each patch sent by the point cloud block information generation module 101, and then according to the packaging information of each patch and each The depth map of the patch generates the depth map of the current point cloud frame. Then the generated depth map of the current point cloud frame is sent to the depth map filling module 105 to fill the blank pixels in the depth map of the current point cloud frame to obtain the filled depth map of the current point cloud frame. The depth map filling module 105 may send the obtained filled depth map of the current point cloud frame to the image or video-based encoding module 107 to perform image or video-based encoding on the filled depth map of the current point cloud frame, Obtain the reconstructed current point cloud depth map and the code stream including the encoded current point cloud frame depth map, and may send the obtained reconstructed current point cloud frame depth map to the point cloud reconstruction module 113 , And send the code stream including the encoded depth map of the current point cloud frame to the multiplexing module 110.
点云占用图下采样模块111可以将接收到的由打包模块102发送的当前点云帧的占用图进行下采样处理，以得到低分辨率的当前点云帧的占用图。之后点云占用图下采样模块111还可以将该低分辨率的当前点云帧的占用图发送给占用图编码模块108和点云占用图填充模块112。占用图编码模块108可以对接收到的低分辨率的当前点云帧的占用图进行编码，得到包括经编码的低分辨率的当前点云帧的占用图的码流，占用图编码模块108还可以将包括 经编码的低分辨率的当前点云帧的占用图的码流发送给复用模块110。点云占用图填充模块112根据接收到的低分辨率的当前点云帧的占用图对于具有原始分辨率的当前点云帧的占用图进行填充，得到经填充的当前点云帧的占用图，该经填充的当前点云帧的占用图具有原始分辨率。The point cloud occupancy map down-sampling module 111 may perform down-sampling processing on the received occupancy map of the current point cloud frame sent by the packing module 102 to obtain a low-resolution occupancy map of the current point cloud frame. After that, the point cloud occupancy map down-sampling module 111 may also send the occupancy map of the current low-resolution point cloud frame to the occupancy map encoding module 108 and the point cloud occupancy map filling module 112. The occupancy map encoding module 108 can encode the received occupancy map of the current low-resolution point cloud frame to obtain a code stream including the occupancy map of the encoded low-resolution current point cloud frame, and the occupancy map encoding module 108 can also The code stream including the coded low-resolution occupancy map of the current point cloud frame may be sent to the multiplexing module 110. The point cloud occupancy map filling module 112 fills the occupancy map of the current point cloud frame with the original resolution according to the received occupancy map of the low-resolution current point cloud frame to obtain the filled occupancy map of the current point cloud frame, The occupancy map of the filled current point cloud frame has the original resolution.
点云重构模块113可以根据接收到的由点云占用图填充模块112发送的经填充的当前点云帧的占用图、由基于图像或视频的编码模块107发送的重构的当前点云帧的深度图、patch的打包信息和patch的辅助信息对当前点云帧的几何进行重构，以输出重构点云，另外，点云重构模块113还可以输出重构点云中重构点和patch的对应关系，以及重构点云中重构点的打包位置。点云重构模块113可以将重构点云、重构点云中重构点和patch的对应关系发送给点云滤波模块114，点云重构模块113还可以将重构点云中重构点的打包位置发送给纹理生成模块104。The point cloud reconstruction module 113 may be based on the received occupancy map of the filled current point cloud frame sent by the point cloud occupancy map filling module 112, and the reconstructed current point cloud frame sent by the image or video-based encoding module 107. The depth map, patch packing information and patch auxiliary information reconstruct the geometry of the current point cloud frame to output the reconstructed point cloud. In addition, the point cloud reconstruction module 113 can also output the reconstructed points in the reconstructed point cloud. Correspondence with patch, and the packaging position of the reconstructed point in the reconstructed point cloud. The point cloud reconstruction module 113 can send the reconstructed point cloud, the corresponding relationship between the reconstructed point in the reconstructed point cloud and the patch to the point cloud filtering module 114, and the point cloud reconstruction module 113 can also reconstruct the reconstructed point cloud. The packing positions of the points are sent to the texture generation module 104.
点云滤波模块114在接收到由点云重构模块113发送的重构点云、重构点云中重构点和patch的对应关系之后，可以对重构点云进行滤波。具体地，可以是去除重构点云中明显的噪声点和缝隙等瑕疵以得到经滤波的重构点云，也可以称为平滑的重构点云。或者也可以说，点云滤波块114可以对重构点云进行平滑处理。The point cloud filtering module 114 may filter the reconstructed point cloud after receiving the reconstructed point cloud sent by the point cloud reconstruction module 113 and the corresponding relationship between the reconstructed points in the reconstructed point cloud and the patch. Specifically, defects such as obvious noise points and gaps in the reconstructed point cloud can be removed to obtain a filtered reconstructed point cloud, which can also be referred to as a smooth reconstructed point cloud. Or it can be said that the point cloud filtering block 114 can perform smoothing processing on the reconstructed point cloud.
纹理图生成模块104在接收到由点云滤波模块114发送的平滑的重构点云、由点云重构模块113发送的重构点云中重构点的打包位置，以及由数据源120发送的当前点云帧之后，可以根据平滑的重构点云、重构点云中重构点的打包位置和当前点云帧生成当前点云帧的纹理图，并可以将所生成的当前点云帧的纹理图发送给纹理图填充模块106，以对当前点云帧的纹理图中的空白像素点进行填充，得到经填充的当前点云帧的纹理图。纹理图填充模块106可以将得到的经填充的当前点云帧的纹理图发送给基于图像或视频的编码模块107，以对经填充的当前点云帧的纹理图进行基于图像或视频的编码，得到包括重构的当前点云帧的纹理图的码流。基于图像或视频的编码模块107还可以将得到的包括重构的当前点云帧的纹理图的码流发送给复用模块110。The texture map generating module 104 receives the smooth reconstructed point cloud sent by the point cloud filtering module 114, the packed position of the reconstructed point in the reconstructed point cloud sent by the point cloud reconstruction module 113, and is sent by the data source 120 After the current point cloud frame, the texture map of the current point cloud frame can be generated according to the smooth reconstructed point cloud, the packaging position of the reconstructed point in the reconstructed point cloud and the current point cloud frame, and the generated current point cloud The texture map of the frame is sent to the texture map filling module 106 to fill the blank pixels in the texture map of the current point cloud frame to obtain the filled texture map of the current point cloud frame. The texture map filling module 106 may send the obtained filled texture map of the current point cloud frame to the image or video-based encoding module 107 to perform image or video-based encoding on the filled texture map of the current point cloud frame, Obtain the code stream including the texture map of the reconstructed current point cloud frame. The image or video-based encoding module 107 may also send the obtained code stream including the reconstructed texture map of the current point cloud frame to the multiplexing module 110.
通过上述描述，基于图像或视频的编码模块107、占用图编码模块108、辅助信息编码模块109可以将得到的码流发送给复用模块110，复用模块110可以将接收到的这些码流合并成一个合并的码流，并将该合并的码流发送给输出接口140。输出接口140可以将该合并的码流发送给解码器200。Through the above description, the image or video-based encoding module 107, the occupancy map encoding module 108, and the auxiliary information encoding module 109 can send the obtained code streams to the multiplexing module 110, and the multiplexing module 110 can combine the received code streams To form a combined code stream, and send the combined code stream to the output interface 140. The output interface 140 may send the combined code stream to the decoder 200.
应当理解的是，图2所示的编码器100仅为本申请提供的一种实施例，在具体的实现方式中，编码器100可以包括比图2中所示的模块更多或更少的模块。本申请实施例对此不做具体限定。It should be understood that the encoder 100 shown in FIG. 2 is only an embodiment provided by the present application. In a specific implementation manner, the encoder 100 may include more or less modules than those shown in FIG. 2 Module. The embodiments of the present application do not specifically limit this.
图6为本申请实施例提供的一种解码器200的示意性框图。图6是以MPEG PCC解码框架为例进行说明的。参见图6，解码器200可以包括解复用模块201、基于图像或视频的解码模块202、占用图解码模块203、辅助信息解码模块204、点云占用图填充模块205、点云重构模块206、点云滤波模块207和点云的纹理信息重构模块208。FIG. 6 is a schematic block diagram of a decoder 200 according to an embodiment of the application. Figure 6 illustrates the MPEG PCC decoding framework as an example. Referring to FIG. 6, the decoder 200 may include a demultiplexing module 201, an image or video-based decoding module 202, an occupancy map decoding module 203, an auxiliary information decoding module 204, a point cloud occupancy map filling module 205, and a point cloud reconstruction module 206 , Point cloud filtering module 207 and point cloud texture information reconstruction module 208.
解复用模块201可以通过输入接口204接收由编码器100的输出接口140发送的合并的码流，并将该合并的码流发送到相应解码模块。具体地，解复用模块201将包括经编码的当前点云帧的纹理图的码流和经编码的当前点云帧的深度图的码流，发送给基于图像或视频的 解码模块202，将包括经编码的低分辨率的当前点云帧的占用图的码流发送给占用图解码模块203，将包括经编码的辅助信息的码流发送给辅助信息解码模块204。The demultiplexing module 201 may receive the combined code stream sent by the output interface 140 of the encoder 100 through the input interface 204, and send the combined code stream to the corresponding decoding module. Specifically, the demultiplexing module 201 sends the code stream of the texture map of the current point cloud frame encoded and the code stream of the depth map of the encoded current point cloud frame to the image or video-based decoding module 202, and The code stream including the occupancy map of the current point cloud frame with the encoded low resolution is sent to the occupancy map decoding module 203, and the code stream including the encoded auxiliary information is sent to the auxiliary information decoding module 204.
基于图像或视频的解码模块202可以对接收到的包括经编码的当前点云帧的纹理图的码流和包括经编码的当前点云帧的深度图的码流进行解码，得到重构的当前点云帧的纹理图信息和重构的当前点云帧的深度图信息，并可以将该重构的当前点云帧的纹理图信息发送给点云的纹理信息重构模块208，将该重构的当前点云帧的深度图信息发送给点云重构模块206。占用图解码模块203可以对接收到的包括经编码的低分辨率的当前点云帧的占用图的码流进行解码，得到重构的低分辨率的当前点云帧的占用图信息，并将该重构的低分辨率的当前点云帧的占用图信息发送给点云占用图填充模块205。点云占用图填充模块205根据重构的低分辨率的当前点云帧的占用图信息，可以得到重构的具有原始分辨率的当前点云帧的占用图信息，然后将该重构的具有原始分辨率的当前点云帧的占用图信息发送给点云重构模块206。为了方便描述，后续统一将重构的具有原始分辨率的当前点云帧的占用图信息，简称为重构的当前点云帧的占用图信息。The image or video-based decoding module 202 can decode the received bitstream including the texture map of the encoded current point cloud frame and the bitstream including the depth map of the encoded current point cloud frame to obtain the reconstructed current The texture map information of the point cloud frame and the depth map information of the reconstructed current point cloud frame, and the reconstructed texture map information of the current point cloud frame can be sent to the point cloud texture information reconstruction module 208, The constructed depth map information of the current point cloud frame is sent to the point cloud reconstruction module 206. The occupancy map decoding module 203 can decode the received bitstream including the occupancy map of the current point cloud frame with encoded low resolution to obtain the occupancy map information of the reconstructed current point cloud frame with low resolution, and The reconstructed low-resolution current point cloud frame occupancy map information is sent to the point cloud occupancy map filling module 205. The point cloud occupancy map filling module 205 can obtain the reconstructed occupancy map information of the current point cloud frame with the original resolution according to the occupancy map information of the reconstructed low-resolution current point cloud frame, and then the reconstructed has The occupancy map information of the current point cloud frame at the original resolution is sent to the point cloud reconstruction module 206. For the convenience of description, the reconstructed occupancy map information of the current point cloud frame with the original resolution will be uniformly referred to as the occupancy map information of the reconstructed current point cloud frame.
辅助信息解码模块204可以对接收到的包括经编码的辅助信息的码流进行解码，得到辅助信息，并可以将该辅助信息发送给点云重构模块206。具体的解码方法会在后续实施例中进行说明。The auxiliary information decoding module 204 may decode the received code stream including the encoded auxiliary information to obtain auxiliary information, and may send the auxiliary information to the point cloud reconstruction module 206. The specific decoding method will be described in subsequent embodiments.
点云重构模块206可以根据接收到的由基于图像或视频的解码模块202发送的重构的当前点云帧的深度图信息、由点云占用图填充模块205发送的重构的当前点云帧的占用图信息，以及由辅助信息解码模块204发送的辅助信息对当前点云帧的几何进行重构，得到重构点云。该重构点云与编码器100中的点云重构模块112得到的重构点云类似，且具体的重构过程可以参考编码器100中的点云重构模块112的重构过程，此处不再赘述。点云重构模块206还可以将该重构点云发送给点云滤波模块207。点云滤波模块207可以根据接收到的重构点云，对重构点云进行滤波，得到平滑的重构点云，具体的滤波过程可以参考编码器100中的点云滤波模块114的滤波过程，此处不再赘述。点云滤波模块207可以将该平滑的重构点云发送给点云的纹理信息重构模块208。点云的纹理信息重构模块208在接收到由点云滤波模块207发送的平滑的重构点云，以及由基于图像或视频的解码模块202发送的重构的当前点云帧的纹理图信息之后，可以对重构点云的纹理信息进行重构，得到经纹理信息重构的重构点云。The point cloud reconstruction module 206 can be based on the received depth map information of the reconstructed current point cloud frame sent by the image or video-based decoding module 202, and the reconstructed current point cloud sent by the point cloud occupancy map filling module 205. The occupancy map information of the frame and the auxiliary information sent by the auxiliary information decoding module 204 reconstruct the geometry of the current point cloud frame to obtain the reconstructed point cloud. The reconstructed point cloud is similar to the reconstructed point cloud obtained by the point cloud reconstruction module 112 in the encoder 100, and the specific reconstruction process can refer to the reconstruction process of the point cloud reconstruction module 112 in the encoder 100. I won't repeat it here. The point cloud reconstruction module 206 may also send the reconstructed point cloud to the point cloud filtering module 207. The point cloud filtering module 207 can filter the reconstructed point cloud according to the received reconstructed point cloud to obtain a smooth reconstructed point cloud. The specific filtering process can refer to the filtering process of the point cloud filtering module 114 in the encoder 100 , I won’t repeat it here. The point cloud filtering module 207 may send the smooth reconstructed point cloud to the texture information reconstruction module 208 of the point cloud. The texture information reconstruction module 208 of the point cloud receives the smooth reconstructed point cloud sent by the point cloud filtering module 207 and the texture map information of the reconstructed current point cloud frame sent by the image or video-based decoding module 202. Afterwards, the texture information of the reconstructed point cloud can be reconstructed to obtain the reconstructed point cloud reconstructed by the texture information.
应当理解的是，图6所示的解码器200仅为示例，具体实现时，解码器200可以包括比图6中所示的更多或更少的模块。本申请实施例对此不进行限定。It should be understood that the decoder 200 shown in FIG. 6 is only an example, and in a specific implementation, the decoder 200 may include more or less modules than those shown in FIG. 6. The embodiment of the present application does not limit this.
为了便于理解，接下来先对本申请实施例中涉及的技术术语进行说明。For ease of understanding, the technical terms involved in the embodiments of the present application will be described first.
在一种示例中，当前点云组可以是一个帧组(group of frame，GOF)，一个帧组可以包括32帧点云，也即是当前点云组可以包括32帧点云。当然当前点云组包括的点云的帧数也可以为其他数值，本申请实施例对此不做限定。在一种示例中，一个帧组还可以划分为多个子组，每个子组可以包括一帧或多帧点云，当前点云组可以为该多个子组中的任意一个子组。例如，参见图7，图7为本申请实施例提供的一种帧组的示意图，如图7所示，一个帧组可以划分为10个子组，第一个子组包括第1,2,3帧点云，第二个子组包括第4帧点云，第三个子组包括第5,6,7,8帧点云，……，第10个子组包括第30,31,32帧点云，当前点云组即可以是这10个子组中的任意一个子组。图7中仅是以示例性地表示出第一个子组、第二个子组、 第三个子组和第十个子组中包括的点云，其他子组中包括的点云不作具体说明。综上，当前点云组可以是一个帧组，或者，当前点云组可以是一个帧组中的一个子组，当然，当前点云组还可以是其他形式的点云组，比如自定义的点云组，本申请实施例对此不做限定。In an example, the current point cloud group may be a group of frame (GOF), and a frame group may include 32 frames of point clouds, that is, the current point cloud group may include 32 frames of point clouds. Of course, the number of frames of the point cloud included in the current point cloud group may also be other values, which is not limited in the embodiment of the present application. In an example, a frame group may also be divided into multiple subgroups, each subgroup may include one or more frames of point clouds, and the current point cloud group may be any one of the multiple subgroups. For example, referring to FIG. 7, FIG. 7 is a schematic diagram of a frame group provided by an embodiment of the application. As shown in FIG. 7, a frame group can be divided into 10 subgroups, and the first subgroup includes the first, second, and third subgroups. Frame point cloud, the second subgroup includes the 4th frame point cloud, the third subgroup includes the 5th, 6, 7, and 8th frame point clouds,..., the 10th subgroup includes the 30th, 31st, and 32th frame point clouds, The current point cloud group can be any one of these 10 subgroups. FIG. 7 only exemplarily shows the point clouds included in the first subgroup, the second subgroup, the third subgroup, and the tenth subgroup, and the point clouds included in other subgroups are not described in detail. In summary, the current point cloud group can be a frame group, or the current point cloud group can be a subgroup of a frame group. Of course, the current point cloud group can also be other forms of point cloud groups, such as custom ones. The point cloud group is not limited in this embodiment of the application.
基于上述描述，patch可以分为三种类型，分别为第一类patch、第二类patch和第三类patch。示例性地，根据patch在当前点云组内的匹配情况，patch也可以分为全局匹配patch(SKIP_patch)、局部匹配patch(local_patch)和非匹配patch(I_INTRA，或者P_INTRA)。也即是，第一类patch可以为全局匹配patch，第二类patch可以为局部匹配patch，第三类patch可以为非匹配patch。为了便于描述，以待编码patch为例，对这三类patch进行说明，应当理解的是，待编码patch为当前点云帧中的任一patch，当前点云帧为当前点云组内的任一帧点云。如果待编码patch为第一类patch，则待编码patch在当前点云组内的所有点云帧中都存在具有匹配关系的patch，这种可以称为全局匹配patch；如果待编码patch为第二类patch，则待编码patch在当前点云帧的参考帧中存在具有匹配关系的patch，但只在当前点云组内的部分点云帧中存在具有匹配关系的patch，这种可以称为局部匹配patch；如果待编码patch为第三类patch，则待编码patch在当前点云组内的任意点云帧中都不存在具有匹配关系的patch，这种可以称为非匹配patch。Based on the above description, patches can be divided into three types, namely the first type of patch, the second type of patch and the third type of patch. Exemplarily, according to the matching situation of the patch in the current point cloud group, the patch can also be divided into a global matching patch (SKIP_patch), a local matching patch (local_patch), and a non-matching patch (I_INTRA, or P_INTRA). That is, the first type of patch can be a global matching patch, the second type of patch can be a local matching patch, and the third type of patch can be a non-matching patch. For ease of description, taking the patch to be coded as an example, the three types of patches are described. It should be understood that the patch to be coded is any patch in the current point cloud frame, and the current point cloud frame is any of the current point cloud groups. A frame of point cloud. If the patch to be coded is the first type of patch, the patch to be coded has a matching relationship in all the point cloud frames in the current point cloud group, this can be called a global matching patch; if the patch to be coded is the second For the patch type, the patch to be encoded has a patch with a matching relationship in the reference frame of the current point cloud frame, but only a patch with a matching relationship exists in a part of the point cloud frame in the current point cloud group. This can be called local Matching patch; if the patch to be coded is the third type of patch, there is no patch with a matching relationship in any point cloud frame in the current point cloud group for the patch to be coded. This can be called a non-matching patch.
在一种可能的实现方式中，参见图8，当前点云帧的参考帧可以位于当前点云组内。示例性地，当前点云帧的参考帧可以是当前点云帧的前一帧点云。另外，当前点云组中的所有点云包括的patch可以按照第一类patch、第二类patch和第三类patch的顺序依次进行排列，其中，当前点云组内的第一类patch在每帧点云中的个数是相同的，并且当第一类patch的数量大于1时，当前点云组内的每帧点云中的第一类patch还可以按照相同的顺序依次排列，也即是，每帧点云中相同位置的第一类patch之间具备匹配关系。比如，每帧点云中的第w个第一类patch相互之间具有匹配关系，w小于等于第一类patch的数量。In a possible implementation manner, referring to FIG. 8, the reference frame of the current point cloud frame may be located in the current point cloud group. Exemplarily, the reference frame of the current point cloud frame may be the point cloud of the previous frame of the current point cloud frame. In addition, the patches included in all point clouds in the current point cloud group can be arranged in the order of the first type patch, the second type patch, and the third type patch. Among them, the first type patch in the current point cloud group The number of point clouds in the frame is the same, and when the number of the first-type patches is greater than 1, the first-type patches in each frame of the current point cloud group can also be arranged in the same order, that is, Yes, there is a matching relationship between the first-type patches at the same position in the point cloud of each frame. For example, the w-th first-type patch in each frame of point cloud has a matching relationship with each other, and w is less than or equal to the number of first-type patches.
通常情况下，第一类patch在当前点云帧的占用图中的二维信息、旋转变换信息和法向坐标轴信息相同，而三维信息不同。但是在某些情况下，第一类patch的二维信息也可以不同，或者三维信息也可以相同。也即是，第一类patch的旋转变换信息和法向坐标轴信息可以相同，而二维信息和三维信息可能相同，也可能不同。这样，对于二维信息和三维信息而言，可能存在三种情况。为了便于描述，接下来以待编码patch为例进行说明。在第一种可能的情况中，当待编码patch为第一类patch时，待编码patch的参考patch的二维信息与待编码patch的二维信息相同，并且待编码patch的参考patch的三维信息与待编码patch的三维信息相同。在第二种可能的情况中，当待编码patch为第一类patch时，待编码patch的参考patch的二维信息和待编码patch的二维信息相同，待编码patch的参考patch的三维信息与待编码patch的三维信息不相同。在第三种可能的情况中，当待编码patch为第一类patch时，待编码patch的参考patch的二维信息与待编码patch的二维信息不相同，并且待编码patch的参考patch的三维信息与待编码patch的三维信息不相同。Generally, the two-dimensional information, rotation transformation information, and normal coordinate axis information of the first-type patch in the occupancy map of the current point cloud frame are the same, but the three-dimensional information is different. However, in some cases, the two-dimensional information of the first-type patch can also be different, or the three-dimensional information can also be the same. That is, the rotation transformation information and the normal coordinate axis information of the first type patch may be the same, and the two-dimensional information and the three-dimensional information may be the same or different. In this way, for two-dimensional information and three-dimensional information, there may be three situations. For ease of description, the following takes the patch to be coded as an example. In the first possible situation, when the patch to be encoded is the first type of patch, the two-dimensional information of the reference patch of the patch to be encoded is the same as the two-dimensional information of the patch to be encoded, and the three-dimensional information of the reference patch of the patch to be encoded Same as the three-dimensional information of the patch to be encoded. In the second possible situation, when the patch to be encoded is the first type of patch, the two-dimensional information of the reference patch of the patch to be encoded is the same as the two-dimensional information of the patch to be encoded, and the three-dimensional information of the reference patch of the patch to be encoded is the same as The three-dimensional information of the patch to be encoded is different. In the third possible situation, when the patch to be encoded is the first type of patch, the two-dimensional information of the reference patch of the patch to be encoded is different from the two-dimensional information of the patch to be encoded, and the three-dimensional information of the reference patch of the patch to be encoded The information is different from the three-dimensional information of the patch to be encoded.
当按照图8所示的方式对当前点云组中的所有点云包括的patch进行排列时，待解码patch的参考patch可以为待解码patch所在的当前点云帧的参考帧中处于同一位置的patch。比如，待解码patch在当前点云帧中处于第三个位置，那么，当前点云帧的参考帧中的第三个patch即为待解码patch的参考patch。When the patches included in all the point clouds in the current point cloud group are arranged in the manner shown in FIG. 8, the reference patch of the patch to be decoded may be the same position in the reference frame of the current point cloud frame where the patch to be decoded is located. patch. For example, if the patch to be decoded is in the third position in the current point cloud frame, then the third patch in the reference frame of the current point cloud frame is the reference patch of the patch to be decoded.
旋转变换信息可以为旋转角度信息、坐标轴变换信息或镜像变换信息。其中，旋转角度 信息包括具体的旋转角度值或旋转角度索引。比如若待编码patch的旋转角度值为0时，则表示该待编码patch不旋转；若待编码patch的旋转角度值为20，则表示该待编码patch旋转20°；若待编码patch的旋转角度值为80，则表示该待编码patch旋转80°。再比如若待编码patch的旋转角度索引为0，则表示该待编码patch不旋转；若待编码patch的旋转角度索引为1，则表示该待编码patch旋转15°；若待编码patch的旋转角度索引为2，则表示该待编码patch旋转30°；若待编码patch的旋转角度索引为3，则表示该待编码patch旋转45°。旋转角度信息也可以用其他形式进行表征，在此不做具体限定。例如也可以用矩阵的形式进行描述，特别是在GPU(Graphic Procession Unit，图像处理器)中进行加速处理时，采用矩阵形式会提高处理速度。The rotation transformation information may be rotation angle information, coordinate axis transformation information, or mirror transformation information. Among them, the rotation angle information includes a specific rotation angle value or a rotation angle index. For example, if the rotation angle value of the patch to be coded is 0, it means that the patch to be coded does not rotate; if the rotation angle value of the patch to be coded is 20, it means that the patch to be coded is rotated by 20°; if the rotation angle of the patch to be coded is A value of 80 means that the patch to be coded is rotated by 80°. For another example, if the rotation angle index of the patch to be coded is 0, it means that the patch to be coded does not rotate; if the rotation angle index of the patch to be coded is 1, it means that the patch to be coded is rotated by 15°; if the rotation angle of the patch to be coded is If the index is 2, it means that the patch to be coded is rotated by 30°; if the rotation angle index of the patch to be coded is 3, it means that the patch to be coded is rotated by 45°. The rotation angle information can also be characterized in other forms, which is not specifically limited here. For example, it can also be described in the form of a matrix, especially when acceleration processing is performed in a GPU (Graphic Processing Unit, image processor), the use of the matrix form will increase the processing speed.
需要说明的是，待编码patch的旋转可以是顺时针旋转，也可以是逆时针旋转。It should be noted that the rotation of the patch to be encoded can be clockwise or counterclockwise.
若旋转变换信息为坐标轴交换信息时，坐标轴交换信息可以表示为坐标轴交换标识，比如坐标轴交换标识为false或者0时，表示待编码patch的切线坐标轴和待编码patch的双切线坐标轴不交换；比如坐标轴交换标识为ture或者1时，待编码patch的切线坐标轴和待编码patch的双切线坐标轴交换。例如，待编码patch的坐标轴的索引可以为0，1,或者2。假设X轴的索引为0，Y轴的索引为1，Z轴的索引为2，则待编码patch的切线坐标轴索引和待编码patch的双切线坐标轴索引可以是0，1和2中的任意2个。一般情况下，patch的切线坐标轴可以称为U轴，patch的双切线坐标轴可以称为V轴。假设在某种情况下，获得的当前patch的切线坐标轴索引和待编码patch的双切线坐标轴索引分别为0和2，如果坐标轴交换标识为false或者0，则不交换待编码patch的切线坐标轴的索引和待编码patch的双切线坐标轴的索引；如果坐标轴交换标识为ture或者1，则交换待编码patch的切线坐标轴的索引和待编码patch的双切线坐标轴的索引，即待编码patch的切线坐标轴的索引更新为2，待编码patch的双切线坐标轴的索引更新为0。If the rotation transformation information is the coordinate axis exchange information, the coordinate axis exchange information can be expressed as the coordinate axis exchange flag. For example, when the coordinate axis exchange flag is false or 0, it indicates the tangent coordinate axis of the patch to be encoded and the double tangent coordinate of the patch to be encoded The axes are not exchanged; for example, when the coordinate axis exchange flag is ture or 1, the tangent axis of the patch to be encoded and the double tangent axis of the patch to be encoded are exchanged. For example, the index of the coordinate axis of the patch to be encoded can be 0, 1, or 2. Assuming that the index of the X axis is 0, the index of the Y axis is 1, and the index of the Z axis is 2, then the tangent axis index of the patch to be encoded and the double tangent axis index of the patch to be encoded can be 0, 1, and 2. Any two. In general, the tangent axis of the patch can be called the U axis, and the double tangent axis of the patch can be called the V axis. Assuming that under certain circumstances, the obtained tangent axis index of the current patch and the double tangent axis index of the patch to be encoded are 0 and 2, respectively. If the axis exchange flag is false or 0, the tangent of the patch to be encoded is not exchanged The index of the coordinate axis and the index of the double tangent axis of the patch to be encoded; if the axis exchange flag is true or 1, then the index of the tangent axis of the patch to be encoded and the index of the double tangent axis of the patch to be encoded are exchanged, namely The index of the tangent axis of the patch to be coded is updated to 2, and the index of the double tangent axis of the patch to be coded is updated to 0.
接下来，对本申请实施例提供的第一种点云编码方法进行说明。需要说明的是，结合图1所示的点云译码***，以及图2所示的编码器100的示意性框图，下文中的任一种点云编码方法可以是点云译码***中的编码器100执行的，更具体地，可以是编码器100中的辅助信息编码模块109执行的。Next, the first point cloud coding method provided by the embodiment of the present application will be described. It should be noted that, in combination with the point cloud decoding system shown in FIG. 1 and the schematic block diagram of the encoder 100 shown in FIG. 2, any of the following point cloud encoding methods may be the point cloud decoding system. The execution by the encoder 100, more specifically, may be executed by the auxiliary information encoding module 109 in the encoder 100.
图9是本申请实施例提供的一种点云编码方法的流程图，该方法应用于点云译码***。参见图9，该方法包括：FIG. 9 is a flowchart of a point cloud encoding method provided by an embodiment of the present application, and the method is applied to a point cloud decoding system. Referring to Figure 9, the method includes:
S901：获取待编码patch的辅助信息。S901: Obtain auxiliary information of the patch to be encoded.
需要说明的是，待编码patch可以是待编码的当前点云帧包括的patch中的任一patch。It should be noted that the patch to be encoded may be any patch included in the current point cloud frame to be encoded.
S902：将待编码patch的辅助信息和待编码patch的第一标识编入码流。S902: Encode the auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded into the code stream.
第一标识可以指示待编码patch的类型。第一标识的语法元素可以是patch_mode或patch_type等。第一标识的取值不同，待编码patch的类型不同。The first identifier may indicate the type of patch to be encoded. The first identified syntax element may be patch_mode or patch_type, etc. The value of the first identifier is different, and the type of the patch to be encoded is different.
其中，第一标识为第一值时，可以指示待编码patch为第一类patch，编入码流的待编码patch的辅助信息包括三维信息，而不包括二维信息、旋转变换信息和法向坐标轴信息；和/或，第一标识为第二值时，可以指示待编码patch为第二类patch，编入码流的待编码patch的辅助信息包括二维信息、三维信息和待编码patch的参考patch的标识信息，而不包括旋转变换信息和法向坐标轴信息；和/或，第一标识为第三值时，可以指示待编码patch为第三类 patch，编入码流的待编码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。Wherein, when the first identifier is the first value, it can indicate that the patch to be encoded is the first type of patch. The auxiliary information of the patch to be encoded encoded into the code stream includes three-dimensional information, but does not include two-dimensional information, rotation transformation information, and normal direction. Coordinate axis information; and/or, when the first identifier is the second value, it can indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded that is encoded into the code stream includes two-dimensional information, three-dimensional information, and the patch to be encoded The identification information of the reference patch does not include the rotation transformation information and the normal coordinate axis information; and/or, when the first identification is the third value, it can indicate that the patch to be encoded is a third-type patch, and is encoded into the waiting stream of the code stream. The auxiliary information of the encoded patch includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
通常情况下，第一类patch在当前点云组内的所有点云中都存在具有匹配关系的patch，且当前点云组内的第一类patch之间的二维信息、旋转变换信息和法向坐标轴信息相同，而三维信息可能不相同，第二类patch在当前点云帧的参考帧中存在具有匹配关系的patch，且第二类patch与其参考patch之间的旋转变换信息和法向坐标轴信息相同，而二维信息和三维信息可能不相同，第三类patch在当前点云帧的参考帧中不存在具有匹配关系的patch，因此，为了减少码流的比特开销，进而提高编码的效率，当第一标识为第一值时，待编码patch的辅助信息可以包括三维信息，而不包括二维信息、旋转变换信息和法向坐标轴信息。当第一标识为第二值时，待编码patch的辅助信息可以包括二维信息、三维信息和待编码patch的参考patch的标识信息，而不包括旋转变换信息和法向坐标轴信息。当第一标识为第三值时，待编码patch的辅助信息可以包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。Generally, the first-type patch has matching patches in all point clouds in the current point cloud group, and the two-dimensional information, rotation transformation information and method between the first-type patches in the current point cloud group The direction axis information is the same, but the three-dimensional information may be different. The second type of patch has a matching patch in the reference frame of the current point cloud frame, and the rotation transformation information and normal direction between the second type of patch and its reference patch The coordinate axis information is the same, but the two-dimensional information and the three-dimensional information may not be the same. The third type of patch does not have a matching patch in the reference frame of the current point cloud frame. Therefore, in order to reduce the bit overhead of the bit stream, and then improve the coding When the first identifier is the first value, the auxiliary information of the patch to be encoded may include three-dimensional information, but not two-dimensional information, rotation transformation information, and normal coordinate axis information. When the first identifier is the second value, the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded, but does not include rotation transformation information and normal coordinate axis information. When the first identifier is the third value, the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
需要说明的是，待编码patch的三维信息可以包括待编码patch在3D空间中沿着切线方向的偏移(3d_shift_tangent_axis)、待编码patch在3D空间中沿着双切线方向的偏移(3d_shift_bitangent_axis)，以及待编码patch在3D空间中沿着法线方向的偏移(3d_shift_normal_axis)。待编码patch的二维信息可以包括待编码patch在当前点云帧占用图中沿u轴方向的偏移(2d_shift_u)、待编码patch在当前点云帧占用图中沿v轴方向的偏移(2d_shift_v)、待编码patch在当前点云帧占用图中的宽度(2d_size_u)，以及待编码patch在当前点云帧占用图中的高度(2d_size_v)。另外，待编码patch的参考patch的语法元素可以为patchIndex，用于指示参考patch的标识信息；语法元素还可以包括参考patch所在参考帧的标识信息：frameIndex。待编码patch的旋转变换信息的语法元素可以是rotation。再者，待编码patch的参考patch的标识信息可以用于指示待编码patch的参考patch。法向坐标轴信息可以为待编码patch的法向轴索引，例如normalAxis法向轴索引可以为0,1或者2，分别对于X轴、Y轴和Z轴。It should be noted that the three-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the 3D space along the tangent direction (3d_shift_tangent_axis), and the offset of the patch to be encoded in the 3D space along the bitangent direction (3d_shift_bitangent_axis), And the offset of the patch to be encoded in the 3D space along the normal direction (3d_shift_normal_axis). The two-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the current point cloud frame occupancy map along the u-axis direction (2d_shift_u), and the offset of the patch to be encoded in the current point cloud frame occupancy map along the v-axis direction ( 2d_shift_v), the width of the patch to be encoded in the current point cloud frame occupancy map (2d_size_u), and the height of the patch to be encoded in the current point cloud frame occupancy map (2d_size_v). In addition, the syntax element of the reference patch of the patch to be encoded may be patchIndex, which is used to indicate the identification information of the reference patch; the syntax element may also include the identification information of the reference frame where the reference patch is located: frameIndex. The syntax element of the rotation transformation information of the patch to be encoded may be rotation. Furthermore, the identification information of the reference patch of the patch to be encoded may be used to indicate the reference patch of the patch to be encoded. The normal axis information may be the normal axis index of the patch to be encoded, for example, the normal axis index of the normalAxis may be 0, 1, or 2, for the X axis, Y axis, and Z axis, respectively.
作为一种示例，当第一标识为第一值时，可以根据待编码patch的参考patch的三维信息，将待编码patch的三维信息编入码流，具体可以采用差分编码方式将待编码patch的三维信息编入码流，也可以直接编入码流，具体的编码方式不做限定。采用定长编码方式将待编码patch的第一标识编入码流。当第一标识为第二值时，可以根据待编码patch的参考patch的二维信息和三维信息，将待编码patch的二维信息和三维信息编入码流，具体可以采用差分编码方式将待编码patch的二维信息和三维信息编入码流，也可以采用更加精确的预测差分方式将待编码patch的二维信息和三维信息编入码流，具体的编码方式不做限定。采用定长编码方式将待编码patch的参考patch的标识信息和第一标识编入码流。当第一标识为第三值时，可以采用定长编码方式将待编码patch的二维信息、三维信息、旋转变换信息、法向坐标轴信息和第一标识编入码流。As an example, when the first identifier is the first value, the three-dimensional information of the patch to be encoded can be encoded into the code stream according to the three-dimensional information of the reference patch of the patch to be encoded. Specifically, the differential encoding method can be used to encode the three-dimensional information of the patch to be encoded. The 3D information is encoded into the code stream, or directly into the code stream, and the specific encoding method is not limited. The fixed-length encoding method is adopted to encode the first identifier of the patch to be encoded into the code stream. When the first identifier is the second value, the two-dimensional information and three-dimensional information of the patch to be encoded can be encoded into the code stream according to the two-dimensional information and three-dimensional information of the reference patch to be encoded. Specifically, the differential encoding method can be used to encode the two-dimensional information and three-dimensional information of the patch to be encoded. The two-dimensional information and three-dimensional information of the encoded patch are encoded into the code stream, and the two-dimensional information and the three-dimensional information of the patch to be encoded can also be encoded into the code stream by using a more accurate predictive difference method. The specific encoding method is not limited. The identification information and the first identification of the reference patch of the patch to be encoded are encoded into the code stream using a fixed-length encoding method. When the first identifier is the third value, the fixed-length encoding method can be used to encode the two-dimensional information, three-dimensional information, rotation transformation information, normal coordinate axis information, and the first identifier of the patch to be encoded into the code stream.
其中，采用差分编码方式将待编码patch的三维信息编入码流是指将待编码patch的三维信息与待解码patch的参考patch的三维信息之间的差值编入码流。采用差分编码方式将待编码patch的二维信息和三维信息编入码流是指将待编码patch的二维信息与待解码patch的参考patch的二维信息之间的差值，以及待编码patch的三维信息与待解码patch的参考patch的三维信息之间的差值编入码流。Wherein, encoding the three-dimensional information of the patch to be encoded into the code stream using the differential encoding method refers to encoding the difference between the three-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch of the patch to be decoded into the code stream. Using differential encoding to encode the two-dimensional information and three-dimensional information of the patch to be encoded into the code stream refers to the difference between the two-dimensional information of the patch to be encoded and the two-dimensional information of the reference patch of the patch to be decoded, and the patch to be encoded The difference between the 3D information of the reference patch and the 3D information of the patch to be decoded is encoded into the code stream.
在一种实现方式下，可以将当前点云帧中的多个patch的第一标识一起打包并编入码流。例如，可以将当前点云帧中的所有patch的第一标识一起打包并编入码流。在另一种实现方式下，可以将当前点云帧中的每个patch的第一标识各自打包并编入码流，示例性地，可以将待编码patch[i-1]的第一标识和待编码patch[i-1]的辅助信息、待编码patch[i]的第一标识和待编码patch[i]的辅助信息，以及待编码patch[i+1]的第一标识和待编码patch[i+1]的辅助信息等依序编入码流中。也即是，对于当前点云帧来说，当前点云帧中的所有patch的第一标识可以各自打包并编入码流，也可以打包在一起并编入码流。In an implementation manner, the first identifiers of multiple patches in the current point cloud frame can be packaged together and compiled into a code stream. For example, the first identifiers of all patches in the current point cloud frame can be packaged together and compiled into the code stream. In another implementation manner, the first identifier of each patch in the current point cloud frame can be individually packaged and compiled into the code stream. For example, the first identifier of the patch[i-1] to be encoded can be combined with The auxiliary information of the patch[i-1] to be encoded, the first identification of the patch[i] to be encoded and the auxiliary information of the patch[i] to be encoded, and the first identification of the patch[i+1] to be encoded and the patch to be encoded [i+1] Auxiliary information, etc. are sequentially encoded into the code stream. That is, for the current point cloud frame, the first identifiers of all patches in the current point cloud frame can be packaged separately and compiled into a code stream, or they can be packed together and compiled into a code stream.
其中，第一值、第二值和第三值可以分别是二进制的10、11和00，或者，第一值、第二值和第三值可以分别是二进制的11、10和00等，待编码patch的参考patch包括于当前点云帧的参考帧中，本申请实施例对此不做限定。Among them, the first value, the second value, and the third value may be binary 10, 11, and 00 respectively, or the first value, the second value, and the third value may be binary 11, 10, and 00, etc., respectively. The reference patch of the coded patch is included in the reference frame of the current point cloud frame, which is not limited in the embodiment of the present application.
作为一种示例，本文示例一种待编码patch的第一标识为第一值时的部分语法结构，参见表1和表2所示。As an example, this article exemplifies a partial syntax structure when the first identifier of the patch to be encoded is the first value, as shown in Table 1 and Table 2.
表1Table 1
Figure PCTCN2020080140-appb-000001
Figure PCTCN2020080140-appb-000001
表2Table 2
Figure PCTCN2020080140-appb-000002
Figure PCTCN2020080140-appb-000002
Figure PCTCN2020080140-appb-000003
Figure PCTCN2020080140-appb-000003
其中，表1中的skip_patch_data_unit即是待编码patch的第一标识为第一值时，编入码流的待编码patch的信息，也可以称为待编码patch的数据单元。Wherein, skip_patch_data_unit in Table 1 is the information of the patch to be encoded that is encoded into the code stream when the first identifier of the patch to be encoded is the first value, and can also be referred to as the data unit of the patch to be encoded.
为了在解码时确定当前点云帧是否为当前点云组内的首帧点云，进而在解码过程中对当前点云组与其他点云组进行划分，参见图10，本申请实施例可以在辅助信息编码模块109接收到当前点云组的首帧点云之后，将当前点云组的首帧点云包括的所有patch的第一标识设置为第三值。这样，当当前点云帧为当前点云组内的首帧点云时，当前点云帧中的首个patch的第一标识即为第三值。当当前点云帧为当前点云组内的非首帧点云时，当前点云帧中的首个patch的第一标识即为第一值。也即是，待编码patch的第一标识为第三值，且待编码patch为当前点云帧中的首个patch时，即可确定当前点云帧为当前点云组内的首帧点云。这样就无需在码流中编入当前点云组的点云帧的长度，进而提高了编码效率，同时减少了码流的比特开销。In order to determine whether the current point cloud frame is the first point cloud in the current point cloud group during decoding, and then divide the current point cloud group from other point cloud groups during the decoding process, referring to FIG. 10, the embodiment of the present application may After receiving the first frame point cloud of the current point cloud group, the auxiliary information encoding module 109 sets the first identifiers of all patches included in the first frame point cloud of the current point cloud group to the third value. In this way, when the current point cloud frame is the first point cloud in the current point cloud group, the first identifier of the first patch in the current point cloud frame is the third value. When the current point cloud frame is a non-first frame point cloud in the current point cloud group, the first identifier of the first patch in the current point cloud frame is the first value. That is, when the first identifier of the patch to be encoded is the third value, and the patch to be encoded is the first patch in the current point cloud frame, it can be determined that the current point cloud frame is the first point cloud in the current point cloud group . In this way, there is no need to include the length of the point cloud frame of the current point cloud group in the code stream, thereby improving the coding efficiency and reducing the bit overhead of the code stream.
基于上述描述，每帧点云中的第一类patch、第二类patch和第三类patch可以按照图8所示的方式进行排列，然后编码的时候也按照图8所示的顺序进行编码。但是，在某些情况下，可能需要对每帧点云包括的patch的顺序进行打乱编码，也即是，不再按照图8中的顺序进行编码，此时，为了便于解码，当第一标识为第一值时，编入码流的待编码patch的辅助信息中还可以包括待编码patch的参考patch的标识信息。其中，参考patch的标识信息用于在待编码patch的参考帧中唯一标识待编码patch的参考patch。示例性地，参考patch的标识信息可以为参考patch在参考帧中的位置编号。Based on the above description, the first-type patch, the second-type patch, and the third-type patch in each frame of point cloud can be arranged in the manner shown in FIG. 8, and then encoded in the order shown in FIG. 8 when encoding. However, in some cases, it may be necessary to scramble the order of the patches included in each frame of point cloud, that is, no longer code according to the order in Figure 8. At this time, in order to facilitate decoding, when the first When the identifier is the first value, the auxiliary information of the patch to be encoded encoded into the code stream may also include identification information of the reference patch of the patch to be encoded. The identification information of the reference patch is used to uniquely identify the reference patch of the patch to be encoded in the reference frame of the patch to be encoded. Exemplarily, the identification information of the reference patch may be the position number of the reference patch in the reference frame.
为了进一步减少码流的比特开销，当按照图8所示的顺序进行编码时，对于第三类patch而言，还可以采用除前文所述的编码方式之外的其他方式进行编码，比如，如果待编码patch为首个第三类patch，那么，可以按照前文所述的方式进行编码，如果待编码patch不是首个第三类patch，此时，可以根据首个第三类patch的二维信息和三维信息，采用差分编码方式将待编码patch的二维信息和三维信息编入码流，而其他的信息还采用前文所述的方式进行编码。其中，根据首个第三类patch的二维信息和三维信息，采用差分编码方式将待编码patch 的二维信息和三维信息编入码流是指：将待编码patch的二维信息和首个第三类patch的二维信息之间的差值，以及待编码patch的三维信息和首个第三类patch的三维信息之间的差值编入码流。In order to further reduce the bit overhead of the code stream, when encoding is performed in the order shown in Figure 8, for the third type of patch, other methods other than the encoding methods described above can also be used for encoding, for example, if The patch to be coded is the first type 3 patch, then it can be coded as described above. If the patch to be coded is not the first type 3 patch, at this time, it can be based on the two-dimensional information of the first type 3 patch and For the three-dimensional information, the two-dimensional information and the three-dimensional information of the patch to be encoded are encoded into the code stream using the differential encoding method, and other information is also encoded in the manner described above. Among them, according to the two-dimensional information and three-dimensional information of the first third-type patch, using differential encoding to encode the two-dimensional information and three-dimensional information of the patch to be encoded into the code stream means: the two-dimensional information and the first The difference between the two-dimensional information of the third-type patch and the difference between the three-dimensional information of the patch to be encoded and the first three-dimensional patch of the third type are encoded into the code stream.
在本申请实施例中，先获取待编码patch的辅助信息，然后将待编码patch的辅助信息和第一标识编入码流。由于第一标识的取值可以为第一值、第二值和第三值。不同的取值指示不同类型的patch。因此，可以通过第一标识将不同类型的patch进行区分，这样就不需要在码流中写入第一类patch的数量，减少码流的比特开销。而且，针对不同类型的patch，编入码流中的辅助信息包括的内容可以不相同，且对于第一类patch和第二类patch而言，只有部分辅助信息编入码流。这样，可以简化编入码流的格式，进一步减少码流的比特开销，从而提高编码的效率。In the embodiment of the present application, first obtain the auxiliary information of the patch to be encoded, and then encode the auxiliary information and the first identifier of the patch to be encoded into the code stream. Because the value of the first identifier can be the first value, the second value, and the third value. Different values indicate different types of patches. Therefore, different types of patches can be distinguished by the first identifier, so that there is no need to write the number of the first type of patches in the code stream, which reduces the bit overhead of the code stream. Moreover, for different types of patches, the content of the auxiliary information compiled into the code stream may be different, and for the first type of patch and the second type of patch, only part of the auxiliary information is compiled into the code stream. In this way, the format of the code stream can be simplified, and the bit overhead of the code stream can be further reduced, thereby improving the coding efficiency.
接下来对基于上述图9所示的实施例的点云编码方法对应的点云解码方法进行说明。需要说明的是，结合图1所示的点云译码***，以及图6所示的解码器200的示意性框图，下文中的任一种点云编码方法可以是点云译码***中的解码器200执行的，更具体地，可以是解码器200中的辅助信息解码模块204执行的。参见图11，该解码方法包括：Next, the point cloud decoding method corresponding to the point cloud encoding method based on the embodiment shown in FIG. 9 will be described. It should be noted that, in combination with the point cloud decoding system shown in FIG. 1 and the schematic block diagram of the decoder 200 shown in FIG. 6, any of the following point cloud encoding methods may be the point cloud decoding system. What is executed by the decoder 200, more specifically, may be executed by the auxiliary information decoding module 204 in the decoder 200. Referring to Figure 11, the decoding method includes:
S1101：从码流中解析待解码patch的第一标识。S1101: Parse the first identifier of the patch to be decoded from the code stream.
由于当前点云帧中的所有patch的第一标识可以各自打包并编入码流，也可以打包在一起并编入码流。因此，对于各自打包并编入码流的方式而言，如果待解码patch为当前点云帧中的首个patch，那么，可以从码流中解析待解码patch的第一标识。当然，对于非首个patch，也可以从码流中解析待解码patch的第一标识。也即是，可以依次解析当前点云帧中待解码的每个patch的第一标识。例如，依次解析出待解码patch[i-1]的第一标识和待解码patch[i-1]的辅助信息，待解码patch[i]的第一标识和待解码patch[i]的辅助信息，以及待解码patch[i+1]的第一标识和待解码patch[i+1]的辅助信息。对于打包在一起并编入码流的方式而言，如果待解码patch为当前点云帧中的首个patch，那么，可以从码流中解析当前点云帧包括的多个patch的第一标识，其中解析出的多个第一标识中包括待解码patch的第一标识。也即是，通过一次解析即可得到当前点云帧包括的多个patch的第一标识，而无需分多次进行解析。Because the first identifiers of all patches in the current point cloud frame can be individually packaged and compiled into a code stream, or they can be packaged together and compiled into a code stream. Therefore, for the respective methods of packaging and encoding into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, then the first identifier of the patch to be decoded can be parsed from the code stream. Of course, for the non-first patch, the first identifier of the patch to be decoded can also be parsed from the code stream. That is, the first identifier of each patch to be decoded in the current point cloud frame can be analyzed sequentially. For example, the first identifier of the patch[i-1] to be decoded and the auxiliary information of the patch[i-1] to be decoded are sequentially parsed, the first identifier of the patch[i] to be decoded and the auxiliary information of the patch[i] to be decoded , And the first identifier of the patch[i+1] to be decoded and the auxiliary information of the patch[i+1] to be decoded. For the method of packing together and compiling into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, then the first identification of multiple patches included in the current point cloud frame can be parsed from the code stream , Wherein the multiple first identifiers parsed include the first identifier of the patch to be decoded. That is, the first identifiers of multiple patches included in the current point cloud frame can be obtained through one analysis, without the need for multiple analysis.
作为一种示例，可以采用与编码端匹配的方法从码流中解析待解码patch的第一标识。例如可以采用定长解码方式从码流中解析待解码patch的第一标识。As an example, a method of matching with the encoding end may be used to parse the first identifier of the patch to be decoded from the code stream. For example, a fixed-length decoding method can be used to parse the first identifier of the patch to be decoded from the code stream.
S1102：当第一标识为第一值时，从码流中解析待解码patch的辅助信息，该辅助信息包括待解码patch的三维信息。S1102: When the first identifier is the first value, parse the auxiliary information of the patch to be decoded from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded.
需要说明的是，当第一标识为第一值时，表明待解码patch为第一类patch，也即待解码patch在当前点云组内的所有点云中都存在相匹配的patch。此时，可以从码流中解析待解码patch的辅助信息，该辅助信息包括待解码patch的三维信息，但不包括待解码patch的二维信息、旋转变换信息和法向坐标轴信息。It should be noted that when the first identifier is the first value, it indicates that the patch to be decoded is the first type of patch, that is, the patch to be decoded has a matching patch in all point clouds in the current point cloud group. At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream. The auxiliary information includes the three-dimensional information of the patch to be decoded, but does not include the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.
由于第一类patch在编码的过程中编入码流的辅助信息包括三维信息，但不包括二维信息、旋转变换信息和法向坐标轴信息，所以，当待解码patch为第一类patch时，从码流中解析的辅助信息包括待解码patch的三维信息，但不包括待解码patch的二维信息、旋转变换信息和法向坐标轴信息。此时，还可以根据待解码patch的参考patch的辅助信息，获取待解码patch的二维信息、旋转变换信息和法向坐标轴信息。Since the auxiliary information of the first type of patch that is encoded into the code stream during the encoding process includes three-dimensional information, but does not include two-dimensional information, rotation transformation information and normal coordinate axis information, when the patch to be decoded is the first type of patch The auxiliary information parsed from the code stream includes the three-dimensional information of the patch to be decoded, but does not include the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded. At this time, the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded can also be obtained according to the auxiliary information of the reference patch of the patch to be decoded.
作为一种示例，可以采用与编码端匹配的方法进行解码，例如可以采用差分解码方式从码流中解析待解码patch的三维信息。对于待解码patch的其他辅助信息，例如，待解码patch的二维信息、旋转变换信息和法向坐标轴信息，可以从已解码的点云中获取位置与待解码patch在当前点云帧中位置相同的patch的二维信息、旋转变换信息和法向坐标轴信息，并将获取的二维信息、旋转变换信息和法向坐标轴信息作为待解码patch的二维信息、旋转变换信息和法向坐标轴信息。或者，可以从当前点云帧的参考帧中获取待解码patch的参考patch的二维信息、旋转变换信息和法向坐标轴信息，并将参考patch的二维信息、旋转变换信息和法向坐标轴信息作为待解码patch的二维信息、旋转变换信息和法向坐标轴信息。As an example, a method matching the encoding end may be used for decoding, for example, a differential decoding method may be used to parse the three-dimensional information of the patch to be decoded from the code stream. For other auxiliary information of the patch to be decoded, for example, the two-dimensional information of the patch to be decoded, rotation transformation information and normal axis information, the position of the patch to be decoded can be obtained from the decoded point cloud and the position of the patch to be decoded in the current point cloud frame The two-dimensional information, rotation transformation information and normal coordinate axis information of the same patch, and the obtained two-dimensional information, rotation transformation information and normal coordinate axis information are used as the two-dimensional information, rotation transformation information and normal direction of the patch to be decoded Coordinate axis information. Alternatively, the two-dimensional information, rotation transformation information and normal coordinate axis information of the reference patch of the patch to be decoded can be obtained from the reference frame of the current point cloud frame, and the two-dimensional information, rotation transformation information and normal coordinate information of the reference patch can be obtained The axis information is used as the two-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded.
当点云中的patch按照图8所示的排列方式进行编码时，待解码patch的参考patch可以为当前点云帧的参考帧中位置与待解码patch在当前点云帧中的位置相同的patch。但是，基于上述描述，在某些情况下，当前点云组内的每帧点云中的第一类patch、第二类patch和第三类patch并没有按照图8所示的方式排列，此时，当第一标识为第一值时，编入码流的待解码patch的辅助信息中还可以包括待解码patch的参考patch的标识信息。这样，在进行解码时，当解析出的第一标识为第一值时，从码流中解析出的辅助信息中还可以包括待解码patch的参考patch的标识信息。之后，便可以根据待解码patch的参考patch的标识信息确定待解码patch的参考patch。When the patch in the point cloud is encoded according to the arrangement shown in Figure 8, the reference patch of the patch to be decoded can be the patch whose position in the reference frame of the current point cloud frame is the same as the position of the patch to be decoded in the current point cloud frame . However, based on the above description, in some cases, the first type of patch, the second type of patch, and the third type of patch in each frame of point cloud in the current point cloud group are not arranged in the manner shown in Figure 8. When the first identifier is the first value, the auxiliary information of the patch to be decoded encoded into the code stream may also include the identification information of the reference patch of the patch to be decoded. In this way, during decoding, when the parsed first identifier is the first value, the auxiliary information parsed from the code stream may also include identification information of the reference patch of the patch to be decoded. After that, the reference patch of the patch to be decoded can be determined according to the identification information of the reference patch of the patch to be decoded.
关于采用差分解码方式从码流中解析待解码patch的三维信息的方式，具体可以为：从已解码的点云中获取位置与待解码patch在当前点云帧中位置相同的patch的三维信息，或者，从当前点云帧的参考帧中获取待解码patch的参考patch的三维信息。然后，将获取的三维信息与从码流中解析出的三维信息差值进行相加，从而得到待解码patch的三维信息。Regarding the method of using differential decoding to parse the 3D information of the patch to be decoded from the code stream, it may specifically be: obtaining the 3D information of the patch whose position is the same as the position of the patch to be decoded in the current point cloud frame from the decoded point cloud. Or, obtain the three-dimensional information of the reference patch of the patch to be decoded from the reference frame of the current point cloud frame. Then, the obtained three-dimensional information is added with the difference of the three-dimensional information parsed from the code stream to obtain the three-dimensional information of the patch to be decoded.
S1103：当第一标识为第二值时，从码流中解析待解码patch的辅助信息，该辅助信息包括待解码patch的二维信息、三维信息和待解码patch的参考patch的标识信息。S1103: When the first identifier is the second value, parse the auxiliary information of the patch to be decoded from the code stream. The auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded and identification information of the reference patch of the patch to be decoded.
需要说明的是，当第一标识为第二值时，表明待解码patch为第二类patch，也即当前点云帧的参考帧中存在与待解码patch相匹配的参考patch。此时，可以从码流中解析待解码patch的辅助信息，该辅助信息包括二维信息、三维信息和待解码patch的参考patch的标识信息，但不包括旋转变换信息和法向坐标轴信息。It should be noted that when the first identifier is the second value, it indicates that the patch to be decoded is a second type of patch, that is, there is a reference patch matching the patch to be decoded in the reference frame of the current point cloud frame. At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream. The auxiliary information includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be decoded, but does not include rotation transformation information and normal coordinate axis information.
由于第二类patch在编码的过程中编入码流的辅助信息包括二维信息、三维信息和参考patch的标识信息，但不包括旋转变换信息和法向坐标轴信息，所以，当待解码patch为第二类patch时，从码流中解析的辅助信息包括待解码patch的二维信息、三维信息和待解码patch的参考patch的标识信息，但不包括待解码patch的旋转变换信息和法向坐标轴信息。此时，还可以根据待解码patch的参考patch的标识信息，获取待解码patch的旋转变换信息和法向坐标轴信息。Because the auxiliary information of the second type of patch that is encoded into the code stream during the encoding process includes two-dimensional information, three-dimensional information and identification information of the reference patch, but does not include the rotation transformation information and the normal coordinate axis information, so when the patch to be decoded When it is the second type of patch, the auxiliary information parsed from the code stream includes the two-dimensional information, three-dimensional information of the patch to be decoded and the identification information of the reference patch of the patch to be decoded, but does not include the rotation transformation information and normal direction of the patch to be decoded Coordinate axis information. At this time, the rotation transformation information and normal coordinate axis information of the patch to be decoded can also be obtained according to the identification information of the reference patch of the patch to be decoded.
其中，对于待解码patch的参考patch的标识信息，可以采用与编码端匹配的方法从码流中解析，例如可以采用定长解码方式来解析。对于待解码patch的旋转变换信息和法向坐标轴信息，可以通过待解码patch的参考patch的标识信息确定待解码patch的参考patch，进而获取待解码patch的参考patch的旋转变换信息和法向坐标轴信息，并将参考patch的旋转变换信息和法向坐标轴信息作为待解码patch的旋转变换信息和法向坐标轴信息。对于待解码patch的二维信息和三维信息，作为一种示例，可以采用与编码端匹配的方法进行解码，例如可以差分解码方式从码流中解析待解码patch的二维信息和三维信息。具体地，可以通过待 解码patch的参考patch的标识信息确定待解码patch的参考patch，进而获取待解码patch的参考patch的二维信息和三维信息。然后，将获取的二维信息和三维信息分别与从码流中解析出的二维信息差值和三维信息差值对应相加，从而得到待解码patch的二维信息和三维信息。Among them, the identification information of the reference patch of the patch to be decoded can be parsed from the code stream by a method matching the encoding end, for example, a fixed-length decoding method can be used for analysis. For the rotation transformation information and normal coordinate axis information of the patch to be decoded, the reference patch of the reference patch to be decoded can be determined through the identification information of the reference patch to be decoded, and then the rotation transformation information and normal coordinates of the reference patch to be decoded are obtained. Axis information, and use the rotation transformation information and normal coordinate axis information of the reference patch as the rotation transformation information and normal coordinate axis information of the patch to be decoded. For the two-dimensional information and three-dimensional information of the patch to be decoded, as an example, a method that matches the encoding end can be used for decoding. For example, the two-dimensional information and three-dimensional information of the patch to be decoded can be parsed from the code stream in a differential decoding manner. Specifically, the reference patch of the patch to be decoded can be determined by the identification information of the reference patch of the patch to be decoded, and then the two-dimensional information and three-dimensional information of the reference patch of the patch to be decoded can be obtained. Then, the obtained two-dimensional information and three-dimensional information are respectively added with the two-dimensional information difference and the three-dimensional information difference parsed from the code stream, so as to obtain the two-dimensional information and the three-dimensional information of the patch to be decoded.
S1104：当第一标识为第三值时，从码流中解析待解码patch的辅助信息，该辅助信息包括待解码patch的二维信息、三维信息、旋转变换信息和法向坐标轴信息。S1104: When the first identifier is the third value, parse the auxiliary information of the patch to be decoded from the code stream. The auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.
需要说明的是，当第一标识为第三值时，表明待解码patch为第三类patch，也即当前点云组中不存在与待解码patch相匹配的参考patch。此时，可以从码流中解析出待解码patch的二维信息、三维信息、旋转变换信息和法向坐标轴信息。It should be noted that when the first identifier is the third value, it indicates that the patch to be decoded is a third type of patch, that is, there is no reference patch matching the patch to be decoded in the current point cloud group. At this point, the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded can be parsed from the code stream.
其中，第一值、第二值和第三值可以分别是二进制的10、11和00，或者，第一值、第二值和第三值可以分别是二进制的11、10和00等，本申请实施例对此不做限定。Among them, the first value, the second value, and the third value may be binary 10, 11, and 00 respectively, or the first value, the second value, and the third value may be binary 11, 10, and 00, etc., respectively. The application embodiment does not limit this.
基于上述描述，当前点云组内的每帧点云中的第一类patch、第二类patch和第三类patch可以按照如图8所示的方式排列，此时，如果待解码patch为当前点云帧中的首个patch，那么，从码流中解析待解码patch的第一标识之后，即可根据第一标识的取值确定当前点云帧是否为当前点云组内的首帧点云。也即是，当待解码patch为当前点云帧中的首个patch，且第一标识为第三值时，可以确定当前点云帧为当前点云组内的首帧点云。Based on the above description, the first type patch, the second type patch, and the third type patch in each frame of point cloud in the current point cloud group can be arranged as shown in Figure 8. At this time, if the patch to be decoded is the current The first patch in the point cloud frame, then, after parsing the first identifier of the patch to be decoded from the code stream, you can determine whether the current point cloud frame is the first point in the current point cloud group according to the value of the first identifier cloud. That is, when the patch to be decoded is the first patch in the current point cloud frame, and the first identifier is the third value, it can be determined that the current point cloud frame is the first point cloud in the current point cloud group.
但是，在某些情况下，当前点云组内的每帧点云中的第一类patch、第二类patch和第三类patch并没有按照图8所示的方式排列，也即是，当前点云组中的任一帧点云中的首个patch的第一标识可以为第一值、第二值或第三值。这样，当待解码patch为当前点云帧中的首个patch时，无法根据从码流中解析的第一标识确定当前点云帧是否为当前点云组内的首帧点云。此时，可以确定当前点云帧包括的多个patch的第一标识。在一种可能的实现方式中，该多个patch即是待解码patch包括的所有patch。如果该多个patch的第一标识均为第三值时，则可以确定当前点云帧为当前点云组内的首帧点云。However, in some cases, the first type of patch, the second type of patch, and the third type of patch in each frame of point cloud in the current point cloud group are not arranged in the way shown in Figure 8, that is, the current point cloud group The first identifier of the first patch in the point cloud of any frame in the point cloud group may be the first value, the second value, or the third value. In this way, when the patch to be decoded is the first patch in the current point cloud frame, it is impossible to determine whether the current point cloud frame is the first point cloud in the current point cloud group according to the first identifier parsed from the code stream. At this time, the first identifiers of multiple patches included in the current point cloud frame can be determined. In a possible implementation, the multiple patches are all the patches included in the patch to be decoded. If the first identifiers of the multiple patches are all the third value, it can be determined that the current point cloud frame is the first point cloud in the current point cloud group.
对于第一标识为第三值的情况，如果首个第三类patch的辅助信息按照定长编码方式进行编码，非首个第三类patch的二维信息和三维信息采用差分编码方式进行编码，其他辅助信息采用定长编码方式进行编码。那么，在本步骤中，对于非首个第三类patch来说，可以采用差分解码方式从码流中解析待解码patch的二维信息和三维信息，采用定长解码方式从码流中解析待解码patch的旋转变换信息和法向坐标轴信息。其中，对于非首个第三类patch来说，采用差分解码方式从码流中解析待解码patch的二维信息和三维信息的方式为：从码流中解析待解码patch的二维信息与首个第三类patch的二维信息之间的差值，将该差值与首个第三类patch的二维信息相加，得到待解码patch的二维信息。从码流中解析待解码patch的三维信息与首个第三类patch的三维信息之间的差值，将该差值与首个第三类patch的三维信息相加，得到待解码patch的三维信息。For the case where the first identifier is the third value, if the auxiliary information of the first third-type patch is encoded according to the fixed-length encoding method, the two-dimensional information and three-dimensional information of the non-first third-type patch are encoded by differential encoding. Other auxiliary information is coded using fixed-length coding. Then, in this step, for non-first type III patches, differential decoding can be used to parse the two-dimensional information and three-dimensional information of the patch to be decoded from the code stream, and fixed-length decoding can be used to parse the to-be-decoded patch from the code stream. Decode the rotation transformation information and normal axis information of the patch. Among them, for the non-first third-type patch, the method of using differential decoding to parse the two-dimensional information and three-dimensional information of the patch to be decoded from the code stream is: parse the two-dimensional information and the first information of the patch to be decoded from the code stream. The difference between the two-dimensional information of the third-type patch is added to the two-dimensional information of the first third-type patch to obtain the two-dimensional information of the patch to be decoded. Analyze the difference between the 3D information of the patch to be decoded and the 3D information of the first third type patch from the code stream, and add the difference to the 3D information of the first third type patch to obtain the 3D information of the patch to be decoded information.
S1105：根据待解码patch的辅助信息，重构待解码patch。S1105: According to the auxiliary information of the patch to be decoded, reconstruct the patch to be decoded.
需要说明的是，在获取待解码patch的辅助信息之后，也即获取到了待解码patch的二维信息、三维信息、旋转变换信息和法向坐标轴信息之后，可以根据但不限于这些辅助信息重构待解码patch，本申请对重构待解码patch需要的具体信息不做限定。It should be noted that after obtaining the auxiliary information of the patch to be decoded, that is, after obtaining the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded, the auxiliary information can be reproduced according to but not limited to these auxiliary information. To construct the patch to be decoded, this application does not limit the specific information needed to reconstruct the patch to be decoded.
在本申请实施例中，当第一标识为第一值时，表明待解码patch为第一类patch，此时可以从码流中解析待解码patch的辅助信息，该辅助信息包括待解码patch的三维信息。当第一 标识为第二值时，表明待解码patch为第二类patch，此时可以从码流中解析待解码patch的辅助信息，该辅助信息包括待解码patch的二维信息、三维信息和待解码patch的参考patch的标识信息。当第一标识为第三值时，表明待解码patch为第三类patch，此时可以从码流中解析待解码patch的辅助信息，该辅助信息包括待解码patch的二维信息、三维信息、旋转变换信息和法向坐标轴信息。也即是，对于第一类patch和第二类patch而言，只需要从码流中解析部分辅助信息，其他的辅助信息可以推导或者获取得到，这样，可以简化解析码流的格式，减少码流的比特开销，从而提高解码的效率。而且，本申请实施例可以通过第一标识的取值指示不同类型的patch，也即是，可以区分第一类patch、第二类patch和第三类patch，这样就不需要在码流中解析第一类patch的数量，进一步减少码流的比特开销，提高解码的效率。In the embodiment of the present application, when the first identifier is the first value, it indicates that the patch to be decoded is the first type of patch. At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream. The auxiliary information includes the patch to be decoded. Three-dimensional information. When the first identifier is the second value, it indicates that the patch to be decoded is a second type of patch. At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream. The auxiliary information includes the two-dimensional information, three-dimensional information and information of the patch to be decoded. The identification information of the reference patch of the patch to be decoded. When the first identifier is the third value, it indicates that the patch to be decoded is a third type of patch. At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream. The auxiliary information includes the two-dimensional information, three-dimensional information, and Rotation transformation information and normal coordinate axis information. That is, for the first type of patch and the second type of patch, only part of the auxiliary information needs to be parsed from the code stream, and other auxiliary information can be derived or obtained. In this way, the format of the parsed code stream can be simplified and the code can be reduced. Stream bit overhead, thereby improving decoding efficiency. Moreover, the embodiment of the present application can indicate different types of patches through the value of the first identifier, that is, it can distinguish between the first type of patch, the second type of patch, and the third type of patch, so there is no need to parse in the code stream. The number of the first type of patch further reduces the bit overhead of the code stream and improves the decoding efficiency.
以下，对本申请实施例提供的第二种点云编码方法进行说明。需要说明的是，结合图1所示的点云译码***，以及图2所示的编码器100的示意性框图，下文中的任一种点云编码方法可以是点云译码***中的编码器100执行的，更具体地，可以是编码器100中的辅助信息编码模块109执行的。Hereinafter, the second point cloud coding method provided by the embodiment of the present application will be described. It should be noted that, in combination with the point cloud decoding system shown in FIG. 1 and the schematic block diagram of the encoder 100 shown in FIG. 2, any of the following point cloud encoding methods may be the point cloud decoding system. The execution by the encoder 100, more specifically, may be executed by the auxiliary information encoding module 109 in the encoder 100.
参见图12，该方法包括：Referring to Figure 12, the method includes:
S1201：获取待编码patch的辅助信息。S1201: Obtain auxiliary information of the patch to be encoded.
需要说明的时，S1201的内容可以参考上述图11所示的实施例中的S1101，本申请实施例对此不再赘述。When it needs to be explained, the content of S1201 can refer to S1101 in the embodiment shown in FIG. 11, which will not be repeated in this embodiment of the present application.
S1202：将待编码patch的辅助信息和待编码patch的第一标识编入码流。S1202: Coding the auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded into the code stream.
由于通常情况下，第一类patch在当前点云组内的所有点云中都存在具有匹配关系的patch，且当前点云组内的第一类patch之间的二维信息、旋转变换信息和法向坐标轴信息相同，而三维信息可能不相同，但是在某些情况下，第一类patch的三维信息也可以相同。在这种情况下，当第一标识为第一值时，可以指示待编码patch为第一类patch，且编入码流的信息还可以包括第二标识。Generally, the first-type patch has matching patches in all the point clouds in the current point cloud group, and the two-dimensional information, rotation transformation information, and information between the first-type patches in the current point cloud group The normal coordinate axis information is the same, but the three-dimensional information may be different, but in some cases, the three-dimensional information of the first type of patch may also be the same. In this case, when the first identifier is the first value, it may indicate that the patch to be encoded is a first-type patch, and the information encoded into the code stream may also include the second identifier.
需要说明的是，第二标识用于指示待编码patch的三维信息是否已编入码流。第二标识的语法元素可以是override_3d_shift_data_flag，也即是override_3d_shift_data_flag可以称为第三语法元素。第二标识为第四值时，可以指示待编码patch的三维信息未编入码流。也即是，待编码patch的三维信息与待编码patch的参考patch的三维信息相同，因此可以不用将待编码patch的三维信息再次编入码流，也即待编码patch的辅助信息可以不包括三维信息。第二标识为第五值时，可以指示待编码patch的三维信息已编入码流。也即是，待编码patch的三维信息与待编码patch的参考patch的三维信息不相同，因此需要将待编码patch的三维信息编入码流，也即待编码patch的辅助信息可以包括三维信息。具体编码方法在此不做限定，例如可以采用差分编码方式，也可以采用定长编码方式。其中，第四值的语法元素可以是false，第五值的语法元素可以是true。It should be noted that the second identifier is used to indicate whether the three-dimensional information of the patch to be encoded has been encoded into the code stream. The syntax element of the second identifier may be override_3d_shift_data_flag, that is, override_3d_shift_data_flag may be referred to as the third syntax element. When the second identifier is the fourth value, it can indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is the same as the three-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the three-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may not include the three-dimensional information. information. When the second identifier is the fifth value, it can indicate that the three-dimensional information of the patch to be encoded has been encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is different from the three-dimensional information of the reference patch of the patch to be encoded. Therefore, it is necessary to encode the three-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may include three-dimensional information. The specific encoding method is not limited here. For example, a differential encoding method may be used, or a fixed-length encoding method may be used. The syntax element of the fourth value may be false, and the syntax element of the fifth value may be true.
作为一种示例，本文示例一种待编码patch的第一标识为第一值，且编入码流的信息包括第二标识时的部分语法结构，参见表1和表3所示。As an example, this article exemplifies a partial syntax structure when the first identifier of the patch to be encoded is the first value, and the information encoded into the code stream includes the second identifier, see Table 1 and Table 3.
表3table 3
Figure PCTCN2020080140-appb-000004
Figure PCTCN2020080140-appb-000004
Figure PCTCN2020080140-appb-000005
Figure PCTCN2020080140-appb-000005
进一步地，在一种可能的情况中，待编码patch的第一标识为第一值时，也即待编码patch为第一类patch时，待编码patch的三维信息与参考patch的三维信息，以及待编码patch的二维信息与参考patch的二维信息都不相同。这样，当第一标识为第一值时，编入码流的信息除了包括第二标识之外，还可以包括用于指示待编码patch的二维信息是否已编入码流的第三标识。第三标识的语法元素可以是override_2d_shift_data_flag，也即是override_2d_shift_data_flag可以称为第二语法元素。其中，第二标识已做过描述，所以此处不再赘述。Further, in a possible situation, when the first identifier of the patch to be encoded is the first value, that is, when the patch to be encoded is the first type of patch, the three-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch, and The two-dimensional information of the patch to be encoded is different from the two-dimensional information of the reference patch. In this way, when the first identifier is the first value, in addition to the second identifier, the information encoded into the code stream may also include a third identifier for indicating whether the two-dimensional information of the patch to be encoded has been encoded into the code stream. The syntax element of the third identifier may be override_2d_shift_data_flag, that is, override_2d_shift_data_flag may be referred to as the second syntax element. Among them, the second identifier has already been described, so it will not be repeated here.
第三标识为第六值时，可以指示待编码patch的二维信息未编入码流。也即是，待编码patch的二维信息与待编码patch的参考patch的二维信息相同，因此可以不用将待编码patch的二维信息再次编入码流，也即待编码patch的辅助信息可以不包括二维信息。第三标识为第七值时，可以指示待编码patch的二维信息已编入码流。也即是，待编码patch的二维信息与待编码patch的参考patch的二维信息不相同，因此需要将待编码patch的二维信息编入码流，也即待编码patch的辅助信息可以包括二维信息。其中，第六值的语法元素可以是false，第七值的语法元素可以是true。When the third identifier is the sixth value, it may indicate that the two-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the two-dimensional information of the patch to be encoded is the same as the two-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the two-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded can be Does not include two-dimensional information. When the third identifier is the seventh value, it can indicate that the two-dimensional information of the patch to be encoded has been encoded into the code stream. That is, the two-dimensional information of the patch to be encoded is not the same as the two-dimensional information of the reference patch of the patch to be encoded. Therefore, it is necessary to encode the two-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may include Two-dimensional information. The syntax element of the sixth value may be false, and the syntax element of the seventh value may be true.
作为一种示例，本文示例一种待编码patch的第一标识为第一值，且编入码流的信息包括第二标识和第三标识时的部分语法结构，参见表1和表4所示。As an example, this article illustrates a partial syntax structure when the first identifier of the patch to be encoded is the first value, and the information encoded into the code stream includes the second identifier and the third identifier, see Table 1 and Table 4 .
表4Table 4
Figure PCTCN2020080140-appb-000006
Figure PCTCN2020080140-appb-000006
Figure PCTCN2020080140-appb-000007
Figure PCTCN2020080140-appb-000007
当待编码patch的三维信息与参考patch的三维信息，以及待编码patch的二维信息与参考patch的二维信息都不相同时，不仅可以通过上述第二标识和第三标识来指示编入码流的信息中是否包括待解码patch的二维信息和三维信息。还可以通过其他的方式来指示，比如，还可以通过第四标识来指示待编码patch的二维信息和三维信息是否均已编入码流，也即是，当第一标识为第一值时，可以指示待编码patch为第一类patch，且编入码流的信息还可以包括第四标识。第四标识的语法元素可以是override_patch_data_flag。当第四标识为第八值时，可以指示待解码patch的二维信息和三维信息均与待解码patch的参考patch的二维信息和三维信息相同，且未编入码流。当第四标识为第九值时，可以指示待解码patch的二维信息和三维信息均与待解码patch的参考patch的二维信息和三维信息不相同，且均已编入码流。其中，第八值的语法元素可以为flase，第九值的语法元素可以为true。When the three-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch, and the two-dimensional information of the patch to be encoded and the two-dimensional information of the reference patch are not the same, not only the above-mentioned second and third marks can be used to indicate the coding Whether the information of the stream includes 2D information and 3D information of the patch to be decoded. It can also be indicated in other ways. For example, the fourth identifier can also be used to indicate whether the two-dimensional information and three-dimensional information of the patch to be encoded have been encoded into the code stream, that is, when the first identifier is the first value , It can indicate that the patch to be encoded is the first type of patch, and the information encoded into the code stream can also include a fourth identifier. The fourth identified syntax element may be override_patch_data_flag. When the fourth identifier is the eighth value, it may indicate that the two-dimensional information and three-dimensional information of the patch to be decoded are the same as the two-dimensional information and three-dimensional information of the reference patch of the patch to be decoded, and the code stream is not included. When the fourth identifier is a ninth value, it can indicate that the two-dimensional information and three-dimensional information of the patch to be decoded are different from the two-dimensional information and three-dimensional information of the reference patch of the patch to be decoded, and both have been encoded into the code stream. The syntax element of the eighth value may be flase, and the syntax element of the ninth value may be true.
其中，由于待编码patch的三维信息可以包括待编码patch在3D空间中沿着切线方向的偏移(3d_shift_tangent_axis)、待编码patch在3D空间中沿着双切线方向的偏移(3d_shift_bitangent_axis)，以及待编码patch在3D空间中沿着法线方向的偏移(3d_shift_normal_axis)。待编码patch的二维信息可以包括待编码patch在当前点云帧占用图中沿u轴方向的偏移(2d_shift_u)、待编码patch在当前点云帧占用图中沿v轴方向的偏移(2d_shift_v)、待编码patch在当前点云帧占用图中的宽度(2d_size_u)，以及待编码patch在当前点云帧占用图中的高度(2d_size_v)。所以，还可以通过用于指示3d_shift_tangent_axis是否已编入码流的标识(override_3d_shift_tangent_axis)、用于指示3d_shift_bitangent_axis是否已编入码流的标识(override_3d_shift_bitangent_axis)和用于指示3d_shift_normal_axis是否已编入码流的标识(override_3d_shift_normal_axis)来指示待解码patch的三维信息是否已编入码流，以及通过用于指示2d_shift_u是否已编入码流的标识(override_2d_shift_u)、用于 指示2d_shift_v是否已编入码流的标识(override_2d_shift_v)、用于指示2d_size_u是否已编入码流的标识(override_2d_size_u)和用于指示2d_size_v是否已编入码流的标识(override_2d_size_v)来指示待解码patch的二维信息是否已编入码流。与第二标识、第三标识和第四标识类似地，上述各个标识均可以有两种取值，这两种取值的语法元素可以为true和false。每种取值的含义可以参考第二标识、第三标识或第四标识的不同取值的含义，此处不再赘述。Among them, because the three-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the 3D space along the tangent direction (3d_shift_tangent_axis), the offset of the patch to be encoded in the 3D space along the bitangent direction (3d_shift_bitangent_axis), and The offset of the encoded patch in the 3D space along the normal direction (3d_shift_normal_axis). The two-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the current point cloud frame occupancy map along the u-axis direction (2d_shift_u), and the offset of the patch to be encoded in the current point cloud frame occupancy map along the v-axis direction ( 2d_shift_v), the width of the patch to be encoded in the current point cloud frame occupancy map (2d_size_u), and the height of the patch to be encoded in the current point cloud frame occupancy map (2d_size_v). Therefore, it is also possible to pass the flag (override_3d_shift_tangent_axis) used to indicate whether 3d_shift_tangent_axis has been coded into the code stream, the flag used to indicate whether 3d_shift_bitangent_axis has been coded into the code stream (override_3d_shift_bitangent_axis), and the flag used to indicate whether 3d_shift_normal_axis has been coded override_3d_shift_normal_axis) to indicate whether the three-dimensional information of the patch to be decoded has been encoded into the code stream, and through the identifier (override_2d_shift_u) used to indicate whether 2d_shift_u has been encoded into the code stream, and the identifier used to indicate whether 2d_shift_v has been encoded into the code stream (override_2d_shift_v) An identifier (override_2d_size_u) used to indicate whether 2d_size_u has been encoded into the code stream and an identifier (override_2d_size_v) used to indicate whether 2d_size_v has been encoded into the code stream to indicate whether the two-dimensional information of the patch to be decoded has been encoded into the code stream. Similar to the second identifier, the third identifier, and the fourth identifier, each of the above identifiers can have two values, and the syntax elements of the two values can be true and false. The meaning of each value can refer to the meaning of different values of the second identifier, the third identifier or the fourth identifier, which will not be repeated here.
作为一种示例，本文示例一种待编码patch的第一标识为第一值，且编入码流的信息包括第四标识，以及分别用于指示不同的二维信息和三维信息的标识时的部分语法结构，参见表1和表5所示。As an example, this article exemplifies when the first identifier of the patch to be encoded is the first value, and the information encoded into the code stream includes the fourth identifier, and the identifier used to indicate different two-dimensional information and three-dimensional information. Part of the grammatical structure, see Table 1 and Table 5.
表5table 5
Figure PCTCN2020080140-appb-000008
Figure PCTCN2020080140-appb-000008
Figure PCTCN2020080140-appb-000009
Figure PCTCN2020080140-appb-000009
其中，第一标识为第二值时，可以指示待编码patch为第二类patch。也即当前点云帧的参考帧中存在与待编码patch相匹配的参考patch。待编码patch的辅助信息包括二维信息、三维信息和待编码patch的参考patch的标识信息。Wherein, when the first identifier is the second value, it may indicate that the patch to be encoded is a second-type patch. That is, there is a reference patch that matches the patch to be encoded in the reference frame of the current point cloud frame. The auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded.
其中，第一标识为第三值时，可以指示待编码patch为第三类patch。也即当前点云帧的参考帧中不存在与待编码patch相匹配的参考patch。待编码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。Wherein, when the first identifier is a third value, it may indicate that the patch to be encoded is a third-type patch. That is, there is no reference patch that matches the patch to be encoded in the reference frame of the current point cloud frame. The auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
在本申请实施例中，第一标识的取值可以为第一值、第二值和第三值。不同的取值指示不同类型的patch。因此，可以通过第一标识将不同类型的patch进行区分，这样就不需要在码流中写入第一类patch的数量，减少码流的比特开销。而且，当第一标识为第一值时，编入码流的信息还可以包括第二标识，以指示待编码patch的三维信息是否已编入码流。当第二标识为第四值时，表明待编码patch的三维信息未编入码流。也即是，待编码patch的三维信息与待编码patch的参考patch的三维信息相同，因此可以不用将待编码patch的三维信息再次编入码流，也即待编码patch的辅助信息可以不包括三维信息，这样，可以进一步减少码流的比特开销。且对于第一类patch和第二类patch而言，只有部分辅助信息编入码流。这样，可以简化编入码流的格式，进一步减少码流的比特开销，从而提高编码的效率。In the embodiment of the present application, the value of the first identifier may be the first value, the second value, and the third value. Different values indicate different types of patches. Therefore, different types of patches can be distinguished by the first identifier, so that there is no need to write the number of the first type of patches in the code stream, which reduces the bit overhead of the code stream. Moreover, when the first identifier is the first value, the information encoded into the code stream may also include a second identifier to indicate whether the three-dimensional information of the patch to be encoded has been encoded into the code stream. When the second identifier is the fourth value, it indicates that the three-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is the same as the three-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the three-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may not include the three-dimensional information. Information, in this way, can further reduce the bit overhead of the code stream. And for the first type of patch and the second type of patch, only part of the auxiliary information is encoded into the code stream. In this way, the format of the code stream can be simplified, and the bit overhead of the code stream can be further reduced, thereby improving the coding efficiency.
接下来对基于上述图12所示的实施例的点云编码方法的点云解码方法进行说明。需要说明的是，结合图1所示的点云译码***，以及图6所示的解码器200的示意性框图，下文中的任一种点云编码方法可以是点云译码***中的解码器200执行的，更具体地，可以是解码器200中的辅助信息解码模块204执行的。参见图13，该解码方法包括：Next, the point cloud decoding method based on the point cloud encoding method of the embodiment shown in FIG. 12 will be described. It should be noted that, in combination with the point cloud decoding system shown in FIG. 1 and the schematic block diagram of the decoder 200 shown in FIG. 6, any of the following point cloud encoding methods may be the point cloud decoding system. What is executed by the decoder 200, more specifically, may be executed by the auxiliary information decoding module 204 in the decoder 200. Referring to Figure 13, the decoding method includes:
S1301：从码流中解析待解码patch的第一标识。S1301: Parse the first identifier of the patch to be decoded from the code stream.
需要说明的是，S1301的内容可以参考上述S1101的内容，此处不再赘述。It should be noted that the content of S1301 can refer to the content of S1101, which will not be repeated here.
S1302：当第一标识为第一值时，从码流中解析待解码patch的第二标识。S1302: When the first identifier is the first value, parse the second identifier of the patch to be decoded from the code stream.
需要说明的是，第二标识已在上述S1201中做过说明，所以此处不再赘述。It should be noted that the second identifier has been described in S1201 above, so it will not be repeated here.
S1303：当第二标识为第四值时，将待解码patch的参考patch的辅助信息作为待解码patch的辅助信息；当第二标识为第五值时，从码流中解析待解码patch的辅助信息，该辅助信息包括待解码patch的三维信息。S1303: When the second identifier is the fourth value, use the auxiliary information of the reference patch of the patch to be decoded as the auxiliary information of the patch to be decoded; when the second identifier is the fifth value, parse the auxiliary information of the patch to be decoded from the code stream The auxiliary information includes the three-dimensional information of the patch to be decoded.
需要说明的是，第二标识为第四值时，可以指示待解码patch的三维信息未编入码流。也即是，待解码patch的三维信息与待解码patch的参考patch的三维信息相同，此时可以将待解码patch的参考patch的辅助信息作为待解码patch的辅助信息。第二标识为第五值时， 可以指示待解码patch的三维信息已编入码流，待解码patch的辅助信息包括三维信息。此时，从码流中解析待解码patch的辅助信息，辅助信息包括待解码patch的三维信息。作为一种示例，可以采用差分解码方式从码流中解析待解码patch的辅助信息。具体方式可以参考前文。It should be noted that when the second identifier is the fourth value, it may indicate that the three-dimensional information of the patch to be decoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be decoded is the same as the three-dimensional information of the reference patch of the patch to be decoded. In this case, the auxiliary information of the reference patch of the patch to be decoded can be used as the auxiliary information of the patch to be decoded. When the second identifier is the fifth value, it may indicate that the three-dimensional information of the patch to be decoded has been encoded into the bitstream, and the auxiliary information of the patch to be decoded includes the three-dimensional information. At this time, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded. As an example, a differential decoding method can be used to parse the auxiliary information of the patch to be decoded from the code stream. The specific method can refer to the previous article.
在一些实施例中，基于步骤1202的描述，在待编码patch的三维信息与参考patch的三维信息，以及待编码patch的二维信息与参考patch的二维信息都不相同的情况下，编入码流的信息除了包括第二标识之外，还可以包括用于指示待编码patch的二维信息是否已编入码流的第三标识。也即是，当第一标识为第一值时，除了从码流中解析待解码patch的第二标识之外，还可以从码流中解析待解码patch的第三标识。第三标识为第六值时，可以指示待编码patch的二维信息未编入码流。也即是，待编码patch的二维信息与待编码patch的参考patch的二维信息相同，因此可以将待编码patch的参考patch的二维信息作为待编码patch的二维信息。当第三标识为第七值时，可以指示待编码patch的二维信息已编入码流。也即是，待编码patch的二维信息与待编码patch的参考patch的二维信息不相同，因此可以从码流中解析待解码patch二维信息。具体解码方式可以为与编码方法匹配的解码方法，此处不再赘述。In some embodiments, based on the description of step 1202, when the three-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch, and the two-dimensional information of the patch to be encoded and the two-dimensional information of the reference patch are not the same, the In addition to the second identifier, the information of the codestream may also include a third identifier for indicating whether the two-dimensional information of the patch to be encoded has been encoded into the codestream. That is, when the first identifier is the first value, in addition to parsing the second identifier of the patch to be decoded from the code stream, the third identifier of the patch to be decoded can also be parsed from the code stream. When the third identifier is the sixth value, it may indicate that the two-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the two-dimensional information of the patch to be encoded is the same as the two-dimensional information of the reference patch of the patch to be encoded, so the two-dimensional information of the reference patch of the patch to be encoded can be used as the two-dimensional information of the patch to be encoded. When the third identifier is the seventh value, it can indicate that the two-dimensional information of the patch to be encoded has been encoded into the code stream. That is, the two-dimensional information of the patch to be encoded is different from the two-dimensional information of the reference patch of the patch to be encoded, so the two-dimensional information of the patch to be decoded can be parsed from the code stream. The specific decoding method may be a decoding method that matches the encoding method, which will not be repeated here.
在另一些实施例中，基于步骤1202的描述，在待编码patch的三维信息与参考patch的三维信息，以及待编码patch的二维信息与参考patch的二维信息都不相同的情况下，不仅可以通过上述第二标识和第三标识来指示编入码流的信息中是否包括待解码patch的二维信息和三维信息。还可以通过其他的方式来指示，比如，还可以通过第四标识来指示待编码patch的二维信息和三维信息是否均已编入码流。也即是，当第一标识为第一值时，可以从码流中解析待解码patch的第四标识。第四标识为第八值时，可以指示待解码patch的二维信息和三维信息均与待解码patch的参考patch的二维信息和三维信息相同，且未编入码流，因此可以将待编码patch的参考patch的二维信息和三维信息作为待编码patch的二维信息和三维信息。当第四标识为第九值时，可以指示待解码patch的二维信息和三维信息均与待解码patch的参考patch的二维信息和三维信息不相同，且均已编入码流，因此可以从码流中解析待解码patch二维信息和三维信息。具体解码方式可以为与编码方法匹配的解码方法，此处不再赘述。In other embodiments, based on the description of step 1202, when the three-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch, and the two-dimensional information of the patch to be encoded and the two-dimensional information of the reference patch are different, not only The above-mentioned second identifier and third identifier can be used to indicate whether the information compiled into the code stream includes the two-dimensional information and three-dimensional information of the patch to be decoded. It can also be indicated in other ways, for example, the fourth identifier can also be used to indicate whether the two-dimensional information and three-dimensional information of the patch to be encoded have been encoded into the code stream. That is, when the first identifier is the first value, the fourth identifier of the patch to be decoded can be parsed from the code stream. When the fourth identifier is the eighth value, it can indicate that the two-dimensional information and three-dimensional information of the patch to be decoded are the same as the two-dimensional information and three-dimensional information of the reference patch of the patch to be decoded, and the code stream is not encoded. The two-dimensional information and three-dimensional information of the reference patch of the patch are used as the two-dimensional information and three-dimensional information of the patch to be encoded. When the fourth identifier is the ninth value, it can indicate that the two-dimensional information and three-dimensional information of the patch to be decoded are different from the two-dimensional information and three-dimensional information of the reference patch of the patch to be decoded, and both have been encoded into the code stream. Parse the two-dimensional information and three-dimensional information of the patch to be decoded from the code stream. The specific decoding method may be a decoding method that matches the encoding method, which will not be repeated here.
并且，对于通过用于指示3d_shift_tangent_axis是否已编入码流的标识(override_3d_shift_tangent_axis)、用于指示3d_shift_bitangent_axis是否已编入码流的标识(override_3d_shift_bitangent_axis)和用于指示3d_shift_normal_axis是否已编入码流的标识(override_3d_shift_normal_axis)来指示待解码patch的三维信息是否已编入码流，以及通过用于指示2d_shift_u是否已编入码流的标识(override_2d_shift_u)、用于指示2d_shift_v是否已编入码流的标识(override_2d_shift_v)、用于指示2d_size_u是否已编入码流的标识(override_2d_size_u)和用于指示2d_size_v是否已编入码流的标识(override_2d_size_v)来指示待解码patch的二维信息是否已编入码流的方式，与上述第二标识、第三标识和第四标识类似，此处不再赘述。In addition, the identification (override_3d_shift_tangent_axis) used to indicate whether 3d_shift_tangent_axis has been coded into the code stream, the flag (override_3d_shift_bitangent_axis) used to indicate whether the 3d_shift_bitangent_axis has been coded into the code stream, and the flag (override_shift_normal_axis) used to indicate whether 3d_shift_normal_axis has been coded into the code stream ) To indicate whether the three-dimensional information of the patch to be decoded has been encoded into the code stream, and the identifier used to indicate whether 2d_shift_u has been encoded into the code stream (override_2d_shift_u), the identifier used to indicate whether 2d_shift_v has been encoded into the code stream (override_2d_shift_v), The flag used to indicate whether 2d_size_u has been encoded into the code stream (override_2d_size_u) and the flag used to indicate whether 2d_size_v has been encoded into the code stream (override_2d_size_v) to indicate whether the two-dimensional information of the patch to be decoded has been encoded into the code stream, and The above-mentioned second identification, third identification and fourth identification are similar and will not be repeated here.
S1304：当第一标识为第二值时，从码流中解析待解码patch的辅助信息，该辅助信息包括待解码patch的二维信息、三维信息和待解码patch的参考patch的标识信息。S1304: When the first identifier is the second value, parse the auxiliary information of the patch to be decoded from the code stream. The auxiliary information includes the two-dimensional information, the three-dimensional information of the patch to be decoded, and the identification information of the reference patch of the patch to be decoded.
需要说明的是，S1304的内容可以参考上述S1103的内容，此处不再赘述。It should be noted that the content of S1304 can refer to the content of S1103, which will not be repeated here.
S1305：当第一标识为第三值时，从码流中解析待解码patch的辅助信息，该辅助信息包括待解码patch二维信息、三维信息、旋转变换信息和法向坐标轴信息。S1305: When the first identifier is the third value, parse the auxiliary information of the patch to be decoded from the code stream. The auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded.
需要说明的是，S1305的内容可以参考上述S1104的内容，此处不再赘述。It should be noted that the content of S1305 can refer to the content of S1104, which will not be repeated here.
S1306：根据待解码patch的辅助信息，重构待解码patch。S1306: According to the auxiliary information of the patch to be decoded, reconstruct the patch to be decoded.
需要说明的是，S1306的内容可以参考上述S1105的内容，此处不再赘述。It should be noted that the content of S1306 can refer to the content of S1105 above, which will not be repeated here.
在本申请实施例中，第一标识的取值可以为第一值、第二值和第三值。不同的取值指示不同类型的patch。因此，可以通过第一标识将不同类型的patch进行区分，这样就不需要在码流中解析第一类patch的数量，减少码流的比特开销。而且，当第一标识为第一值时，从码流中解析待解码patch的第二标识。当第二标识为第四值时，表明待解码patch的三维信息未编入码流。也即是，待解码patch的三维信息与待解码patch的参考patch的三维信息相同，此时可以将待解码patch的参考patch的三维信息作为待解码patch的三维信息，而不需要从码流中解析待解码patch的三维信息，可以简化解析码流的格式，减少码流的比特开销，从而提高解码的效率。In the embodiment of the present application, the value of the first identifier may be the first value, the second value, and the third value. Different values indicate different types of patches. Therefore, different types of patches can be distinguished by the first identifier, so that there is no need to parse the number of the first type of patches in the code stream, and the bit overhead of the code stream is reduced. Moreover, when the first identifier is the first value, the second identifier of the patch to be decoded is parsed from the code stream. When the second identifier is the fourth value, it indicates that the three-dimensional information of the patch to be decoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be decoded is the same as the three-dimensional information of the reference patch of the patch to be decoded. At this time, the three-dimensional information of the reference patch of the patch to be decoded can be used as the three-dimensional information of the patch to be decoded, without the need to extract from the code stream. Parsing the three-dimensional information of the patch to be decoded can simplify the format of the parsing code stream, reduce the bit overhead of the code stream, and improve the efficiency of decoding.
参见图14，图14为本申请实施例提供的第一种点云编码装置的结构示意图。如图14所示，该编码装置1400可以包括：Refer to FIG. 14, which is a schematic structural diagram of the first point cloud coding apparatus provided by an embodiment of the application. As shown in FIG. 14, the encoding device 1400 may include:
点云块patch信息获取模块1401，用于待编码patch的辅助信息；Point cloud block patch information acquisition module 1401, used for auxiliary information of the patch to be encoded;
辅助信息编码模块1402，用于将待编码patch的辅助信息和待编码patch的第一标识编入码流；An auxiliary information encoding module 1402, configured to encode auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded into a code stream;
其中，第一标识为第一值，以指示待编码patch为第一类patch，待编码patch的辅助信息包括三维信息；Wherein, the first identifier is a first value to indicate that the patch to be encoded is a first type of patch, and the auxiliary information of the patch to be encoded includes three-dimensional information;
其中，第一标识为第二值，以指示待编码patch为第二类patch，待编码patch的辅助信息包括二维信息、三维信息和待编码patch的参考patch的标识信息；Wherein, the first identifier is a second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded;
其中，第一标识为第三值，以指示待编码patch为第三类patch，待编码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。Wherein, the first identifier is a third value to indicate that the patch to be encoded is a third-type patch. The auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
其中，第一值、第二值和第三值分别是二进制的10、11和00，或者，第一值、第二值和第三值分别是二进制的11、10和00。Wherein, the first value, the second value and the third value are binary 10, 11 and 00 respectively, or the first value, the second value and the third value are binary 11, 10 and 00 respectively.
其中，待编码patch的第一标识为第三值，待编码patch为当前点云帧中的首个patch。The first identifier of the patch to be encoded is the third value, and the patch to be encoded is the first patch in the current point cloud frame.
其中，待编码patch的参考patch包括于当前点云帧的参考帧中。Wherein, the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.
其中，第一类patch为当前组中的全局匹配patch，第二类patch为当前组中的局部匹配patch，第三类patch为当前组中的非匹配patch。Among them, the first type of patch is a global matching patch in the current group, the second type of patch is a local matching patch in the current group, and the third type of patch is a non-matching patch in the current group.
需要说明的是，patch信息获取模块1401用于执行图9所示实施例的步骤S901的相关内容，辅助信息编码模块1402用于执行图9所示实施例的步骤S902的相关内容。作为一个示例，patch信息获取模块1401可以对应于图2中的点云块信息生成模块101和打包模块102的组合，辅助信息编码模块1402可以对应于图2中的辅助信息编码模块109，换言之，patch信息获取模块1401的功能可以由图2中的点云块信息生成模块101和打包模块102实现，辅助信息生成模块1402的功能可以由图2中的辅助信息编码模块109来实现。在一个示例中，点云块信息生成模块101用于获取待编码patch的三维信息、法向坐标轴信息和第一标识等，打包模块102用于获取待编码patch的旋转变换信息、参考patch的标识信息和二维信息等。当然本申请不限于此。其他细节可以参考现有技术或上文中对图2所示的编码器的原理的解释，此处不再赘述。It should be noted that the patch information acquisition module 1401 is used to execute the related content of step S901 of the embodiment shown in FIG. 9, and the auxiliary information encoding module 1402 is used to execute the related content of step S902 of the embodiment shown in FIG. 9. As an example, the patch information acquisition module 1401 may correspond to the combination of the point cloud block information generation module 101 and the packing module 102 in FIG. 2, and the auxiliary information encoding module 1402 may correspond to the auxiliary information encoding module 109 in FIG. 2, in other words, The function of the patch information acquisition module 1401 may be implemented by the point cloud block information generating module 101 and the packing module 102 in FIG. 2, and the function of the auxiliary information generating module 1402 may be implemented by the auxiliary information encoding module 109 in FIG. 2. In one example, the point cloud block information generation module 101 is used to obtain the three-dimensional information, normal coordinate axis information, and first identification of the patch to be encoded, and the packaging module 102 is used to obtain the rotation transformation information of the patch to be encoded and the reference patch Identification information and two-dimensional information, etc. Of course, this application is not limited to this. For other details, reference can be made to the prior art or the above explanation of the principle of the encoder shown in FIG. 2, which will not be repeated here.
参见图15，图15为本申请实施例提供的第一种点云解码装置的结构示意图。如图15所示，该解码装置1500可以包括：Refer to FIG. 15, which is a schematic structural diagram of a first point cloud decoding apparatus provided by an embodiment of the application. As shown in FIG. 15, the decoding apparatus 1500 may include:
辅助信息解码模块1501，用于从码流中解析待解码patch的第一标识；The auxiliary information decoding module 1501 is used to parse the first identifier of the patch to be decoded from the code stream;
辅助信息解码模块1501还用于当第一标识为第一值时，从码流中解析待解码patch的辅助信息，辅助信息包括待解码patch的三维信息；当第一标识为第二值时，从码流中解析待解码patch的辅助信息，辅助信息包括待解码patch的二维信息、三维信息和待解码patch的参考patch的标识信息；当第一标识为第三值时，从码流中解析待解码patch的辅助信息，辅助信息包括待解码patch的二维信息、三维信息、旋转变换信息和法向坐标轴信息；The auxiliary information decoding module 1501 is further configured to parse auxiliary information of the patch to be decoded from the code stream when the first identifier is the first value, and the auxiliary information includes the three-dimensional information of the patch to be decoded; when the first identifier is the second value, The auxiliary information of the patch to be decoded is parsed from the code stream. The auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded and the identification information of the reference patch of the patch to be decoded; when the first identifier is the third value, from the code stream Parse the auxiliary information of the patch to be decoded. The auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded;
重构模块1502，用于根据待解码patch的辅助信息，重构待解码patch。The reconstruction module 1502 is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.
第一值、第二值和第三值分别是二进制的10、11和00，或者，第一值、第二值和第三值分别是二进制的11、10和00。The first value, the second value, and the third value are binary 10, 11, and 00 respectively, or the first value, the second value, and the third value are binary 11, 10, and 00, respectively.
其中，辅助信息解码模块1501还用于：Among them, the auxiliary information decoding module 1501 is also used for:
当待解码patch为首个patch，且第一标识为第三值时，确定待解码patch所属的当前点云帧为首个点云帧。When the patch to be decoded is the first patch and the first identifier is the third value, it is determined that the current point cloud frame to which the patch to be decoded belongs is the first point cloud frame.
其中，辅助信息解码模块1501还用于：Among them, the auxiliary information decoding module 1501 is also used for:
当待解码patch为当前点云帧中的首个patch时，从码流中解析待解码patch的第一标识；或者，当待解码patch为当前点云帧中的首个patch时，从码流中解析当前点云帧包括的多个patch的多个第一标识，其中解析出的多个第一标识中包括待解码patch的第一标识。When the patch to be decoded is the first patch in the current point cloud frame, the first identifier of the patch to be decoded is parsed from the code stream; or, when the patch to be decoded is the first patch in the current point cloud frame, from the code stream Analyze multiple first identifiers of multiple patches included in the current point cloud frame, and the multiple first identifiers parsed include the first identifier of the patch to be decoded.
其中，当第一标识为第一值时，辅助信息解码模块1501还用于：Wherein, when the first identifier is the first value, the auxiliary information decoding module 1501 is further configured to:
根据待解码patch的参考patch的辅助信息，获取待解码patch的二维信息、旋转变换信息和法向坐标轴信息；Obtain the two-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded;
重构模块1502还用于：The reconstruction module 1502 is also used to:
根据待解码patch的二维信息、三维信息、法向坐标轴信息和旋转变换信息重构待解码patch。According to the two-dimensional information, three-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded, the patch to be decoded is reconstructed.
其中，当第一标识为第二值时，辅助信息解码模块1501还用于：Wherein, when the first identifier is the second value, the auxiliary information decoding module 1501 is further configured to:
根据待解码patch的参考patch的标识信息，获得待解码patch的旋转变换信息和法向坐标轴信息；Obtain the rotation transformation information and normal coordinate axis information of the patch to be decoded according to the identification information of the reference patch of the patch to be decoded;
重构模块1502还用于：The reconstruction module 1502 is also used to:
根据待解码patch的二维信息、三维信息、法向坐标轴信息和旋转变换信息重构待解码patch。According to the two-dimensional information, three-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded, the patch to be decoded is reconstructed.
其中，待解码patch的参考patch包括于当前点云帧的参考帧中。Wherein, the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.
需要说明的是，辅助信息解码模块1501用于执行图11所示实施例的步骤S1101-S1104的相关内容，重构模块1502用于执行图11所示实施例的步骤S1105的相关内容。作为一个示例，图15中的辅助信息解码模块1501对应于图6中的辅助信息解码模块204，图15中的重构模块1502对应于图5中的点云的几何信息重构模块206，换言之，辅助信息解码模块1501的功能可以由图6中的辅助信息解码模块204来实现，重构模块1502的功能可以由图6中的点云的几何信息重构模块206来实现。当然本申请不限于此。其他细节可以参考现有技术或上文中对图6所示的解码器的原理的解释，此处不再赘述。It should be noted that the auxiliary information decoding module 1501 is used to execute the related content of steps S1101-S1104 in the embodiment shown in FIG. 11, and the reconstruction module 1502 is used to execute the related content of step S1105 in the embodiment shown in FIG. As an example, the auxiliary information decoding module 1501 in FIG. 15 corresponds to the auxiliary information decoding module 204 in FIG. 6, and the reconstruction module 1502 in FIG. 15 corresponds to the geometric information reconstruction module 206 of the point cloud in FIG. 5. In other words The function of the auxiliary information decoding module 1501 may be implemented by the auxiliary information decoding module 204 in FIG. 6, and the function of the reconstruction module 1502 may be implemented by the point cloud geometric information reconstruction module 206 in FIG. 6. Of course, this application is not limited to this. For other details, reference may be made to the prior art or the above explanation of the principle of the decoder shown in FIG. 6, which will not be repeated here.
参见图16，图16为本申请实施例提供的第二种点云编码装置的结构示意图。如图16所示，该编码装置1600可以包括：Refer to FIG. 16, which is a schematic structural diagram of a second point cloud encoding device provided by an embodiment of this application. As shown in FIG. 16, the encoding apparatus 1600 may include:
点云块patch信息获取模块1601，用于获取待编码patch的辅助信息；The point cloud block patch information acquisition module 1601 is used to acquire auxiliary information of the patch to be encoded;
辅助信息编码模块1602，用于将待编码patch的辅助信息和待编码patch的第一标识编入码流；An auxiliary information encoding module 1602, configured to encode auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded into a code stream;
其中第一标识为第一值，以指示待编码patch为第一类patch，且编入码流的信息还包括第二标识，其中第二标识为第四值，以指示待编码patch的三维信息未编入码流；或者，第二标识为第五值，以指示待编码patch的三维信息已编入码流，待编码patch的辅助信息包括三维信息；The first identifier is a first value to indicate that the patch to be encoded is a first-type patch, and the information encoded into the code stream also includes a second identifier, where the second identifier is a fourth value to indicate the three-dimensional information of the patch to be encoded The code stream is not coded; or, the second identifier is the fifth value to indicate that the three-dimensional information of the patch to be coded has been coded into the code stream, and the auxiliary information of the patch to be coded includes three-dimensional information;
其中第一标识为第二值，以指示待编码patch为第二类patch，待编码patch的辅助信息包括二维信息、三维信息和待编码patch的参考patch的标识信息；The first identifier is the second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded;
其中第一标识为第三值，以指示待编码patch为第三类patch，待编码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。The first identifier is a third value to indicate that the patch to be encoded is a third type of patch. The auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
需要说明的是，patch信息获取模块1601用于执行图12所示实施例的步骤S1201的相关内容，辅助信息编码模块1602用于执行图12所示实施例的步骤S1202的相关内容。作为一个示例，patch信息获取模块1601可以对应于图2中的点云块信息生成模块101和打包模块102的组合，辅助信息编码模块1602可以对应于图2中的辅助信息编码模块109，换言之，patch信息获取模块1601的功能可以由图2中的点云块信息生成模块101和打包模块102实现，辅助信息生成模块1602的功能可以由图2中的辅助信息编码模块109来实现。在一个示例中，点云块信息生成模块101用于获取待编码patch的三维信息、法向坐标轴信息、第一标识和第二标识等，打包模块102用于获取待编码patch的旋转变换信息、参考patch的标识信息和二维信息等。当然本申请不限于此。其他细节可以参考现有技术或上文中对图2所示的编码器的原理的解释，此处不再赘述。It should be noted that the patch information acquisition module 1601 is used to execute the related content of step S1201 in the embodiment shown in FIG. 12, and the auxiliary information encoding module 1602 is used to execute the related content of step S1202 in the embodiment shown in FIG. As an example, the patch information acquisition module 1601 may correspond to the combination of the point cloud block information generation module 101 and the packing module 102 in FIG. 2, and the auxiliary information encoding module 1602 may correspond to the auxiliary information encoding module 109 in FIG. 2, in other words, The function of the patch information acquisition module 1601 may be implemented by the point cloud block information generating module 101 and the packing module 102 in FIG. 2, and the function of the auxiliary information generating module 1602 may be implemented by the auxiliary information encoding module 109 in FIG. 2. In an example, the point cloud block information generation module 101 is used to obtain three-dimensional information, normal coordinate axis information, first identification and second identification of the patch to be encoded, and the packaging module 102 is used to obtain rotation transformation information of the patch to be encoded , Refer to patch identification information and two-dimensional information, etc. Of course, this application is not limited to this. For other details, reference can be made to the prior art or the above explanation of the principle of the encoder shown in FIG. 2, which will not be repeated here.
参见图17，图17为本申请实施例提供的第二种点云解码装置的结构示意图。如图17所示，该解码装置1700可以包括：Referring to FIG. 17, FIG. 17 is a schematic structural diagram of a second point cloud decoding apparatus provided by an embodiment of this application. As shown in FIG. 17, the decoding apparatus 1700 may include:
辅助信息解码模块1701，用于从码流中解析待解码patch的第一标识；The auxiliary information decoding module 1701 is used to parse the first identifier of the patch to be decoded from the code stream;
辅助信息解码模块1701还用于当第一标识为第一值时，从码流中解析待解码patch的第二标识；当第二标识为第四值时，将待解码patch的参考patch的辅助信息作为待解码patch的辅助信息；当第二标识为第五值时，从码流中解析待解码patch的辅助信息，辅助信息包括待解码patch的三维信息；当第一标识为第二值时，从码流中解析待解码patch的辅助信息，辅助信息包括待解码patch的二维信息、三维信息和待解码patch的参考patch的标识信息；当第一标识为第三值时，从码流中解析待解码patch的辅助信息，辅助信息包括待解码patch的二维信息、三维信息、旋转变换信息和法向坐标轴信息；The auxiliary information decoding module 1701 is also used to parse the second identifier of the patch to be decoded from the code stream when the first identifier is the first value; when the second identifier is the fourth value, to add the auxiliary information of the reference patch of the patch to be decoded The information is used as auxiliary information of the patch to be decoded; when the second identifier is the fifth value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded; when the first identifier is the second value Analyze the auxiliary information of the patch to be decoded from the code stream. The auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded and the identification information of the reference patch of the patch to be decoded; when the first identifier is the third value, the auxiliary information Parse the auxiliary information of the patch to be decoded. The auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded;
重构模块1702，用于根据待解码patch的辅助信息，重构待解码patch。The reconstruction module 1702 is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.
需要说明的是，辅助信息解码模块1701用于执行图13所示实施例的步骤S1301-S1305的相关内容，重构模块1702用于执行图13所示实施例的步骤S1306的相关内容。作为一个示例，图17中的辅助信息解码模块1701对应于图6中的辅助信息解码模块204，图17中的 重构模块1702对应于图5中的点云的几何信息重构模块206，换言之，辅助信息解码模块1701的功能可以由图6中的辅助信息解码模块204来实现，重构模块1702的功能可以由图6中的点云的几何信息重构模块206来实现。当然本申请不限于此。其他细节可以参考现有技术或上文中对图6所示的解码器的原理的解释，此处不再赘述。It should be noted that the auxiliary information decoding module 1701 is used to execute related content of steps S1301 to S1305 of the embodiment shown in FIG. 13, and the reconstruction module 1702 is used to execute related content of step S1306 of the embodiment shown in FIG. 13. As an example, the auxiliary information decoding module 1701 in FIG. 17 corresponds to the auxiliary information decoding module 204 in FIG. 6, and the reconstruction module 1702 in FIG. 17 corresponds to the geometric information reconstruction module 206 of the point cloud in FIG. 5. In other words The function of the auxiliary information decoding module 1701 may be implemented by the auxiliary information decoding module 204 in FIG. 6, and the function of the reconstruction module 1702 may be implemented by the geometric information reconstruction module 206 of the point cloud in FIG. 6. Of course, this application is not limited to this. For other details, reference may be made to the prior art or the above explanation of the principle of the decoder shown in FIG. 6, which will not be repeated here.
图18为用于本申请实施例的译码装置1800的一种实现方式的示意性框图。其中，获取装置1800可以包括处理器1801、存储器1802和总线***1803。其中，处理器1801和存储器1802通过总线***1803相连，该存储器1802用于存储指令，该处理器1801用于执行该存储器1802存储的指令，以执行本申请描述的各种点云编码或解码方法，尤其是基于当前图像块的块尺寸对当前图像块进行滤波的方法。为避免重复，这里不再详细描述。FIG. 18 is a schematic block diagram of an implementation manner of a decoding device 1800 used in an embodiment of the present application. Wherein, the obtaining apparatus 1800 may include a processor 1801, a memory 1802, and a bus system 1803. The processor 1801 and the memory 1802 are connected through a bus system 1803, the memory 1802 is used to store instructions, and the processor 1801 is used to execute instructions stored in the memory 1802 to execute various point cloud encoding or decoding methods described in this application , Especially the method of filtering the current image block based on the block size of the current image block. To avoid repetition, it will not be described in detail here.
在本申请实施例中，该处理器1801可以是中央处理单元(central processing unit，CPU)，该处理器1801还可以是其他通用处理器、DSP、ASIC、FPGA或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。In the embodiment of the present application, the processor 1801 may be a central processing unit (CPU), and the processor 1801 may also be other general-purpose processors, DSP, ASIC, FPGA or other programmable logic devices, discrete gates. Or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
该存储器1802可以包括ROM设备或者RAM设备。任何其他适宜类型的存储设备也可以用作存储器1802。存储器1802可以包括由处理器1801使用总线1803访问的代码和数据18021。存储器1802可以进一步包括操作***18023和应用程序18022，该应用程序18022包括允许处理器1801执行本申请描述的点云编码或解码方法(尤其是本申请描述的点云的编解码方法)的至少一个程序。例如，应用程序18022可以包括应用1至N，其进一步包括执行在本申请描述的点云编码或解码方法的点云编码或解码应用(简称点云译码应用)。The memory 1802 may include a ROM device or a RAM device. Any other suitable type of storage device can also be used as the memory 1802. The memory 1802 may include code and data 18021 accessed by the processor 1801 using the bus 1803. The memory 1802 may further include an operating system 18023 and an application program 18022. The application program 18022 includes at least one of the point cloud encoding or decoding methods described in this application (especially the point cloud encoding and decoding methods described in this application) that allows the processor 1801 to execute program. For example, the application 18022 may include applications 1 to N, which further include a point cloud encoding or decoding application (referred to as a point cloud decoding application) that executes the point cloud encoding or decoding method described in this application.
该总线***1803除包括数据总线之外，还可以包括电源总线、控制总线和状态信号总线等。但是为了清楚说明起见，在图中将各种总线都标为总线***1803。In addition to the data bus, the bus system 1803 may also include a power bus, a control bus, and a status signal bus. However, for clear description, various buses are marked as the bus system 1803 in the figure.
可选的，编码装置1800还可以包括一个或多个输出设备，诸如显示器1804。在一个示例中，显示器1804可以是触感显示器，其将显示器与可操作地感测触摸输入的触感单元合并。显示器1804可以经由总线1803连接到处理器1801。Optionally, the encoding apparatus 1800 may further include one or more output devices, such as a display 1804. In one example, the display 1804 may be a touch-sensitive display that merges the display with a touch-sensitive unit operable to sense touch input. The display 1804 may be connected to the processor 1801 via a bus 1803.
需要指出的是，译码装置1800可以执行本申请中的点云的编码方法，也可执行本申请中的点云的解码方法。It should be pointed out that the decoding device 1800 can execute the point cloud encoding method in this application, and can also execute the point cloud decoding method in this application.
本领域技术人员能够领会，结合本文公开描述的各种说明性逻辑框、模块和算法步骤所描述的功能可以硬件、软件、固件或其任何组合来实施。如果以软件来实施，那么各种说明性逻辑框、模块、和步骤描述的功能可作为一或多个指令或代码在计算机可读媒体上存储或传输，且由基于硬件的处理单元执行。计算机可读媒体可包含计算机可读存储媒体，其对应于有形媒体，例如数据存储媒体，或包括任何促进将计算机程序从一处传送到另一处的媒体(例如，根据通信协议)的通信媒体。以此方式，计算机可读媒体大体上可对应于(1)非暂时性的有形计算机可读存储媒体，或(2)通信媒体，例如信号或载波。数据存储媒体可为可由一或多个计算机或一或多个处理器存取以检索用于实施本申请中描述的技术的指令、代码和/或数据结构的任何可用媒体。计算机程序产品可包含计算机可读媒体。Those skilled in the art can understand that the functions described in conjunction with the various illustrative logical blocks, modules, and algorithm steps disclosed herein can be implemented by hardware, software, firmware, or any combination thereof. If implemented in software, the functions described by various illustrative logical blocks, modules, and steps may be stored or transmitted as one or more instructions or codes on a computer-readable medium, and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium, such as a data storage medium, or a communication medium that includes any medium that facilitates the transfer of a computer program from one place to another (for example, according to a communication protocol) . In this manner, computer-readable media may generally correspond to (1) non-transitory tangible computer-readable storage media, or (2) communication media, such as signals or carrier waves. Data storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and/or data structures for implementing the techniques described in this application. The computer program product may include a computer-readable medium.
作为实例而非限制，此类计算机可读存储媒体可包括RAM、ROM、EEPROM、CD-ROM或其它光盘存储装置、磁盘存储装置或其它磁性存储装置、快闪存储器或可用来存储指令或数据结构的形式的所要程序代码并且可由计算机存取的任何其它媒体。并且，任何连接被恰 当地称作计算机可读媒体。举例来说，如果使用同轴缆线、光纤缆线、双绞线、数字订户线(DSL)或例如红外线、无线电和微波等无线技术从网站、服务器或其它远程源传输指令，那么同轴缆线、光纤缆线、双绞线、DSL或例如红外线、无线电和微波等无线技术包含在媒体的定义中。但是，应理解，所述计算机可读存储媒体和数据存储媒体并不包括连接、载波、信号或其它暂时媒体，而是实际上针对于非暂时性有形存储媒体。如本文中所使用，磁盘和光盘包含压缩光盘(CD)、激光光盘、光学光盘、DVD和蓝光光盘，其中磁盘通常以磁性方式再现数据，而光盘利用激光以光学方式再现数据。以上各项的组合也应包含在计算机可读媒体的范围内。By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or structures that can be used to store instructions or data Any other media that can be accessed by the computer in the form of desired program code. And, any connection is appropriately called a computer-readable medium. For example, if you use coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave to transmit instructions from a website, server, or other remote source, then the coaxial cable Wire, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of media. However, it should be understood that the computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other temporary media, but are actually directed to non-transitory tangible storage media. As used herein, magnetic disks and optical disks include compact disks (CDs), laser disks, optical disks, DVDs, and Blu-ray disks, where disks usually reproduce data magnetically, and optical disks use lasers to reproduce data optically. Combinations of the above should also be included in the scope of computer-readable media.
可通过例如一或多个数字信号处理器(DSP)、通用微处理器、专用集成电路(ASIC)、现场可编程逻辑阵列(FPGA)或其它等效集成或离散逻辑电路等一或多个处理器来执行指令。因此，如本文中所使用的术语“处理器”可指前述结构或适合于实施本文中所描述的技术的任一其它结构中的任一者。另外，在一些方面中，本文中所描述的各种说明性逻辑框、模块、和步骤所描述的功能可以提供于经配置以用于编码和解码的专用硬件和/或软件模块内，或者并入在组合编解码器中。而且，所述技术可完全实施于一或多个电路或逻辑元件中。在一种示例下，编码器100及解码器200中的各种说明性逻辑框、单元、模块可以理解为对应的电路器件或逻辑元件。It can be processed by one or more digital signal processors (DSP), general-purpose microprocessors, application-specific integrated circuits (ASIC), field programmable logic arrays (FPGA), or other equivalent integrated or discrete logic circuits, for example To execute instructions. Therefore, the term "processor" as used herein may refer to any of the foregoing structure or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functions described by the various illustrative logical blocks, modules, and steps described herein may be provided in dedicated hardware and/or software modules configured for encoding and decoding, or combined Into the combined codec. Moreover, the technology may be fully implemented in one or more circuits or logic elements. In an example, various illustrative logical blocks, units, and modules in the encoder 100 and the decoder 200 can be understood as corresponding circuit devices or logical elements.
本申请的技术可在各种各样的装置或设备中实施，包含无线手持机、集成电路(IC)或一组IC(例如，芯片组)。本申请中描述各种组件、模块或单元是为了强调用于执行所揭示的技术的装置的功能方面，但未必需要由不同硬件单元实现。实际上，如上文所描述，各种单元可结合合适的软件和/或固件组合在编码解码器硬件单元中，或者通过互操作硬件单元(包含如上文所描述的一或多个处理器)来提供。The technology of this application can be implemented in a variety of devices or devices, including wireless handsets, integrated circuits (ICs), or a set of ICs (for example, chipsets). Various components, modules, or units are described in this application to emphasize the functional aspects of the device for performing the disclosed technology, but they do not necessarily need to be implemented by different hardware units. In fact, as described above, various units can be combined with appropriate software and/or firmware in the codec hardware unit, or by interoperating hardware units (including one or more processors as described above). provide.
以上所述，仅为本申请示例性的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到的变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应该以权利要求的保护范围为准。The above are only exemplary specific implementations of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or changes within the technical scope disclosed in this application. Replacement shall be covered within the scope of protection of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims (52)

  1. 一种点云编码方法，其特征在于，所述方法包括：A point cloud coding method, characterized in that the method includes:
    获取待编码点云块patch的辅助信息；Obtain auxiliary information of the point cloud patch to be coded;
    将所述待编码patch的辅助信息和所述待编码patch的第一标识编入码流；Encoding the auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded into a code stream;
    其中所述第一标识为第一值，以指示所述待编码patch为第一类patch，所述待编码patch的辅助信息包括三维信息；The first identifier is a first value to indicate that the patch to be encoded is a first type of patch, and the auxiliary information of the patch to be encoded includes three-dimensional information;
    其中所述第一标识为第二值，以指示所述待编码patch为第二类patch，所述待编码patch的辅助信息包括二维信息、三维信息和所述待编码patch的参考patch的标识信息；The first identifier is a second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and the identifier of the reference patch of the patch to be encoded information;
    其中所述第一标识为第三值，以指示所述待编码patch为第三类patch，所述待编码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。The first identifier is a third value to indicate that the patch to be encoded is a third-type patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  2. 如权利要求1所述的方法，其特征在于，所述第一值、所述第二值和所述第三值分别是二进制的10、11和00，The method of claim 1, wherein the first value, the second value, and the third value are binary 10, 11, and 00, respectively,
    或者，所述第一值、所述第二值和所述第三值分别是二进制的11、10和00。Alternatively, the first value, the second value, and the third value are 11, 10, and 00 in binary, respectively.
  3. 如权利要求1或2所述的方法，其特征在于，所述待编码patch的第一标识为第三值，所述待编码patch为所属的当前点云帧中的首个patch，所述当前点云帧为首个点云帧。The method according to claim 1 or 2, wherein the first identifier of the patch to be encoded is a third value, the patch to be encoded is the first patch in the current point cloud frame to which it belongs, and the current The point cloud frame is the first point cloud frame.
  4. 如权利要求1-3任一项所述的方法，其特征在于，所述待编码patch的参考patch包括于当前点云帧的参考帧中。The method according to any one of claims 1 to 3, wherein the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.
  5. 如权利要求1-4任一项所述的方法，其特征在于，所述第一类patch为当前组中的全局匹配patch，所述第二类patch为当前组中的局部匹配patch，所述第三类patch为当前组中的非匹配patch。The method according to any one of claims 1 to 4, wherein the first type of patch is a global matching patch in the current group, the second type of patch is a local matching patch in the current group, and the The third type of patch is the non-matching patch in the current group.
  6. 一种点云解码方法，其特征在于，所述方法包括：A point cloud decoding method, characterized in that the method includes:
    从码流中解析待解码点云块patch的第一标识；Parse the first identifier of the point cloud block patch to be decoded from the code stream;
    当所述第一标识为第一值时，从所述码流中解析所述待解码patch的辅助信息，所述辅助信息包括所述待解码patch的三维信息；When the first identifier is the first value, parse the auxiliary information of the patch to be decoded from the code stream, where the auxiliary information includes the three-dimensional information of the patch to be decoded;
    当所述第一标识为第二值时，从所述码流中解析所述待解码patch的辅助信息，所述辅助信息包括所述待解码patch的二维信息、三维信息和所述待解码patch的参考patch的标识信息；When the first identifier is the second value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded, and the patch to be decoded. Reference patch identification information for patch;
    当所述第一标识为第三值时，从所述码流中解析所述待解码patch的辅助信息，所述辅助信息包括所述待解码patch的二维信息、三维信息、旋转变换信息和法向坐标轴信息；When the first identifier is the third value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and information of the patch to be decoded. Normal axis information;
    根据所述待解码patch的辅助信息，重构所述待解码patch。According to the auxiliary information of the patch to be decoded, reconstruct the patch to be decoded.
  7. 如权利要求6所述的方法，其特征在于，所述第一值、所述第二值和所述第三值分别 是二进制的10、11和00，The method according to claim 6, wherein said first value, said second value and said third value are binary 10, 11 and 00, respectively,
    或者，所述第一值、所述第二值和所述第三值分别是二进制的11、10和00。Alternatively, the first value, the second value, and the third value are 11, 10, and 00 in binary, respectively.
  8. 如权利要求6或7所述的方法，其特征在于，所述方法还包括：The method according to claim 6 or 7, wherein the method further comprises:
    当所述待解码patch为首个patch，且所述第一标识为第三值时，确定所述待解码patch所属的当前点云帧为首个点云帧。When the patch to be decoded is the first patch and the first identifier is the third value, it is determined that the current point cloud frame to which the patch to be decoded belongs is the first point cloud frame.
  9. 如权利要求6-8任一项所述的方法，其特征在于，所述从码流中解析待解码patch的第一标识，包括：8. The method of any one of claims 6-8, wherein the parsing the first identifier of the patch to be decoded from the code stream comprises:
    当所述待解码patch为当前点云帧中的首个patch时，从所述码流中解析所述待解码patch的第一标识；或者，When the patch to be decoded is the first patch in the current point cloud frame, parse the first identifier of the patch to be decoded from the code stream; or,
    当所述待解码patch为当前点云帧中的首个patch时，从所述码流中解析所述当前点云帧包括的多个patch的多个第一标识，其中解析出的多个第一标识中包括所述待解码patch的第一标识。When the patch to be decoded is the first patch in the current point cloud frame, the multiple first identifiers of the multiple patches included in the current point cloud frame are parsed from the code stream, and the multiple first identifiers parsed are An identifier includes the first identifier of the patch to be decoded.
  10. 如权利要求6-9任一项所述的方法，其特征在于，当所述第一标识为第一值时，所述方法还包括：9. The method according to any one of claims 6-9, wherein when the first identifier is a first value, the method further comprises:
    根据所述待解码patch的参考patch的辅助信息，获取所述待解码patch的二维信息、旋转变换信息和法向坐标轴信息；Acquiring the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded;
    根据所述待解码patch的辅助信息，重构所述待解码patch，包括：According to the auxiliary information of the patch to be decoded, reconstructing the patch to be decoded includes:
    根据所述待解码patch的二维信息、三维信息、法向坐标轴信息和旋转变换信息重构所述待解码patch。The patch to be decoded is reconstructed according to the two-dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information of the patch to be decoded.
  11. 如权利要求6-9任一项所述的方法，其特征在于，当所述第一标识为第二值时，所述方法还包括：9. The method according to any one of claims 6-9, wherein when the first identifier is a second value, the method further comprises:
    根据所述待解码patch的参考patch的标识信息，获取所述待解码patch的旋转变换信息和法向坐标轴信息；Obtaining the rotation transformation information and normal coordinate axis information of the patch to be decoded according to the identification information of the reference patch of the patch to be decoded;
    根据所述待解码patch的辅助信息，重构所述待解码patch，包括：According to the auxiliary information of the patch to be decoded, reconstructing the patch to be decoded includes:
    根据所述待解码patch的二维信息、三维信息、法向坐标轴信息和旋转变换信息重构所述待解码patch。The patch to be decoded is reconstructed according to the two-dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information of the patch to be decoded.
  12. 如权利要求6-11任一项所述的方法，其特征在于，所述待解码patch的参考patch包括于当前点云帧的参考帧中。The method according to any one of claims 6-11, wherein the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.
  13. 一种点云编码方法，其特征在于，所述方法包括：A point cloud coding method, characterized in that the method includes:
    获取待编码点云块patch的辅助信息；Obtain auxiliary information of the point cloud patch to be coded;
    将所述待编码patch的辅助信息和所述待编码patch的第一标识编入码流；Encoding the auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded into a code stream;
    其中所述第一标识为第一值，以指示所述待编码patch为第一类patch，且编入所述码流的信息还包括第二标识，其中所述第二标识为第四值，以指示所述待编码patch的三维信息 未编入所述码流；或者，所述第二标识为第五值，以指示所述待编码patch的三维信息已编入所述码流，所述待编码patch的辅助信息包括所述三维信息；The first identifier is a first value to indicate that the patch to be encoded is a first-type patch, and the information compiled into the code stream also includes a second identifier, where the second identifier is a fourth value, To indicate that the three-dimensional information of the patch to be encoded has not been encoded into the code stream; or, the second identifier is a fifth value to indicate that the three-dimensional information of the patch to be encoded has been encoded into the code stream, The auxiliary information of the patch to be encoded includes the three-dimensional information;
    其中所述第一标识为第二值，以指示所述待编码patch为第二类patch，所述待编码patch的辅助信息包括二维信息、三维信息和所述待编码patch的参考patch的标识信息；The first identifier is a second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and the identifier of the reference patch of the patch to be encoded information;
    其中所述第一标识为第三值，以指示所述待编码patch为第三类patch，所述待编码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。The first identifier is a third value to indicate that the patch to be encoded is a third-type patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  14. 一种点云解码方法，其特征在于，所述方法包括：A point cloud decoding method, characterized in that the method includes:
    从码流中解析待解码点云块patch的第一标识；Parse the first identifier of the point cloud block patch to be decoded from the code stream;
    当所述第一标识为第一值时，从所述码流中解析所述待解码patch的第二标识；When the first identifier is the first value, parse the second identifier of the patch to be decoded from the code stream;
    当所述第二标识为第四值时，将所述待解码patch的参考patch的辅助信息作为所述待解码patch的辅助信息；当所述第二标识为第五值时，从所述码流中解析所述待解码patch的辅助信息，所述辅助信息包括所述待解码patch的三维信息；When the second identifier is the fourth value, the auxiliary information of the reference patch of the patch to be decoded is used as the auxiliary information of the patch to be decoded; when the second identifier is the fifth value, from the code Parse the auxiliary information of the patch to be decoded in the stream, where the auxiliary information includes the three-dimensional information of the patch to be decoded;
    当所述第一标识为第二值时，从所述码流中解析所述待解码patch的辅助信息，所述辅助信息包括所述待解码patch的二维信息、三维信息和所述待解码patch的参考patch的标识信息；When the first identifier is the second value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded, and the patch to be decoded. Reference patch identification information for patch;
    当所述第一标识为第三值时，从所述码流中解析所述待解码patch的辅助信息，所述辅助信息包括所述待解码patch的二维信息、三维信息、旋转变换信息和法向坐标轴信息；When the first identifier is the third value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and information of the patch to be decoded. Normal axis information;
    根据所述待解码patch的辅助信息，重构所述待解码patch。According to the auxiliary information of the patch to be decoded, reconstruct the patch to be decoded.
  15. 一种点云编码装置，其特征在于，所述装置包括：A point cloud coding device, characterized in that the device comprises:
    点云块patch信息获取模块，用于待编码patch的辅助信息；Point cloud block patch information acquisition module for auxiliary information of the patch to be encoded;
    辅助信息编码模块，用于将所述待编码patch的辅助信息和所述待编码patch的第一标识编入码流；Auxiliary information encoding module, configured to encode auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded into a code stream;
    其中所述第一标识为第一值，以指示所述待编码patch为第一类patch，所述待编码patch的辅助信息包括三维信息；The first identifier is a first value to indicate that the patch to be encoded is a first type of patch, and the auxiliary information of the patch to be encoded includes three-dimensional information;
    其中所述第一标识为第二值，以指示所述待编码patch为第二类patch，所述待编码patch的辅助信息包括二维信息、三维信息和所述待编码patch的参考patch的标识信息；The first identifier is a second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and the identifier of the reference patch of the patch to be encoded information;
    其中所述第一标识为第三值，以指示所述待编码patch为第三类patch，所述待编码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。The first identifier is a third value to indicate that the patch to be encoded is a third-type patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  16. 如权利要求15所述的装置，其特征在于，所述第一值、所述第二值和所述第三值分别是二进制的10、11和00，The device of claim 15, wherein the first value, the second value and the third value are binary 10, 11 and 00, respectively,
    或者，所述第一值、所述第二值和所述第三值分别是二进制的11、10和00。Alternatively, the first value, the second value, and the third value are 11, 10, and 00 in binary, respectively.
  17. 如权利要求15或16所述的装置，其特征在于，所述待编码patch的第一标识为第三值，所述待编码patch为所属的当前点云帧中的首个patch，所述当前点云帧为首个点云帧。The device according to claim 15 or 16, wherein the first identifier of the patch to be encoded is a third value, the patch to be encoded is the first patch in the current point cloud frame to which it belongs, and the current The point cloud frame is the first point cloud frame.
  18. 如权利要求15-17任一项所述的装置，其特征在于，所述待编码patch的参考patch 包括于当前点云帧的参考帧中。The device according to any one of claims 15-17, wherein the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.
  19. 如权利要求15-18任一项所述的装置，其特征在于，所述第一类patch为当前组中的全局匹配patch，所述第二类patch为当前组中的局部匹配patch，所述第三类patch为当前组中的非匹配patch。The device according to any one of claims 15-18, wherein the first type of patch is a global matching patch in the current group, and the second type of patch is a local matching patch in the current group. The third type of patch is the non-matching patch in the current group.
  20. 一种点云解码装置，其特征在于，所述装置包括：A point cloud decoding device, characterized in that the device includes:
    辅助信息解码模块，用于从码流中解析待解码点云块patch的第一标识；The auxiliary information decoding module is used to parse the first identifier of the point cloud block patch to be decoded from the code stream;
    所述辅助信息解码模块，还用于当所述第一标识为第一值时，从所述码流中解析所述待解码patch的辅助信息，所述辅助信息包括所述待解码patch的三维信息；当所述第一标识为第二值时，从所述码流中解析所述待解码patch的辅助信息，所述辅助信息包括所述待解码patch的二维信息、三维信息和所述待解码patch的参考patch的标识信息；当所述第一标识为第三值时，从所述码流中解析所述待解码patch的辅助信息，所述辅助信息包括所述待解码patch的二维信息、三维信息、旋转变换信息和法向坐标轴信息；The auxiliary information decoding module is further configured to parse the auxiliary information of the patch to be decoded from the code stream when the first identifier is the first value, and the auxiliary information includes the three-dimensional information of the patch to be decoded. Information; when the first identifier is the second value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded and the The identification information of the reference patch of the patch to be decoded; when the first identifier is a third value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the second information of the patch to be decoded Dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information;
    重构模块，用于根据所述待解码patch的辅助信息，重构所述待解码patch。The reconstruction module is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.
  21. 如权利要求20所述的装置，其特征在于，所述第一值、所述第二值和所述第三值分别是二进制的10、11和00，The device of claim 20, wherein the first value, the second value, and the third value are binary 10, 11, and 00, respectively,
    或者，所述第一值、所述第二值和所述第三值分别是二进制的11、10和00。Alternatively, the first value, the second value, and the third value are 11, 10, and 00 in binary, respectively.
  22. 如权利要求20或21所述的装置，其特征在于，所述辅助信息解码模块还用于：The device according to claim 20 or 21, wherein the auxiliary information decoding module is further configured to:
    当所述待解码patch为首个patch，且所述第一标识为第三值时，确定所述待解码patch所属的当前点云帧为首个点云帧。When the patch to be decoded is the first patch and the first identifier is the third value, it is determined that the current point cloud frame to which the patch to be decoded belongs is the first point cloud frame.
  23. 如权利要求20-22任一项所述的装置，其特征在于，所述辅助信息解码模块还用于：The device according to any one of claims 20-22, wherein the auxiliary information decoding module is further configured to:
    当所述待解码patch为当前点云帧中的首个patch时，从所述码流中解析所述待解码patch的第一标识；或者，当所述待解码patch为当前点云帧中的首个patch时，从所述码流中解析所述当前点云帧包括的多个patch的多个第一标识，其中解析出的多个第一标识中包括所述待解码patch的第一标识。When the patch to be decoded is the first patch in the current point cloud frame, the first identifier of the patch to be decoded is parsed from the code stream; or, when the patch to be decoded is the first patch in the current point cloud frame For the first patch, the multiple first identifiers of the multiple patches included in the current point cloud frame are parsed from the code stream, and the multiple first identifiers parsed include the first identifier of the patch to be decoded .
  24. 如权利要求20-23任一项所述的装置，其特征在于，当所述第一标识为第一值时，所述辅助信息解码模块还用于：The device according to any one of claims 20-23, wherein when the first identifier is a first value, the auxiliary information decoding module is further configured to:
    根据所述待解码patch的参考patch的辅助信息，获取所述待解码patch的二维信息、旋转变换信息和法向坐标轴信息；Acquiring the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded;
    所述重构模块还用于：The reconstruction module is also used for:
    根据所述待解码patch的二维信息、三维信息、法向坐标轴信息和旋转变换信息重构所述待解码patch。The patch to be decoded is reconstructed according to the two-dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information of the patch to be decoded.
  25. 如权利要求20-23任一项所述的装置，其特征在于，当所述第一标识为第二值时， 所述辅助信息解码模块还用于：The device according to any one of claims 20-23, wherein when the first identifier is a second value, the auxiliary information decoding module is further configured to:
    根据所述待解码patch的参考patch的标识信息，获取所述待解码patch的旋转变换信息和法向坐标轴信息；Obtaining the rotation transformation information and normal coordinate axis information of the patch to be decoded according to the identification information of the reference patch of the patch to be decoded;
    所述重构模块还用于：The reconstruction module is also used for:
    根据所述待解码patch的二维信息、三维信息、法向坐标轴信息和旋转变换信息重构所述待解码patch。The patch to be decoded is reconstructed according to the two-dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information of the patch to be decoded.
  26. 如权利要求20-25任一项所述的装置，其特征在于，所述待解码patch的参考patch包括于当前点云帧的参考帧中。The device according to any one of claims 20-25, wherein the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.
  27. 一种点云编码装置，其特征在于，所述装置包括：A point cloud coding device, characterized in that the device comprises:
    点云块patch信息获取模块，用于获取待编码patch的辅助信息；Point cloud block patch information acquisition module for acquiring auxiliary information of the patch to be encoded;
    辅助信息编码模块，用于将所述待编码patch的辅助信息和所述待编码patch的第一标识编入码流；Auxiliary information encoding module, configured to encode auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded into a code stream;
    其中所述第一标识为第一值，以指示所述待编码patch为第一类patch，且编入所述码流的信息还包括第二标识，其中所述第二标识为第四值，以指示所述待编码patch的三维信息未编入所述码流；或者，所述第二标识为第五值，以指示所述待编码patch的三维信息已编入所述码流，所述待编码patch的辅助信息包括所述三维信息；The first identifier is a first value to indicate that the patch to be encoded is a first-type patch, and the information compiled into the code stream also includes a second identifier, where the second identifier is a fourth value, To indicate that the three-dimensional information of the patch to be encoded has not been encoded into the code stream; or, the second identifier is a fifth value to indicate that the three-dimensional information of the patch to be encoded has been encoded into the code stream, The auxiliary information of the patch to be encoded includes the three-dimensional information;
    其中所述第一标识为第二值，以指示所述待编码patch为第二类patch，所述待编码patch的辅助信息包括二维信息、三维信息和待编码patch的参考patch的标识信息；The first identifier is a second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded;
    其中所述第一标识为第三值，以指示所述待编码patch为第三类patch，所述待编码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。The first identifier is a third value to indicate that the patch to be encoded is a third-type patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  28. 一种点云解码装置，其特征在于，所述装置包括：A point cloud decoding device, characterized in that the device includes:
    辅助信息解码模块，用于从码流中解析待解码点云块patch的第一标识；The auxiliary information decoding module is used to parse the first identifier of the point cloud block patch to be decoded from the code stream;
    所述辅助信息解码模块，还用于当所述第一标识为第一值时，从所述码流中解析所述待解码patch的第二标识；当所述第二标识为第四值时，将所述待解码patch的参考patch的辅助信息作为所述待解码patch的辅助信息；当所述第二标识为第五值时，从所述码流中解析所述待解码patch的辅助信息，所述辅助信息包括所述待解码patch的三维信息；当所述第一标识为第二值时，从所述码流中解析所述待解码patch的辅助信息，所述辅助信息包括所述待解码patch的二维信息、三维信息和所述待解码patch的参考patch的标识信息；当所述第一标识为第三值时，从所述码流中解析所述待解码patch的辅助信息，所述辅助信息包括所述待解码patch的二维信息、三维信息、旋转变换信息和法向坐标轴信息；The auxiliary information decoding module is further configured to parse the second identifier of the patch to be decoded from the code stream when the first identifier is a first value; when the second identifier is a fourth value , Use the auxiliary information of the reference patch of the patch to be decoded as the auxiliary information of the patch to be decoded; when the second identifier is the fifth value, parse the auxiliary information of the patch to be decoded from the code stream The auxiliary information includes the three-dimensional information of the patch to be decoded; when the first identifier is a second value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the The two-dimensional information, three-dimensional information of the patch to be decoded, and identification information of the reference patch of the patch to be decoded; when the first identifier is a third value, the auxiliary information of the patch to be decoded is parsed from the code stream , The auxiliary information includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded;
    重构模块，用于根据所述待解码patch的辅助信息，重构所述待解码patch。The reconstruction module is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.
  29. 一种点云编码方法，其特征在于，所述方法包括：A point cloud coding method, characterized in that the method includes:
    获取待编码点云块patch的辅助信息；Obtain auxiliary information of the point cloud patch to be coded;
    将所述待编码patch的辅助信息和所述待编码patch的语法元素编入码流，所述语法元素包括第一语法元素，其中：The auxiliary information of the patch to be coded and the syntax element of the patch to be coded are encoded into a code stream, and the syntax element includes a first syntax element, wherein:
    当所述第一语法元素指示所述待编码patch具有参考patch时，When the first syntax element indicates that the patch to be encoded has a reference patch,
    编入所述码流的语法元素还包括第二语法元素，所述第二语法元素的取值为真，以指示所述待编码patch的二维信息编入所述码流，但所述待编码patch的三维信息未编入所述码流，所述待编码patch的辅助信息包括所述二维信息；或者The syntax element incorporated into the code stream further includes a second syntax element, and the value of the second syntax element is true to indicate that the two-dimensional information of the patch to be encoded is incorporated into the code stream, but the The three-dimensional information of the encoded patch is not encoded into the code stream, and the auxiliary information of the patch to be encoded includes the two-dimensional information; or
    编入所述码流的语法元素还包括第二语法元素和第三语法元素，所述第二语法元素的取值为假且所述第三语法元素的取值为真，以指示所述待编码patch的二维信息未编入所述码流，且所述待编码patch的三维信息编入所述码流，所述待编码patch的辅助信息包括所述三维信息；或者The syntax elements compiled into the code stream further include a second syntax element and a third syntax element, the value of the second syntax element is false and the value of the third syntax element is true to indicate the waiting The two-dimensional information of the encoded patch is not encoded into the code stream, and the three-dimensional information of the patch to be encoded is encoded into the code stream, and the auxiliary information of the patch to be encoded includes the three-dimensional information; or
    编入所述码流的语法元素还包括第二语法元素和第三语法元素，所述第二语法元素的取值为假且所述第三语法元素的取值为假，以指示所述待编码patch的二维信息未编入所述码流，且所述待编码patch的三维信息未编入所述码流。The syntax element compiled into the code stream further includes a second syntax element and a third syntax element, the value of the second syntax element is false and the value of the third syntax element is false to indicate the waiting The two-dimensional information of the encoded patch is not encoded into the code stream, and the three-dimensional information of the patch to be encoded is not encoded into the code stream.
  30. 如权利要求29所述的方法，其特征在于，当所述第一语法元素指示所述待编码patch不具有参考patch时，所述待编码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。The method according to claim 29, wherein when the first syntax element indicates that the patch to be encoded does not have a reference patch, the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and rotation transformation. Information and normal axis information.
  31. 如权利要求29或30所述的方法，其特征在于，所述第一语法元素指示所述待编码patch不具有参考patch，所述待编码patch为所属的当前点云帧中的首个patch，所述当前点云帧为首个点云帧。The method according to claim 29 or 30, wherein the first syntax element indicates that the patch to be encoded does not have a reference patch, and the patch to be encoded is the first patch in the current point cloud frame to which it belongs, The current point cloud frame is the first point cloud frame.
  32. 如权利要求29-31任一项所述的方法，其特征在于，所述待编码patch的参考patch包括于当前点云帧的参考帧中。The method according to any one of claims 29-31, wherein the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.
  33. 如权利要求32所述的方法，其特征在于，所述待编码patch在所述当前点云帧中的位置与所述待编码patch的参考patch在所述当前点云帧的参考帧的位置相同。The method according to claim 32, wherein the position of the patch to be encoded in the current point cloud frame is the same as the position of the reference patch of the patch to be encoded in the reference frame of the current point cloud frame .
  34. 一种点云解码方法，其特征在于，所述方法包括：A point cloud decoding method, characterized in that the method includes:
    从码流中解析待解码点云块patch的第一语法元素；Parse the first syntax element of the point cloud block patch to be decoded from the code stream;
    当所述第一语法元素指示所述待解码patch具有参考patch时，从所述码流中解析所述待解码patch的第二语法元素，或者，从所述码流中解析所述待解码patch的第二语法元素和第三语法元素；When the first syntax element indicates that the patch to be decoded has a reference patch, parse the second syntax element of the patch to be decoded from the code stream, or parse the patch to be decoded from the code stream The second and third syntax elements of;
    当所述第二语法元素的取值为真时，从所述码流中解析所述待解码patch的二维信息，将所述待解码patch的参考patch的三维信息作为所述待解码patch的三维信息；或者When the value of the second syntax element is true, the two-dimensional information of the patch to be decoded is parsed from the code stream, and the three-dimensional information of the reference patch of the patch to be decoded is used as the patch to be decoded. Three-dimensional information; or
    当所述第二语法元素的取值为假且所述第三语法元素的取值为真时，从所述码流中解析所述待解码patch的三维信息，将所述待解码patch的参考patch的二维信息作为所述待解码patch的二维信息；或者When the value of the second syntax element is false and the value of the third syntax element is true, the three-dimensional information of the patch to be decoded is parsed from the code stream, and the reference of the patch to be decoded The two-dimensional information of the patch is used as the two-dimensional information of the patch to be decoded; or
    当所述第二语法元素的取值为假且所述第三语法元素的取值为假时，将所述待解码patch的参考patch的二维信息作为所述待解码patch的二维信息，将所述待解码patch的参考patch的三维信息作为所述待解码patch的三维信息；When the value of the second syntax element is false and the value of the third syntax element is false, use the two-dimensional information of the reference patch of the patch to be decoded as the two-dimensional information of the patch to be decoded, Using the three-dimensional information of the reference patch of the patch to be decoded as the three-dimensional information of the patch to be decoded;
    根据所述待解码patch的辅助信息重构所述待解码patch，所述待解码patch的辅助信息包括所述待解码patch的二维信息和所述待解码patch的三维信息。The patch to be decoded is reconstructed according to the auxiliary information of the patch to be decoded. The auxiliary information of the patch to be decoded includes the two-dimensional information of the patch to be decoded and the three-dimensional information of the patch to be decoded.
  35. 如权利要求34所述的方法，其特征在于，所述方法还包括：The method of claim 34, wherein the method further comprises:
    当所述第一语法元素指示所述待解码patch不具有参考patch时，从所述码流中解析所述待解码patch的辅助信息，所述待解码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。When the first syntax element indicates that the patch to be decoded does not have a reference patch, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information of the patch to be decoded includes two-dimensional information and three-dimensional information , Rotation transformation information and normal coordinate axis information.
  36. 如权利要求34或35所述的方法，其特征在于，所述方法还包括：The method according to claim 34 or 35, wherein the method further comprises:
    当所述待解码patch为首个patch，且所述第一语法元素指示所述待解码patch不具有参考patch时，确定所述待解码patch所属的当前点云帧为首个点云帧。When the patch to be decoded is the first patch and the first syntax element indicates that the patch to be decoded does not have a reference patch, it is determined that the current point cloud frame to which the patch to be decoded belongs is the first point cloud frame.
  37. 如权利要求34-36任一项所述的方法，其特征在于，所述从码流中解析待解码点云块patch的第一语法元素，包括：The method according to any one of claims 34-36, wherein the parsing the first syntax element of the point cloud block patch to be decoded from the code stream comprises:
    当所述待解码patch为当前点云帧中的首个patch时，从所述码流中解析所述待解码patch的第一语法元素；或者，When the patch to be decoded is the first patch in the current point cloud frame, parse the first syntax element of the patch to be decoded from the code stream; or,
    当所述待解码patch为当前点云帧中的首个patch时，从所述码流中解析所述当前点云帧包括的多个patch的多个第一语法元素，其中解析出的多个第一语法元素中包括所述待解码patch的第一语法元素。When the patch to be decoded is the first patch in the current point cloud frame, the multiple first syntax elements of multiple patches included in the current point cloud frame are parsed from the code stream, and the multiple parsed The first syntax element includes the first syntax element of the patch to be decoded.
  38. 如权利要求34-37任一项所述的方法，其特征在于，所述待解码patch的参考patch包括于当前点云帧的参考帧中。The method according to any one of claims 34-37, wherein the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.
  39. 如权利要求38所述的方法，其特征在于，所述待解码patch在所述当前点云帧中的位置与所述待解码patch的参考patch在所述当前点云帧的参考帧的位置相同。The method of claim 38, wherein the position of the patch to be decoded in the current point cloud frame is the same as the position of the reference patch of the patch to be decoded in the reference frame of the current point cloud frame .
  40. 一种点云编码装置，其特征在于，所述装置包括：A point cloud coding device, characterized in that the device comprises:
    点云块patch信息获取模块，用于获取待编码patch的辅助信息；Point cloud block patch information acquisition module for acquiring auxiliary information of the patch to be encoded;
    辅助信息编码模块，用于将所述待编码patch的辅助信息和所述待编码patch的语法元素编入码流，所述语法元素包括第一语法元素，其中：The auxiliary information encoding module is used to encode the auxiliary information of the patch to be encoded and the syntax element of the patch to be encoded into a code stream, the syntax element includes a first syntax element, wherein:
    当所述第一语法元素指示所述待编码patch具有参考patch时，When the first syntax element indicates that the patch to be encoded has a reference patch,
    编入所述码流的语法元素还包括第二语法元素，所述第二语法元素的取值为真，以指示所述待编码patch的二维信息编入所述码流，但所述待编码patch的三维信息未编入所述码流，所述待编码patch的辅助信息包括所述二维信息；或者The syntax element incorporated into the code stream further includes a second syntax element, and the value of the second syntax element is true to indicate that the two-dimensional information of the patch to be encoded is incorporated into the code stream, but the The three-dimensional information of the encoded patch is not encoded into the code stream, and the auxiliary information of the patch to be encoded includes the two-dimensional information; or
    编入所述码流的语法元素还包括第二语法元素和第三语法元素，所述第二语法元素的取值为假且所述第三语法元素的取值为真，以指示所述待编码patch的二维信息未编入所述码流，且所述待编码patch的三维信息编入所述码流，所述待编码patch的辅助信息包括所述三维信息；或者The syntax elements compiled into the code stream further include a second syntax element and a third syntax element, the value of the second syntax element is false and the value of the third syntax element is true to indicate the waiting The two-dimensional information of the encoded patch is not encoded into the code stream, and the three-dimensional information of the patch to be encoded is encoded into the code stream, and the auxiliary information of the patch to be encoded includes the three-dimensional information; or
    编入所述码流的语法元素还包括第二语法元素和第三语法元素，所述第二语法元素的取 值为假且所述第三语法元素的取值为假，以指示所述待编码patch的二维信息未编入所述码流，且所述待编码patch的三维信息未编入所述码流。The syntax element compiled into the code stream further includes a second syntax element and a third syntax element, the value of the second syntax element is false and the value of the third syntax element is false to indicate the waiting The two-dimensional information of the encoded patch is not encoded into the code stream, and the three-dimensional information of the patch to be encoded is not encoded into the code stream.
  41. 如权利要求40所述的装置，其特征在于，当所述第一语法元素指示所述待编码patch不具有参考patch时，所述待编码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。The device according to claim 40, wherein when the first syntax element indicates that the patch to be encoded does not have a reference patch, the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and rotation transformation. Information and normal axis information.
  42. 如权利要求40或41所述的装置，其特征在于，所述第一语法元素指示所述待编码patch不具有参考patch，所述待编码patch为所属的当前点云帧中的首个patch，所述当前点云帧为首个点云帧。The device according to claim 40 or 41, wherein the first syntax element indicates that the patch to be encoded does not have a reference patch, and the patch to be encoded is the first patch in the current point cloud frame to which it belongs, The current point cloud frame is the first point cloud frame.
  43. 如权利要求40-42任一项所述的装置，其特征在于，所述待编码patch的参考patch包括于当前点云帧的参考帧中。The apparatus according to any one of claims 40-42, wherein the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.
  44. 如权利要求43所述的装置，其特征在于，所述待编码patch在所述当前点云帧中的位置与所述待编码patch的参考patch在所述当前点云帧的参考帧的位置相同。The apparatus of claim 43, wherein the position of the patch to be coded in the current point cloud frame is the same as the position of the reference patch of the patch to be coded in the reference frame of the current point cloud frame .
  45. 一种点云解码装置，其特征在于，所述装置包括：A point cloud decoding device, characterized in that the device includes:
    辅助信息解码模块，用于从码流中解析待解码点云块patch的第一语法元素；The auxiliary information decoding module is used to parse the first syntax element of the point cloud block patch to be decoded from the code stream;
    所述辅助信息解码模块，还用于当所述第一语法元素指示所述待解码patch具有参考patch时，从所述码流中解析所述待解码patch的第二语法元素，或者，从所述码流中解析所述待解码patch的第二语法元素和第三语法元素；The auxiliary information decoding module is further configured to parse the second syntax element of the patch to be decoded from the code stream when the first syntax element indicates that the patch to be decoded has a reference patch, or Parsing the second syntax element and the third syntax element of the patch to be decoded in the code stream;
    当所述第二语法元素的取值为真时，从所述码流中解析所述待解码patch的二维信息，将所述待解码patch的参考patch的三维信息作为所述待解码patch的三维信息；或者When the value of the second syntax element is true, the two-dimensional information of the patch to be decoded is parsed from the code stream, and the three-dimensional information of the reference patch of the patch to be decoded is used as the patch to be decoded. Three-dimensional information; or
    当所述第二语法元素的取值为假且所述第三语法元素的取值为真时，从所述码流中解析所述待解码patch的三维信息，将所述待解码patch的参考patch的二维信息作为所述待解码patch的二维信息；或者When the value of the second syntax element is false and the value of the third syntax element is true, the three-dimensional information of the patch to be decoded is parsed from the code stream, and the reference of the patch to be decoded The two-dimensional information of the patch is used as the two-dimensional information of the patch to be decoded; or
    当所述第二语法元素的取值为假且所述第三语法元素的取值为假时，将所述待解码patch的参考patch的二维信息作为所述待解码patch的二维信息，将所述待解码patch的参考patch的三维信息作为所述待解码patch的三维信息；When the value of the second syntax element is false and the value of the third syntax element is false, use the two-dimensional information of the reference patch of the patch to be decoded as the two-dimensional information of the patch to be decoded, Using the three-dimensional information of the reference patch of the patch to be decoded as the three-dimensional information of the patch to be decoded;
    重构模块，用于根据所述待解码patch的辅助信息重构所述待解码patch，所述待解码patch的辅助信息包括所述待解码patch的二维信息和所述待解码patch的三维信息。The reconstruction module is configured to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded, and the auxiliary information of the patch to be decoded includes the two-dimensional information of the patch to be decoded and the three-dimensional information of the patch to be decoded .
  46. 如权利要求45所述的装置，其特征在于，所述辅助信息解码模块还用于：The device of claim 45, wherein the auxiliary information decoding module is further configured to:
    当所述第一语法元素指示所述待解码patch不具有参考patch时，从所述码流中解析所述待解码patch的辅助信息，所述待解码patch的辅助信息包括二维信息、三维信息、旋转变换信息和法向坐标轴信息。When the first syntax element indicates that the patch to be decoded does not have a reference patch, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information of the patch to be decoded includes two-dimensional information and three-dimensional information , Rotation transformation information and normal coordinate axis information.
  47. 如权利要求45或46所述的装置，其特征在于，所述辅助信息解码模块还用于：The device according to claim 45 or 46, wherein the auxiliary information decoding module is further configured to:
    当所述待解码patch为首个patch，且所述第一语法元素指示所述待解码patch不具有参考patch时，确定所述待解码patch所属的当前点云帧为首个点云帧。When the patch to be decoded is the first patch and the first syntax element indicates that the patch to be decoded does not have a reference patch, it is determined that the current point cloud frame to which the patch to be decoded belongs is the first point cloud frame.
  48. 如权利要求45-47任一项所述的装置，其特征在于，所述辅助信息解码模块还用于：The device according to any one of claims 45-47, wherein the auxiliary information decoding module is further configured to:
    当所述待解码patch为当前点云帧中的首个patch时，从所述码流中解析所述待解码patch的第一语法元素；或者，When the patch to be decoded is the first patch in the current point cloud frame, parse the first syntax element of the patch to be decoded from the code stream; or,
    当所述待解码patch为当前点云帧中的首个patch时，从所述码流中解析所述当前点云帧包括的多个patch的多个第一语法元素，其中解析出的多个第一语法元素中包括所述待解码patch的第一语法元素。When the patch to be decoded is the first patch in the current point cloud frame, the multiple first syntax elements of multiple patches included in the current point cloud frame are parsed from the code stream, and the multiple parsed The first syntax element includes the first syntax element of the patch to be decoded.
  49. 如权利要求45-48任一项所述的装置，其特征在于，所述待解码patch的参考patch包括于当前点云帧的参考帧中。The device according to any one of claims 45-48, wherein the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.
  50. 如权利要求49所述的装置，其特征在于，所述待解码patch在所述当前点云帧中的位置与所述待解码patch的参考patch在所述当前点云帧的参考帧的位置相同。The apparatus of claim 49, wherein the position of the patch to be decoded in the current point cloud frame is the same as the position of the reference patch of the patch to be decoded in the reference frame of the current point cloud frame .
  51. 一种计算机可读存储介质，其特征在于，包括程序代码，所述程序代码在计算机上运行时，使得所述计算机执行如权利要求1至5任一项、权利要求13或者权利要求29至33任一项所述的点云编码方法。A computer-readable storage medium, characterized by comprising program code, which when run on a computer, causes the computer to execute any one of claims 1 to 5, claim 13 or claims 29 to 33 Any of the point cloud coding methods.
  52. 一种计算机可读存储介质，其特征在于，包括程序代码，所述程序代码在计算机上运行时，使得所述计算机执行如权利要求6至12任一项、权利要求14或者权利要求34至39任一项所述的点云解码方法。A computer-readable storage medium, characterized by comprising program code, which when running on a computer, causes the computer to execute any one of claims 6 to 12, claim 14 or claims 34 to 39 Any one of the point cloud decoding methods.
PCT/CN2020/080140 2019-03-19 2020-03-19 Point cloud encoding method, point cloud decoding method, apparatus, and storage medium WO2020187283A1 (en)

Priority Applications (6)

Application Number Priority Date Filing Date Title
JP2021556407A JP7453992B2 (en) 2019-03-19 2020-03-19 Point cloud encoding method and device, point cloud decoding method and device, and storage medium
BR112021018435A BR112021018435A2 (en) 2019-03-19 2020-03-19 Point cloud encoding method and apparatus, point cloud decoding method and apparatus, and storage medium
SG11202110220RA SG11202110220RA (en) 2019-03-19 2020-03-19 Point cloud encoding method and apparatus, point cloud decoding method and apparatus, and storage medium
EP20774250.3A EP3934250A4 (en) 2019-03-19 2020-03-19 Point cloud encoding method, point cloud decoding method, apparatus, and storage medium
KR1020217033214A KR20210134401A (en) 2019-03-19 2020-03-19 Point cloud encoding method and apparatus, point cloud decoding method and apparatus, and storage medium
US17/477,812 US20220007037A1 (en) 2019-03-19 2021-09-17 Point cloud encoding method and apparatus, point cloud decoding method and apparatus, and storage medium

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN201910210181 2019-03-19
CN201910210181.3 2019-03-19
CN201910214557.8A CN111726616B (en) 2019-03-19 2019-03-20 Point cloud encoding method, point cloud decoding method, device and storage medium
CN201910214557.8 2019-03-20

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/477,812 Continuation US20220007037A1 (en) 2019-03-19 2021-09-17 Point cloud encoding method and apparatus, point cloud decoding method and apparatus, and storage medium

Publications (1)

Publication Number Publication Date
WO2020187283A1 true WO2020187283A1 (en) 2020-09-24

Family

ID=72519591

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/080140 WO2020187283A1 (en) 2019-03-19 2020-03-19 Point cloud encoding method, point cloud decoding method, apparatus, and storage medium

Country Status (1)

Country Link
WO (1) WO2020187283A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170214943A1 (en) * 2016-01-22 2017-07-27 Mitsubishi Electric Research Laboratories, Inc. Point Cloud Compression using Prediction and Shape-Adaptive Transforms
CN108322742A (en) * 2018-02-11 2018-07-24 北京大学深圳研究生院 A kind of point cloud genera compression method based on intra prediction
CN108833927A (en) * 2018-05-03 2018-11-16 北京大学深圳研究生院 A kind of point cloud genera compression method based on 0 element in deletion quantization matrix

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170214943A1 (en) * 2016-01-22 2017-07-27 Mitsubishi Electric Research Laboratories, Inc. Point Cloud Compression using Prediction and Shape-Adaptive Transforms
CN108322742A (en) * 2018-02-11 2018-07-24 北京大学深圳研究生院 A kind of point cloud genera compression method based on intra prediction
CN108833927A (en) * 2018-05-03 2018-11-16 北京大学深圳研究生院 A kind of point cloud genera compression method based on 0 element in deletion quantization matrix

Similar Documents

Publication Publication Date Title
JP7057453B2 (en) Point cloud coding method, point cloud decoding method, encoder, and decoder
WO2020151496A1 (en) Point cloud encoding/decoding method and apparatus
US11388442B2 (en) Point cloud encoding method, point cloud decoding method, encoder, and decoder
WO2020011265A1 (en) Point cloud coding and decoding method and coder-decoder
US11961265B2 (en) Point cloud encoding and decoding method and apparatus
CN110944187B (en) Point cloud encoding method and encoder
JP2021518694A (en) Methods and devices for coding / decoding point clouds that represent 3D objects
WO2020147379A1 (en) Point cloud filtering method and device and storage medium
US20220007037A1 (en) Point cloud encoding method and apparatus, point cloud decoding method and apparatus, and storage medium
JP2021518681A (en) Methods and devices for coding / decoding point clouds that represent 3D objects
WO2020187283A1 (en) Point cloud encoding method, point cloud decoding method, apparatus, and storage medium
WO2020220941A1 (en) Point cloud encoding method, point cloud decoding method, apparatuses, and storage medium
US20230156229A1 (en) Methods and apparatus to signal enabled views per atlas in immersive video
WO2020015517A1 (en) Point cloud encoding method, point cloud decoding method, encoder and decoder
US20240129537A1 (en) Method and apparatus for signaling cmaf switching sets in isobmff
WO2020187191A1 (en) Point cloud encoding and decoding method and codec
WO2020057338A1 (en) Point cloud coding method and encoder
WO2023071462A1 (en) Point cloud encoding method and apparatus, point cloud decoding method and apparatus, and device, storage medium and program product
KR20240093609A (en) A transmitting device for point cloud data and a method performed in the transmitting device, and a receiving device for point cloud data and a method performed in the receiving device
KR20240113797A (en) Point cloud coding method, devices and media
CN114760525A (en) Video generation and playing method, device, equipment and medium
CN117751387A (en) Face mesh connectivity coding
US20120263390A1 (en) Decoding a quilted graphic 2d image representation into a digital asset

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20774250

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021556407

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

REG Reference to national code

Ref country code: BR

Ref legal event code: B01A

Ref document number: 112021018435

Country of ref document: BR

ENP Entry into the national phase

Ref document number: 2020774250

Country of ref document: EP

Effective date: 20210930

ENP Entry into the national phase

Ref document number: 20217033214

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 112021018435

Country of ref document: BR

Kind code of ref document: A2

Effective date: 20210916