CN111866484A

CN111866484A - Point cloud encoding method, point cloud decoding method, point cloud encoding device, point cloud decoding device, and storage medium

Info

Publication number: CN111866484A
Application number: CN201910364032.2A
Authority: CN
Inventors: 蔡康颖; 张德军
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2019-04-30
Filing date: 2019-04-30
Publication date: 2020-10-30
Anticipated expiration: 2039-04-30
Also published as: WO2020220941A1; CN111866484B

Abstract

The application discloses a point cloud encoding method, a point cloud decoding device and a storage medium, and belongs to the field of data processing. The method comprises the following steps: the auxiliary information of the patch to be coded and the first identifier of the patch to be coded are obtained first. Because the number of the first type of patch in each point cloud frame in the current point cloud group can be the same, when the first identifier is a first value and the to-be-encoded patch is the first patch in the current point cloud frame, the auxiliary information of the to-be-encoded patch and the first identifier can be encoded into the code stream. When the first identifier is the first value and the to-be-encoded patch is a non-first patch in the current point cloud frame, the auxiliary information of the to-be-encoded patch can be encoded into the code stream, and the first identifier of the to-be-encoded patch does not need to be encoded into the code stream, so that the format of the encoded code stream can be simplified, the bit cost of the code stream is further reduced, and the encoding efficiency is improved.

Description

Point cloud encoding method, point cloud decoding method, point cloud encoding device, point cloud decoding device, and storage medium

Technical Field

The present disclosure relates to the field of data processing technologies, and in particular, to a point cloud encoding method, a point cloud decoding method, a point cloud encoding device, and a point cloud decoding device.

Background

With the continuous development of the three-dimensional sensing technology, the point cloud is more and more convenient to acquire, the quality of the acquired point cloud is higher and higher, and the scale of the acquired point cloud is larger and larger. Therefore, how to effectively encode and decode the point cloud becomes a problem to be solved urgently at present.

Disclosure of Invention

The application provides a point cloud encoding method, a point cloud decoding device and a storage medium, which can solve the problem of how to effectively encode and decode point clouds in the related art. The technical scheme is as follows:

in a first aspect, a point cloud encoding method is provided, which includes: acquiring auxiliary information of a patch (point cloud block) to be encoded and a first identifier of the patch to be encoded; when the first identification is a first value, encoding auxiliary information and the first identification of the patch to be encoded into a code stream, wherein the patch to be encoded is the first patch in the current point cloud frame; or when the first identifier is a first value, encoding auxiliary information of a patch to be encoded into the code stream, wherein the patch to be encoded is a non-first patch in the current point cloud frame; the first identifier is a first value to indicate that the to-be-encoded patch is a first type of patch, and the auxiliary information of the to-be-encoded patch includes three-dimensional information.

It should be noted that the patch to be encoded may be any one of the patches included in the current point cloud frame to be encoded. The first identification may indicate a type of patch to be encoded. The syntax element of the first flag may be a patch _ mode or a patch _ type, etc. The values of the first identification are different, and the types of the patch to be coded are different.

When the first identification is the first value, the to-be-encoded patch can be indicated to be the first type of patch, the first type of patch can be a global matching patch in the current point cloud set, and the auxiliary information of the to-be-encoded patch includes three-dimensional information but does not include two-dimensional information, rotation transformation information and normal coordinate axis information.

When the first type of patch exists in the current point cloud group, because the number of the first type of patch in each point cloud frame in the current point cloud group can be the same, and when the number of the first type of patch is greater than 1, the first type of patch in each point cloud frame in the current point cloud group can be sequentially arranged according to the same sequence, when the first identifier is a first value and the to-be-encoded patch is a first patch in the current point cloud frame, it indicates that the to-be-encoded patch is a first type of patch in the current point cloud frame, and at this time, the auxiliary information of the to-be-encoded patch and the first identifier can be encoded into a code stream; when the first identifier is a first value and the to-be-encoded patch is a non-first patch in the current point cloud frame, it is indicated that the to-be-encoded patch is a non-first type patch in the current point cloud frame, and at this time, the auxiliary information of the to-be-encoded patch can be encoded into the code stream, and the first identifier of the to-be-encoded patch does not need to be encoded into the code stream, so that the format of the encoded code stream can be simplified.

Because the first type of patch has the patches with matching relations in all the point cloud frames in the current point cloud group, the two-dimensional information, the rotation transformation information and the normal coordinate axis information of the first type of patch in the current point cloud group are the same, and the three-dimensional information may be different. Therefore, in order to reduce the bit overhead of the code stream and further improve the encoding efficiency, when the first identifier is the first value, the auxiliary information of the patch to be encoded may include three-dimensional information, but not include two-dimensional information, rotation transformation information, and normal coordinate axis information.

In a possible case, the patches in each point cloud frame may be arranged according to the order of the first type of patch, the second type of patch, and the third type of patch, and then encoded according to the order of the first type of patch, the second type of patch, and the third type of patch. Therefore, if the first type of patch exists in the current point cloud group, the first patch in each point cloud frame is the first type of patch. The number of the first type of patch in each point cloud frame in the current point cloud group may be the same, and when the number of the first type of patch is greater than 1, the first type of patch in each point cloud frame in the current point cloud group may be sequentially arranged in the same order, that is, the first type of patch at the same position in each point cloud frame has a matching relationship.

However, in order to distinguish which point cloud frames may form a point cloud group, as shown in fig. 10, for a first point cloud frame in the current point cloud group, the first identifiers of all the patches included in the first point cloud frame may be set as a third value, and the patches in the other point cloud frames are arranged according to the order of the first type of patch, the second type of patch, and the third type of patch. Thus, when the first identifier of the patch to be encoded is a first value, it may be determined that the current point cloud frame is a non-first point cloud frame within the current point cloud group. Moreover, the current point cloud frame may be a kth point cloud frame in the current point cloud group, where K is an integer greater than or equal to 3. That is, the 3 rd point cloud frame in the current point cloud group and each point cloud frame after the 3 rd point cloud frame may be encoded in the manner of S902.

The point cloud frames in the current point cloud group can be numbered according to two numbering modes. The first numbering scheme: point cloud frame 0, point cloud frame 1, point cloud frame 2 … …; the second numbering scheme: point cloud frame 1, point cloud frame 2, point cloud frame 3 … …. The kth point cloud frame may refer to the point cloud frame 2 and any point cloud frame after the point cloud frame 2 in the first numbering manner, or may refer to the point cloud frame 3 and any point cloud frame after the point cloud frame 3 in the second numbering manner. No matter what numbering method the point cloud frames in the current point cloud group are numbered, the Kth point cloud frame can refer to the actual point cloud frame 3 and the point cloud frames after the point cloud frame 3 in the current point cloud group.

Based on the above description, the 3 rd point cloud frame in the current point cloud group and each point cloud frame after the 3 rd point cloud frame can be encoded in the above manner, and for the 2 nd point cloud frame in the current point cloud group, the following two manners can be encoded:

in a first possible implementation manner, if the current point cloud frame is the 2 nd point cloud frame in the current point cloud group, then, when the first identifier of the to-be-encoded patch is the first value and the to-be-encoded patch is the first patch in the current point cloud frame, the first identifier of the to-be-encoded patch, the number of the first type of patches in the current point cloud group, and the auxiliary information of the to-be-encoded patch may be encoded into the code stream. When the first identifier of the to-be-encoded patch is a first value and the to-be-encoded patch is a non-first patch in the current point cloud frame, the auxiliary information of the to-be-encoded patch can be encoded into the code stream without encoding the first identifier of the to-be-encoded patch into the code stream.

In the process of setting the first identifiers of all the patches included in the first point cloud frame in the current point cloud group as the third value, the number of the first-type patches in the current point cloud group can be acquired.

Moreover, the operation of encoding the first identifier of the patch to be encoded, the number of the first type of patch in the current point cloud group, and the auxiliary information of the patch to be encoded into the code stream may be: and encoding the first identification of the patch to be encoded into a code stream, encoding the number of the first type of patch in the current point cloud group into the code stream, and finally encoding the auxiliary information of the patch to be encoded into the code stream. That is, the number of the first type of patch in the current point cloud group is coded into the code stream after the first identifier of the patch to be coded.

It should be noted that the number of the first type of patch may be coded into the code stream by using a fixed-length coding method, or the number of the first type of patch may be coded into the code stream by using a variable-length coding method, which is not limited in this embodiment of the present application.

In a second possible implementation manner, if the current point cloud frame is the 2 nd point cloud frame in the current point cloud group, when the first identifier of the patch to be encoded is the first value, the first identifier of the patch to be encoded and the auxiliary information may be encoded into the code stream. That is, the first identifiers and the auxiliary information of all the first type patches in the 2 nd point cloud frame need to be coded into the code stream, so that the number of the first type patches in the current point cloud group can be indicated, and the number of the first type patches in the current point cloud group does not need to be coded into the code stream.

No matter what implementation method is used for coding the first type of patch in the 2 nd point cloud frame, the number of the first type of patch in the current point cloud group can be determined through the 2 nd point cloud frame. Therefore, for the 3 rd point cloud frame in the current point cloud group and the point cloud frames after the 3 rd point cloud frame, only the first patch, that is, the first identifier of the first type of patch, can be encoded into the code stream, and the first identifiers of the non-first type of patch are not encoded into the code stream.

In summary, when the first identifier of the patch to be encoded is the third value and the patch to be encoded is the first patch in the current point cloud frame, it can be determined that the current point cloud frame is the first point cloud frame in the current point cloud group. Therefore, the length of the point cloud frame of the current point cloud group is not required to be coded in the code stream. Moreover, for the point cloud frame 3 and the point cloud frames after the point cloud frame 3 in the current point cloud group, only the first identifier of the first type of patch can be coded into the code stream, and the first identifiers of the non-first type of patch are not required to be coded into the code stream, so that the coding efficiency is improved, and the bit cost of the code stream is reduced.

It should be noted that, in a possible implementation manner, the first patch in the current point cloud frame may correspond to an identification information, which may also be referred to as an index, and the first patch in the current point cloud frame may be uniquely indicated by the identification information, and the identification information of the first patch may be usually encoded in the code stream. For other patches in the current point cloud frame, a counter may be employed to determine that the other patches are the next patches in the current point cloud frame. Wherein, the counter starts counting when determining the first patch of the current point cloud frame.

It should be noted that the three-dimensional information of the patch to be encoded may include a shift of the patch to be encoded in the 3D space along the tangential direction (3D _ shift _ tangent _ axis), a shift of the patch to be encoded in the 3D space along the bi-tangential direction (3D _ shift _ bitangent _ axis), and a shift of the patch to be encoded in the 3D space along the normal direction (3D _ shift _ normal _ axis).

As an example, when the first identifier is the first value, the three-dimensional information of the to-be-encoded patch may be encoded into the code stream according to the three-dimensional information of the reference patch of the to-be-encoded patch, specifically, the three-dimensional information of the to-be-encoded patch may be encoded into the code stream by using a differential encoding method, or may be directly encoded into the code stream, where the specific encoding method is not limited. And adopting a fixed-length coding mode to code the first identifier of the patch to be coded into the code stream.

The encoding of the three-dimensional information of the to-be-encoded patch into the code stream by adopting a differential encoding mode refers to encoding a difference value between the three-dimensional information of the to-be-encoded patch and the three-dimensional information of the reference patch of the to-be-encoded patch into the code stream.

When the first identification is a second value or a third value, the auxiliary information of the patch to be coded and the first identification are coded into the code stream.

When the first identification is the second value, the to-be-encoded patch is indicated to be the second type of patch, the second type of patch can be a local matching patch in the current point cloud set, and the auxiliary information of the to-be-encoded patch comprises two-dimensional information, three-dimensional information and identification information of a reference patch of the to-be-encoded patch. When the first identification is a third value, the to-be-encoded patch is indicated to be a third type of patch, the third type of patch can be a non-matching patch in the current point cloud set, and the auxiliary information of the to-be-encoded patch comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.

Because the second type of patch has the patch with the matching relation in the reference frame of the current point cloud frame, the rotational transformation information between the second type of patch and the reference patch is the same as the normal coordinate axis information, and the two-dimensional information and the three-dimensional information may not be the same, and the third type of patch does not have the patch with the matching relation in the reference frame of the current point cloud frame, therefore, in order to reduce the bit overhead of the code stream, and further improve the coding efficiency, when the first identifier is the first value, the auxiliary information of the patch to be coded may include the three-dimensional information, but does not include the two-dimensional information, the rotational transformation information and the normal coordinate axis information. When the first flag is the second value, the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, and flag information of a reference patch of the patch to be encoded, but does not include rotation transformation information and normal coordinate axis information. When the first identifier is the third value, the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.

It should be noted that the two-dimensional information of the patch to be encoded may include a shift (2d _ shift _ u) of the patch to be encoded in the current point cloud frame occupancy graph along the u-axis direction, a shift (2d _ shift _ v) of the patch to be encoded in the current point cloud frame occupancy graph along the v-axis direction, a width (2d _ size _ u) of the patch to be encoded in the current point cloud frame occupancy graph, and a height (2d _ size _ v) of the patch to be encoded in the current point cloud frame occupancy graph. In addition, the syntax element of the reference patch of the patch to be encoded may be a patch index for indicating identification information of the reference patch; the syntax element may further include identification information of a reference frame in which the patch is located: the frameIndex. The syntax element of the rotation transform information of the patch to be encoded may be rotation. Furthermore, the identification information of the reference patch of the patch to be encoded may be used to indicate the reference patch of the patch to be encoded. The normal coordinate axis information may be the normal axis index of the patch to be encoded, e.g., the normalAxis normal axis index may be 0,1, or 2 for the X, Y, and Z axes, respectively.

As an example, when the first identifier is the second value, the two-dimensional information and the three-dimensional information of the to-be-encoded patch may be encoded into the code stream according to the two-dimensional information and the three-dimensional information of the reference patch of the to-be-encoded patch, specifically, the two-dimensional information and the three-dimensional information of the to-be-encoded patch may be encoded into the code stream by using a differential encoding method, or the two-dimensional information and the three-dimensional information of the to-be-encoded patch may be encoded into the code stream by using a more accurate prediction differential method, where a specific encoding method is not limited. And encoding the identification information and the first identification of the reference patch of the patch to be encoded into the code stream by adopting a fixed-length encoding mode. When the first identifier is the third value, the two-dimensional information, the three-dimensional information, the rotation transformation information, the normal coordinate axis information, and the first identifier of the patch to be encoded may be encoded into the code stream by using a fixed length encoding method, or the two-dimensional information, the three-dimensional information, the rotation transformation information, the normal coordinate axis information, and the first identifier of the patch to be encoded may be encoded into the code stream by using a variable length encoding method.

The encoding of the two-dimensional information and the three-dimensional information of the to-be-encoded patch into the code stream by adopting a differential encoding mode refers to encoding of a difference value between the two-dimensional information of the to-be-encoded patch and the two-dimensional information of the reference patch of the to-be-encoded patch into the code stream, and encoding of a difference value between the three-dimensional information of the to-be-encoded patch and the three-dimensional information of the reference patch of the to-be-encoded patch into the code stream.

Based on the above description, for the current point cloud frame, the first identifier of the first type of patch, the first identifier of each second type of patch, and the first identifier of each third type of patch in the current point cloud frame need to be encoded into the code stream. The auxiliary information of each patch included in the current point cloud frame also needs to be coded into the code stream. There may be three possible implementations for the way the first identification and the auxiliary information are encoded as follows.

The first possible implementation manner may pack and encode the first identifiers of the multiple patches included in the current point cloud frame together into the code stream, where the first identifiers of the multiple patches include a first identifier of a patch to be encoded, and the first identifiers of the patches whose remaining values are the second value or the third value except the patch to be encoded in the multiple patches. And for the auxiliary information of each patch included in the current point cloud frame, the auxiliary information can be respectively packed and coded into a code stream. For example, the first identifier of the first patch in the current point cloud frame and the first identifier of the patch other than the first patch, which takes the second value or the third value as the value, may be packed together and encoded into the code stream. Then, packing the auxiliary information of each patch included in the current point cloud frame one by one and encoding the auxiliary information into a code stream.

In a second possible implementation manner, the first identifier of the first patch in the current point cloud frame and the first identifier of the patch whose value is the second value or the third value except the first patch may be respectively packed and encoded into the code stream, and the auxiliary information of each patch included in the current point cloud frame may also be respectively packed and encoded into the code stream. For example, the first identifier of the first patch in the current point cloud frame may be encoded into the code stream, and then the auxiliary information of the first patch may be encoded into the code stream. Then, the auxiliary information of other first-type patches in the current point cloud frame is coded into the code stream one by one. Finally, the remaining patches, i.e., the second-type patch and the third-type patch, may be encoded in a manner of encoding the first identifier first and then encoding the auxiliary information.

In a third possible implementation manner, the first identifiers of the multiple patches included in the current point cloud frame may be packed together and coded into the code stream, where the first identifiers of the multiple patches include a first identifier of a patch to be encoded, and the first identifiers of the patches whose values are the second value or the third value are left in the multiple patches except for the patch to be encoded. For the auxiliary information of each patch included in the current point cloud frame, the same type of information in the auxiliary information of each patch may be packed together and encoded into the code stream. Different types of information in the auxiliary information of each patch can be packed and coded into the code stream respectively.

For example, the first identifier of the first patch in the current point cloud frame and the first identifier of the patch other than the first patch, which takes the second value or the third value as the value, may be packed together and encoded into the code stream. If the auxiliary information of each patch includes rotation transformation information and normal coordinate axis information, while some auxiliary information of the patches includes two-dimensional information or three-dimensional information, and some auxiliary information of the patches does not include two-dimensional information or three-dimensional information, then the rotation transformation information and the normal coordinate axis information in the auxiliary information of each patch are packed together and encoded into the code stream. And respectively packaging the two-dimensional information and the three-dimensional information in the auxiliary information of each patch and coding the two-dimensional information and the three-dimensional information into a code stream.

The first value, the second value, and the third value may be respectively 10, 11, and 00 of a binary system, or the first value, the second value, and the third value may be respectively 11, 10, and 00 of a binary system, and the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame, which is not limited in this embodiment of the application.

In order to further reduce the bit overhead of the code stream, the patches in each point cloud frame in the current point cloud group may be sequentially arranged according to the order of the first type of patch, the second type of patch, and the third type of patch, and at this time, for the third type of patch, if the patch to be encoded is the first third type of patch, then the encoding may be performed by using a fixed-length encoding method, or a variable-length encoding method may be used, which is not limited in this embodiment of the present application. If the patch to be coded is not the first third type of patch, at this time, the two-dimensional information and the three-dimensional information of the patch to be coded can be coded into the code stream by adopting a differential coding mode according to the three-dimensional information of the first third type of patch, and other information is coded by adopting the fixed length coding mode.

Since different values of the first flag indicate different types of patch. Therefore, different types of patches can be distinguished through the first identifier, and for the different types of patches, the content included in the auxiliary information coded into the code stream may be different, and for the first type of patch and the second type of patch, only part of the auxiliary information is coded into the code stream. Therefore, the format of the code stream can be simplified, the bit overhead of the code stream is further reduced, and the coding efficiency is improved.

In a second aspect, a point cloud decoding method is provided, which includes: when the point cloud block patch to be decoded is the first patch in the current point cloud frame, analyzing a first identifier of the patch to be decoded from the code stream; when the first identification is a first value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded; when the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patches in the reference point cloud frame of the current point cloud frame, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded; and reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

Because the first identifier of the first patch in the current point cloud frame is coded into the code stream, when the point cloud block patch to be decoded is the first patch in the current point cloud frame, the first identifier of the patch to be decoded can be analyzed from the code stream.

In a possible implementation manner, whether the code stream includes the identification information of the to-be-decoded patch or not may be determined, and if the code stream includes the identification information of the to-be-decoded patch, it may be determined that the to-be-decoded patch is the first patch in the current point cloud frame. If the identification information of the patch to be decoded is not included, it may be determined that the patch to be decoded is a non-first patch in the current point cloud frame. At this time, it may be determined from the counter that the patch to be decoded is the next patch in the current point cloud frame. Wherein, the counter starts counting when determining the first patch of the current point cloud frame.

It should be noted that, when the first identifier is a first value, it indicates that the to-be-decoded patch is a first type of patch, that is, the to-be-decoded patch has a matching patch in all point cloud frames in the current point cloud group. At this time, auxiliary information of the patch to be decoded, which includes three-dimensional information of the patch to be decoded but does not include two-dimensional information, rotation conversion information, and normal coordinate axis information of the patch to be decoded, may be parsed from the codestream.

As an example, the number of the first type of patch may be parsed from the code stream by a method matching with the encoding end. For example, the number of the first type of patch may be analyzed from the code stream by adopting a fixed-length decoding manner or a variable-length decoding manner.

As can be seen from the method provided by the first aspect, when the first identifier is the first value, it may indicate that the current point cloud frame is a non-first point cloud frame within the current point cloud group. At this time, if the current point cloud frame is the 2 nd point cloud frame in the current point cloud group, the patch to be decoded may be decoded through the following two possible implementation manners.

In a first possible implementation manner, the first identifier of the first patch in the 2 nd point cloud frame in the current point cloud group and the number of the first type of patches included in each point cloud frame in the current point cloud group are jointly encoded into the code stream. At this time, if the current point cloud frame is the 2 nd point cloud frame in the current point cloud group and the patch to be decoded is the first patch in the current point cloud frame, the first identifier of the patch to be decoded may be analyzed from the code stream, and when the first identifier is the first value, the number of the first type of patches included in each point cloud frame in the current point cloud group may be analyzed from the code stream. And then analyzing the auxiliary information of the patch to be decoded from the code stream.

In a second possible implementation manner, the code stream is encoded with the first identifiers of the first types of patches included in the 2 nd point cloud frame in the current point cloud group, at this time, if the current point cloud frame is the 2 nd point cloud frame in the current point cloud group, and the to-be-decoded patch is the first patch, the first identifier of the to-be-decoded patch can be analyzed from the code stream, and when the first identifier is the first value, the auxiliary information of the to-be-decoded patch is analyzed from the code stream.

According to the method provided by the first aspect, in the encoding process of the first type of patch, the first identifier of the first type of patch that is not the first type of patch is not encoded into the code stream, so that when the to-be-decoded patch is the nth patch in the current point cloud frame and n is greater than 1 and less than or equal to the number of the first type of patches in the reference point cloud frame of the current point cloud frame, it is indicated that the to-be-decoded patch is the first type of patch in the current point cloud frame, and at this time, the auxiliary information of the to-be-decoded patch can be analyzed from the code stream, and the auxiliary information includes three-dimensional information of the to-be-decoded patch.

Since the auxiliary information of the first type of patch coded into the code stream in the coding process includes three-dimensional information, but does not include two-dimensional information, rotation transformation information and normal coordinate axis information, when the to-be-decoded patch is the first type of patch, the auxiliary information analyzed from the code stream includes the three-dimensional information of the to-be-decoded patch, but does not include the two-dimensional information, the rotation transformation information and the normal coordinate axis information of the to-be-decoded patch. At this time, two-dimensional information, rotation transformation information, and normal coordinate axis information of the to-be-decoded patch may also be acquired according to the auxiliary information of the reference patch of the to-be-decoded patch.

As an example, the decoding may be performed by a method matching with the encoding end, for example, the three-dimensional information of the patch to be decoded may be analyzed from the code stream by using a differential decoding method. For other auxiliary information of the patch to be decoded, for example, the two-dimensional information, the rotational transformation information, and the normal coordinate axis information of the patch to be decoded, the identification information of the reference frame of the current point cloud frame may be analyzed from the code stream, so as to determine the reference frame of the current point cloud frame according to the identification information of the reference frame, then the two-dimensional information, the rotational transformation information, and the normal coordinate axis information of the reference patch of the patch to be decoded are obtained from the reference frame of the current point cloud frame, and the two-dimensional information, the rotational transformation information, and the normal coordinate axis information of the reference patch are used as the two-dimensional information, the rotational transformation information, and the normal coordinate axis information of the patch to be decoded. For convenience of description, the reference frame of the current point cloud frame is marked as a first reference frame, and the reference frame corresponding to the reference frame of the current point cloud frame is marked as a second reference frame, so that the auxiliary information of the reference patch in the first reference frame can be determined according to the auxiliary information of the corresponding reference patch in the second reference frame. By the method, the auxiliary information of the patch to be decoded can be acquired step by step according to the reference frame of the current point cloud frame, the reference frame corresponding to the reference frame of the current point cloud frame and the like. In addition, the two-dimensional information, the rotation transformation information and the normal coordinate axis information of the patch with the same position as that of the to-be-decoded patch in the current point cloud frame can be acquired from the decoded point cloud, and the acquired two-dimensional information, the rotation transformation information and the normal coordinate axis information can be used as the two-dimensional information, the rotation transformation information and the normal coordinate axis information of the to-be-decoded patch.

In other words, when the first identifier is the first value, after the three-dimensional information of the to-be-decoded patch is analyzed from the code stream, the auxiliary information of the to-be-decoded patch can be acquired according to the auxiliary information of the reference patch of the to-be-decoded patch, where the auxiliary information includes two-dimensional information, rotation transformation information, and normal coordinate axis information. Thus, reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded includes: and reconstructing the patch to be decoded according to the two-dimensional information, the three-dimensional information, the normal coordinate axis information and the rotation transformation information of the decoding patch.

When the patches in the point cloud are encoded according to the way that the first type of patches, the second type of patches, and the third type of patches are sequentially arranged, the reference patch of the to-be-decoded patch may be a patch whose position in the reference frame of the current point cloud frame is the same as the position of the to-be-decoded patch in the current point cloud frame.

Based on the above description, if the to-be-decoded patch is the non-first patch in the current point cloud frame, it may be determined by the counter that the to-be-decoded patch is the second patch in the current point cloud frame. Therefore, when the to-be-decoded patch is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first-type patches in the reference point cloud frame of the current point cloud frame, the to-be-decoded patch can be determined to be the non-first-type patch in the current point cloud frame, so that the auxiliary information of the to-be-decoded patch can be analyzed from the code stream, and the auxiliary information comprises three-dimensional information of the to-be-decoded patch.

The number of the first type of patch included in each point cloud frame in the current point cloud group can be determined in two ways. In a first possible implementation manner, after the first identifier of the first patch in the 2 nd point cloud frame in the current point cloud group is decoded, the number of the first type of patches included in each point cloud frame in the current point cloud group can be directly analyzed from the code stream. In a second possible implementation manner, after the first identifiers of all the patches of the 2 nd point cloud frame in the current point cloud group are decoded, the number of the patches of which the first identifier is the first value may be determined, so that the number of the first type of patches included in the 2 nd point cloud frame, that is, the number of the first type of patches included in each point cloud frame in the current point cloud group, may be determined. That is, the number of the first type patches included in each point cloud frame in the current point cloud group can be determined by the 2 nd point cloud frame.

In one possible scenario, the 2 nd point cloud frame within the current point cloud group may serve as a reference point cloud frame for the 3 rd point cloud frame, the 3 rd point cloud frame may serve as a reference point cloud frame for the 4 th point cloud frame, and so on. Therefore, the 3 rd point cloud frame and the point cloud frames after the 3 rd point cloud frame can determine whether n in the nth patch in the current point cloud frame is larger than 1 and smaller than or equal to the number of the first type of patches in the reference point cloud frame of the current point cloud frame according to the number of the first type of patches in the reference point cloud frame, so as to determine whether the to-be-decoded patches are the first type of patches.

In addition, the first identifier of the first patch in the current point cloud frame and the first identifier of the patch, which takes the second value or the third value, except the first patch can be packed and coded into the code stream respectively, or packed together and coded into the code stream. Therefore, for the way of packing and encoding the code stream respectively, if the patch to be decoded is the first patch in the current point cloud frame, the first identifier of the patch to be decoded can be analyzed from the code stream. Of course, for the patch whose value is the second value or the third value except the first patch, the first identifier of the patch to be decoded may also be analyzed from the code stream. That is, the first identifier of the first patch to be decoded in the current point cloud frame and the first identifier of the patch whose value is the second value or the third value other than the first patch may be sequentially analyzed. For the way of packing together and encoding into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, the first identifiers of the multiple patches included in the current point cloud frame may be analyzed from the code stream, where the analyzed first identifiers include the first identifier of the patch to be decoded. That is, the first identifiers of the plurality of patches included in the current point cloud frame can be obtained through one-time analysis, and the analysis does not need to be performed for multiple times. Similarly, when the same type of information in the auxiliary information of each patch included in the current point cloud frame is also packed and coded into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, the same type of information in the auxiliary information of each patch included in the current point cloud frame can be analyzed from the code stream. That is, the same type of information in the auxiliary information of each patch included in the current point cloud frame can be obtained through one-time analysis, and the analysis is not required to be performed for multiple times.

As an example, the first identifier of the patch to be decoded may be parsed from the code stream by using a method matching with the encoding end. For example, the first identifier of the patch to be decoded may be parsed from the code stream by using a fixed-length decoding method.

When the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and greater than the number of the first type of patches in the reference point cloud frame of the current point cloud frame, analyzing a first identifier of the patch to be decoded from the code stream.

In the encoding process of the second type of patch and the third type of patch, the first identifier is encoded into the code stream, so when the to-be-decoded patch is the nth patch in the current point cloud frame, and n is greater than 1 and greater than the number of the first type of patches in the reference point cloud frame of the current point cloud frame, it indicates that the to-be-decoded patch is the second type of patch or the third type of patch, and at this time, the first identifier of the to-be-decoded patch can be analyzed from the code stream.

When the first identification is the second value, the auxiliary information of the to-be-decoded patch is analyzed from the code stream, and the auxiliary information comprises two-dimensional information and three-dimensional information of the to-be-decoded patch and identification information of a reference patch of the to-be-decoded patch.

It should be noted that, when the first identifier is the second value, it indicates that the patch to be decoded is the second type of patch, that is, a reference patch matching the patch to be decoded exists in the reference frame of the current point cloud frame. At this time, the auxiliary information of the to-be-decoded patch, which includes the two-dimensional information, the three-dimensional information, and the identification information of the reference patch of the to-be-decoded patch but does not include the rotation transformation information and the normal coordinate axis information, may be parsed from the codestream.

When the first identifier is the second value, for the identifier information of the reference patch of the to-be-decoded patch, a method matching with the encoding end may be adopted to analyze from the code stream, for example, a fixed-length decoding manner may be adopted to analyze. For the rotation transformation information and the normal coordinate axis information of the to-be-decoded patch, the reference patch of the to-be-decoded patch can be determined through the identification information of the reference patch of the to-be-decoded patch, so that the rotation transformation information and the normal coordinate axis information of the reference patch of the to-be-decoded patch are acquired, and the rotation transformation information and the normal coordinate axis information of the reference patch of the to-be-decoded patch are used as the rotation transformation information and the normal coordinate axis information of the to-be-decoded patch. As an example, the two-dimensional information and the three-dimensional information of the patch to be decoded may be decoded by a method matching with the encoding end, for example, the two-dimensional information and the three-dimensional information of the patch to be decoded may be analyzed from the code stream in a differential decoding manner. That is, for the second type of patch, the two-dimensional information and the three-dimensional information can be analyzed from the code stream, and the rotation transformation information and the normal coordinate axis information are derived or obtained, so that the rotation transformation information and the normal coordinate axis information are prevented from being coded in the code stream, the bit overhead of the code stream is reduced, and the decoding efficiency is improved.

In other words, when the first identifier is the second value, after the two-dimensional information and the three-dimensional information of the to-be-decoded patch and the identifier information of the reference patch of the to-be-decoded patch are analyzed from the code stream, the rotation transformation information and the normal coordinate axis information of the to-be-decoded patch can be acquired according to the identifier information of the reference patch of the to-be-decoded patch. Thus, reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded includes: and reconstructing the patch to be decoded according to the two-dimensional information, the three-dimensional information, the normal coordinate axis information and the rotation transformation information of the patch to be decoded.

When the first identification is a third value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded.

It should be noted that, when the first identifier is a third value, it indicates that the patch to be decoded is a third type of patch, that is, there is no reference patch matching the patch to be decoded in the current point cloud set. At this time, the two-dimensional information, the three-dimensional information, the rotation transformation information and the normal coordinate axis information of the patch to be decoded can be analyzed from the code stream.

As an example, the decoding may be performed by using a method matching with the encoding end, for example, the fixed length decoding method may be used to analyze the two-dimensional information, the three-dimensional information, the rotation transformation information, and the normal coordinate axis information of the to-be-decoded patch from the code stream, or the variable length decoding method may be used to analyze the two-dimensional information, the three-dimensional information, the rotation transformation information, and the normal coordinate axis information of the to-be-decoded patch from the code stream, which is not limited in this embodiment of the present application. However, if the first auxiliary information of the third type of patch is encoded by a fixed-length encoding method or a variable-length encoding method, the two-dimensional information and the three-dimensional information of the third type of patch are encoded by a differential encoding method, and the other auxiliary information is encoded by a fixed-length encoding method. Then, for the non-first third type patch, according to the two-dimensional information and the three-dimensional information of the first third type patch, the two-dimensional information and the three-dimensional information of the patch to be decoded are analyzed from the code stream by adopting a differential decoding mode, and the rotation transformation information and the normal coordinate axis information of the patch to be decoded are analyzed from the code stream by adopting a fixed-length decoding mode. That is, for the third type patch, the non-first third type patch two-dimensional information and the non-first third type patch three-dimensional information can be analyzed from the code stream, and the rotation transformation information and the normal coordinate axis information are derived or obtained, so that the rotation transformation information and the normal coordinate axis information are prevented from being coded in the code stream, the bit overhead of the code stream is reduced, and the decoding efficiency is improved.

The first value, the second value, and the third value may be binary 10, 11, and 00, or the first value, the second value, and the third value may be binary 11, 10, and 00, respectively, which is not limited in this embodiment of the application.

Based on the above description, the first type of patch, the second type of patch, and the third type of patch in each point cloud frame in the current point cloud group may be sequentially arranged, at this time, if the to-be-decoded patch is the first patch in the current point cloud frame, then, after the first identifier of the to-be-decoded patch is analyzed from the code stream, it may be determined whether the current point cloud frame is the first frame point cloud in the current point cloud group according to the value of the first identifier. That is, when the patch to be decoded is the first patch in the current point cloud frame and the first identifier is the third value, it may be determined that the current point cloud frame is the first frame point cloud in the current point cloud group. Thus, the length of a point cloud group does not need to be coded in the code stream.

It should be noted that after the auxiliary information of the to-be-decoded patch is acquired, that is, after the two-dimensional information, the three-dimensional information, the rotation transformation information, and the normal coordinate axis information of the to-be-decoded patch are acquired, the to-be-decoded patch may be reconstructed according to, but not limited to, the auxiliary information, and the specific information required for reconstructing the to-be-decoded patch is not limited in this application.

In summary, for the first type of patch and the second type of patch, only a part of auxiliary information needs to be analyzed from the code stream, and other auxiliary information can be derived or obtained, so that the format of the analyzed code stream can be simplified, the bit overhead of the code stream is reduced, and the decoding efficiency is improved.

In a third aspect, a point cloud encoding method is provided, which includes: acquiring auxiliary information of a cloud patch to be coded and a first identifier of the patch to be coded; when the first identification is a first value, encoding auxiliary information of a patch to be encoded, the first identification and a second identification of the patch to be encoded into a code stream, wherein the patch to be encoded is the first patch in the current point cloud frame; or when the first identifier is a first value, encoding auxiliary information of the patch to be encoded and a second identifier of the patch to be encoded into the code stream, wherein the patch to be encoded is a non-first patch in the current point cloud frame; the first identification is a first value to indicate that the patch to be coded is a first type of patch; the second mark is a fourth value to indicate that the three-dimensional information of the patch to be coded is not coded into the code stream; or the second identifier is a fifth value to indicate that the three-dimensional information of the to-be-encoded patch is encoded into the code stream, and the auxiliary information of the to-be-encoded patch comprises the three-dimensional information.

The first type of patch has a matching relationship among all point cloud frames in the current point cloud group, and the two-dimensional information, the rotational transformation information and the normal coordinate axis information of the first type of patch in the current point cloud group are the same, while the three-dimensional information may not be the same, but in some cases, the three-dimensional information of the first type of patch may also be the same. In this case, when the first identifier is a first value, the to-be-encoded patch may be indicated as a first type of patch, and the information encoded into the codestream may further include a second identifier.

It should be noted that the second identifier is used to indicate whether the three-dimensional information of the patch to be encoded has been coded into the code stream. The syntax element of the second flag may be override _3d _ shift _ data _ flag. When the second identifier is a fourth value, it may indicate that the three-dimensional information of the patch to be encoded is not coded into the code stream. That is, the three-dimensional information of the to-be-encoded patch is the same as the three-dimensional information of the reference patch of the to-be-encoded patch, so that the three-dimensional information of the to-be-encoded patch does not need to be encoded into the code stream again, that is, the auxiliary information of the to-be-encoded patch may not include the three-dimensional information. When the second identifier is a fifth value, it may indicate that the three-dimensional information of the patch to be encoded has been coded into the code stream. That is, the three-dimensional information of the to-be-encoded patch is different from the three-dimensional information of the reference patch of the to-be-encoded patch, so that the three-dimensional information of the to-be-encoded patch needs to be encoded into the code stream, that is, the auxiliary information of the to-be-encoded patch may include the three-dimensional information. The specific encoding method is not limited herein, and for example, a differential encoding method may be adopted, and a fixed-length encoding method may also be adopted. Wherein, the syntax element of the fourth value may be false, and the syntax element of the fifth value may be true.

When the first identifier is a second value, it may indicate that the patch to be encoded is a second type of patch. That is, a reference patch matching the patch to be encoded exists in the reference frame of the current point cloud frame. The auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of a reference patch of the patch to be encoded.

When the first flag is a third value, it may indicate that the patch to be encoded is a third type of patch. That is, no reference patch matched with the patch to be encoded exists in the reference frame of the current point cloud frame. The auxiliary information of the patch to be encoded comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.

Because the number of the first type of patch in each point cloud frame in the current point cloud group can be the same, when the first identifier is a first value and the to-be-encoded patch is the first patch in the current point cloud frame, it indicates that the to-be-encoded patch is the first type of patch in the current point cloud frame, and at this time, the auxiliary information, the first identifier and the second identifier of the to-be-encoded patch can be encoded into the code stream; when the first identifier is a first value and the to-be-encoded patch is a non-first patch in the current point cloud frame, it is indicated that the to-be-encoded patch is a non-first type patch in the current point cloud frame, and the auxiliary information and the second identifier of the to-be-encoded patch can be encoded into the code stream without encoding the first identifier of the to-be-encoded patch into the code stream, so that the format of the encoded code stream can be simplified. The second identifier is an identifier used for indicating whether the three-dimensional information of the patch to be encoded is coded into the code stream. And when the second identifier is a fourth value, the three-dimensional information of the patch to be coded is indicated to be not coded into the code stream. That is, the three-dimensional information of the to-be-encoded patch is the same as the three-dimensional information of the reference patch of the to-be-encoded patch, so that the three-dimensional information of the to-be-encoded patch does not need to be encoded into the code stream again, that is, the auxiliary information of the to-be-encoded patch may not include the three-dimensional information, and thus, the bit overhead of the code stream can be further reduced, and the encoding efficiency is improved.

In a fourth aspect, a point cloud decoding method is provided, the method comprising: when the point cloud block patch to be decoded is the first patch in the current point cloud frame, analyzing a first identifier of the patch to be decoded from the code stream; when the first identification is a first value, a second identification of the patch to be decoded is analyzed from the code stream; when the second identifier is a fourth value, determining auxiliary information of the to-be-decoded patch according to the auxiliary information of the reference patch of the to-be-decoded patch; when the second identifier is a fifth value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded; when the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patches in the reference point cloud frame of the current point cloud frame, analyzing a second identifier of the patch to be decoded from the code stream; when the second identifier is a fourth value, determining auxiliary information of the to-be-decoded patch according to the auxiliary information of the reference patch of the to-be-decoded patch; when the second identifier is a fifth value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded; and reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

It should be noted that, when the second identifier is a fourth value, it may indicate that the three-dimensional information of the patch to be decoded is not coded into the code stream. That is, the three-dimensional information of the to-be-decoded patch is the same as the three-dimensional information of the reference patch of the to-be-decoded patch, and at this time, the auxiliary information of the to-be-decoded patch may be determined according to the auxiliary information of the reference patch of the to-be-decoded patch. Specifically, the side information of the reference patch of the patch to be decoded may be taken as the side information of the patch to be decoded. When the second identifier is a fifth value, the three-dimensional information of the to-be-decoded patch can be indicated to be coded into the code stream, and the auxiliary information of the to-be-decoded patch comprises the three-dimensional information. At this time, the auxiliary information of the patch to be decoded is analyzed from the code stream, and the auxiliary information includes three-dimensional information of the patch to be decoded. As an example, the auxiliary information of the patch to be decoded may be parsed from the code stream by adopting a differential decoding manner. Reference may be made to the foregoing.

Because the second identifiers of the non-first type patches in the current point cloud frame are all coded into the code stream, when the to-be-decoded patch is the nth patch in the current point cloud frame, and n is greater than 1 and greater than the number of the first type patches in the reference point cloud frame of the current point cloud frame, that is, when the to-be-decoded patch is the non-first type patch, the second identifiers of the to-be-decoded patches can be analyzed from the code stream.

Because the first identification of the first patch in the current point cloud frame is coded into the code stream, when the point cloud block patch to be decoded is the first patch in the current point cloud frame, the first identification of the patch to be decoded is analyzed from the code stream, when the first identification is a first value, the patch to be decoded is shown to be the first type of patch, and at the moment, the second identification of the patch to be decoded can be analyzed from the code stream. When the to-be-decoded patch is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patches in the reference point cloud frame of the current point cloud frame, it indicates that the to-be-decoded patch is the non-first type of patch, and at this time, the second identifier of the to-be-decoded patch can be analyzed from the code stream. And when the second identifier is a fourth value, the three-dimensional information of the patch to be decoded is indicated to be not coded into the code stream. That is, the three-dimensional information of the to-be-decoded patch is the same as the three-dimensional information of the reference patch of the to-be-decoded patch, and at this time, the three-dimensional information of the reference patch of the to-be-decoded patch can be used as the three-dimensional information of the to-be-decoded patch, and the three-dimensional information of the to-be-decoded patch does not need to be analyzed from the code stream, so that the format of the analyzed code stream can be simplified, the bit overhead of the code stream is reduced, and the decoding efficiency is improved.

In a fifth aspect, a point cloud encoding method is provided, which includes: acquiring auxiliary information of a cloud patch to be coded and a first identifier of the patch to be coded; when the first identification is a first value, the first identification of the patch to be coded is coded into a code stream; when the first identification is a second value or a third value, encoding auxiliary information of the patch to be encoded and the first identification into a code stream; the first identification is a first value to indicate that the patch to be coded is a first type of patch; the first identification is a second value to indicate that the patch to be coded is a second type of patch, and the auxiliary information of the patch to be coded comprises two-dimensional information, three-dimensional information and identification information of a reference patch of the patch to be coded; the first identification is a third value to indicate that the patch to be coded is a third type of patch, and the auxiliary information of the patch to be coded comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.

When the first identifier is a first value, it indicates that the patch to be encoded is a first type of patch. In a possible implementation manner, the first type of patch has a matching relationship among all point cloud frames in the current point cloud set, and the two-dimensional information, the three-dimensional information, the rotation transformation information, and the normal coordinate axis information of the first type of patch in the current point cloud set are all the same. Therefore, in order to reduce the bit overhead of the code stream and further improve the coding efficiency, when the first identifier is the first value, the first identifier of the patch to be coded is coded into the code stream, and the auxiliary information of the patch to be coded is not required to be coded into the code stream.

Because the two-dimensional information, the rotation transformation information, the normal coordinate axis information and the three-dimensional information among the first types of patches in the current point cloud group can be the same, when the first identifier of the to-be-encoded patch is a first value, the to-be-encoded patch is indicated to be the first type of patch, at the moment, the first identifier of the to-be-encoded patch can be encoded into a code stream, and the auxiliary information of the to-be-encoded patch does not need to be encoded into the code stream, so that the format of the encoded code stream can be simplified. In addition, the value of the first identifier may be a first value, a second value and a third value. Different values indicate different types of patch. Therefore, different types of patches can be distinguished through the first identifier, and for the different types of patches, the content included in the auxiliary information coded into the code stream may be different, and for the first type of patch and the second type of patch, only part of the auxiliary information is coded into the code stream. Therefore, the format of the code stream can be simplified, the bit overhead of the code stream is further reduced, and the coding efficiency is improved.

In a sixth aspect, a point cloud decoding method is provided, which includes: analyzing a first identifier of a point cloud block patch to be decoded from the code stream; when the first identification is a first value, determining auxiliary information of the to-be-decoded patch according to the auxiliary information of the reference patch of the to-be-decoded patch, wherein the auxiliary information of the to-be-decoded patch comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information; when the first identification is a second value, analyzing auxiliary information of the to-be-decoded patch from the code stream, wherein the auxiliary information of the to-be-decoded patch comprises two-dimensional information, three-dimensional information and identification information of a reference patch of the to-be-decoded patch; when the first identification is a third value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information of the patch to be decoded comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information; and reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

The first identifications of all the patches in the current point cloud frame can be packed and coded into the code stream respectively, or packed together and coded into the code stream. Therefore, for the way of packing and encoding the code stream respectively, if the patch to be decoded is the first patch in the current point cloud frame, the first identifier of the patch to be decoded can be analyzed from the code stream. Of course, for the non-first patch, the first identifier of the patch to be decoded may also be parsed from the code stream. That is, the first identifier of each patch to be decoded in the current point cloud frame may be sequentially parsed. For the way of packing together and encoding into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, the first identifiers of the multiple patches included in the current point cloud frame may be analyzed from the code stream, where the analyzed first identifiers include the first identifier of the patch to be decoded. That is, the first identifiers of the plurality of patches included in the current point cloud frame can be obtained through one-time analysis, and the analysis does not need to be performed for multiple times.

Because the first type of patch does not encode auxiliary information into the code stream in the encoding process, when the first identifier is the first value, that is, when the to-be-decoded patch is the first type of patch, the two-dimensional information, the three-dimensional information, the rotation transformation information and the normal coordinate axis information of the to-be-decoded patch can be acquired according to the auxiliary information of the reference patch of the to-be-decoded patch.

As an example, the identification information of the reference frame of the current point cloud frame may be analyzed from the code stream, so as to determine the reference frame of the current point cloud frame according to the identification information of the reference frame, then the two-dimensional information, the three-dimensional information, the rotational transformation information, and the normal coordinate axis information of the reference patch of the patch to be decoded are obtained from the reference frame of the current point cloud frame, and the two-dimensional information, the three-dimensional information, the rotational transformation information, and the normal coordinate axis information of the reference patch are used as the two-dimensional information, the three-dimensional information, the rotational transformation information, and the normal coordinate axis information of the patch to be decoded. For convenience of description, the reference frame of the current point cloud frame is marked as a first reference frame, and the reference frame corresponding to the reference frame of the current point cloud frame is marked as a second reference frame, so that the auxiliary information of the reference patch in the first reference frame can be determined according to the auxiliary information of the corresponding reference patch in the second reference frame. By the method, the auxiliary information of the patch to be decoded can be acquired step by step according to the reference frame of the current point cloud frame, the reference frame corresponding to the reference frame of the current point cloud frame and the like. In addition, for the two-dimensional information, the three-dimensional information, the rotation transformation information and the normal coordinate axis information of the patch to be decoded, the two-dimensional information, the three-dimensional information, the rotation transformation information and the normal coordinate axis information of the patch with the same position as that of the patch to be decoded in the current point cloud frame can be acquired from the decoded point cloud, and the acquired two-dimensional information, the three-dimensional information, the rotation transformation information and the normal coordinate axis information can be used as the two-dimensional information, the rotation transformation information and the normal coordinate axis information of the patch to be decoded.

When the patches in each point cloud frame in the current point cloud group are arranged according to the order of the first type of patches, the second type of patches, and the third type of patches for encoding, the reference patch of the patch to be decoded may be a patch whose position in the reference frame of the current point cloud frame is the same as the position of the patch to be decoded in the current point cloud frame.

The auxiliary information of the code stream coded by the second type of patch in the coding process comprises two-dimensional information, three-dimensional information and identification information of a reference patch, but does not comprise rotation transformation information and normal coordinate axis information, so when the to-be-decoded patch is the second type of patch, the auxiliary information analyzed from the code stream comprises the two-dimensional information and the three-dimensional information of the to-be-decoded patch and the identification information of the reference patch of the to-be-decoded patch, but does not comprise the rotation transformation information and the normal coordinate axis information of the to-be-decoded patch. At this time, the rotation transformation information and the normal coordinate axis information of the to-be-decoded patch may also be acquired according to the identification information of the reference patch of the to-be-decoded patch.

In summary, for the first type of patch, only the first identifier needs to be analyzed from the code stream, other auxiliary information may be derived or obtained, for the second type of patch, only a part of auxiliary information needs to be analyzed from the code stream, and other auxiliary information may also be derived or obtained. Therefore, the format of the analyzed code stream can be simplified, the bit overhead of the code stream is reduced, and the decoding efficiency is improved.

In a seventh aspect, a point cloud encoding apparatus is provided, which includes: the point cloud patch information acquisition module is used for acquiring auxiliary information of a to-be-encoded point cloud patch and a first identifier of the to-be-encoded patch; the auxiliary information encoding module is used for encoding the auxiliary information and the first identifier of the patch to be encoded into the code stream when the first identifier is a first value, wherein the patch to be encoded is the first patch in the current point cloud frame; or the auxiliary information coding module is used for coding the auxiliary information of the patch to be coded into the code stream when the first identifier is a first value, wherein the patch to be coded is a non-first patch in the current point cloud frame; the first identifier is a first value to indicate that the to-be-encoded patch is a first type of patch, and the auxiliary information of the to-be-encoded patch includes three-dimensional information.

And the current point cloud frame is the Kth point cloud frame in the current group, and K is an integer greater than or equal to 3.

Wherein the auxiliary information encoding module is further configured to:

when the first identification is a second value or a third value, encoding auxiliary information of the patch to be encoded and the first identification into a code stream;

the first identification is a second value to indicate that the patch to be coded is a second type of patch, and the auxiliary information of the patch to be coded comprises two-dimensional information, three-dimensional information and identification information of a reference patch of the patch to be coded;

The first identification is a third value to indicate that the patch to be coded is a third type of patch, and the auxiliary information of the patch to be coded comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.

Wherein the auxiliary information encoding module is further configured to:

encoding first identifications of a plurality of patches included by the current point cloud frame into a code stream, wherein the first identifications of the plurality of patches include first identifications of the patches to be encoded, and the remaining first identifications except the first identifications of the patches to be encoded in the first identifications of the plurality of patches are second values or third values.

Wherein the first value, the second value and the third value are binary 10, 11 and 00, respectively,

alternatively, the first, second and third values are binary 11, 10 and 00, respectively.

The first identifier of the patch to be encoded is a third value, and the patch to be encoded is the first patch in the current point cloud frame.

Wherein, the reference patch of the patch to be coded is included in the reference frame of the current point cloud frame.

The first type of patch is a global matching patch in the current group, the second type of patch is a local matching patch in the current group, and the third type of patch is a non-matching patch in the current group.

In an eighth aspect, there is provided a point cloud decoding apparatus including: the auxiliary information decoding module is used for analyzing a first identifier of the to-be-decoded patch from the code stream when the to-be-decoded point cloud patch is the first patch in the current point cloud frame; the auxiliary information decoding module is further used for analyzing the auxiliary information of the patch to be decoded from the code stream when the first identifier is a first value, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded; the auxiliary information decoding module is further used for analyzing the auxiliary information of the patch to be decoded from the code stream when the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patches in the reference point cloud frame of the current point cloud frame, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded; and the reconstruction module is used for reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

Wherein the side information decoding module is further configured to:

when the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and greater than the number of the first type of patches in the reference point cloud frame of the current point cloud frame, analyzing a first identifier of the patch to be decoded from the code stream;

when the first identification is a second value, analyzing auxiliary information of the to-be-decoded patch from the code stream, wherein the auxiliary information comprises two-dimensional information and three-dimensional information of the to-be-decoded patch and identification information of a reference patch of the to-be-decoded patch;

and when the first identification is a third value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded.

Wherein the side information decoding module is further configured to:

when the patch to be decoded is the first patch in the current point cloud frame, analyzing first identifications of a plurality of patches included in the current point cloud frame from the code stream, wherein the analyzed first identifications include the first identification of the patch to be decoded, and the remaining first identifications except the first identification of the patch to be decoded in the first identifications are second values or third values.

When the patch to be decoded is the first patch in the current point cloud frame and the first identifier is the third value, the current point cloud frame is the first point cloud frame.

Wherein, when the first identifier is a first value, the auxiliary information decoding module is further configured to:

acquiring two-dimensional information, rotation transformation information and normal coordinate axis information of the to-be-decoded patch according to auxiliary information of a reference patch of the to-be-decoded patch;

the reconstruction module is further configured to:

and reconstructing the patch to be decoded according to the two-dimensional information, the three-dimensional information, the normal coordinate axis information and the rotation transformation information of the patch to be decoded.

Wherein, when the first flag is the second value, the auxiliary information decoding module is further configured to:

acquiring the rotation transformation information and the normal coordinate axis information of the to-be-decoded patch according to the identification information of the reference patch of the to-be-decoded patch;

the reconstruction module is further configured to:

Wherein, the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.

In a ninth aspect, there is provided a point cloud encoding apparatus, the apparatus including: the point cloud patch information acquisition module is used for acquiring auxiliary information of a to-be-encoded point cloud patch and a first identifier of the to-be-encoded patch; the auxiliary information encoding module is used for encoding the auxiliary information of the patch to be encoded, the first identifier and the second identifier of the patch to be encoded into the code stream when the first identifier is a first value, wherein the patch to be encoded is the first patch in the current point cloud frame; or the auxiliary information coding module is used for coding the auxiliary information of the patch to be coded and the second identification of the patch to be coded into the code stream when the first identification is the first value, wherein the patch to be coded is a non-first patch in the current point cloud frame; the first identification is a first value to indicate that the patch to be coded is a first type of patch; the second mark is a fourth value to indicate that the three-dimensional information of the patch to be coded is not coded into the code stream; or the second identifier is a fifth value to indicate that the three-dimensional information of the to-be-encoded patch is encoded into the code stream, and the auxiliary information of the to-be-encoded patch comprises the three-dimensional information.

In a tenth aspect, there is provided a point cloud decoding apparatus including: the auxiliary information decoding module is used for analyzing a first identifier of the to-be-decoded patch from the code stream when the to-be-decoded point cloud patch is the first patch in the current point cloud frame; the auxiliary information decoding module is further used for analyzing a second identifier of the patch to be decoded from the code stream when the first identifier is a first value; the auxiliary information decoding module is further configured to determine auxiliary information of the to-be-decoded patch according to the auxiliary information of the reference patch of the to-be-decoded patch when the second identifier is a fourth value; when the second identifier is a fifth value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded; the auxiliary information decoding module is further used for analyzing a second identifier of the patch to be decoded from the code stream when the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patches in the reference point cloud frame of the current point cloud frame; the auxiliary information decoding module is further configured to determine auxiliary information of the to-be-decoded patch according to the auxiliary information of the reference patch of the to-be-decoded patch when the second identifier is a fourth value; when the second identifier is a fifth value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded; and the reconstruction module is used for reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

In an eleventh aspect, there is provided a point cloud encoding apparatus, including: the point cloud patch information acquisition module is used for acquiring auxiliary information of a to-be-encoded point cloud patch and a first identifier of the to-be-encoded patch; the auxiliary information coding module is used for coding the first identifier of the patch to be coded into the code stream when the first identifier is a first value; the auxiliary information coding module is further used for coding the auxiliary information of the patch to be coded and the first identifier into a code stream when the first identifier is a second value or a third value; the first identification is a first value to indicate that the patch to be coded is a first type of patch; the first identification is a second value to indicate that the patch to be coded is a second type of patch, and the auxiliary information of the patch to be coded comprises two-dimensional information, three-dimensional information and identification information of a reference patch of the patch to be coded; the first identification is a third value to indicate that the patch to be coded is a third type of patch, and the auxiliary information of the patch to be coded comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.

In a twelfth aspect, there is provided a point cloud decoding apparatus, including: the auxiliary information decoding module is used for analyzing a first identifier of the point cloud block patch to be decoded from the code stream; the auxiliary information decoding module is further used for determining auxiliary information of the to-be-decoded patch according to the auxiliary information of the reference patch of the to-be-decoded patch when the first identifier is a first value, wherein the auxiliary information of the to-be-decoded patch comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information; the auxiliary information decoding module is further used for analyzing the auxiliary information of the to-be-decoded patch from the code stream when the first identifier is a second value, wherein the auxiliary information of the to-be-decoded patch comprises two-dimensional information, three-dimensional information and identification information of a reference patch of the to-be-decoded patch; the auxiliary information decoding module is further used for analyzing the auxiliary information of the patch to be decoded from the code stream when the first identifier is a third value, wherein the auxiliary information of the patch to be decoded comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information; and the reconstruction module is used for reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

the reconstruction module is further configured to:

In a thirteenth aspect, there is provided an apparatus for encoding point cloud data, which may include: a memory and an encoder. The memory is used for storing point cloud data. The encoder is configured to perform the point cloud encoding method provided by the first aspect or any one of the possible designs of the first aspect, or the encoder is configured to perform the point cloud encoding method provided by the third aspect or any one of the possible designs of the third aspect, or the encoder is configured to perform the point cloud encoding method provided by any one of the possible designs of the fifth aspect or the fifth aspect.

In a fourteenth aspect, an apparatus for decoding point cloud data is provided, which may include: a memory and a decoder. The memory is used for storing point cloud data in a code stream form. The decoder is configured to perform the point cloud decoding method provided by any one of the above-mentioned second aspect or any one of the above-mentioned possible designs of the second aspect, or the decoder is configured to perform the point cloud decoding method provided by any one of the above-mentioned fourth aspect or any one of the above-mentioned possible designs of the sixth aspect, or the decoder is configured to perform the point cloud decoding method provided by any one of the above-mentioned possible designs of the sixth aspect or the sixth aspect.

In a fifteenth aspect, an encoding apparatus is provided, including: a memory and a processor; wherein the memory is used for storing program codes; the processor is configured to invoke the program code to execute the point cloud encoding method provided in the first aspect, the third aspect, or the fifth aspect.

In a sixteenth aspect, there is provided a decoding apparatus comprising: a memory and a processor; wherein the memory is used for storing program codes; the processor is configured to invoke the program code to execute the point cloud decoding method provided in the second aspect, the fourth aspect, or the sixth aspect.

The present application also provides a computer-readable storage medium comprising program code which, when run on a computer, causes the computer to perform any one of the point cloud encoding methods provided in the first aspect and its possible designs, or the third aspect and its possible designs, or the fifth aspect and its possible designs, as described above.

The present application also provides a computer-readable storage medium comprising program code which, when run on a computer, causes the computer to perform any one of the point cloud decoding methods provided by the second aspect and its possible designs, or the fourth aspect and its possible designs, or the sixth aspect and its possible designs.

The present application further provides a computer program product which, when run on a computer, causes the computer to perform the first aspect or any one of the possible designs of the first aspect, or to perform the third aspect or any one of the possible designs of the third aspect, or to perform some or all of the steps of the point cloud encoding method provided by any one of the possible designs of the fifth aspect or the fifth aspect.

The present application also provides a computer program product which, when run on a computer, causes the computer to perform any one of the possible designs of the second aspect or the second aspect, or, alternatively, any one of the possible designs of the fourth aspect or the fourth aspect, or, alternatively, some or all of the steps of the point cloud decoding method provided by any one of the possible designs of the sixth aspect or the sixth aspect.

It should be understood that beneficial effects of any one of the coding and decoding apparatuses, the computer readable storage medium and the computer program product provided above may correspond to beneficial effects of the method embodiments provided with reference to the above corresponding aspects, and are not described again.

The technical scheme provided by the application can at least bring the following beneficial effects:

In the embodiment of the application, the auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded are obtained first. Because the number of the first type of patch in each point cloud frame in the current point cloud group can be the same, when the first identifier is a first value and the to-be-encoded patch is the first patch in the current point cloud frame, it indicates that the to-be-encoded patch is the first type of patch in the current point cloud frame, and at this time, the auxiliary information of the to-be-encoded patch and the first identifier can be encoded into the code stream; when the first identifier is a first value and the to-be-encoded patch is a non-first patch in the current point cloud frame, it is indicated that the to-be-encoded patch is a non-first type patch in the current point cloud frame, and at this time, the auxiliary information of the to-be-encoded patch can be encoded into the code stream, and the first identifier of the to-be-encoded patch does not need to be encoded into the code stream, so that the format of the encoded code stream can be simplified. The bit overhead of the code stream is further reduced, so that the coding efficiency is improved.

Drawings

Fig. 1 is a schematic block diagram of a point cloud decoding system according to an embodiment of the present disclosure.

Fig. 2 is a schematic block diagram of an encoder that may be used in embodiments of the present application.

Fig. 3 is a schematic diagram of a point cloud applicable to the embodiment of the present application.

Fig. 4 is a schematic diagram of a patch of a point cloud applicable to the embodiment of the present application.

Fig. 5 is a schematic diagram of an occupancy map of a point cloud applicable to an embodiment of the present application.

Fig. 6 is a schematic block diagram of a decoder that may be used with embodiments of the present application.

Fig. 7 is a diagram of a frame group that can be used in an embodiment of the present application.

Fig. 8 is a schematic diagram of a first current point cloud set according to an embodiment of the present disclosure.

Fig. 9 is a flowchart of a first point cloud encoding method according to an embodiment of the present disclosure.

Fig. 10 is a schematic diagram of a second current point cloud set provided in the embodiment of the present application.

Fig. 11 is a flowchart of a first point cloud decoding method according to an embodiment of the present disclosure.

Fig. 12 is a flowchart of a second point cloud encoding method according to an embodiment of the present disclosure.

Fig. 13 is a flowchart of a second point cloud decoding method according to an embodiment of the present disclosure.

Fig. 14 is a flowchart of a third point cloud encoding method according to an embodiment of the present application.

Fig. 15 is a flowchart of a third point cloud decoding method according to an embodiment of the present application.

Fig. 16 is a schematic block diagram of a first encoding apparatus according to an embodiment of the present application.

Fig. 17 is a schematic block diagram of a first decoding apparatus provided in an embodiment of the present application.

Fig. 18 is a schematic block diagram of a second encoding apparatus provided in an embodiment of the present application.

Fig. 19 is a schematic block diagram of a second decoding apparatus provided in an embodiment of the present application.

Fig. 20 is a schematic block diagram of a third encoding apparatus provided in an embodiment of the present application.

Fig. 21 is a schematic block diagram of a third decoding apparatus provided in an embodiment of the present application.

FIG. 22 is a schematic block diagram of one implementation of a decoding apparatus for embodiments of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

The term "plurality" in the embodiments of the present application means two or more. In the description of the present application, "/" indicates an OR meaning, for example, A/B may indicate A or B; "and/or" herein is merely an association describing an associated object, and means that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, in order to facilitate clear description of technical solutions of the embodiments of the present application, in the embodiments of the present application, terms such as "first" and "second" are used to distinguish the same items or similar items having substantially the same functions and actions. Those skilled in the art will appreciate that the terms "first," "second," etc. do not denote any order or quantity, nor do the terms "first," "second," etc. denote any order or importance.

Before explaining the embodiments of the present application in detail, an implementation environment of the embodiments of the present application is described:

an implementation environment provided by an embodiment of the application includes a point cloud decoding system. Fig. 1 is a schematic block diagram of a point cloud decoding system according to an embodiment of the present disclosure. Wherein the terms "point cloud coding" or "coding" may generally refer to point cloud encoding or point cloud decoding. Referring to fig. 1, a point cloud decoding system includes a source device 10, a destination device 20, a link 30, and a storage device 40. Source device 10 may generate, among other things, encoded point cloud data. Therefore, the source apparatus 10 may also be referred to as a point cloud encoding apparatus. Destination device 20 may decode the encoded point cloud data generated by source device 10. Therefore, the destination device 20 may also be referred to as a point cloud decoding device. The link 30 may receive encoded point cloud data generated by the source device 10 and may transmit the encoded point cloud data to the destination device 20. The storage device 40 may receive the encoded point cloud data generated by the source device 10 and may store the encoded point cloud data, such that the destination device 20 may retrieve the encoded point cloud data directly from the storage device 40. Alternatively, the storage device 40 may correspond to a file server or another intermediate storage device that may hold the encoded point cloud data generated by the source device 10, such that the destination device 20 may stream or download the encoded point cloud data stored by the storage device 40.

Source device 10 and destination device 20 may each include one or more processors and memory coupled to the one or more processors, which may include Random Access Memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, any other medium operable to store desired program code in the form of computer-accessible instructions or data structures, and the like. For example, source device 10 and destination device 20 may each comprise a desktop computer, a mobile computing device, a notebook (e.g., laptop) computer, a tablet computer, a set-top box, a telephone handset such as a so-called "smart" phone, a television, a camera, a display device, a digital media player, a video game console, an on-board computer, or the like.

The link 30 may include one or more media or devices capable of transmitting the encoded point cloud data from the source device 10 to the destination device 20. In one possible implementation, the link 30 may include one or more communication media that enable the source device 10 to send the encoded point cloud data directly to the destination device 20 in real-time. In the present embodiment, the source device 10 may modulate the encoded point cloud data according to a communication standard, which may be a wireless communication protocol or the like, and may transmit the modulated point cloud data to the destination device 20. The one or more communication media may include wireless and/or wired communication media, for example, the one or more communication media may include a Radio Frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, which may be a local area network, a wide area network, or a global network (e.g., the internet), among others. The one or more communication media may include a router, a switch, a base station, or other devices that facilitate communication from source device 10 to destination device 20, and the like, which is not specifically limited in this embodiment.

In one possible implementation, the storage device 40 may store the received encoded point cloud data transmitted by the source device 10, and the destination device 20 may retrieve the encoded point cloud data directly from the storage device 40. As such, storage device 40 may include any of a variety of distributed or locally accessed data storage media, which may be, for example, a hard drive, a Blu-ray disc, a Digital Versatile Disc (DVD), a compact disc read-only memory (CD-ROM), a flash memory, a volatile or non-volatile memory, or any other suitable digital storage media for storing encoded point cloud data, among others.

In one possible implementation, the storage device 40 may correspond to a file server or another intermediate storage device that may hold the encoded point cloud data generated by the source device 10, and the destination device 20 may stream or download the point cloud data stored by the storage device 40. The file server may be any type of server capable of storing the encoded point cloud data and transmitting the encoded point cloud data to the destination device 20. In one possible implementation, the file server may include a network server, a File Transfer Protocol (FTP) server, a Network Attached Storage (NAS) device, a local disk drive, or the like. The destination device 20 may acquire the encoded point cloud data through any standard data connection, including an internet connection. Any standard data connection may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a Digital Subscriber Line (DSL), cable modem, etc.), or a combination of both suitable for acquiring encoded point cloud data stored on a file server. The transmission of the encoded point cloud data from the storage device 40 may be a streaming transmission, a download transmission, or a combination of both.

The point cloud decoding system shown in fig. 1 is only one possible implementation manner, and the technology of the present application may be applied to not only the source device 10 that can encode the point cloud and the destination device 20 that can decode the encoded point cloud data shown in fig. 1, but also other devices that can encode and decode the point cloud, which is not specifically limited in this embodiment of the present application.

In the point cloud decoding system shown in fig. 1, the source apparatus 10 includes a data source 120, an encoder 100, and an output interface 140. In some embodiments, output interface 140 may include a regulator/demodulator (modem) and/or a transmitter, which may also be referred to as a transmitter. The data source 120 may include a point cloud capture device (e.g., a camera, etc.), a point cloud archive containing previously captured point cloud data, a point cloud feed interface for receiving point cloud data from a point cloud content provider, and/or a computer graphics system for generating point cloud data, or a combination of these sources of point cloud data.

The data source may send a point cloud to the encoder 100, and the encoder 100 may encode the received point cloud sent by the data source 120, resulting in encoded point cloud data. The encoder may send the encoded point cloud data to an output interface. In some embodiments, source device 10 sends the encoded point cloud data directly to destination device 20 via output interface 140. In other embodiments, the encoded point cloud data may also be stored onto the storage device 40 for later retrieval by the destination device 20 and use in decoding and/or playback.

In the embodiment of fig. 1, destination device 20 includes an input interface 240, decoder 200, and display device 220. In some embodiments, input interface 240 includes a receiver and/or a modem. The input interface 240 may receive the encoded point cloud data via the link 30 and/or from the storage device 40 and then send it to the decoder 200, and the decoder 200 may decode the received encoded point cloud data to obtain decoded point cloud data. The decoder may send the decoded point cloud data to the display device 220. Display device 220 may be integrated with destination device 20 or may be external to destination device 20. In general, the display device 220 displays the decoded point cloud data. The display device 220 may be any one of a plurality of types of display devices, for example, the display device 220 may be a Liquid Crystal Display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

Although not shown in fig. 1, in some aspects, encoder 100 and decoder 200 may each be integrated with an audio encoder and decoder, and may include appropriate multiplexer-demultiplexer (MUX-DEMUX) units or other hardware and software for encoding both audio and video in a common data stream or separate data streams. In some embodiments, the MUX-DEMUX unit may conform to the ITU h.223 multiplexer protocol, or other protocols such as User Datagram Protocol (UDP), if applicable.

Encoder 100 and decoder 200 may each be any of the following circuits: one or more microprocessors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combinations thereof. If the present application is implemented in part in software, a device may store instructions for the software in a suitable non-volatile computer-readable storage medium and may execute the instructions in hardware using one or more processors to implement the techniques of the present application. Any of the foregoing, including hardware, software, a combination of hardware and software, etc., can be considered as one or more processors. Each of the encoder 100 and decoder 200 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder/decoder (codec) in the respective device.

This application may refer generally to encoder 100 as "signaling" or "sending" certain information to another device, such as decoder 200. The terms "signaling" or "sending" may generally refer to the transfer of syntax elements and/or other data used to decode the compressed point cloud data. This transfer may occur in real time or near real time. Alternatively, such communication may occur over a period of time, such as may occur when, at the time of encoding, syntax elements are stored in the encoded bitstream to a computer-readable storage medium, which the decoding device may then retrieve at any time after the syntax elements are stored to such medium.

Fig. 2 is a schematic block diagram of an encoder 100 according to an embodiment of the present disclosure. Fig. 2 is an example of an mpeg (moving picture Expert group) Point Cloud Compression (PCC) encoding framework. Referring to fig. 2, the encoder 100 may include a point cloud block information generating module 101, a packing module 102, a depth map generating module 103, a texture map generating module 104, a depth map padding module 105, a texture map padding module 106, an image or video-based encoding module 107, an occupancy map encoding module 108, an auxiliary information encoding module 109, a multiplexing module 110, a point cloud occupancy map downsampling module 111, a point cloud occupancy map padding module 112, a point cloud reconstructing module 113, and a point cloud filtering module 114.

The point cloud block information generating module 101 may receive one or more point cloud groups transmitted by the data source 120, where each point cloud group may include one or more frames of point clouds, and for convenience of description, the following unification takes a current point cloud frame in the current point cloud group as an example for illustration. The point cloud block information generating module 101 may determine a three-dimensional coordinate of each point included in the current point cloud frame in a three-dimensional space coordinate system and a normal direction vector of each point in the three-dimensional space, and partition the current point cloud frame into a plurality of patches (point cloud blocks) according to the determined normal direction vector of each point in the three-dimensional space and a predefined projection plane. Each patch includes one or more points in the current point cloud frame. The point cloud patch information generation module 101 may also determine an occupancy map for each patch and a depth map for each patch. The point cloud block information generating module 101 may further determine three-dimensional information of each patch and normal coordinate axis information of each patch, and these pieces of information may be used as auxiliary information of each patch. In addition, the point cloud patch information generating module 101 may divide the patch into three types, which are a first type patch, a second type patch, and a third type patch. And indicates the type of each patch by a first identification. The point cloud block information generating module 101 may send the first identifier of each patch and the auxiliary information of each patch in the current point cloud frame to the auxiliary information encoding module 109 for encoding, which may also be referred to as compression encoding. Wherein the three-dimensional information of each patch may include an offset of the patch to be encoded in the 3D space along the tangential direction, an offset of the patch to be encoded in the 3D space along the bi-tangential direction, and an offset of the patch to be encoded in the 3D space along the normal direction. The point cloud chunk information generation module 101 may also send the occupancy map of each patch and the auxiliary information of each patch to the packing module 102. In addition, the point cloud patch information generation module 101 may also send the depth map of each patch to the depth map generation module 103.

The packing module 102 may pack the received occupancy map of each patch and the received auxiliary information of each patch sent by the point cloud block information generating module 101, to obtain the occupancy map of the current point cloud frame. Specifically, the packing module 102 may arrange the occupancy maps of each patch in a specific order, for example, in a descending order (or ascending order) of the width/height of the occupancy map of each patch, and then sequentially insert the occupancy map of each patch into an available area of the occupancy map of the current point cloud frame according to the order of the arranged occupancy maps of each patch, so as to obtain the occupancy map of the current point cloud frame and the packing information of the patch. The packing module 102 may send the packing information of the occupancy map and the patch of the current point cloud frame to the auxiliary information encoding module 109 for compression encoding. The packing information of the patches may include two-dimensional information of each patch in an occupancy map of the current point cloud frame, identification information of a reference patch of each patch, rotation transformation information of each patch, and identification information of a reference frame of the current point cloud frame. The two-dimensional information of each patch in the occupancy map of the current point cloud frame may be referred to as the two-dimensional information of each patch for short. The two-dimensional information of each patch may include an offset of each patch in the current point cloud frame occupancy map in the u-axis direction, an offset of each patch in the current point cloud frame occupancy map in the v-axis direction, a width of each patch in the current point cloud frame occupancy map, and a height of each patch in the current point cloud frame occupancy map. In addition, the packing module 102 may also send the packing information of patch to the depth map generation module 103 and the point cloud occupancy map downsampling module 111.

It should be noted that, in order to more intuitively understand the point cloud, the patch of the point cloud, and the occupancy map of the point cloud related in the present application technology, refer to fig. 3, fig. 4, and fig. 5, fig. 3 is a schematic diagram of a frame of point cloud applicable to the embodiment of the present application, fig. 4 is a schematic diagram of the patch of the frame of point cloud, fig. 5 is a schematic diagram of the occupancy map of each patch of the frame of point cloud shown in fig. 4, and the occupancy map of the frame of point cloud obtained by packing by the packing module 102. The point cloud shown in fig. 3 may be a current point cloud frame in the embodiment of the present application, the patch of the point cloud shown in fig. 4 may be the patch of the current point cloud frame in the embodiment of the present application, and the occupancy map of the point cloud shown in fig. 5 may be the occupancy map of the current point cloud frame in the embodiment of the present application.

After receiving the occupancy map of the current point cloud frame and the packing information of the patches sent by the packing module 102, and the first identifier of each patch and the auxiliary information of each patch sent by the point cloud block information generating module 101, the auxiliary information encoding module 109 may encode the current point cloud frame and each patch included in the current point cloud frame to obtain a code stream including the encoded auxiliary information, and then send the obtained code stream including the encoded auxiliary information to the multiplexing module 110. In the packing information of each patch, the two-dimensional information of each patch and the identification information of the reference patch of each patch may also be referred to as auxiliary information of each patch.

The depth map generation module 103 may generate a depth map of the current point cloud frame according to the packing information of each patch and the depth map of each patch after receiving the packing information of the patch transmitted by the packing module 102 and the depth map packing of each patch transmitted by the point cloud block information generation module 101. Then, the generated depth map of the current point cloud frame is sent to the depth map filling module 105, so as to fill blank pixel points in the depth map of the current point cloud frame, and obtain the filled depth map of the current point cloud frame. The depth map filling module 105 may send the obtained depth map of the filled current point cloud frame to the image or video based encoding module 107, to perform image or video based encoding on the depth map of the filled current point cloud frame, to obtain a reconstructed depth map of the current point cloud and a code stream including the depth map of the encoded current point cloud frame, and may send the obtained depth map of the reconstructed current point cloud frame to the point cloud reconstruction module 113, and to send the code stream including the depth map of the encoded current point cloud frame to the multiplexing module 110.

The point cloud occupancy map downsampling module 111 may downsample the received occupancy map of the current point cloud frame sent by the packing module 102 to obtain the occupancy map of the current point cloud frame with low resolution. The point cloud occupancy map downsampling module 111 may then send the occupancy map of the low-resolution current point cloud frame to the occupancy map encoding module 108 and the point cloud occupancy map populating module 112. The occupancy map encoding module 108 may encode the occupancy map of the received low-resolution current point cloud frame to obtain a code stream including the occupancy map of the encoded low-resolution current point cloud frame, and the occupancy map encoding module 108 may further send the code stream including the occupancy map of the encoded low-resolution current point cloud frame to the multiplexing module 110. The point cloud occupancy map filling module 112 fills the occupancy map of the current point cloud frame with the original resolution according to the received occupancy map of the current point cloud frame with the low resolution, so as to obtain a filled occupancy map of the current point cloud frame, where the occupancy map of the filled current point cloud frame has the original resolution.

The point cloud reconstruction module 113 may reconstruct the geometry of the current point cloud frame according to the occupied map of the populated current point cloud frame sent by the point cloud occupied map populating module 112, the depth map of the reconstructed current point cloud frame sent by the image or video-based encoding module 107, the packing information of the patch, and the auxiliary information of the patch, so as to output a reconstructed point cloud, and in addition, the point cloud reconstruction module 113 may also output the corresponding relationship between the reconstructed point in the reconstructed point cloud and the patch, and the packing position of the reconstructed point in the reconstructed point cloud. The point cloud reconstruction module 113 may send the reconstructed point cloud, the corresponding relationship between the reconstructed points in the reconstructed point cloud and the patch to the point cloud filtering module 114, and the point cloud reconstruction module 113 may also send the packing positions of the reconstructed points in the reconstructed point cloud to the texture generation module 104.

The point cloud filtering module 114 may filter the reconstructed point cloud after receiving the reconstructed point cloud transmitted by the point cloud reconstruction module 113 and the corresponding relationship between the reconstructed point and the patch in the reconstructed point cloud. Specifically, flaws such as obvious noise points and gaps in the reconstructed point cloud may be removed to obtain a filtered reconstructed point cloud, which may also be referred to as a smoothed reconstructed point cloud. Alternatively, the point cloud filtering block 114 may smooth the reconstructed point cloud.

After receiving the smoothed reconstructed point cloud sent by the point cloud filtering module 114, the packing position of the reconstructed point in the reconstructed point cloud sent by the point cloud reconstructing module 113, and the current point cloud frame sent by the data source 120, the texture map generating module 104 may generate a texture map of the current point cloud frame according to the smoothed reconstructed point cloud, the packing position of the reconstructed point in the reconstructed point cloud, and the current point cloud frame, and may send the generated texture map of the current point cloud frame to the texture map filling module 106 to fill the blank pixel points in the texture map of the current point cloud frame, so as to obtain a filled texture map of the current point cloud frame. The texture map filling module 106 may send the obtained texture map of the filled current point cloud frame to the image or video-based encoding module 107, so as to perform image or video-based encoding on the texture map of the filled current point cloud frame, and obtain a code stream including the reconstructed texture map of the current point cloud frame. The image or video based encoding module 107 may also send the resulting code stream including the texture map of the reconstructed current point cloud frame to the multiplexing module 110.

Through the above description, the image or video-based encoding module 107, the occupancy map encoding module 108, and the auxiliary information encoding module 109 may send the obtained code streams to the multiplexing module 110, and the multiplexing module 110 may combine the received code streams into a combined code stream and send the combined code stream to the output interface 140. The output interface 140 may send the merged codestream to the decoder 200.

It should be understood that the encoder 100 shown in fig. 2 is merely one example provided herein, and in particular implementations, the encoder 100 may include more or fewer modules than those shown in fig. 2. This is not particularly limited in the embodiments of the present application.

Fig. 6 is a schematic block diagram of a decoder 200 according to an embodiment of the present application. Fig. 6 illustrates an example of an MPEG PCC decoding framework. Referring to fig. 6, the decoder 200 may include a demultiplexing module 201, an image or video based decoding module 202, an occupancy map encoding module 203, an auxiliary information decoding module 204, a point cloud occupancy map filling module 205, a geometrical information reconstruction module 206 of the point cloud, a point cloud filtering module 207, and a texture information reconstruction module 208 of the point cloud.

The demultiplexing module 201 may receive the combined code stream transmitted by the output interface 140 of the encoder 100 through the input interface 204 and transmit the combined code stream to a corresponding decoding module. Specifically, the demultiplexing module 201 sends a code stream including a texture map of the encoded current point cloud frame and a depth map of the encoded current point cloud frame to the image or video-based decoding module 202, sends a code stream including an occupancy map of the encoded low-resolution current point cloud frame to the occupancy map decoding module 203, and sends a code stream including encoded auxiliary information to the auxiliary information decoding module 204.

The image or video-based decoding module 202 may decode the received code stream including the texture map of the encoded current point cloud frame and the code stream including the depth map of the encoded current point cloud frame to obtain texture map information of the reconstructed current point cloud frame and depth map information of the reconstructed current point cloud frame, send the texture map information of the reconstructed current point cloud frame to the point cloud texture information reconstruction module 208, and send the depth map information of the reconstructed current point cloud frame to the point cloud geometric information reconstruction module 206. The occupancy graph code module 203 may decode the received code stream including the occupancy graph of the encoded low-resolution current point cloud frame to obtain the occupancy graph information of the reconstructed low-resolution current point cloud frame, and send the occupancy graph information of the reconstructed low-resolution current point cloud frame to the point cloud occupancy graph filling module 205. The point cloud occupancy map filling module 205 may obtain the occupancy map information of the reconstructed current point cloud frame with the original resolution according to the occupancy map information of the reconstructed current point cloud frame with the low resolution, and then send the occupancy map information of the reconstructed current point cloud frame with the original resolution to the point cloud geometric information reconstruction module 206. For convenience of description, the occupancy map information of the reconstructed current point cloud frame with the original resolution is subsequently unified, and is simply referred to as the occupancy map information of the reconstructed current point cloud frame.

The auxiliary information decoding module 204 may decode the received code stream including the encoded auxiliary information to obtain auxiliary information, and may send the auxiliary information to the geometrical information reconstruction module 206 of the point cloud. The specific decoding method will be described in the following embodiments.

The point cloud geometry information reconstruction module 206 may reconstruct the geometry of the current point cloud frame according to the received depth map information of the reconstructed current point cloud frame sent by the image or video based decoding module 202, the occupancy map information of the reconstructed current point cloud frame sent by the point cloud occupancy map filling module 205, and the auxiliary information sent by the auxiliary information decoding module 204, so as to obtain a reconstructed point cloud. The reconstructed point cloud is similar to the reconstructed point cloud obtained by the point cloud reconstruction module 112 in the encoder 100, and the specific reconstruction process may refer to the reconstruction process of the point cloud reconstruction module 112 in the encoder 100, which is not described herein again. The point cloud geometry information reconstruction module 206 may also send the reconstructed point cloud to the point cloud filtering module 207. The point cloud filtering module 207 may filter the reconstructed point cloud according to the received reconstructed point cloud to obtain a smooth reconstructed point cloud, and the specific filtering process may refer to the filtering process of the point cloud filtering module 114 in the encoder 100, which is not described herein again. The point cloud filtering module 207 may send the smoothed reconstructed point cloud to the texture information reconstruction module 208 of the point cloud. The texture information reconstruction module 208 of the point cloud may reconstruct the texture information of the reconstructed point cloud after receiving the smoothed reconstructed point cloud sent by the point cloud filtering module 207 and the texture map information of the reconstructed current point cloud frame sent by the image or video-based decoding module 202, so as to obtain the reconstructed point cloud reconstructed by the texture information.

It should be understood that the decoder 200 shown in fig. 6 is merely an example, and in particular implementations, the decoder 200 may include more or fewer modules than shown in fig. 6. This is not limited in the embodiments of the present application.

For the sake of easy understanding, technical terms referred to in the embodiments of the present application will be described below.

In one example, the current point cloud group may be a group of frames (GOF), and one group of frames may include 32 frame point clouds, i.e., the current point cloud group may include 32 frame point clouds. Of course, the number of frames of the point clouds included in the current point cloud group may also be other values, which is not limited in this embodiment. In one example, a frame group may be further divided into a plurality of subgroups, each subgroup may include one or more frames of point clouds, and the current point cloud group may be any one of the plurality of subgroups. For example, referring to fig. 7, fig. 7 is a schematic diagram of a frame group provided in this embodiment, as shown in fig. 7, one frame group may be divided into 10 subgroups, where the first subgroup includes the 1 st, 2,3 rd frame point clouds, the second subgroup includes the 4 th frame point cloud, the third subgroup includes the 5 th, 6 th, 7 th, 8 th frame point clouds, … …, the 10 th subgroup includes the 30 th, 31 th, 32 th frame point clouds, and the current point cloud group may be any one of the 10 subgroups. The point clouds included in the first, second, third and tenth sub-groups are shown in fig. 7 by way of example only, and the point clouds included in the other sub-groups are not specifically described. In summary, the current point cloud group may be a frame group, or the current point cloud group may be a sub-group in a frame group, and of course, the current point cloud group may also be a point cloud group in other forms, such as a custom point cloud group, which is not limited in this embodiment of the present application.

Based on the above description, the types of the patch can be classified into a first type of patch, a second type of patch, and a third type of patch. Illustratively, the patch may also be divided into a global matching patch (SKIP _ patch), a local matching patch (local _ patch), and a non-matching patch (I _ INTRA, or P _ INTRA) according to the matching condition of the patch in the current point cloud set. That is, the first type of patch may be a global matching patch, the second type of patch may be a local matching patch, and the third type of patch may be a non-matching patch. For convenience of description, the three types of patches are described by taking the patch to be encoded as an example, and it should be understood that the patch to be encoded is any one of the patches in the current point cloud frame, and the current point cloud frame is any one of the frame point clouds in the current point cloud group. If the patch to be encoded is the first type of patch, the patch to be encoded has a matching relationship in all point cloud frames in the current point cloud group, which may be referred to as a global matching patch; if the patch to be encoded is the second type of patch, the patch to be encoded has a matching relationship in the reference frame of the current point cloud frame, but only a part of the point cloud frames in the current point cloud group has a patch having a matching relationship, which may be called a local matching patch; if the patch to be encoded is a patch of the third type, the patch to be encoded does not have a patch having a matching relationship in any point cloud frame in the current point cloud group, and this may be referred to as a non-matching patch.

In one possible implementation, referring to fig. 8, the reference frame of the current point cloud frame may be located within the current point cloud group. For example, the reference frame of the current point cloud frame may be a point cloud frame previous to the current point cloud frame. In addition, the patches included in all the point cloud frames in the current point cloud group may be sequentially arranged according to the order of the first type of patch, the second type of patch, and the third type of patch, wherein the number of the first type of patch in each point cloud frame in the current point cloud group is the same, and when the number of the first type of patch is greater than 1, the first type of patch in each point cloud frame in the current point cloud group may also be sequentially arranged according to the same order, that is, the first type of patch at the same position in each point cloud frame has a matching relationship. For example, w-th first-type patches in each point cloud frame have a matching relationship with each other, and w is less than or equal to the number of the first-type patches.

In general, the two-dimensional information, the rotation transformation information, and the normal coordinate axis information of the patch of the first type in the occupancy map of the current point cloud frame are the same, and the three-dimensional information is different. In some cases, however, the two-dimensional information of the first type of patch may be different, or the three-dimensional information may be the same. That is, the rotation transformation information and the normal coordinate axis information of the first type patch may be the same, and the two-dimensional information and the three-dimensional information may be the same or different. Thus, there may be three cases for two-dimensional information and three-dimensional information. For convenience of description, the following takes the patch to be encoded as an example. In a first possible case, when the to-be-encoded patch is a first type of patch, the two-dimensional information of the reference patch of the to-be-encoded patch is the same as the two-dimensional information of the to-be-encoded patch, and the three-dimensional information of the reference patch of the to-be-encoded patch is the same as the three-dimensional information of the to-be-encoded patch. In a second possible case, when the to-be-encoded patch is the first type of patch, the two-dimensional information of the reference patch of the to-be-encoded patch is the same as the two-dimensional information of the to-be-encoded patch, and the three-dimensional information of the reference patch of the to-be-encoded patch is different from the three-dimensional information of the to-be-encoded patch. In a third possible case, when the to-be-encoded patch is the first type of patch, the two-dimensional information of the reference patch of the to-be-encoded patch is different from the two-dimensional information of the to-be-encoded patch, and the three-dimensional information of the reference patch of the to-be-encoded patch is different from the three-dimensional information of the to-be-encoded patch.

When the patches included in all the point cloud frames in the current point cloud group are arranged in the manner shown in fig. 8, the reference patch of the patch to be encoded may be a patch located at the same position in the reference frame of the current point cloud frame where the patch to be encoded is located. For example, the patch to be encoded is located at the third position in the current point cloud frame, and then the third patch in the reference frame of the current point cloud frame is the reference patch of the patch to be encoded.

The rotation transformation information may be rotation angle information, coordinate axis transformation information, or mirror transformation information. Wherein the rotation angle information includes a specific rotation angle value or a rotation angle index. For example, if the rotation angle value of the patch to be encoded is 0, it indicates that the patch to be encoded does not rotate; if the rotation angle value of the patch to be coded is 20, the patch to be coded is rotated by 20 degrees; if the rotation angle value of the patch to be encoded is 80, it indicates that the patch to be encoded rotates by 80 °. If the rotation angle index of the patch to be coded is 0, the patch to be coded does not rotate; if the rotation angle index of the patch to be coded is 1, the patch to be coded is rotated by 15 degrees; if the rotation angle index of the patch to be coded is 2, the patch to be coded is rotated by 30 degrees; if the rotation angle index of the to-be-encoded patch is 3, it indicates that the to-be-encoded patch rotates by 45 °. The rotation angle information may also be characterized in other forms, and is not limited in particular. For example, the description may be performed in a matrix form, and particularly, when acceleration processing is performed in a GPU (graphics processing Unit), the matrix form may increase the processing speed.

It should be noted that the rotation of the patch to be encoded may be clockwise rotation or counterclockwise rotation.

If the rotation transformation information is coordinate axis exchange information, the coordinate axis exchange information may be represented as a coordinate axis exchange identifier, for example, when the coordinate axis exchange identifier is false or 0, it represents that a tangent coordinate axis of the patch to be encoded and a double tangent coordinate axis of the patch to be encoded are not exchanged; for example, when the coordinate axis exchange identifier is "tune" or "1", the tangent coordinate axis of the patch to be encoded and the bi-tangent coordinate axis of the patch to be encoded are exchanged. For example, the index of the coordinate axis of the patch to be encoded may be 0, 1, or 2. Assuming that the index of the X axis is 0, the index of the Y axis is 1, and the index of the Z axis is 2, the tangent coordinate axis index of the patch to be encoded and the double tangent coordinate axis index of the patch to be encoded may be any 2 of 0, 1, and 2. In general, the tangential coordinate axis of the patch may be referred to as the U-axis, and the bi-tangential coordinate axis of the patch may be referred to as the V-axis. If the obtained tangent coordinate axis index of the current patch and the obtained double tangent coordinate axis index of the to-be-coded patch are respectively 0 and 2 under a certain condition, if the coordinate axis exchange identifier is false or 0, the index of the tangent coordinate axis of the to-be-coded patch and the index of the double tangent coordinate axis of the to-be-coded patch are not exchanged; if the coordinate axis exchange identification is tune or 1, exchanging the index of the tangent coordinate axis of the patch to be encoded and the index of the double tangent coordinate axis of the patch to be encoded, namely updating the index of the tangent coordinate axis of the patch to be encoded to 2, and updating the index of the double tangent coordinate axis of the patch to be encoded to 0.

Next, a first point cloud encoding method provided in the embodiment of the present application will be described. It should be noted that, in conjunction with the point cloud decoding system shown in fig. 1 and the schematic block diagram of the encoder 100 shown in fig. 2, any of the point cloud encoding methods below may be performed by the encoder 100 in the point cloud decoding system, and more specifically, may be performed by the auxiliary information encoding module 109 in the encoder 100.

Fig. 9 is a flowchart of a point cloud encoding method, which is applied to a point cloud decoding system according to an embodiment of the present disclosure. Referring to fig. 9, the method includes:

s901: the auxiliary information of the patch to be encoded and the first identification of the patch to be encoded are obtained.

S902: when the first identification is a first value, encoding auxiliary information and the first identification of the patch to be encoded into a code stream, wherein the patch to be encoded is the first patch in the current point cloud frame; or when the first identifier is a first value, encoding auxiliary information of a patch to be encoded into the code stream, wherein the patch to be encoded is a non-first patch in the current point cloud frame.

In general, because the first type of patch has a matching relationship in all the point cloud frames in the current point cloud group, the two-dimensional information, the rotational transformation information, and the normal coordinate axis information of the first type of patch in the current point cloud group are the same, and the three-dimensional information may not be the same. Therefore, in order to reduce the bit overhead of the code stream and further improve the encoding efficiency, when the first identifier is the first value, the auxiliary information of the patch to be encoded may include three-dimensional information, but not include two-dimensional information, rotation transformation information, and normal coordinate axis information.

In a possible case, the patches in each point cloud frame may be arranged according to the order of the first type of patch, the second type of patch, and the third type of patch, and then encoded according to the order of the first type of patch, the second type of patch, and the third type of patch. Therefore, if the first type of patch exists in the current point cloud group, the first patch in each point cloud frame is the first type of patch. The number of the first type of patch in each point cloud frame in the current point cloud group may be the same, and when the number of the first type of patch is greater than 1, the first type of patch in each point cloud frame in the current point cloud group may be sequentially arranged in the same order, that is, the first type of patch at the same position in each point cloud frame has a matching relationship. If the first type of patch does not exist in the current point cloud group, the first patch in each point cloud frame may be the second type of patch or the third type of patch. If the first patch in each point cloud frame is a third type of patch, it may indicate that only this one point cloud frame is included in the current point cloud group, and the first identifiers of all the patches in this point cloud frame are third values.

For the case that the first patch in each point cloud frame in the current point cloud group is the first type of patch or the second type of patch, in order to facilitate distinguishing which point cloud frames can form a point cloud group, as shown in fig. 10, for the first point cloud frame in the current point cloud group, the first identifiers of all the patches included in the first point cloud frame can be set as the third value, and the patches in other point cloud frames are still arranged according to the sequence of the first type of patch, the second type of patch and the third type of patch. Thus, when the first identifier of the patch to be encoded is a first value, it may be determined that the current point cloud frame is a non-first point cloud frame within the current point cloud group. Moreover, the current point cloud frame may be a kth point cloud frame in the current point cloud group, where K is an integer greater than or equal to 3. That is, the 3 rd point cloud frame in the current point cloud group and each point cloud frame after the 3 rd point cloud frame may be encoded in the manner of S902.

Based on the above description, the 3 rd point cloud frame in the current point cloud group and each point cloud frame after the 3 rd point cloud frame may be encoded in the manner of S902, and for the 2 nd point cloud frame in the current point cloud group, the following two manners may be encoded:

As an example, a partial syntax structure when the first flag of the patch to be encoded is the first value is illustrated herein, see table 1 and table 2.

TABLE 1

TABLE 2

The skip _ patch _ data _ unit in table 1 is information of the to-be-encoded patch coded into the code stream when the first identifier of the to-be-encoded patch is the first value, and may also be referred to as a data unit of the to-be-encoded patch.

When the first identifier is the second value or the third value, the auxiliary information of the patch to be encoded and the first identifier may be encoded into the code stream. That is, for each patch in the first point cloud frame in the current point cloud group, the auxiliary information and the first identifier of each patch may be encoded into the code stream. For the non-first type patch in the non-first point cloud frame in the current point cloud group, the auxiliary information and the first identifier of each patch can be coded into the code stream.

In general, a second type of patch has a patch having a matching relationship in a reference frame of a current point cloud frame, and rotational transformation information between the second type of patch and the reference patch is the same as normal coordinate axis information, but two-dimensional information and three-dimensional information may not be the same, and a third type of patch does not have a patch having a matching relationship in a reference frame of the current point cloud frame, so as to reduce bit overhead of a code stream and further improve encoding efficiency, when a first identifier is a first value, auxiliary information of the patch to be encoded may include three-dimensional information, but does not include two-dimensional information, rotational transformation information, and normal coordinate axis information. When the first flag is the second value, the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, and flag information of a reference patch of the patch to be encoded, but does not include rotation transformation information and normal coordinate axis information. When the first identifier is the third value, the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.

In a third possible implementation manner, the first identifiers of the multiple patches included in the current point cloud frame may be packed together and coded into the code stream, where the first identifiers of the multiple patches include a first identifier of a patch to be encoded, and the first identifiers of the patches whose values are the second value or the third value are left in the multiple patches except for the patch to be encoded. For the auxiliary information of each patch included in the current point cloud frame, the same type of information in each patch auxiliary information may be packed together and encoded into the code stream. Different types of information in the auxiliary information of each patch can be packed and coded into the code stream respectively.

In order to further reduce the bit overhead of the code stream, when encoding is performed according to the sequence shown in fig. 8, for the third type of patch, encoding may also be performed by using other methods besides the encoding method described above, for example, if the to-be-encoded patch is the first third type of patch, encoding may be performed by using a fixed-length encoding method, or may also be performed by using a variable-length encoding method, which is not limited in this embodiment of the present application. If the patch to be coded is not the first third type of patch, at this time, the two-dimensional information and the three-dimensional information of the patch to be coded can be coded into the code stream by adopting a differential coding mode according to the two-dimensional information and the three-dimensional information of the first third type of patch, and other information is coded by adopting the mode described above. According to the two-dimensional information and the three-dimensional information of the first third type of patch, coding the two-dimensional information and the three-dimensional information of the patch to be coded into the code stream in a differential coding mode refers to: and coding the difference between the two-dimensional information of the patch to be coded and the two-dimensional information of the first third type of patch and the difference between the three-dimensional information of the patch to be coded and the three-dimensional information of the first third type of patch into the code stream.

In the embodiment of the application, the auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded are obtained first. Because the number of the first type of patch in each point cloud frame in the current point cloud group can be the same, when the first identifier is a first value and the to-be-encoded patch is the first patch in the current point cloud frame, it indicates that the to-be-encoded patch is the first type of patch in the current point cloud frame, and at this time, the auxiliary information of the to-be-encoded patch and the first identifier can be encoded into the code stream; when the first identifier is a first value and the to-be-encoded patch is a non-first patch in the current point cloud frame, it is indicated that the to-be-encoded patch is a non-first type patch in the current point cloud frame, and at this time, the auxiliary information of the to-be-encoded patch can be encoded into the code stream, and the first identifier of the to-be-encoded patch does not need to be encoded into the code stream, so that the format of the encoded code stream can be simplified, the bit cost of the code stream is further reduced, and the encoding efficiency is improved.

Next, a point cloud decoding method corresponding to the point cloud encoding method according to the embodiment shown in fig. 9 will be described. It should be noted that, in conjunction with the point cloud decoding system shown in fig. 1 and the schematic block diagram of the decoder 200 shown in fig. 6, any of the point cloud decoding methods below may be performed by the decoder 200 in the point cloud decoding system, and more specifically, may be performed by the auxiliary information decoding module 204 in the decoder 200. Referring to fig. 11, the decoding method includes:

S1101: and when the patch to be decoded is the first patch in the current point cloud frame, analyzing a first identifier of the patch to be decoded from the code stream.

S1102: and when the first identification is a first value, analyzing the auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded.

As can be seen from the embodiment shown in fig. 9, when the first flag is the first value, it may indicate that the current point cloud frame is a non-first point cloud frame in the current point cloud group. At this time, if the current point cloud frame is the 2 nd point cloud frame in the current point cloud group, the patch to be decoded may be decoded through the following two possible implementation manners.

In a first possible implementation manner, the first identifier of the first patch in the 2 nd point cloud frame in the current point cloud group and the number of the first type of patches included in each point cloud frame in the current point cloud group are jointly encoded into the code stream. At this time, if the current point cloud frame is the 2 nd point cloud frame in the current point cloud group and the to-be-decoded patch is the first patch in the current point cloud frame, the first identifier of the to-be-decoded patch may be analyzed from the code stream, and when the first identifier is the first value, the number of the first type of patches in the current point cloud group may be analyzed from the code stream. And then analyzing the auxiliary information of the patch to be decoded from the code stream.

Next, a case where the to-be-decoded patch is the nth patch in the current point cloud frame will be described with reference to S1103 and S1104.

S1103: when the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patches in the reference point cloud frame of the current point cloud frame, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded.

As can be seen from the embodiment shown in fig. 9, in the encoding process of the first type of patch, the first identifier of the first type of patch that is not the first type of patch is not encoded into the code stream, so when the to-be-decoded patch is the nth patch in the current point cloud frame and n is greater than 1 and less than or equal to the number of the first type of patches in the reference point cloud frame of the current point cloud frame, it indicates that the to-be-decoded patch is the first type of patch in the current point cloud frame, and at this time, the auxiliary information of the to-be-decoded patch may be analyzed from the code stream, where the auxiliary information includes three-dimensional information of the to-be-decoded patch.

When the patch in each point cloud frame in the current point cloud group is encoded according to the arrangement shown in fig. 8, the reference patch of the patch to be decoded may be a patch whose position in the reference frame of the current point cloud frame is the same as the position of the patch to be decoded in the current point cloud frame.

Regarding the manner of analyzing the three-dimensional information of the patch to be decoded from the code stream by adopting the differential decoding manner, the method may specifically be: and acquiring the three-dimensional information of the patch with the same position as that of the to-be-decoded patch in the current point cloud frame from the decoded point cloud, or acquiring the three-dimensional information of the reference patch of the to-be-decoded patch from the reference frame of the current point cloud frame. And then, adding the acquired three-dimensional information and the three-dimensional information difference value analyzed from the code stream to obtain the three-dimensional information of the patch to be decoded.

Based on the description of S1102, the number of the first type patches included in each point cloud frame in the current point cloud group may be determined in two ways. In a first possible implementation manner, after the first identifier of the first patch in the 2 nd point cloud frame in the current point cloud group is decoded, the number of the first type of patches in the current point cloud group can be directly analyzed from the code stream. In a second possible implementation manner, after the first identifiers of all the patches of the 2 nd point cloud frame in the current point cloud group are decoded, the number of the patches of which the first identifier is the first value may be determined, so that the number of the first type of patches included in the 2 nd point cloud frame, that is, the number of the first type of patches in the current point cloud group, may be determined. That is, the number of the first type patches included in each point cloud frame in the current point cloud group can be determined by the 2 nd point cloud frame.

In one possible scenario, the 2 nd point cloud frame within the current point cloud group may serve as a reference point cloud frame for the 3 rd point cloud frame, the 3 rd point cloud frame may serve as a reference point cloud frame for the 4 th point cloud frame, and so on. Therefore, the 3 rd point cloud frame and the point cloud frames after the 3 rd point cloud frame can determine whether the nth patch in the current point cloud frame is the first type of patch according to the number of the first type of patches in the reference point cloud frame.

S1104: when the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and greater than the number of the first type of patches in the reference point cloud frame of the current point cloud frame, analyzing a first identifier of the patch to be decoded from the code stream.

S1105: and when the first identification is the second value, analyzing auxiliary information of the to-be-decoded patch from the code stream, wherein the auxiliary information comprises two-dimensional information and three-dimensional information of the to-be-decoded patch and identification information of a reference patch of the to-be-decoded patch.

Since the auxiliary information of the second type of patch, which is coded into the code stream in the coding process, includes the two-dimensional information, the three-dimensional information and the identification information of the reference patch, but does not include the rotational transformation information and the normal coordinate axis information, when the to-be-decoded patch is the second type of patch, the auxiliary information analyzed from the code stream includes the two-dimensional information and the three-dimensional information of the to-be-decoded patch and the identification information of the reference patch of the to-be-decoded patch, but does not include the rotational transformation information and the normal coordinate axis information of the to-be-decoded patch. At this time, the rotation transformation information and the normal coordinate axis information of the to-be-decoded patch may also be acquired according to the identification information of the reference patch of the to-be-decoded patch.

For the identification information of the reference patch of the to-be-decoded patch, a method matched with the encoding end may be adopted to analyze from the code stream, for example, a fixed-length decoding manner may be adopted to analyze. For the rotation transformation information and the normal coordinate axis information of the to-be-decoded patch, the reference patch of the to-be-decoded patch can be determined through the identification information of the reference patch of the to-be-decoded patch, so that the rotation transformation information and the normal coordinate axis information of the reference patch of the to-be-decoded patch are acquired, and the rotation transformation information and the normal coordinate axis information of the reference patch of the to-be-decoded patch are used as the rotation transformation information and the normal coordinate axis information of the to-be-decoded patch. As an example, the two-dimensional information and the three-dimensional information of the patch to be decoded may be decoded by a method matching with the encoding end, for example, the two-dimensional information and the three-dimensional information of the patch to be decoded may be analyzed from the code stream in a differential decoding manner. Specifically, the reference patch of the to-be-decoded patch may be determined by the identification information of the reference patch of the to-be-decoded patch, and then the two-dimensional information and the three-dimensional information of the reference patch of the to-be-decoded patch may be acquired. And then, correspondingly adding the acquired two-dimensional information and three-dimensional information with a two-dimensional information difference value and a three-dimensional information difference value which are analyzed from the code stream respectively, thereby obtaining the two-dimensional information and the three-dimensional information of the patch to be decoded.

S1106: and when the first identification is a third value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded.

Based on the above description, the first type of patch, the second type of patch, and the third type of patch in each point cloud frame in the current point cloud group may be arranged as shown in fig. 8, at this time, if the to-be-decoded patch is the first patch in the current point cloud frame, then, after the first identifier of the to-be-decoded patch is analyzed from the code stream, it may be determined whether the current point cloud frame is the first frame point cloud in the current point cloud group according to the value of the first identifier. That is, when the patch to be decoded is the first patch in the current point cloud frame and the first identifier is the third value, it may be determined that the current point cloud frame is the first frame point cloud in the current point cloud group.

As an example, the fixed-length decoding manner may be adopted to analyze the two-dimensional information, the three-dimensional information, the rotation transformation information, and the normal coordinate axis information of the to-be-decoded patch from the code stream, or the variable-length decoding manner may be adopted to analyze the two-dimensional information, the three-dimensional information, the rotation transformation information, and the normal coordinate axis information of the to-be-decoded patch from the code stream, which is not limited in this embodiment of the application. However, if the first auxiliary information of the third type of patch is encoded by a fixed-length encoding method or a variable-length encoding method, the two-dimensional information and the three-dimensional information of the third type of patch are encoded by a differential encoding method, and the other auxiliary information is encoded by a fixed-length encoding method. Then, in this step, for the non-first type of patch, a differential decoding manner may be adopted to analyze the two-dimensional information and the three-dimensional information of the patch to be decoded from the code stream, and a fixed-length decoding manner may be adopted to analyze the rotation transformation information and the normal coordinate axis information of the patch to be decoded from the code stream. For the non-first type of patch, the mode of analyzing the two-dimensional information and the three-dimensional information of the patch to be decoded from the code stream by adopting a differential decoding mode is as follows: and analyzing the difference between the two-dimensional information of the to-be-decoded patch and the two-dimensional information of the first third type of patch from the code stream, and adding the difference and the two-dimensional information of the first third type of patch to obtain the two-dimensional information of the to-be-decoded patch. And analyzing the difference value between the three-dimensional information of the to-be-decoded patch and the three-dimensional information of the first third type of patch from the code stream, and adding the difference value and the three-dimensional information of the first third type of patch to obtain the three-dimensional information of the to-be-decoded patch.

S1107: and reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

In the embodiment of the application, because the first identifier of the first patch in the current point cloud frame is coded into the code stream, when the point cloud block patch to be decoded is the first patch in the current point cloud frame, the first identifier of the patch to be decoded can be analyzed from the code stream, when the first identifier is a first value, the patch to be decoded is indicated to be the first type of patch, at the moment, the auxiliary information of the patch to be decoded can be analyzed from the code stream, and the auxiliary information comprises the three-dimensional information of the patch to be decoded. When the to-be-decoded patch is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patches in the reference point cloud frame of the current point cloud frame, it indicates that the to-be-decoded patch is a non-first type of patch, and at this time, the auxiliary information of the to-be-decoded patch can be analyzed from the code stream, and the auxiliary information includes three-dimensional information of the to-be-decoded patch. When the first identifier is the second value, it indicates that the to-be-decoded patch is the second type of patch, and at this time, the auxiliary information of the to-be-decoded patch may be analyzed from the code stream, where the auxiliary information includes two-dimensional information and three-dimensional information of the to-be-decoded patch and identification information of a reference patch of the to-be-decoded patch. When the first identifier is a third value, it indicates that the patch to be decoded is a third type of patch, and at this time, the auxiliary information of the patch to be decoded can be analyzed from the code stream, where the auxiliary information includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded. That is, the first type of patch needs to analyze the first identifier from the code stream, and the first identifier does not need to be analyzed from the code stream for the non-first type of patch, and for the first type of patch and the second type of patch, only part of auxiliary information needs to be analyzed from the code stream, and other auxiliary information can be derived or obtained, so that the format of the analyzed code stream can be simplified, the bit overhead of the code stream is reduced, and the decoding efficiency is improved.

A second point cloud encoding method provided in the embodiments of the present application is described below. It should be noted that, in conjunction with the point cloud decoding system shown in fig. 1 and the schematic block diagram of the encoder 100 shown in fig. 2, any of the point cloud encoding methods below may be performed by the encoder 100 in the point cloud decoding system, and more specifically, may be performed by the auxiliary information encoding module 109 in the encoder 100. Referring to fig. 12, the method includes:

s1201: the auxiliary information of the patch to be encoded and the first identification of the patch to be encoded are obtained.

When it needs to be described, reference may be made to S901 in the embodiment shown in fig. 9 for the content of S1201, which is not described again in this embodiment of the application.

S1202: when the first identification is a first value, encoding auxiliary information of a patch to be encoded, the first identification and a second identification of the patch to be encoded into a code stream, wherein the patch to be encoded is the first patch in the current point cloud frame; or when the first identifier is a first value, encoding the auxiliary information of the patch to be encoded and the second identifier of the patch to be encoded into the code stream, wherein the patch to be encoded is a non-first patch in the current point cloud frame.

In general, there are patches having matching relationships in all point cloud frames in the current point cloud group of the first type of patch, and the two-dimensional information, the rotational transformation information, and the normal coordinate axis information of the first type of patch in the current point cloud group are the same, and the three-dimensional information may not be the same, but in some cases, the three-dimensional information of the first type of patch may also be the same. In this case, when the first identifier is a first value, the to-be-encoded patch may be indicated as a first type of patch, and the information encoded into the codestream may further include a second identifier.

As an example, a partial syntax structure when the first flag of the patch to be encoded is a first value and the information encoded into the codestream includes a second flag is illustrated herein, see tables 1 and 3.

TABLE 3

Further, in a possible case, when the to-be-encoded patch is the first value, that is, when the to-be-encoded patch is the first type of patch, the three-dimensional information of the to-be-encoded patch is different from the three-dimensional information of the reference patch, and the two-dimensional information of the to-be-encoded patch is different from the two-dimensional information of the reference patch. In this way, when the first identifier is the first value, the information coded into the code stream may include, in addition to the second identifier, a third identifier for indicating whether the two-dimensional information of the patch to be encoded has been coded into the code stream. The syntax element of the third flag may be override _3d _ shift _ data _ flag. The second identifier has already been described, and therefore is not described herein again.

And when the third identifier is a sixth value, the two-dimensional information of the patch to be coded can be indicated to be not coded into the code stream. That is, the two-dimensional information of the to-be-encoded patch is the same as the two-dimensional information of the reference patch of the to-be-encoded patch, so that the two-dimensional information of the to-be-encoded patch does not need to be encoded into the code stream again, and the auxiliary information of the to-be-encoded patch does not include the two-dimensional information. And when the third identifier is a seventh value, the two-dimensional information of the patch to be coded can be indicated to be coded into the code stream. That is, the two-dimensional information of the to-be-encoded patch is different from the two-dimensional information of the reference patch of the to-be-encoded patch, and therefore the two-dimensional information of the to-be-encoded patch needs to be encoded into the code stream, that is, the auxiliary information of the to-be-encoded patch may include the two-dimensional information. Wherein, the syntax element of the sixth value may be false, and the syntax element of the seventh value may be true.

As an example, a partial syntax structure when the first flag of the patch to be encoded is a first value and the information coded into the code stream includes the second flag and the third flag is illustrated herein, see table 1 and table 4.

TABLE 4

When the three-dimensional information of the patch to be encoded is different from the three-dimensional information of the reference patch, and the two-dimensional information of the patch to be encoded is different from the two-dimensional information of the reference patch, whether the two-dimensional information and the three-dimensional information of the patch to be encoded are included in the information of the code stream can be indicated through the second identifier and the third identifier. The indication may also be performed in other manners, for example, whether the two-dimensional information and the three-dimensional information of the to-be-encoded patch are both encoded into the code stream may also be indicated by the fourth identifier, that is, when the first identifier is the first value, the to-be-encoded patch may be indicated as the first type of patch, and the information encoded into the code stream may also include the fourth identifier. The syntax element of the fourth flag may be override _ patch _ data _ flag. When the fourth flag is an eighth value, it may be indicated that both the two-dimensional information and the three-dimensional information of the to-be-encoded patch are the same as both the two-dimensional information and the three-dimensional information of the reference patch of the to-be-encoded patch, and are not encoded into the code stream. When the fourth identifier is a ninth value, it may be indicated that both the two-dimensional information and the three-dimensional information of the to-be-encoded patch are different from both the two-dimensional information and the three-dimensional information of the reference patch of the to-be-encoded patch, and both the two-dimensional information and the three-dimensional information are already encoded in the code stream. Wherein, the syntax element of the eighth value may be a flash, and the syntax element of the ninth value may be a true.

The three-dimensional information of the patch to be encoded may include a shift of the patch to be encoded in the 3D space along a tangential direction (3D _ shift _ vary _ axis), a shift of the patch to be encoded in the 3D space along a bi-tangential direction (3D _ shift _ bitangent _ axis), and a shift of the patch to be encoded in the 3D space along a normal direction (3D _ shift _ normal _ axis). The two-dimensional information of the patch to be encoded may include a shift (2d _ shift _ u) of the patch to be encoded in the u-axis direction in the current point cloud frame occupancy map, a shift (2d _ shift _ v) of the patch to be encoded in the v-axis direction in the current point cloud frame occupancy map, a width (2d _ size _ u) of the patch to be encoded in the current point cloud frame occupancy map, and a height (2d _ size _ v) of the patch to be encoded in the current point cloud frame occupancy map. Therefore, whether the three-dimensional information of the patch to be encoded has been coded into the codestream can also be indicated by an indication (override _3d _ shift _ range _ axis) indicating whether 3d _ shift _ range _ axis has been coded into the codestream, and an indication (override _3d _ shift _ normal _ axis) indicating whether 3d _ shift _ normal _ axis has been coded into the codestream, and indicating whether the two-dimensional information of the patch to be encoded is coded into the code stream by an indication (override _2d _ shift _ u) indicating whether 2d _ shift _ u is coded into the code stream, an indication (override _2d _ shift _ v) indicating whether 2d _ shift _ v is coded into the code stream, an indication (override _2d _ size _ u) indicating whether 2d _ size _ u is coded into the code stream, and an indication (override _2d _ size _ v) indicating whether 2d _ size _ v is coded into the code stream. Similarly to the second identifier, the third identifier, and the fourth identifier, each identifier may have two values, and syntax elements of the two values may be true and false. The meaning of each value can refer to the meaning of a different value of the second identifier, the third identifier or the fourth identifier, which is not described herein again.

As an example, this document illustrates a partial syntax structure when the first flag of the patch to be encoded is a first value, and the information coded into the code stream includes a fourth flag and flags for indicating different two-dimensional information and three-dimensional information, respectively, as shown in table 1 and table 5.

TABLE 5

It should be noted that the same contents in the embodiment of the present application and the embodiment shown in fig. 9 are not repeated, and reference may be made to the description in the embodiment shown in fig. 9.

In the embodiment of the application, the auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded are obtained first. Because the number of the first type of patch in each point cloud frame in the current point cloud group can be the same, when the first identifier is a first value and the to-be-encoded patch is the first patch in the current point cloud frame, it indicates that the to-be-encoded patch is the first type of patch in the current point cloud frame, and at this time, the auxiliary information, the first identifier and the second identifier of the to-be-encoded patch can be encoded into the code stream; when the first identifier is a first value and the to-be-encoded patch is a non-first patch in the current point cloud frame, it is indicated that the to-be-encoded patch is a non-first type patch in the current point cloud frame, and the auxiliary information and the second identifier of the to-be-encoded patch can be encoded into the code stream without encoding the first identifier of the to-be-encoded patch into the code stream, so that the format of the encoded code stream can be simplified. The second identifier is an identifier used for indicating whether the three-dimensional information of the patch to be encoded is coded into the code stream. And when the second identifier is a fourth value, the three-dimensional information of the patch to be coded is indicated to be not coded into the code stream. That is, the three-dimensional information of the to-be-encoded patch is the same as the three-dimensional information of the reference patch of the to-be-encoded patch, so that the three-dimensional information of the to-be-encoded patch does not need to be encoded into the code stream again, that is, the auxiliary information of the to-be-encoded patch may not include the three-dimensional information, and thus, the bit overhead of the code stream can be further reduced, and the encoding efficiency is improved.

Next, a point cloud decoding method based on the point cloud encoding method of the embodiment shown in fig. 12 will be described. It should be noted that, in conjunction with the point cloud decoding system shown in fig. 1 and the schematic block diagram of the decoder 200 shown in fig. 6, any of the point cloud decoding methods below may be performed by the decoder 200 in the point cloud decoding system, and more specifically, may be performed by the auxiliary information decoding module 204 in the decoder 200. Referring to fig. 13, the decoding method includes:

s1301: and when the patch to be decoded is the first patch in the current point cloud frame, analyzing a first identifier of the patch to be decoded from the code stream.

It should be noted that the content of S1301 may refer to the content of S1101 described above, and is not described herein again.

S1302: and when the first identifier is a first value, analyzing a second identifier of the patch to be decoded from the code stream.

It should be noted that the second identifier has already been described in the above step S1202, and therefore is not described herein again.

S1303: when the second identifier is a fourth value, determining auxiliary information of the to-be-decoded patch according to the auxiliary information of the reference patch of the to-be-decoded patch; and when the second identifier is a fifth value, analyzing the auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises the three-dimensional information of the patch to be decoded.

In some embodiments, based on the description of step 1202, in a case that the three-dimensional information of the patch to be encoded is different from the three-dimensional information of the reference patch, and the two-dimensional information of the patch to be encoded is different from the two-dimensional information of the reference patch, the information encoded into the codestream may further include, in addition to the second identifier, a third identifier for indicating whether the two-dimensional information of the patch to be encoded has been encoded into the codestream. That is, when the first identifier is the first value, in addition to parsing the second identifier of the patch to be decoded from the code stream, the third identifier of the patch to be decoded may also be parsed from the code stream. And when the third identifier is a sixth value, the two-dimensional information of the patch to be decoded can be indicated to be not coded into the code stream. That is, the two-dimensional information of the to-be-decoded patch is the same as the two-dimensional information of the reference patch of the to-be-decoded patch, and thus the two-dimensional information of the reference patch of the to-be-decoded patch can be used as the two-dimensional information of the to-be-decoded patch. When the third flag is a seventh value, it may indicate that the two-dimensional information of the patch to be decoded has been coded into the code stream. That is, the two-dimensional information of the to-be-decoded patch is different from the two-dimensional information of the reference patch of the to-be-decoded patch, so that the two-dimensional information of the to-be-decoded patch can be analyzed from the code stream. The specific decoding method may be a decoding method matching the encoding method, and is not described herein again.

In other embodiments, based on the description of step 1202, when the three-dimensional information of the to-be-encoded patch is different from the three-dimensional information of the reference patch, and the two-dimensional information of the to-be-encoded patch is different from the two-dimensional information of the reference patch, it may be indicated whether the two-dimensional information and the three-dimensional information of the to-be-decoded patch are included in the information encoded into the code stream by using the second identifier and the third identifier. The indication may also be performed in other manners, for example, whether the two-dimensional information and the three-dimensional information of the patch to be decoded are both coded into the code stream may also be indicated by the fourth identifier. That is, when the first identifier is the first value, the fourth identifier of the patch to be decoded may be parsed from the code stream. When the fourth flag is the eighth value, it may be indicated that both the two-dimensional information and the three-dimensional information of the to-be-decoded patch are the same as those of the reference patch of the to-be-decoded patch, and the code stream is not encoded, so that the two-dimensional information and the three-dimensional information of the reference patch of the to-be-decoded patch may be used as the two-dimensional information and the three-dimensional information of the to-be-decoded patch. When the fourth identifier is the ninth value, it may be indicated that both the two-dimensional information and the three-dimensional information of the to-be-decoded patch are different from both the two-dimensional information and the three-dimensional information of the reference patch of the to-be-decoded patch, and both the two-dimensional information and the three-dimensional information of the to-be-decoded patch are encoded into the code stream, so that the two-dimensional information and the three-dimensional information of the to-be-decoded patch may be analyzed from the code stream. The specific decoding method may be a decoding method matching the encoding method, and is not described herein again.

And, for indicating whether the three-dimensional information of the to-be-decoded patch has been coded into the codestream by the flag indicating whether 3d _ shift _ range _ axis has been coded into the codestream (override _3d _ shift _ range _ axis), the flag indicating whether 3d _ shift _ bitangent _ axis has been coded into the codestream (override _3d _ shift _ bitangent _ axis), and the flag indicating whether 3d _ shift _ normal _ axis has been coded into the codestream (override _3d _ shift _ normal _ axis), and indicating whether the three-dimensional information of the to-be-decoded patch has been coded into the codestream by the flag indicating whether 2d _ shift _ u has been coded into the codestream (override _2d _ shift _ u), the flag indicating whether 2d _ shift _ v has been coded into the codestream (override _2d _ shift _ v), the flag indicating whether 2d _ shift _ v has been coded into the codestream (override _2d _ shift _ size _ v), and the flag indicating whether 2d _ shift _ size _2d _ size _ flag indicates whether 2d _ shift _ size _2d _ size _ value indicates that the flag indicates that the codestream has been coded into the bitstream has been coded into, similar to the second identifier, the third identifier and the fourth identifier, the description is omitted here.

S1304: and when the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patches in the reference point cloud frame of the current point cloud frame, analyzing a second identifier of the patch to be decoded from the code stream.

S1305: when the second identifier is a fourth value, determining auxiliary information of the to-be-decoded patch according to the auxiliary information of the reference patch of the to-be-decoded patch; and when the second identifier is a fifth value, analyzing the auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises the three-dimensional information of the patch to be decoded.

It should be noted that, the content of S1305 may refer to the content of S1303, which is not described herein again.

S1306: and reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

It should be noted that the content of S1306 may refer to the content of S1107 described above, and is not described herein again.

In the embodiment of the application, because the first identifier of the first patch in the current point cloud frame is coded into the code stream, when the point cloud block patch to be decoded is the first patch in the current point cloud frame, the first identifier of the patch to be decoded is analyzed from the code stream, when the first identifier is a first value, it indicates that the patch to be decoded is the first type of patch, and at this time, the second identifier of the patch to be decoded can be analyzed from the code stream. When the to-be-decoded patch is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patches in the reference point cloud frame of the current point cloud frame, it indicates that the to-be-decoded patch is the non-first type of patch, and at this time, the second identifier of the to-be-decoded patch can be analyzed from the code stream. And when the second identifier is a fourth value, the three-dimensional information of the patch to be decoded is indicated to be not coded into the code stream. That is, the three-dimensional information of the to-be-decoded patch is the same as the three-dimensional information of the reference patch of the to-be-decoded patch, and at this time, the three-dimensional information of the reference patch of the to-be-decoded patch can be used as the three-dimensional information of the to-be-decoded patch, and the three-dimensional information of the to-be-decoded patch does not need to be analyzed from the code stream, so that the format of the analyzed code stream can be simplified, the bit overhead of the code stream is reduced, and the decoding efficiency is improved.

A third point cloud encoding method provided in the embodiments of the present application is described below. It should be noted that, in conjunction with the point cloud decoding system shown in fig. 1 and the schematic block diagram of the encoder 100 shown in fig. 2, any of the point cloud encoding methods below may be performed by the encoder 100 in the point cloud decoding system, and more specifically, may be performed by the auxiliary information encoding module 109 in the encoder 100. Referring to fig. 14, the method includes:

s1401: the auxiliary information of the patch to be encoded and the first identification of the patch to be encoded are obtained.

It should be noted that the content of S1401 may refer to S901 in the embodiment shown in fig. 9, and this is not described again in this embodiment of the application.

S1402: and when the first identifier is a first value, encoding the first identifier of the patch to be encoded into the code stream.

S1403: and when the first identifier is a second value or a third value, encoding the auxiliary information of the patch to be encoded and the first identifier into the code stream.

Based on the descriptions of S1402 and S1403, it can be known that whether the first identifier of the patch to be encoded is the first value, the second value, or the third value, the first identifier needs to be encoded into the code stream. In this case, the first identifications of all the patches in the current point cloud frame may be packed together and encoded into the codestream. Of course, the code streams may be packed individually.

In the embodiment of the application, the auxiliary information of the to-be-encoded cloud block patch and the first identifier of the to-be-encoded patch are obtained first. Because the two-dimensional information, the rotation transformation information, the normal coordinate axis information and the three-dimensional information among the first types of patches in the current point cloud group can be the same, when the first identifier of the to-be-encoded patch is a first value, the to-be-encoded patch is indicated to be the first type of patch, at the moment, the first identifier of the to-be-encoded patch can be encoded into a code stream, and the auxiliary information of the to-be-encoded patch does not need to be encoded into the code stream, so that the format of the encoded code stream can be simplified. In addition, the value of the first identifier may be a first value, a second value and a third value. Different values indicate different types of patch. Therefore, different types of patches can be distinguished through the first identifier, and for the different types of patches, the content included in the auxiliary information coded into the code stream may be different, and for the first type of patch and the second type of patch, only part of the auxiliary information is coded into the code stream. Therefore, the format of the code stream can be simplified, the bit overhead of the code stream is further reduced, and the coding efficiency is improved.

Next, a point cloud decoding method corresponding to the point cloud encoding method according to the embodiment shown in fig. 14 will be described. It should be noted that, in conjunction with the point cloud decoding system shown in fig. 1 and the schematic block diagram of the decoder 200 shown in fig. 6, any of the point cloud decoding methods below may be performed by the decoder 200 in the point cloud decoding system, and more specifically, may be performed by the auxiliary information decoding module 204 in the decoder 200. Referring to fig. 15, the decoding method includes:

s1501: and analyzing the first identifier of the patch to be decoded from the code stream.

S1502: when the first identification is a first value, determining auxiliary information of the to-be-decoded patch according to the auxiliary information of the reference patch of the to-be-decoded patch, wherein the auxiliary information of the to-be-decoded patch comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.

S1503: and when the first identification is a second value, analyzing the auxiliary information of the to-be-decoded patch from the code stream, wherein the auxiliary information of the to-be-decoded patch comprises two-dimensional information, three-dimensional information and identification information of a reference patch of the to-be-decoded patch.

It should be noted that, for the content of S1503, reference may be made to S1105 in the embodiment shown in fig. 11, which is not described again in this embodiment of the application.

S1504: and when the first identification is a third value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information of the patch to be decoded comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.

It should be noted that the content of S1504 may refer to S1106 in the embodiment shown in fig. 11, which is not described again in this embodiment of the application.

S1505: and reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

It should be noted that, reference may be made to S1107 in the embodiment shown in fig. 11 for the content of S1505, which is not described again in this embodiment of the application.

In the embodiment of the application, a first identifier of a point cloud block patch to be decoded is firstly analyzed from a code stream, when the first identifier is a first value, the patch to be decoded is indicated to be a first type of patch, auxiliary information of the patch to be decoded is not included in the code stream, the auxiliary information of the patch to be decoded can be determined according to the auxiliary information of a reference patch of the patch to be decoded, and the auxiliary information of the patch to be decoded comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information. When the first identification is a second value, it indicates that the to-be-decoded patch is a second type of patch, and at this time, the auxiliary information of the to-be-decoded patch can be analyzed from the code stream, and the auxiliary information of the to-be-decoded patch includes two-dimensional information, three-dimensional information, and identification information of a reference patch of the to-be-decoded patch. When the first identification is a third value, it is indicated that the patch to be decoded is a third type of patch, and at this time, the auxiliary information of the patch to be decoded is analyzed from the code stream, and the auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information. That is, for the first type of patch, only the first identifier needs to be analyzed from the code stream, and other auxiliary information may be derived or obtained, and for the second type of patch, only a part of auxiliary information needs to be analyzed from the code stream, and other auxiliary information may also be derived or obtained. Therefore, the format of the analyzed code stream can be simplified, the bit overhead of the code stream is reduced, and the decoding efficiency is improved.

Referring to fig. 16, fig. 16 is a schematic structural diagram of a first point cloud encoding device according to an embodiment of the present disclosure. As shown in fig. 16, the encoding apparatus 1600 may include:

the patch information obtaining module 1601 is configured to obtain auxiliary information of a cloud patch to be encoded and a first identifier of the patch to be encoded;

the auxiliary information encoding module 1602 is configured to, when the first identifier is a first value, encode the auxiliary information of the patch to be encoded and the first identifier into a code stream, where the patch to be encoded is a first patch in the current point cloud frame; or

The auxiliary information encoding module 1602 is configured to, when the first identifier is a first value, encode auxiliary information of a patch to be encoded into a code stream, where the patch to be encoded is a non-first patch in the current point cloud frame;

the first identifier is a first value to indicate that the to-be-encoded patch is a first type of patch, and the auxiliary information of the to-be-encoded patch includes three-dimensional information.

Wherein the auxiliary information encoding module 1602 is further configured to:

It should be noted that the patch information obtaining module 1601 is configured to execute the relevant content of step S901 in the embodiment shown in fig. 9, and the auxiliary information encoding module 1602 is configured to execute the relevant content of step S902 and step S903 in the embodiment shown in fig. 9. As an example, the patch information acquiring module 1601 may correspond to a combination of the point cloud patch information generating module 101 and the packing module 102 in fig. 2, and the auxiliary information encoding module 1602 may correspond to the auxiliary information encoding module 109 in fig. 2, in other words, the function of the patch information acquiring module 1601 may be implemented by the point cloud patch information generating module 101 and the packing module 102 in fig. 2, and the function of the auxiliary information generating module 1602 may be implemented by the auxiliary information encoding module 109 in fig. 2. In one example, the point cloud block information generating module 101 is configured to obtain three-dimensional information, normal coordinate axis information, a first identifier, and the like of the patch to be encoded, and the packing module 102 is configured to obtain rotation transformation information of the patch to be encoded, identifier information and two-dimensional information of the reference patch, and the like. Although the application is not so limited. For further details, reference may be made to the prior art or the above explanation of the principle of the encoder shown in fig. 2, which is not described here in detail.

Referring to fig. 17, fig. 17 is a schematic structural diagram of a first point cloud decoding apparatus provided in the embodiment of the present application. As shown in fig. 17, the decoding apparatus 1700 may include:

the auxiliary information decoding module 1701 is used for analyzing a first identifier of a to-be-decoded patch from a code stream when the to-be-decoded point cloud patch is the first patch in the current point cloud frame;

the auxiliary information decoding module 1701 is further configured to, when the first identifier is a first value, parse auxiliary information of the patch to be decoded from the code stream, where the auxiliary information includes three-dimensional information of the patch to be decoded;

the auxiliary information decoding module 1701 is further configured to, when the to-be-decoded patch is the nth patch in the current point cloud frame and n is greater than 1 and less than or equal to the number of the first type of patches in the reference point cloud frame of the current point cloud frame, analyze auxiliary information of the to-be-decoded patch from the code stream, where the auxiliary information includes three-dimensional information of the to-be-decoded patch;

a reconstructing module 1702, configured to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.

Wherein the side information decoding module 1701 is further configured to:

Wherein, when the first flag is a first value, the side information decoding module 1701 is further configured to:

the reconstruction module 1702 is further configured to:

Wherein, when the first flag is the second value, the side information decoding module 1701 is further configured to:

the reconstruction module 1702 is further configured to:

It should be noted that the side information decoding module 1701 is configured to execute the related content in steps S1101-S1106 of the embodiment shown in fig. 11, and the reconstruction module 1702 is configured to execute the related content in step S1107 of the embodiment shown in fig. 11. As an example, the side information decoding module 1701 in fig. 17 corresponds to the side information decoding module 204 in fig. 6, and the reconstruction module 1702 in fig. 17 corresponds to the geometric information reconstruction module 206 of the point cloud in fig. 5, in other words, the function of the side information decoding module 1701 may be implemented by the side information decoding module 204 in fig. 6, and the function of the reconstruction module 1702 may be implemented by the geometric information reconstruction module 206 of the point cloud in fig. 6. Although the application is not so limited. For further details, reference may be made to the prior art or the above explanation of the principle of the decoder shown in fig. 6, which is not described here in detail.

Referring to fig. 18, fig. 18 is a schematic structural diagram of a first point cloud encoding device according to an embodiment of the present disclosure. As shown in fig. 18, the encoding apparatus 1800 may include:

a patch information obtaining module 1801, configured to obtain auxiliary information of a cloud patch to be encoded and a first identifier of the patch to be encoded;

the auxiliary information encoding module 1802 is configured to, when the first identifier is a first value, encode the auxiliary information of the patch to be encoded, the first identifier, and a second identifier of the patch to be encoded into the code stream, where the patch to be encoded is a first patch in the current point cloud frame; or

The auxiliary information encoding module 1802 is configured to, when the first identifier is a first value, encode the auxiliary information of the patch to be encoded and a second identifier of the patch to be encoded into a code stream, where the patch to be encoded is a non-first patch in the current point cloud frame;

the first identification is a first value to indicate that the patch to be coded is a first type of patch;

the second mark is a fourth value to indicate that the three-dimensional information of the patch to be coded is not coded into the code stream;

or the second identifier is a fifth value to indicate that the three-dimensional information of the to-be-encoded patch is encoded into the code stream, and the auxiliary information of the to-be-encoded patch comprises the three-dimensional information.

It should be noted that the patch information obtaining module 1801 is configured to execute the relevant content of step S1201 in the embodiment shown in fig. 12, and the auxiliary information encoding module 1802 is configured to execute the relevant content of step S1202 in the embodiment shown in fig. 12. As an example, the patch information acquiring module 1801 may correspond to a combination of the point cloud chunk information generating module 101 and the packing module 102 in fig. 2, and the auxiliary information encoding module 1802 may correspond to the auxiliary information encoding module 109 in fig. 2, in other words, the functions of the patch information acquiring module 1801 may be implemented by the point cloud chunk information generating module 101 and the packing module 102 in fig. 2, and the functions of the auxiliary information generating module 1802 may be implemented by the auxiliary information encoding module 109 in fig. 2. In one example, the point cloud block information generating module 101 is configured to obtain three-dimensional information, normal coordinate axis information, a first identifier, a second identifier, and the like of the patch to be encoded, and the packing module 102 is configured to obtain rotation transformation information of the patch to be encoded, identification information of the reference patch, two-dimensional information, and the like. Although the application is not so limited. For further details, reference may be made to the prior art or the above explanation of the principle of the encoder shown in fig. 2, which is not described here in detail.

Referring to fig. 19, fig. 19 is a schematic structural diagram of a first point cloud decoding apparatus according to an embodiment of the present disclosure. As shown in fig. 19, the decoding apparatus 1900 may include:

the auxiliary information decoding module 1901 is configured to, when the point cloud block patch to be decoded is the first patch in the current point cloud frame, analyze a first identifier of the patch to be decoded from the code stream;

the auxiliary information decoding module 1901 is further configured to, when the first identifier is a first value, parse a second identifier of the patch to be decoded from the code stream;

the auxiliary information decoding module 1901 is further configured to determine, when the second identifier is a fourth value, auxiliary information of a to-be-decoded patch according to auxiliary information of a reference patch of the to-be-decoded patch; when the second identifier is a fifth value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded;

the auxiliary information decoding module 1901 is further configured to, when the patch to be decoded is an nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patches in the reference point cloud frame of the current point cloud frame, analyze a second identifier of the patch to be decoded from the code stream;

The reconstructing module 1902 is configured to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.

It should be noted that the auxiliary information decoding module 1901 is configured to execute the related contents of steps S1301-S1305 in the embodiment shown in fig. 13, and the reconstructing module 1902 is configured to execute the related contents of step S1306 in the embodiment shown in fig. 13. As an example, the auxiliary information decoding module 1901 in fig. 19 corresponds to the auxiliary information decoding module 204 in fig. 6, and the reconstruction module 1902 in fig. 19 corresponds to the geometric information reconstruction module 206 of the point cloud in fig. 5, in other words, the function of the auxiliary information decoding module 1901 may be implemented by the auxiliary information decoding module 204 in fig. 6, and the function of the reconstruction module 1902 may be implemented by the geometric information reconstruction module 206 of the point cloud in fig. 6. Although the application is not so limited. For further details, reference may be made to the prior art or the above explanation of the principle of the decoder shown in fig. 6, which is not described here in detail.

Referring to fig. 20, fig. 20 is a schematic structural diagram of a first point cloud encoding device according to an embodiment of the present disclosure. As shown in fig. 20, the encoding apparatus 2000 may include:

the patch information obtaining module 2001 is configured to obtain auxiliary information of the to-be-encoded cloud patch and a first identifier of the to-be-encoded patch;

The auxiliary information encoding module 2002 is configured to, when the first identifier is a first value, encode the first identifier of the patch to be encoded into the code stream;

the auxiliary information encoding module 2002 is further configured to, when the first identifier is the second value or the third value, encode the auxiliary information of the patch to be encoded and the first identifier into a code stream;

It should be noted that the patch information acquiring module 2001 is configured to execute the relevant content of step S1401 in the embodiment shown in fig. 14, and the auxiliary information encoding module 2002 is configured to execute the relevant content of step S1402 and step S1403 in the embodiment shown in fig. 14. As an example, the patch information acquiring module 2001 may correspond to a combination of the point cloud patch information generating module 101 and the packing module 102 in fig. 2, and the auxiliary information encoding module 2002 may correspond to the auxiliary information encoding module 109 in fig. 2, in other words, the function of the patch information acquiring module 2001 may be implemented by the point cloud patch information generating module 101 and the packing module 102 in fig. 2, and the function of the auxiliary information generating module 2002 may be implemented by the auxiliary information encoding module 109 in fig. 2. In one example, the point cloud block information generating module 101 is configured to obtain three-dimensional information, normal coordinate axis information, a first identifier, and the like of the patch to be encoded, and the packing module 102 is configured to obtain rotation transformation information of the patch to be encoded, identifier information and two-dimensional information of the reference patch, and the like. Although the application is not so limited. For further details, reference may be made to the prior art or the above explanation of the principle of the encoder shown in fig. 2, which is not described here in detail.

Referring to fig. 21, fig. 21 is a schematic structural diagram of a first point cloud decoding apparatus provided in the embodiment of the present application. As shown in fig. 21, the decoding apparatus 2100 may include:

the auxiliary information decoding module 2101 is configured to parse a first identifier of the point cloud block patch to be decoded from the code stream;

the auxiliary information decoding module 2101 is further configured to, when the first identifier is the first value, determine auxiliary information of a to-be-decoded patch according to auxiliary information of a reference patch of the to-be-decoded patch, where the auxiliary information of the to-be-decoded patch includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information;

the auxiliary information decoding module 2101 is further configured to, when the first identifier is the second value, analyze the auxiliary information of the to-be-decoded patch from the code stream, where the auxiliary information of the to-be-decoded patch includes two-dimensional information, three-dimensional information, and identifier information of a reference patch of the to-be-decoded patch;

the auxiliary information decoding module 2101 is further configured to, when the first identifier is a third value, analyze auxiliary information of the to-be-decoded patch from the code stream, where the auxiliary information of the to-be-decoded patch includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information;

a reconstructing module 2102 configured to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.

Wherein, when the first flag is the second value, the auxiliary information decoding module 2101 is further configured to:

the reconstruction module 2102 is also for:

It should be noted that the auxiliary information decoding module 2101 is configured to execute the related content of steps S1501 to S1504 in the embodiment shown in fig. 15, and the reconstruction module 2102 is configured to execute the related content of step S1506 in the embodiment shown in fig. 15. As an example, the auxiliary information decoding module 2101 in fig. 21 corresponds to the auxiliary information decoding module 204 in fig. 6, and the reconstruction module 2102 in fig. 21 corresponds to the geometric information reconstruction module 206 of the point cloud in fig. 5, in other words, the function of the auxiliary information decoding module 2101 may be implemented by the auxiliary information decoding module 204 in fig. 6, and the function of the reconstruction module 2102 may be implemented by the geometric information reconstruction module 206 of the point cloud in fig. 6. Although the application is not so limited. For further details, reference may be made to the prior art or the above explanation of the principle of the decoder shown in fig. 6, which is not described here in detail.

Fig. 22 is a schematic block diagram of one implementation of a decoding apparatus 2200 for an embodiment of the present application. The acquisition device 2200 may include, among other things, a processor 2201, a memory 2202, and a bus system 2203. Wherein, the processor 2201 and the memory 2202 are connected through a bus system 2203, the memory 2202 is used for storing instructions, and the processor 2201 is used for executing the instructions stored in the memory 2202 to execute various point cloud encoding or decoding methods described in the present application, especially a method for filtering a current image block based on a block size of the current image block. To avoid repetition, it is not described in detail here.

In this embodiment, the processor 2201 may be a Central Processing Unit (CPU), and the processor 2201 may also be other general purpose processors, DSPs, ASICs, FPGAs, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 2202 may comprise a ROM device or a RAM device. Any other suitable type of memory device may also be used for memory 2202. Memory 2202 may include code and data 22021 accessed by processor 2201 using bus 2203. The memory 2202 may further include an operating system 22023 and application programs 22022, the application programs 22022 including at least one program that allows the processor 2201 to perform the point cloud encoding or decoding methods described herein, and in particular the point cloud encoding or decoding methods described herein. For example, the application programs 22022 may include applications 1 through N, which further include point cloud encoding or decoding applications (referred to as point cloud decoding applications) that perform the point cloud encoding or decoding methods described herein.

The bus system 2203 may include a power bus, a control bus, a status signal bus, and the like, in addition to the data bus. For clarity of illustration, however, the various buses are designated in the figure as the bus system 2203.

Optionally, the encoding apparatus 2200 may also include one or more output devices, such as a display 2204. In one example, the display 2204 may be a touch-sensitive display that incorporates a display with a touch-sensitive unit operable to sense touch input. A display 2204 may be connected to the processor 2201 via the bus 2203.

It should be noted that the decoding apparatus 2200 may perform the method for encoding the point cloud in the present application, and may also perform the method for decoding the point cloud in the present application.

Those of skill in the art will appreciate that the functions described in connection with the various illustrative logical blocks, modules, and algorithm steps described in the disclosure herein may be implemented as hardware, software, firmware, or any combination thereof. If implemented in software, the functions described in the various illustrative logical blocks, modules, and steps may be stored on or transmitted over as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium, such as a data storage medium, or any communication medium including a medium that facilitates transfer of a computer program from one place to another (e.g., according to a communication protocol). In this manner, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium, such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementing the techniques described herein. The computer program product may include a computer-readable medium.

By way of example, and not limitation, such computer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if the instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. It should be understood, however, that the computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but are instead directed to non-transitory tangible storage media. Disk and disc, as used herein, includes Compact Disc (CD), laser disc, optical disc, DVD and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

The instructions may be executed by one or more processors, such as one or more Digital Signal Processors (DSPs), general purpose microprocessors, Application Specific Integrated Circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Thus, the term "processor," as used herein may refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein. Additionally, in some aspects, the functions described by the various illustrative logical blocks, modules, and steps described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or incorporated in a combined codec. Also, the techniques may be fully implemented in one or more circuits or logic elements. In one example, various illustrative logical blocks, units, and modules within the encoder 100 and the decoder 200 may be understood as corresponding circuit devices or logical elements.

The techniques of this application may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an Integrated Circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units are described in this application to emphasize functional aspects of means for performing the disclosed techniques, but do not necessarily require realization by different hardware units. Indeed, as described above, the various units may be combined in a codec hardware unit, in conjunction with suitable software and/or firmware, or provided by an interoperating hardware unit (including one or more processors as described above).

The above description is only an exemplary embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method of point cloud encoding, the method comprising:

acquiring auxiliary information of a cloud block patch to be coded and a first identifier of the patch to be coded;

when the first identification is a first value, encoding the auxiliary information of the patch to be encoded and the first identification into a code stream, wherein the patch to be encoded is the first patch in the current point cloud frame; or

When the first identification is a first value, encoding auxiliary information of the patch to be encoded into a code stream, wherein the patch to be encoded is a non-first patch in the current point cloud frame;

the first identification is a first value to indicate that the to-be-encoded patch is a first type of patch, and the auxiliary information of the to-be-encoded patch comprises three-dimensional information.

2. The method of claim 1, wherein the current point cloud frame is a kth point cloud frame in a current group, K being an integer greater than or equal to 3.

3. The method according to claim 1 or 2, wherein after the obtaining the side information of the to-be-encoded point cloud patch and the first identifier of the to-be-encoded patch, the method further comprises:

when the first identification is a second value or a third value, encoding the auxiliary information of the patch to be coded and the first identification into a code stream;

4. The method of any of claims 1-3, wherein encoding the first identifier into a codestream when the first identifier is a first value comprises:

encoding the first identifications of a plurality of patches included by the current point cloud frame into a code stream, wherein the first identifications of the plurality of patches comprise the first identifications of the patches to be encoded, and the remaining first identifications of the plurality of patches except the first identifications of the patches to be encoded are second values or third values.

5. The method of claim 3 or 4, wherein the first value, the second value, and the third value are binary 10, 11, and 00, respectively,

alternatively, the first, second and third values are 11, 10 and 00 in binary, respectively.

6. The method of any one of claims 3-5, wherein the first identifier of the patch to be encoded is a third value, the patch to be encoded being a first patch in the current point cloud frame.

7. The method of any of claims 3-6, wherein the reference patch of the patch to be encoded is included in a reference frame of the current point cloud frame.

8. The method of any of claims 3-7, wherein the first type of patch is a global match patch in the current set, the second type of patch is a local match patch in the current set, and the third type of patch is a non-match patch in the current set.

9. A method of point cloud decoding, the method comprising:

when the point cloud block patch to be decoded is the first patch in the current point cloud frame, analyzing a first identifier of the patch to be decoded from a code stream;

when the first identification is a first value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded;

When the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patches in the reference point cloud frame of the current point cloud frame, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded;

and reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

10. The method of claim 9, wherein the current point cloud frame is a kth point cloud frame in a current group, K being an integer greater than or equal to 3.

11. The method of claim 9 or 10, wherein the method further comprises:

When the first identification is a third value, the auxiliary information of the patch to be decoded is analyzed from the code stream, and the auxiliary information comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded.

12. The method according to any one of claims 9 to 11, wherein when the to-be-decoded point cloud patch is the first patch in the current point cloud frame, parsing the first identifier of the to-be-decoded patch from the codestream includes:

when the to-be-decoded patch is the first patch in the current point cloud frame, analyzing first identifications of a plurality of patches included in the current point cloud frame from the code stream, wherein the analyzed first identifications comprise the to-be-decoded patch, and the remaining first identifications except the to-be-decoded patch in the first identifications are second values or third values.

13. The method of claim 11 or 12, wherein the first value, the second value, and the third value are binary 10, 11, and 00, respectively,

14. The method of any one of claims 11-13, wherein when the patch to be decoded is a first patch in the current point cloud frame and the first flag is a third value, the current point cloud frame is the first point cloud frame.

15. The method of any of claims 9-14, wherein when the first identifier is a first value, the method further comprises:

acquiring two-dimensional information, rotation transformation information and normal coordinate axis information of the to-be-decoded patch according to the auxiliary information of the reference patch of the to-be-decoded patch;

reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded, including:

16. The method of any of claims 11-14, wherein when the first identifier is a second value, the method further comprises:

17. The method of any one of claims 11-16, wherein the reference patch of the patch to be decoded is included in a reference frame of the current point cloud frame.

18. A method of point cloud encoding, the method comprising:

when the first identification is a first value, encoding auxiliary information of the patch to be encoded, the first identification and a second identification of the patch to be encoded into a code stream, wherein the patch to be encoded is a first patch in a current point cloud frame; or

When the first identification is a first value, encoding auxiliary information of the patch to be encoded and a second identification of the patch to be encoded into a code stream, wherein the patch to be encoded is a non-first patch in the current point cloud frame;

the second identification is a fourth value to indicate that the three-dimensional information of the patch to be coded is not coded into the code stream;

Or the second identification is a fifth value to indicate that the three-dimensional information of the to-be-encoded patch is coded into the code stream, and the auxiliary information of the to-be-encoded patch comprises the three-dimensional information.

19. A method of point cloud decoding, the method comprising:

when the first identifier is a first value, a second identifier of the patch to be decoded is analyzed from the code stream;

when the second identifier is a fourth value, determining the auxiliary information of the to-be-decoded patch according to the auxiliary information of the reference patch of the to-be-decoded patch; when the second identifier is a fifth value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded;

when the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patches in the reference point cloud frame of the current point cloud frame, analyzing a second identifier of the patch to be decoded from the code stream;

20. A method of point cloud encoding, the method comprising:

when the first identification is a first value, the first identification of the patch to be coded is coded into a code stream;

21. A method of point cloud decoding, the method comprising:

Analyzing a first identifier of a point cloud block patch to be decoded from the code stream;

when the first identification is a first value, determining auxiliary information of the to-be-decoded patch according to the auxiliary information of the reference patch of the to-be-decoded patch, wherein the auxiliary information of the to-be-decoded patch comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information;

when the first identification is a second value, analyzing the auxiliary information of the to-be-decoded patch from the code stream, wherein the auxiliary information of the to-be-decoded patch comprises two-dimensional information, three-dimensional information and identification information of a reference patch of the to-be-decoded patch;

when the first identification is a third value, analyzing the auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information of the patch to be decoded comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information;

22. The method of claim 21, wherein when the first identifier is a second value, the method further comprises:

23. A point cloud encoding apparatus, characterized in that the apparatus comprises:

the cloud patch information acquisition module is used for acquiring auxiliary information of a to-be-encoded patch and a first identifier of the to-be-encoded patch;

the auxiliary information encoding module is used for encoding the auxiliary information of the to-be-encoded patch and the first identifier into a code stream when the first identifier is a first value, wherein the to-be-encoded patch is the first patch in the current point cloud frame; or

The auxiliary information encoding module is used for encoding the auxiliary information of the patch to be encoded into a code stream when the first identifier is a first value, wherein the patch to be encoded is a non-first patch in the current point cloud frame;

24. The apparatus of claim 23, wherein the current point cloud frame is a kth point cloud frame in a current group, K being an integer greater than or equal to 3.

25. The apparatus of claim 23 or 24, wherein the side information encoding module is further configured to:

26. The apparatus of any of claims 23-25, wherein the side information encoding module is further to:

27. The apparatus of claim 25 or 26, wherein the first value, the second value, and the third value are binary 10, 11, and 00, respectively,

28. The apparatus of any one of claims 25-27, wherein the first identifier of the patch to be encoded is a third value, the patch to be encoded being a first patch in the current point cloud frame.

29. The apparatus of any one of claims 25-28, wherein the reference patch of the patch to be encoded is included in a reference frame of the current point cloud frame.

30. The apparatus of any of claims 25-29, wherein the first type of patch is a global match patch in the current set, the second type of patch is a local match patch in the current set, and the third type of patch is a non-match patch in the current set.

31. A point cloud decoding apparatus, characterized in that the apparatus comprises:

the auxiliary information decoding module is used for analyzing a first identifier of the to-be-decoded patch from the code stream when the to-be-decoded point cloud patch is the first patch in the current point cloud frame;

The auxiliary information decoding module is further configured to, when the first identifier is a first value, analyze the auxiliary information of the to-be-decoded patch from the code stream, where the auxiliary information includes three-dimensional information of the to-be-decoded patch;

the auxiliary information decoding module is further configured to, when the to-be-decoded patch is an nth patch in the current point cloud frame, where n is greater than 1 and less than or equal to the number of first-type patches in a reference point cloud frame of the current point cloud frame, analyze auxiliary information of the to-be-decoded patch from the code stream, where the auxiliary information includes three-dimensional information of the to-be-decoded patch;

and the reconstruction module is used for reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

32. The apparatus of claim 31, wherein the current point cloud frame is a kth point cloud frame in a current group, K being an integer greater than or equal to 3.

33. The apparatus of claim 31 or 32, wherein the side information decoding module is further configured to:

34. The apparatus of any of claims 31-33, wherein the side information decoding module is further configured to:

35. The apparatus of claim 33 or 34, wherein the first value, the second value, and the third value are binary 10, 11, and 00, respectively,

36. The apparatus of any one of claims 33-35, wherein when the patch to be decoded is a first patch in the current point cloud frame and the first flag is a third value, the current point cloud frame is the first point cloud frame.

37. The apparatus of any of claims 31-36, wherein when the first identifier is a first value, the side information decoding module is further to:

the reconstruction module is further configured to:

38. The apparatus of any of claims 33-36, wherein when the first identifier is a second value, the side information decoding module is further to:

The reconstruction module is further configured to:

39. The apparatus of any one of claims 33-38, wherein the reference patch of the patch to be decoded is included in a reference frame of the current point cloud frame.

40. A point cloud encoding apparatus, characterized in that the apparatus comprises:

the point cloud patch information acquisition module is used for acquiring auxiliary information of a to-be-encoded point cloud patch and a first identifier of the to-be-encoded patch;

the auxiliary information encoding module is used for encoding the auxiliary information of the patch to be encoded, the first identifier and the second identifier of the patch to be encoded into a code stream when the first identifier is a first value, wherein the patch to be encoded is the first patch in the current point cloud frame; or

The auxiliary information encoding module is used for encoding the auxiliary information of the patch to be encoded and the second identifier of the patch to be encoded into a code stream when the first identifier is a first value, wherein the patch to be encoded is a non-first patch in the current point cloud frame;

41. A point cloud decoding apparatus, characterized in that the apparatus comprises:

the auxiliary information decoding module is further configured to, when the first identifier is a first value, parse a second identifier of the patch to be decoded from the code stream;

the auxiliary information decoding module is further configured to determine, when the second identifier is a fourth value, auxiliary information of the to-be-decoded patch according to the auxiliary information of the reference patch of the to-be-decoded patch; when the second identifier is a fifth value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded;

the auxiliary information decoding module is further configured to, when the patch to be decoded is an nth patch in the current point cloud frame, where n is greater than 1 and less than or equal to the number of first-class patches in a reference point cloud frame of the current point cloud frame, analyze a second identifier of the patch to be decoded from the code stream;

42. A point cloud encoding apparatus, characterized in that the apparatus comprises:

the auxiliary information coding module is used for coding the first identifier of the patch to be coded into a code stream when the first identifier is a first value;

the auxiliary information encoding module is further configured to encode the auxiliary information of the patch to be encoded and the first identifier into a code stream when the first identifier is a second value or a third value;

43. A point cloud decoding apparatus, characterized in that the apparatus comprises:

the auxiliary information decoding module is used for analyzing a first identifier of the point cloud block patch to be decoded from the code stream;

the auxiliary information decoding module is further configured to determine auxiliary information of the to-be-decoded patch according to the auxiliary information of the reference patch of the to-be-decoded patch when the first identifier is a first value, where the auxiliary information of the to-be-decoded patch includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information;

the auxiliary information decoding module is further configured to, when the first identifier is a second value, analyze the auxiliary information of the to-be-decoded patch from the code stream, where the auxiliary information of the to-be-decoded patch includes two-dimensional information, three-dimensional information, and identifier information of a reference patch of the to-be-decoded patch;

the auxiliary information decoding module is further configured to, when the first identifier is a third value, analyze the auxiliary information of the to-be-decoded patch from the code stream, where the auxiliary information of the to-be-decoded patch includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information;

44. The apparatus of claim 43, wherein when the first identifier is a second value, the side information decoding module is further to:

the reconstruction module is further configured to:

45. A computer-readable storage medium, characterized by comprising program code which, when run on a computer, causes the computer to perform the point cloud encoding method of any of claims 1 to 8, or claim 18, or claim 20.

46. A computer-readable storage medium, characterized by comprising program code which, when run on a computer, causes the computer to perform the point cloud decoding method of any of claims 9 to 17, or claim 19, or any of claims 21 to 22.