CN116800969A

CN116800969A - Encoding and decoding methods, devices and equipment

Info

Publication number: CN116800969A
Application number: CN202210272443.0A
Authority: CN
Inventors: 吕卓逸; 邹文杰; 张伟; 杨付正
Original assignee: Vivo Mobile Communication Co Ltd
Current assignee: Vivo Mobile Communication Co Ltd
Priority date: 2022-03-18
Filing date: 2022-03-18
Publication date: 2023-09-22
Also published as: WO2023174337A1

Abstract

The application discloses a coding and decoding method, a device and equipment, and relates to the technical field of coding and decoding. The encoding method comprises the following steps: the encoding end quantizes the geometric information of the target three-dimensional grid to obtain first precision geometric information and second precision geometric information; the encoding end obtains a first geometric figure and a occupation map of the first precision geometric information; the encoding end obtains a second geometric figure containing second precision geometric information and first precision geometric information according to the second precision geometric information and the first geometric figure; the encoding end encodes the second geometric figure and the occupation map; the first precision geometric information is the geometric information of the target three-dimensional grid after quantization, and the second precision geometric information is the geometric information lost in the target three-dimensional grid quantization process.

Description

Encoding and decoding methods, devices and equipment

Technical Field

The application belongs to the technical field of encoding and decoding, and particularly relates to an encoding and decoding method, an encoding and decoding device and encoding and decoding equipment.

Background

Three-dimensional Mesh (Mesh) can be considered as the most popular representation of three-dimensional models over the past years, which plays an important role in many applications. The method is simple in representation, and is integrated into Graphic Processing Units (GPU) of computers, tablet computers and smart phones in a large number of hardware algorithms, and is specially used for rendering three-dimensional grids.

Because the vertexes and the point clouds of the Mesh are a group of irregularly distributed discrete point sets in the space, the Mesh has the similar characteristics. Thus, the three-dimensional mesh geometry information may be compressed using a point cloud compression algorithm. However, compared with the point cloud, the vertexes of the three-dimensional grid have the characteristics of sparse spatial distribution and uneven spatial distribution. The geometrical information of the three-dimensional grid model is compressed by using a point cloud compression algorithm, and the compression efficiency is not high.

Disclosure of Invention

The embodiment of the application provides a coding and decoding method, a device and equipment, which can solve the problem of low compression efficiency in the compression mode of three-dimensional grid geometric information in the prior art.

In a first aspect, there is provided an encoding method comprising:

the encoding end quantizes the geometric information of the target three-dimensional grid to obtain first precision geometric information and second precision geometric information;

the encoding end obtains a first geometric figure and a occupation map of the first precision geometric information;

the encoding end obtains a second geometric figure containing second precision geometric information and first precision geometric information according to the second precision geometric information and the first geometric figure;

encoding the second geometric figure and the occupancy map;

The first precision geometric information is the geometric information of the target three-dimensional grid after quantization, and the second precision geometric information is the geometric information lost in the target three-dimensional grid quantization process.

In a second aspect, there is provided an encoding apparatus comprising:

the quantization module is used for quantizing the geometric information of the target three-dimensional grid to obtain first precision geometric information and second precision geometric information;

the first acquisition module is used for acquiring a first geometric figure and a occupation map of the first precision geometric information;

the second acquisition module is used for acquiring a second geometric figure containing second precision geometric information and first precision geometric information according to the second precision geometric information and the first geometric figure;

an encoding module for encoding the second geometric figure and the occupancy map;

In a third aspect, a decoding method is provided, comprising:

the decoding end decomposes the obtained code stream of the target three-dimensional grid to obtain a occupation map and a second geometric map containing second precision geometric information and first precision geometric information;

The decoding end acquires second precision geometric information and a first geometric figure according to the second geometric figure;

the decoding end acquires first precision geometric information according to the first geometric figure and the occupation map;

the decoding end performs inverse quantization according to the second precision geometric information and the first precision geometric information to obtain the geometric information of the target three-dimensional grid;

In a fourth aspect, there is provided a decoding apparatus including:

the third acquisition module is used for decomposing the acquired code stream of the target three-dimensional grid to acquire a occupation map and a second geometric map containing second precision geometric information and first precision geometric information;

the fourth acquisition module is used for acquiring second precision geometric information and the first geometric figure according to the second geometric figure;

a fifth obtaining module, configured to obtain first precision geometric information according to the first geometric diagram and the occupancy map;

the sixth acquisition module is used for performing inverse quantization according to the second precision geometric information and the first precision geometric information to acquire the geometric information of the target three-dimensional grid;

In a fifth aspect, there is provided an encoding apparatus comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, performs the steps of the method according to the first aspect.

In a sixth aspect, an encoding device is provided, including a processor and a communication interface, where the processor is configured to quantize geometric information of a target three-dimensional grid to obtain first precision geometric information and second precision geometric information; acquiring a first geometric figure and a occupation map of the first precision geometric information; acquiring a second geometric figure containing second precision geometric information and first precision geometric information according to the second precision geometric information and the first geometric figure; encoding the second geometric figure and the occupancy map; the first precision geometric information is the geometric information of the target three-dimensional grid after quantization, and the second precision geometric information is the geometric information lost in the target three-dimensional grid quantization process.

In a seventh aspect, there is provided a decoding device comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, implement the steps of the method according to the third aspect.

An eighth aspect provides a decoding device, including a processor and a communication interface, where the processor is configured to decompose a code stream of an obtained target three-dimensional grid, and obtain a occupation map and a second geometric map including second precision geometric information and first precision geometric information; acquiring second precision geometric information and a first geometric figure according to the second geometric figure; acquiring first precision geometric information according to the first geometric figure and the occupancy map; performing inverse quantization according to the second precision geometric information and the first precision geometric information to obtain the geometric information of the target three-dimensional grid;

In a ninth aspect, there is provided a communication system comprising: an encoding device operable to perform the steps of the method as described in the first aspect and a decoding device operable to perform the steps of the method as described in the third aspect.

In a tenth aspect, there is provided a readable storage medium having stored thereon a program or instructions which when executed by a processor, performs the steps of the method according to the first aspect, or performs the steps of the method according to the third aspect.

In an eleventh aspect, there is provided a chip comprising a processor and a communication interface, the communication interface and the processor being coupled, the processor being for running a program or instructions to implement the method according to the first aspect or to implement the method according to the third aspect.

In a twelfth aspect, there is provided a computer program/program product stored in a storage medium, the computer program/program product being executed by at least one processor to implement the steps of the method as described in the first aspect.

In the embodiment of the application, the second geometric figure containing the second precision geometric information and the first precision geometric information is obtained according to the quantized first geometric figure of the second precision geometric information and the first precision geometric information, so that the second precision geometric information can be encoded by utilizing the geometric figure of the first precision geometric information, the encoding efficiency of the second precision geometric information is improved, and the efficiency of compressing the geometric information by using a quantization scheme is improved.

Drawings

FIG. 1 is a flow chart of an encoding method according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a raw patch;

FIG. 3 is a schematic diagram of a grid-based fine partitioning process;

FIG. 4 is a schematic diagram of eight directions of the Patch arrangement;

FIG. 5 is a schematic diagram of a geometric figure after arranging the arrangement values corresponding to the high-precision geometric information according to the positions of the low-precision geometric information;

FIG. 6 is a schematic diagram of a geometric diagram after arranging high-precision arrangement values by shifting the distribution of closely arranged pixels to the left in the horizontal direction;

FIG. 7 is a schematic diagram of a video-based three-dimensional mesh geometry information encoding framework;

FIG. 8 is a block diagram of an encoding apparatus according to an embodiment of the present application;

fig. 9 is a schematic structural view of an encoding apparatus of an embodiment of the present application;

FIG. 10 is a flow chart of a decoding method according to an embodiment of the present application;

FIG. 11 is a geometric information reconstruction block diagram;

FIG. 12 is a schematic diagram of a video-based three-dimensional mesh geometry information decoding framework;

FIG. 13 is a block diagram of a decoding apparatus according to an embodiment of the present application;

fig. 14 is a schematic structural diagram of a communication device according to an embodiment of the present application.

Detailed Description

The technical solutions of the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which are derived by a person skilled in the art based on the embodiments of the application, fall within the scope of protection of the application.

The terms first, second and the like in the description and in the claims, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application are capable of operation in sequences other than those illustrated or otherwise described herein, and that the "first" and "second" distinguishing between objects generally are not limited in number to the extent that the first object may, for example, be one or more. Furthermore, in the description and claims, "and/or" means at least one of the connected objects, and the character "/" generally means a relationship in which the associated object is an "or" before and after.

It should be noted that the techniques described in the embodiments of the present application are not limited to long term evolution (Long Term Evolution, LTE)/LTE evolution (LTE-Advanced, LTE-a) systems, but may also be used in other wireless communication systems, such as code division multiple access (Code Division Multiple Access, CDMA), time division multiple access (Time Division Multiple Access, TDMA), frequency division multiple access (Frequency Division Multiple Access, FDMA), orthogonal frequency division multiple access (Orthogonal Frequency Division Multiple Access, OFDMA), single carrier frequency division multiple access (Single-carrier Frequency Division Multiple Access, SC-FDMA), and other systems. The terms "system" and "network" in embodiments of the application are often used interchangeably, and the techniques described may be used for both the above-mentioned systems and radio technologies, as well as other systems and radio technologies. The following description describes a New air interface (NR) system for purposes of example and uses NR terminology in much of the description that follows, but these techniques are also applicable to applications other than NR system applications, such as generation 6 (6) ^th Generation, 6G) communication system.

The prior art related to the present application is briefly described below.

In recent years, with rapid development of multimedia technology, related research results are rapidly industrialized, and become an essential component in people's life. The three-dimensional model becomes a new generation of digital media following audio, images, video. Three-dimensional grids and point clouds are two commonly used three-dimensional model representations. Compared with traditional multimedia such as images and videos, the three-dimensional grid model has the characteristics of stronger interactivity and reality, so that the three-dimensional grid model is more and more widely applied to various fields such as business, manufacturing industry, construction industry, education, medicine, entertainment, art, military and the like.

With the increasing demands of people on the visual effect of the three-dimensional grid model and the emerging of many more mature three-dimensional scanning technologies and three-dimensional modeling software, the data size and complexity of the three-dimensional grid model obtained by the three-dimensional scanning device or the three-dimensional modeling software are also dramatically increased. Therefore, how to efficiently compress three-dimensional mesh data is a key to achieve convenient transmission, storage, and processing of three-dimensional mesh data.

A three-dimensional grid often contains three main information, namely topology information, geometric information and attribute information. The topology information is used for describing the connection relation between elements such as vertexes and patches in the grid; the geometric information is the three-dimensional coordinates of all vertices in the mesh; the attribute information records other information attached to the grid, such as normal vectors, texture coordinates, colors, etc. Although some conventional general data compression methods can reduce the amount of three-dimensional grid data to some extent, due to the specificity of three-dimensional grid data, the direct use of these compression methods for compressing three-dimensional grid data often cannot achieve the desired effect. Therefore, compression of three-dimensional mesh data faces new challenges. In the data of the three-dimensional grid, the geometric data often occupy more storage space than the topological data, and efficient compression of the geometric data has great significance in reducing the storage space of the three-dimensional grid data. Therefore, compression of three-dimensional mesh geometric information is an important research point.

The three-dimensional mesh geometric information compression algorithm may use a three-dimensional geometric information compression algorithm of a point cloud. In recent years, there are mainly two international standards of point cloud compression, namely V-PCC (Video-based Point Cloud Compression ) and G-PCC (Geometry-based Point Cloud Compression, geometry-based point cloud compression).

The main idea of V-PCC is to project the geometric and attribute information of the point cloud into a two-dimensional video, and compress the two-dimensional video by using the existing video coding technology, thereby achieving the purpose of compressing the point cloud. The geometric coding of the V-PCC is realized by projecting geometric information into a space occupying video and a geometric video and respectively coding the two paths of videos by using a video coder.

The V-PCC geometric information coding process mainly comprises the following steps: first, a three-dimensional patch (3D patch), which is a set of vertices in the pointing cloud that are identical and connected to the projection plane, is generated. The current method for generating the 3D patch is to estimate the normal vector of each vertex by using neighboring points, calculate the projection plane of each vertex according to the normal vector of each point and the normal vector of a preset plane, and form a patch by connecting the vertices with the same projection plane. Then, the 3D patch is projected onto a 2D plane to form a 2D patch, and the 2D patch is arranged on a two-dimensional image, a process called patch packaging (patch packaging). In order to make the patch arrangement more compact and thereby improve compression performance, current arrangement methods have: prioritization, time-domain consistent ranking, global patch allocation, and the like. Then, a occupancy map and a geometry map are generated. The occupancy map is an image representing the occupancy information of vertices in a two-dimensional image, and has a position value of 1 for the projection of the vertices and a remaining position value of 0. The patches are arranged in the two-dimensional image according to a certain rule, and a occupancy map is generated. Stored in the geometry is the distance of each vertex from the projection plane. The depth information of each vertex can be directly calculated by using the three-dimensional coordinates of the vertex, the projection plane of the vertex and the occupation map, so that a geometric figure is generated. For vertices with repeated projection positions, vertex geometric coordinates except for the first projection vertex are arranged in a row patch to be put into a geometric figure or are independently coded. In order to improve compression efficiency, an image filling process is performed on the geometric image. The image filling method comprises a push-pull background filling algorithm, a filling method based on a sparse linear model (Sparse Linear Model), a harmonic background filling (Harmonic Background Filling) and the like. After the image is filled, a final geometric figure is obtained, and the existing video encoder is used for compressing the occupancy map and the geometric figure, so that a video code stream is obtained. And finally, synthesizing the occupied video code stream, the geometric video code stream and the subcode stream containing the patch information into a final total code stream.

The encoding and decoding methods, devices and equipment provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings by some embodiments and application scenarios thereof.

As shown in fig. 1, an embodiment of the present application provides an encoding method, including:

step 101, a coding end quantizes geometric information of a target three-dimensional grid to obtain first precision geometric information and second precision geometric information;

it should be noted that, in the present application, the three-dimensional mesh of the target may be understood as a three-dimensional mesh corresponding to any video frame, and the geometric information of the three-dimensional mesh of the target may be understood as coordinates of vertices in the three-dimensional mesh, where the coordinates generally refer to three-dimensional coordinates.

It should be noted that, the first precision geometric information may be understood as low precision geometric information, that is, low precision geometric information refers to geometric information of the quantized target three-dimensional grid, that is, three-dimensional coordinate information of each vertex included in the quantized target three-dimensional grid. The second precision geometrical information may be understood as high precision geometrical information, which may be regarded as missing geometrical information, i.e. missing three-dimensional coordinate information, during the quantization.

By quantizing the geometric information of the target three-dimensional grid, the pitch of the vertexes of the quantized three-dimensional grid is reduced, and then the pitch of the two-dimensional vertexes after projection is reduced, so that the compression efficiency is improved.

102, the encoding end obtains a first geometric figure and a occupation map of the first precision geometric information;

it should be noted here that the first geometry can be understood as a low-precision geometry.

Step 103, the encoding end obtains a second geometric figure containing second precision geometric information and first precision geometric information according to the second precision geometric information and the first geometric figure;

step 104, the encoding end encodes the second geometric figure and the occupancy map;

after the geometric information of the target three-dimensional grid is quantized to obtain first precision geometric information and second precision geometric information, a geometric figure containing the second precision geometric information and the first precision geometric information is obtained, and then coding is carried out to obtain corresponding subcode streams; according to the scheme, the geometric information of the three-dimensional grid is quantized, so that the distance between the vertexes of the quantized three-dimensional grid is reduced, the distance between the two-dimensional vertexes after projection is shortened, and the compression efficiency of the geometric information of the three-dimensional grid can be improved. In addition, the second precision geometric information is contained in the second geometric figure for encoding, so that the information quantity is low, and the compression efficiency of the geometric information of the three-dimensional grid is high.

Optionally, the specific implementation manner of step 101 is:

the encoding end quantizes each vertex in the target three-dimensional grid according to the quantization parameter of each component to obtain first precision geometric information;

the encoding end obtains second precision geometric information according to the first precision geometric information and the quantization parameter of each component.

It should be noted that, the quantization parameter of each component can be flexibly set according to the use requirement; the quantization parameters mainly include quantization parameters in three components of X-direction, Y-direction, and Z-direction.

In general, for quantization with low precision requirement, only low-precision geometric information can be reserved after quantization; for quantization with higher precision requirement, not only low-precision geometric information but also high-precision geometric information needs to be recorded during quantization, so that accurate grid recovery can be realized during decoding.

For example, assuming that the three-dimensional coordinates of a vertex are (x, y, z), the quantization parameter is (QP _x ,QP _y ,QP _z ) Low-precision geometric information (x _l ,y _l ,z _l ) And high precision geometric information (x _h ,y _h ,z _h ) Calculation of (2)The process is shown in the formulas one to six:

equation one: x is x _l ＝f ₁ (x,QP _x )；

Formula II: y is _l ＝f ₁ (y,QP _y )；

And (3) a formula III: z _l ＝f ₁ (z,QP _z )；

Equation four: x is x _h ＝f ₂ (x,x _l ,QP _x )；

Formula five: y is _h ＝f ₂ (y,y _l ,QP _y )；

Formula six: z _h ＝f ₂ (z,z _l ,QP _z )；

Wherein f in formulas one to three ₁ The function is a quantization function, the input of the quantization function is the coordinate of a certain dimension and the quantization parameter of the dimension, and the output is the quantized coordinate value; f in formulas four to six ₂ The function input is the original coordinate value, the quantized coordinate value and the quantized parameter of the dimension, and the function input is the high-precision coordinate value.

f ₁ The function can be calculated in a plurality of ways, one of which is more general, as shown in formulas seven through nine, and is calculated by dividing the original coordinates of each dimension by the quantization parameter of that dimension. Where/is a division operator, the result of the division operation may be rounded in different ways, such as rounding, rounding down, rounding up, etc. f (f) ₂ There are also various ways of calculating the function, and the implementation corresponding to the formulas seven to nine is shown as the formulas ten to twelve, wherein, the x is the multiplication operator.

Formula seven: x is x _l ＝x/QP _x ；

Formula eight: y is _l ＝y/QP _y ；

Formula nine: z _l ＝z/QP _z ；

Formula ten: x is x _h ＝x-x _l *QP _x ；

Formula eleven: y is _h ＝y-y _l *QP _y ；

Formula twelve: z _h ＝z-z _l *QP _z ；

When the quantization parameter is the integer power of 2, f ₁ Function sum f ₂ The function may be implemented using bit operations, such as equation thirteen through equation eighteen:

Formula thirteen: x is x _l ＝x＞＞log ₂ QP _x ；

Formula fourteen: y is _l ＝y＞＞log ₂ QP _y ；

Formula fifteen: z _l ＝z＞＞log ₂ QP _z ；

Formula sixteen: x is x _h ＝x&(QP _x -1)；

Seventeenth formula: y is _h ＝y&(QP _y -1)；

Equation eighteen: z _h ＝z&(QP _z -1)；

Notably, no matter f ₁ Function sum f ₂ Which calculation mode is adopted by the function, and quantization parameter QP _x 、QP _y And QP (QP) _z Can be flexibly set. First, the quantization parameters of different components are not necessarily equal, and the QP can be established by utilizing the correlation of the quantization parameters of different components _x 、QP _y And QP (QP) _z The relation between the two sets of quantization parameters for different components; and secondly, the quantization parameters of different space regions are not necessarily equal, and the quantization parameters can be adaptively set according to the sparseness degree of the vertex distribution of the local region.

The high-precision geometric information includes detailed information of the outline of the three-dimensional mesh. To further improve compression efficiency, high precision geometric information (x _h ,y _h ,z _h ) And (5) further processing. In a three-dimensional mesh model, the degree of importance of high-precision geometric information of vertices in different regions is different. For areas with sparse vertex distribution, the distortion of high-precision geometric information does not have great influence on the visual effect of the three-dimensional grid. In order to improve the compression efficiency, the high-precision geometric information can be further quantized, or only the high part of points can be reserved Precision geometry information.

Optionally, in the case of quantifying the geometric information of the target three-dimensional grid to obtain the first precision geometric information and the second precision geometric information, the method further includes:

based on quantifying the geometric information of the target three-dimensional grid, obtaining the information of the supplementary points;

the information of the supplemental points refers to information of points requiring additional processing generated in the quantization process, that is, the supplemental points are points requiring additional processing generated in the quantization process, for example, overlapping repeat points of the coordinate positions, and the like, and by processing the repeat points, vertices overlapping the coordinate positions in the quantization can be restored to the original positions after the inverse quantization.

Optionally, the information of the supplemental point includes at least one of:

a11, the index of the vertex in the first precision geometric information corresponding to the supplementary point;

it should be noted that, by identifying the index, it is possible to know which points in the quantized grid are identified by a plurality of points in the three-dimensional grid before quantization, that is, the plurality of points in the three-dimensional grid before quantization are overlapped together after quantization, and the low-precision geometric information of the complementary points can be determined by the index of the vertex.

A12, supplementing third-precision geometric information of the points;

it should be noted that the third precision geometric information may be understood as low precision geometric information of the supplemental point, that is, three-dimensional coordinate information after the supplemental point is quantized.

A13, supplementing fourth-precision geometric information of the points;

it should be noted that the fourth precise geometric information may be understood as high-precise geometric information of the supplemental point, that is, three-dimensional coordinate information of the supplemental point that is lost in the quantization process.

It should be noted that, in specific use, it is possible to determine which of the hidden points after quantization is obtained through a11 and a13 or through a12 and a 13.

Specifically, the specific acquisition mode of the information of the supplementary points is as follows: and the encoding end determines the information of the supplementary points according to the geometric information of the target three-dimensional grid and the first precision geometric information.

That is, after low-precision geometric information of all vertices is obtained, points at which the low-precision geometric information is repeated are used as supplemental points, and encoding is performed individually. The geometric information of the supplementary points can be divided into two parts, namely low-precision geometric information and high-precision geometric information, and all supplementary points or only part of supplementary points can be selected to be reserved according to the requirement of application on compression distortion. Further quantization may also be performed on the high-precision geometric information of the supplemental points, or only high-precision geometric information of a portion of the points may be retained.

Optionally, the geometry of the supplemental points is obtained by:

the encoding end arranges the third precision geometric information of the supplementary points into a first original sheet;

the coding end arranges the fourth-precision geometric information of the supplementary points into a second original sheet according to the arrangement sequence identical to the first original sheet;

the encoding end compresses the first original sheet and the second original sheet to obtain a geometric figure of the supplementary point.

In the embodiment of the present application, the low-precision part and the high-precision part, which are divided into the geometric information of the supplemental points, are respectively encoded. Firstly, arranging low-precision geometric information of supplementary points into a low-precision raw patch of the supplementary points according to any sequence; the first step is to acquire the vertex arrangement sequence, scan the low-precision geometric diagram row by row from left to right, and take the scanning sequence of each vertex as the arrangement sequence of the vertices in the row patch. Second, a raw patch is generated. The raw patch is a rectangular patch formed by arranging three-dimensional coordinates of vertices row by row in a manner as shown in fig. 2. Sequentially arranging the low-precision geometric information of the vertexes according to the vertex arrangement sequence obtained in the first step to obtain low-precision geometric information raw patch; then, arranging the high-precision geometric information into the high-precision raw patch of the supplementary point according to the same sequence as the low-precision raw patch of the supplementary point; finally, the low-precision raw patch and the high-precision raw patch of the supplementary points are compressed, and various compression methods can be adopted. One method is to encode the value in the raw patch in a running code mode, an entropy code mode and the like, and the other method is to add the low-precision raw patch of the supplementary point and the high-precision raw patch of the supplementary point into a blank area in a low-precision geometric figure to obtain the geometric figure of the supplementary point; and finally, utilizing a video encoder to encode the geometric figure to obtain the geometric figure sub-code stream of the supplementary point.

Optionally, the specific implementation procedure of step 102 includes:

step 1021, the encoding end performs three-dimensional slice division on the first precision geometric information;

in this case, the low-precision geometric information is divided into a plurality of three-dimensional slices (Patch); the specific implementation mode of the step is as follows: the coding end determines a projection plane of each vertex contained in the first precision geometric information; the encoding end performs slice division on the vertexes contained in the first precision geometric information according to the projection plane; and the encoding end clusters the vertexes contained in the first precision geometric information to obtain each divided piece. That is, the process for Patch partitioning mainly includes: firstly, estimating the normal vector of each vertex, and selecting a candidate projection plane with the minimum included angle between the plane normal vector and the vertex normal vector as the projection plane of the vertex; then, carrying out initial division on vertexes according to the projection planes, and forming patch by the vertexes which are identical and communicated with the projection planes; and finally, optimizing the clustering result by using a fine division algorithm to obtain a final three-dimensional patch (3D patch).

A detailed description of a specific implementation of the process of obtaining the three-dimensional slice from the first precision geometric information follows.

The normal vector for each point is first estimated. The tangential plane and its corresponding normal are defined at a predefined search distance from the nearest neighbor vertex m of each point. K-D tree is used to separate data and is at point p _i Find adjacent points nearby, the center of gravity of the setFor defining a normal. The center of gravity c is calculated as follows:

nineteenth formula:

the vertex normal vector is estimated by using a characteristic decomposition method, and a calculation process formula twenty is shown as follows:

formula twenty:

in the initial dividing stage, a projection plane of each vertex is initially selected. Let the estimated value of the vertex normal vector beThe normal vector of the candidate projection plane is +.>The plane closest to the normal vector direction of the vertex is selected as the projection plane of the vertex, and the calculation process of plane selection is shown in twenty-one formula:

formula twenty-one:

the fine division process may employ a grid-based algorithm to reduce the time complexity of the algorithm, and the grid-based fine division algorithm flow is shown in fig. 3, and specifically includes:

firstly, setting the circulation number (numlter) to 0, judging whether the circulation number is smaller than the maximum circulation number (the maximum circulation number can be set according to the use requirement), and if so, executing the following processes:

In step S301, the (x, y, z) geometric coordinate space is divided into voxels.

The geometric coordinate space herein refers to a geometric coordinate space formed by the quantized first precision geometric information. For a 10 bit Mesh with a voxel size of 8, for example, the number of voxels per coordinate would be 1024/8=128, the total number of voxels in this coordinate space will be 128 x 128.

In step S302, a filling voxel is searched, where the filling voxel is a voxel containing at least one point in the grid.

In step S303, a smoothing score of each filling voxel on each projection plane is calculated, denoted as voxscore smoothing, and the voxel smoothing score of a voxel on a projection plane is the number of points that are aggregated on the projection plane by the initial segmentation process.

Step S304, using KD-Tree partitioning to find neighboring filler voxels, denoted as nnfiledwoxels, i.e. the nearest filler voxel of each filler voxel (within the search radius and/or limited to the maximum number of neighboring voxels).

Step S305, calculating a voxel smoothing score (score smoothing) of each filling voxel by using the voxel smoothing score of the neighboring filling voxel in each projection plane, wherein the calculation process is as shown in formula twenty two:

Formula twenty-two:

where p is the index of the projection plane and v is the index of the neighbor filling voxels. Score smooth is the same for all points in a voxel.

Step S306, using the normal vector of the vertex and the normal vector algorithm vector score of the candidate projection plane, which is denoted as score normal, the calculation process is shown in the formula twenty-third:

twenty-third formula: score normal [ i ] [ p ] =normal [ i ] & orientation [ p ];

where p is the index of the projection plane and i is the index of the vertex.

Step S307, calculating the final score of each voxel on each projection plane by using the score smooth and score normal, wherein the calculation process is as shown in a twenty-fourth formula:

twenty-four of the formulas:

where i is the vertex index, p is the index of the projection plane, and v is the voxel index where vertex i is located.

Step S308, clustering the vertexes by using the scores in step 307 to obtain finely divided patches.

The above process is iterated multiple times until a more accurate patch is obtained.

Step 1022, the coding end performs two-dimensional projection on the divided three-dimensional slice to obtain a two-dimensional slice;

it should be noted that this process is to project the 3D patch onto a two-dimensional plane to obtain a two-dimensional patch (2D patch).

Step 1023, the encoding end packs the two-dimensional slice to obtain two-dimensional image information;

It should be noted that, this step is implemented by slice packing (Patch packing), where the purpose of Patch packing is to arrange 2D patches on a two-dimensional image, and the basic principle of Patch packing is to arrange patches on a two-dimensional image in a non-overlapping manner or to arrange non-pixel portions of patches on a two-dimensional image in a partially overlapping manner, so that the Patch arrangement is more compact and has time domain consistency, and coding performance is improved by algorithms such as priority arrangement, time domain consistency arrangement, and the like.

Assuming that the resolution of the two-dimensional image is WxH, the minimum block size of the patch arrangement is defined as T, which specifies the minimum distance between different patches placed on this 2D grid.

First, patch is placed on the 2D mesh in an insert-placement manner on a non-overlapping basis. Each patch occupies an area consisting of an integer number of T x T blocks. In addition, at least one T block distance is required between adjacent patches. When there is insufficient space to place the next patch, the height of the image will become 2 times the original, and then continue to place the patch.

To make the patch arrangement more compact, the patch may select a variety of different arrangement orientations. For example, eight different alignment directions may be employed, including 0 degrees, 180 degrees, 90 degrees, 270 degrees, and mirror images of the first four directions, as shown in FIG. 4.

To obtain better adaptation to the inter-frame prediction characteristics of a video encoder, a Patch permutation method with temporal consistency is used. In one GOF (Group of frame), all patches of the first frame are arranged in order from large to small. For other frames in the GOF, the order of the patch is adjusted using a time domain consistency algorithm.

It should be noted that, after the two-dimensional image information is obtained, the patch information can be obtained according to the information in the process of obtaining the two-dimensional image information, and then the encoding of the slice information can be performed to obtain the sub-code stream of the slice information.

Here, it should be noted that, in the process of performing two-dimensional image information, information of a patch division, information of a patch projection plane, and information of a patch packaging position need to be recorded, so that the patch information records information of operations of each step in the process of acquiring the two-dimensional image, that is, the patch information includes: information of patch division, information of a patch projection plane and information of a patch packaging position.

In step 1024, the encoding end obtains the first geometric figure and the occupancy map according to the two-dimensional image information.

It should be noted that, for the process of obtaining the occupancy map, mainly: and setting the position of the vertex in the two-dimensional image as 1 and the rest positions as 0 by using the patch arrangement information obtained by patch packing to obtain a occupation map. For the process of acquiring the first geometry map, mainly: in the process of obtaining the 2D patch through projection, the distance from each vertex to a projection plane is saved, the distance is called depth, and the geometric figure compression part is used for arranging the depth value of each vertex in the 2D patch to the position of the vertex in the occupied figure to obtain a first geometric figure.

Optionally, the implementation manner of the step 103 is:

the coding end arranges coordinate values of three dimensions of each vertex in the second precision geometric information to obtain an arrangement value corresponding to each vertex;

the encoding end arranges the arrangement value corresponding to each vertex in a first area to obtain a second geometric figure containing second precision geometric information and first precision geometric information;

the first area is the area left after the first precision geometric information projection distribution area is removed from the two-dimensional projection distribution area of the geometric information of the target three-dimensional grid.

It should be noted that, by changing the encoding of three components of the three-dimensional coordinates of the high-precision geometric information to the encoding of one component, the encoding efficiency of the video encoder can be improved.

It should be noted that, for the high-precision geometric information, the high-precision geometric information is first arranged by vertex-to-vertex, where the values of xyz in three dimensions are arranged according to a preset arrangement rule (it should be noted that the preset arrangement rule may be pre-agreed, in general, the coding end encodes the preset arrangement rule into a code stream when encoding, and the decoding end obtains the preset arrangement rule when decoding the code stream, so as to use the preset arrangement rule to perform decoding), for example, the sequence of xyz, yzx or zxy is arranged as a value (arrangement value). And placing the arrangement value of each vertex in a region (namely a first region) which is saved after the projection distribution of the quantized low-precision geometric information is reduced, so as to arrange the high-precision arrangement value corresponding to the low-precision vertex in a high-precision geometric information region in the low-precision geometric figure, and obtain the geometric figure containing the low-precision geometric information and the high-precision geometric information.

Optionally, the arranging the arrangement value corresponding to each vertex in the first area, and the implementation manner of obtaining the second geometric diagram including the second precision geometric information and the first precision geometric information may adopt one of the following:

b11, the coding end arranges pixels of the arrangement value corresponding to each vertex in a first area according to the pixel distribution positions of projection points corresponding to the vertices in the first precision geometric information, and a second geometric figure containing second precision geometric information and first precision geometric information is obtained;

the projection points refer to points corresponding to the vertices in the two-dimensional image after the vertices of the three-dimensional mesh are projected on the two-dimensional sheet. One pixel in the two-dimensional image corresponds to the projected point of one vertex. The pixel value corresponding to the projection point in the occupancy map is typically 1, and the depth value corresponding to the projection point in the geometry map is typically greater than 0.

It should be noted that, the pixel distribution of the projection points corresponding to the vertices in the first precision geometric information may be a pixel distribution of the projection points corresponding to the low precision vertices in the occupation map and/or the geometric map, and in this case, the arrangement values of the vertices in the high precision geometric information are correspondingly arranged by using the pixel positions of the projection points corresponding to the low precision vertices in the occupation map and/or the geometric map.

That is, how the first precise geometric information is distributed in this way, the arrangement values of the vertices in the second precise geometric information are arranged in the first area in the same distribution. The high-precision arrangement values corresponding to the low-precision vertexes are arranged in the high-precision geometric information area in the low-precision geometric figure according to the two-dimensional projection distribution of the low-precision geometric information in the area saved after the projection distribution of the quantized low-precision geometric information is reduced, so that the geometric figure further comprising the high-precision geometric information is obtained. As shown in fig. 5, the geometric diagram is a schematic diagram of a geometric figure in which arrangement values corresponding to high-precision geometric information are arranged according to positions of low-precision geometric information, and the arrangement values corresponding to high-precision geometric information are distributed in the lower half of the image in fig. 5.

B12, the coding end arranges pixels of the projection points corresponding to each vertex in the second precision geometric information in a first area according to the pixel distribution positions of the projection points corresponding to the vertices in the first precision geometric information, carries out translation processing on the pixels, arranges the arrangement values of the vertices in the pixel positions of the projection points corresponding to the vertices in the first area after translation processing, and acquires a second geometric figure containing the second precision geometric information and the first precision geometric information;

It should be noted that, in this way, the arrangement values of the vertices in the second precision geometric information are closely arranged in the horizontal direction or the vertical direction, that is, the distribution that is closely arranged in the horizontal direction or the vertical direction translates in the area saved after the projection distribution of the low precision geometric information is reduced after quantization, and the high precision arrangement values corresponding to the low precision vertices are arranged in the high precision geometric information area in the low precision geometric figure, so as to obtain the geometric figure further comprising the high precision geometric information. As shown in fig. 6, which is a schematic diagram of a geometric diagram in which high-precision arrangement values are arranged in a horizontal direction with a distribution in which pixels are closely arranged shifted to the left, the arrangement values corresponding to the high-precision geometric information are distributed in the lower half of the image in fig. 6.

It should also be noted that, optionally, the specific implementation procedure in this implementation manner is:

the coding end scans pixels of projection points corresponding to vertexes in the second precision geometric information row by row or column by column along a first direction, and numbering in the rows or columns is carried out again on position indexes of the pixels in each row or column;

the encoding end arranges the arrangement value corresponding to each vertex in the second precision geometric information at the pixel position designated by the position index of the pixel of the projection point corresponding to the vertex in a first area according to the scanning sequence, and a second geometric figure containing the second precision geometric information and the first precision geometric information is obtained;

Wherein the first direction is a horizontal direction or a vertical direction.

It should be noted that, in general, pixels of projection points corresponding to vertices in the second precision geometric information are not closely adjacent to each other, but have a certain interval, in this embodiment of the present application, the pixels having a pitch in the same row or the same column are renumbered in a manner of being adjacent to each other, for example, in a certain row, there are 5 pixels corresponding to each of which numbers are 1, 3, 5, 7, and 8 from left to right, after scanning from left to right, the 5 pixels are renumbered again, and then the 5 pixels are newly numbered 1, 2, 3, 4, and 5 from left to right after renumbering.

The pixels of the projection points corresponding to the vertexes in the second precision geometric information are scanned row by row or column by column, numbering is carried out again, when the scanning is carried out in the horizontal direction, the scanning is carried out row by row according to the horizontal direction, the numbering of the position indexes in the rows is carried out again on the pixel positions of the projection points corresponding to the vertexes in each row, when the scanning is carried out in the vertical direction, the scanning is carried out column by column according to the vertical direction, and the numbering of the position indexes in the rows is carried out again on the pixel positions of the projection points corresponding to the vertexes in each column; the pixels of the projection points corresponding to the vertexes in the second precision geometric information are translated according to the horizontal or vertical direction, the arrangement values of the vertexes are arranged to the pixel positions of the projection points corresponding to the vertexes, a geometric figure with compact pixel arrangement is obtained, and finally the geometric figure containing high precision geometric information and low precision geometric information is compressed by a video encoder, so that a geometric figure code stream is obtained.

It should be noted that, by changing the encoding of three components of the three-dimensional coordinates into the encoding of one component, the encoding efficiency of the video encoder can be improved after the three components are closely arranged, and the encoding efficiency of high-precision geometric information is improved as a whole.

In summary, it is known that, for the encoding end, the three-dimensional grid is quantized first, and thus low-precision geometric information, high-precision geometric information, and information of the supplemental points may be obtained. For the encoding of low precision geometry information, the occupancy and low precision geometry maps may be generated using a projection method, which is encoded by a video encoder. The application particularly proposes a method for representing and encoding high-precision geometric information. For high-precision geometric information coding, firstly, the high-precision geometric information is arranged into values of three dimensions of xyz vertex by vertex according to a preset arrangement rule. And then arranging the high-precision arrangement values corresponding to the low-precision vertexes in the high-precision geometric information area in the low-precision geometric figure according to the two-dimensional projection distribution of the low-precision geometric information or the distribution which is closely arranged in a horizontal direction (or vertical direction) translation manner in the area which is saved after the projection distribution of the quantized low-precision geometric information is reduced. The geometric information of the complementary points is divided into a low-precision part and a high-precision part, and the low-precision part and the high-precision part can be independently encoded into one path of code stream or can be arranged in a low-precision geometric figure in a raw patch mode. Finally, the occupancy map, the low-precision geometry map containing high-precision geometry information, is encoded using a video encoder. For the decoding end, a video decoder is used for decoding to obtain a occupation map and a low-precision geometric figure containing high-precision geometric information, the occupation map and the low-precision geometric information in the low-precision geometric figure containing the high-precision geometric information can be used for reconstructing a low-precision three-dimensional grid, and the low-precision three-dimensional grid is reconstructed into a high-precision three-dimensional grid by utilizing the high-precision geometric information (arrangement value) in the low-precision geometric figure.

The video-based three-dimensional grid geometric information coding framework of the embodiment of the application is shown in fig. 7, and the overall coding flow is as follows: first, whether to sample the three-dimensional grid for simplification can be selected before quantization; then, the three-dimensional grid is quantized, thereby possibly generating three parts of low-precision geometric information, high-precision geometric information and supplementary point information; for low-precision geometric information, performing patch division in a projection mode, and generating patch sequence compression information (patch division information), a occupation map and a low-precision geometric figure by patch arrangement; for existing high-precision geometric information, the high-precision geometric information can be firstly arranged into one value by vertex according to the values of three dimensions of xyz. Then, in the region saved after the quantized low-precision geometric information projection distribution is reduced, arranging the high-precision arrangement value corresponding to the low-precision vertex in the high-precision geometric information region in the low-precision geometric figure according to the two-dimensional projection distribution of the low-precision geometric information or the distribution closely arranged in a horizontal direction (or a vertical direction) translation mode; for the possible supplementary points, the geometric information of the supplementary points can be divided into a low-precision part and a high-precision part, and the low-precision part and the high-precision part are respectively arranged by a raw patch, and are independently encoded into a code stream, or the raw patch is added into a geometric figure; and finally, encoding the patch sequence compression information, the occupation map and the geometric figure, and mixing the multiple paths of sub-code streams to obtain a final output code stream.

It should be noted that the application provides an implementation mode of how to encode the geometric information of the three-dimensional grid, the three-dimensional coordinate value of the high-precision geometric information obtained by quantizing the geometric information of the three-dimensional grid is changed into one component of encoding, and the encoding is carried out according to the position of the low-precision geometric figure, so that the compression efficiency of the video encoder on the high-precision geometric information can be improved; in addition, the high-precision geometric figures are closely arranged, so that the compression efficiency of the video encoder on the high-precision geometric information can be improved; by the scheme, the efficiency of compressing geometric information by using the quantization scheme can be further improved.

According to the encoding method provided by the embodiment of the application, the execution main body can be an encoding device. In the embodiment of the present application, an encoding method performed by an encoding device is taken as an example, and the encoding device provided in the embodiment of the present application is described.

As shown in fig. 8, an embodiment of the present application provides an encoding apparatus 800, including:

the quantization module 801 is configured to quantize geometric information of the target three-dimensional grid to obtain first precision geometric information and second precision geometric information;

a first obtaining module 802, configured to obtain a first geometric diagram and a occupancy map of the first precise geometric information;

A second obtaining module 803, configured to obtain a second geometric diagram including second precision geometric information and first precision geometric information according to the second precision geometric information and the first geometric diagram;

an encoding module 804, configured to encode the second geometric map and the occupancy map;

Optionally, the second obtaining module 803 includes:

the first acquisition unit is used for arranging coordinate values of three dimensions of each vertex in the second precision geometric information to acquire an arrangement value corresponding to each vertex;

the second acquisition unit is used for arranging the arrangement value corresponding to each vertex in the first area and acquiring a second geometric figure containing second precision geometric information and first precision geometric information;

Optionally, the second obtaining unit is configured to implement one of the following:

Arranging pixels of the arrangement value corresponding to each vertex in a first area according to the pixel distribution positions of projection points corresponding to the vertices in the first precision geometric information, and obtaining a second geometric figure containing second precision geometric information and the first precision geometric information;

and carrying out pixel arrangement of the projection points corresponding to each vertex in the second precision geometric information in the first area according to the pixel distribution positions of the projection points corresponding to the vertices in the first precision geometric information, carrying out translation processing on the pixels, and arranging the arrangement values of the vertices in the pixel positions of the projection points corresponding to the vertices in the first area after translation processing to obtain a second geometric figure containing the second precision geometric information and the first precision geometric information.

Optionally, the performing translation processing on the pixels, arranging the arrangement values of the vertices at pixel positions of projection points corresponding to the vertices in the first area after the translation processing, and obtaining a second geometric figure including second precision geometric information and first precision geometric information includes:

scanning pixels of projection points corresponding to vertexes in the second precision geometric information row by row or column by column along a first direction, and numbering the position indexes of the pixels in each row or column again;

According to the scanning sequence, arranging the arrangement value corresponding to each vertex in the second precision geometric information in a first area at a pixel position designated by a position index of a pixel of a projection point corresponding to the vertex, and obtaining a second geometric figure containing the second precision geometric information and the first precision geometric information;

wherein the first direction is a horizontal direction or a vertical direction.

Optionally, the first obtaining module 802 includes:

the dividing unit is used for dividing the first precision geometric information into three-dimensional slices;

the third acquisition unit is used for carrying out two-dimensional projection on the divided three-dimensional slices to acquire two-dimensional slices;

the fourth acquisition unit is used for packaging the two-dimensional slices to acquire two-dimensional image information;

and a fifth acquisition unit, configured to acquire the first geometric figure and the occupancy map according to the two-dimensional image information.

Optionally, after the fourth obtaining unit packages the two-dimensional slice, the method further includes:

a sixth acquisition unit for acquiring slice information according to information in the process of acquiring the two-dimensional image information;

and a seventh obtaining unit, configured to encode the slice information to obtain a slice information subcode stream.

Optionally, the apparatus further comprises:

the third acquisition module is used for quantizing the geometric information of the target three-dimensional grid to acquire the information of the supplementary points;

the information of the supplementary points is information of points which are generated in the quantization process and need additional processing.

Optionally, the third obtaining module is configured to:

and determining the information of the supplementary points according to the geometric information of the target three-dimensional grid and the first precision geometric information.

Optionally, the information of the supplemental point includes at least one of:

supplementing indexes of vertexes in the first precision geometric information corresponding to the points;

the third precision geometric information of the supplementary points is three-dimensional coordinate information of the quantized supplementary points;

and the fourth precision geometric information of the supplementary point is three-dimensional coordinate information of the supplementary point lost in the quantized process.

Optionally, the quantization module 801 includes:

an eighth obtaining unit, configured to quantize each vertex in the target three-dimensional mesh according to the quantization parameter of each component, and obtain first precision geometric information;

and a ninth acquisition unit, configured to acquire second precision geometric information according to the first precision geometric information and the quantization parameter of each component.

The embodiment of the device corresponds to the embodiment of the encoding method, and each implementation process and implementation manner of the embodiment of the method can be applied to the embodiment of the device, and the same technical effects can be achieved.

The embodiment of the application also provides encoding equipment, which comprises a processor and a communication interface, wherein the processor is used for quantizing the geometric information of the target three-dimensional grid to obtain first precision geometric information and second precision geometric information; acquiring a first geometric figure and a occupation map of the first precision geometric information; acquiring a second geometric figure containing second precision geometric information and first precision geometric information according to the second precision geometric information and the first geometric figure; encoding the second geometric figure and the occupancy map;

Specifically, the embodiment of the present application further provides an encoding apparatus, as shown in fig. 9, where the encoding apparatus 900 includes: a processor 901, a network interface 902, and a memory 903. The network interface 902 is, for example, a common public radio interface (common public radio interface, CPRI).

Specifically, the encoding apparatus 900 of the embodiment of the present application further includes: instructions or programs stored in the memory 903 and executable on the processor 901, the processor 901 invokes the instructions or programs in the memory 903 to execute the method executed by each module shown in fig. 8, and achieve the same technical effects, so that repetition is avoided and thus a description thereof is omitted.

As shown in fig. 10, an embodiment of the present application further provides a decoding method, including:

step 1001, the decoding end decomposes the obtained code stream of the target three-dimensional grid to obtain a occupation map and a second geometric map containing second precision geometric information and first precision geometric information;

step 1002, the decoding end obtains second precision geometric information and a first geometric diagram according to the second geometric diagram;

step 1003, the decoding end obtains first precision geometric information according to the first geometric figure and the occupancy map;

step 1004, the decoding end performs inverse quantization according to the second precision geometric information and the first precision geometric information to obtain the geometric information of the target three-dimensional grid;

Optionally, the decomposing the obtained code stream of the target three-dimensional grid, and obtaining the occupation map and the second geometric map including the second precision geometric information and the first precision geometric information are specifically implemented as follows:

the decoding end obtains a target subcode stream according to the obtained code stream of the target three-dimensional grid, wherein the target subcode stream comprises: a chip information subcode stream, a occupancy map subcode stream, and a geometry map subcode stream;

and the decoding end acquires a occupation map and a second geometric map containing second precision geometric information and first precision geometric information according to the target subcode stream.

Optionally, the obtaining the second precision geometric information and the first geometric diagram according to the second geometric diagram is specifically implemented as follows:

the decoding end respectively acquires an arrangement value corresponding to each vertex in the second precision geometric information and a first geometric figure corresponding to the first precision geometric information from the second geometric figure;

the decoding end restores three dimensional coordinate values of the arrangement value corresponding to each vertex according to the arrangement sequence of the vertices to obtain second precision geometric information;

the arrangement value corresponding to each vertex is arranged in a first area of the second geometric figure, wherein the first area is the area remained after the first precision geometric information projection distribution area is removed from the two-dimensional projection distribution area of the geometric information of the target three-dimensional grid.

Optionally, the implementation manner of obtaining, in the second geometric diagram, the arrangement value corresponding to each vertex in the second precision geometric information includes one of the following:

the decoding end restores the arrangement value corresponding to each vertex in the second precision geometric information in a first area in the second geometric figure according to the pixel distribution position of the projection point corresponding to the vertex in the first precision geometric information;

and the decoding end carries out translation processing on pixels of the projection points in the first area in the second geometric figure, and obtains an arrangement value corresponding to each vertex in the second precision geometric information.

Optionally, the performing a translation process on pixels of the projection points in the first area in the second geometric diagram to obtain an arrangement value corresponding to each vertex in the second precision geometric information includes:

the decoding end scans pixels of projection points of the first geometric figure row by row or column by column along a first direction to obtain a position index in a row or a column where each pixel in each row or column is located;

the decoding end scans pixels of projection points in a first area in the second geometric figure row by row or column by column along a first direction, and numbering in the rows or columns is carried out again on each pixel in each row or column according to a position index corresponding to the pixels of the projection points in the first geometric figure;

It should be noted that, the renumbering herein can be understood as follows: pixels of the projection points corresponding to the vertices in the second precision geometric information are closely adjacent in the first area, while pixels of the projection points corresponding to the actual vertices are at a certain interval, in the embodiment of the present application, the pixels with a pitch in the same row or the same column are renumbered in a manner of adjacent pixels, for example, in a certain row, 5 pixels are correspondingly numbered from left to right, and are respectively numbered 1, 2, 3, 4 and 5, after the pixels are scanned from left to right, the 5 pixels are renumbered according to the pixel distribution of the projection points of the first geometric figure, and the new numbers of the 5 pixels from left to right are: 1. 3, 5, 7, 8.

The decoding end obtains an arrangement value corresponding to each vertex in the second precision geometric information from a first area in the second geometric figure according to a scanning sequence;

wherein the first direction is a horizontal direction or a vertical direction.

It should be noted that, since the pixels of the projection points of the first geometric diagram do not perform pixel translation during the encoding process, during decoding, the positions of the pixels of the projection points corresponding to the vertices in the portion of the other pixel translation need to be restored according to the first geometric diagram that does not perform pixel translation, so as to ensure accurate restoration of the vertices in the second precision geometric information.

Optionally, the decoding end obtains the first precision geometric information according to the first geometric figure and the occupancy map, including:

the decoding end acquires two-dimensional image information according to the first geometric figure and the occupation map;

the decoding end acquires a two-dimensional slice according to the two-dimensional image information;

the decoding end performs three-dimensional back projection on the two-dimensional slice according to slice information corresponding to the slice information subcode stream to obtain a three-dimensional slice;

and the decoding end acquires first precision geometric information according to the three-dimensional slice.

Optionally, the implementation manner of obtaining the geometric information of the target three-dimensional grid by performing inverse quantization according to the second precision geometric information and the first precision geometric information is as follows:

the decoding end determines the coordinates of each vertex in the first precision geometric information according to the first precision geometric information and the quantization parameter of each component;

the decoding end determines the target three-dimensional grid according to the coordinates of each vertex in the first precision geometric information and the second precision geometric information.

Optionally, in the case of decomposing the acquired code stream of the target three-dimensional grid, the method further includes:

Obtaining a geometric figure of the supplementary points;

determining a first original piece corresponding to the third-precision geometric information of the supplementary point and a second original piece corresponding to the fourth-precision geometric information of the supplementary point according to the geometric diagram of the supplementary point;

the decoding end determines the information of the supplementary point according to the first original sheet and the second original sheet;

Optionally, the information of the supplemental point includes at least one of:

In the embodiment of the present application, the low-precision part and the high-precision part, which are separated from the geometric information of the supplemental points, are respectively decoded. First, the geometric map of the supplemental points is decompressed, and a variety of decompression methods may be employed. One method is to decode the geometric figure in a run-length decoding mode, an entropy decoding mode and the like, and the other method is to take the low-precision raw patch of the supplementary point and the high-precision raw patch of the supplementary point out of the low-precision geometric figure. Then, low-precision geometric information of the supplementary points is obtained from the low-precision raw patch of the supplementary points according to a specific sequence, and high-precision geometric information is obtained from the high-precision raw patch of the supplementary points according to the specific sequence; it should be noted that the specific sequence is obtained by the decoding end by parsing the code stream, that is, what sequence is adopted by the encoding end to generate the low-precision raw patch of the supplemental point and the high-precision raw patch of the supplemental point are notified to the decoding end by the code stream.

Optionally, the determining, according to the coordinates of each vertex in the first precision geometric information and the second precision geometric information, the implementation manner of the target three-dimensional grid is as follows:

the decoding end determines the target three-dimensional grid according to the information of the supplementary points, the second precision geometric information and the coordinates of each vertex in the first precision geometric information.

It should be noted that, in the embodiment of the present application, the decoding process of the three-dimensional grid geometric information based on video includes: decomposing the code stream into a latch information subcode stream, a duty bit stream and a geometric figure code stream; then, decoding the three sub-code streams respectively to obtain patch information, a occupation map and a geometric map; finally, the geometry information is reconstructed using the patch information, the occupancy map, and the geometry map. Specifically, as shown in fig. 11, the most critical specific process of geometric information reconstruction is:

step S111, acquiring a 2D patch;

it should be noted that, acquiring the 2D patch refers to dividing the occupation information and the depth information of the 2D patch from the occupation map and the geometric map by using the patch information. The Patch information comprises the position and the size of the bounding box of each 2D Patch in the occupation map and the low-precision geometric map, and the occupation information and the low-precision geometric information of the 2D Patch can be directly obtained by using the Patch information, the occupation map and the low-precision geometric map. And for the high-precision geometric information, using the vertex distribution of the low-precision geometric figure to correspond the high-precision geometric information arrangement value in the high-precision geometric information area with the low-precision geometric figure vertex, and separating xyz three-dimensional geometric information from the high-precision geometric information according to the preset arrangement rule used by the encoding end, thereby obtaining the high-precision geometric information. For the geometric information of the supplementary point, the low-precision geometric information and the high-precision geometric information of the supplementary point can be obtained by directly decoding the low-precision raw patch and the high-precision raw patch of the supplementary point.

Step S112, reconstructing a 3D patch;

it should be noted that reconstructing the 3D patch refers to reconstructing vertices in the 2D patch into the low-precision 3D patch by using the occupancy information and the low-precision geometric information in the 2D patch. The occupation information of the 2D patch comprises the position of the vertex relative to the origin of coordinates in a local coordinate system of the patch projection plane, and the depth information comprises the depth value of the vertex in the normal direction of the projection plane. Thus, the 2D patch can be reconstructed into a low-precision 3D patch in a local coordinate system using the occupancy information and the depth information.

Step S113, reconstructing a low-precision geometric model;

it should be noted that reconstructing the low-precision geometric model refers to reconstructing the entire low-precision three-dimensional geometric model by using the reconstructed low-precision 3D patch. The Patch information comprises a conversion relation of converting the 3D Patch from a local coordinate system to a global coordinate system of the three-dimensional geometric model, and all the 3D patches are converted into the global coordinate system by utilizing the coordinate conversion relation, so that the low-precision three-dimensional geometric model is obtained. In addition, for the supplementary points, the geometric information in the low-precision raw patch is directly utilized to obtain low-precision coordinate values of the supplementary points under the global coordinate system, so that a complete low-precision three-dimensional geometric model is obtained.

Step S114, reconstructing a high-precision geometric model;

the reconstruction of the high-precision geometric model refers to a process of reconstructing the high-precision geometric model by using high-precision geometric information on the basis of the low-precision geometric model. In the process of acquiring the 2D patch, the high-precision geometric information and the low-precision geometric information are corresponding, and the high-precision three-dimensional coordinate of the vertex can be reconstructed according to the high-precision geometric information and the low-precision geometric information of the vertex. According to the application requirements, high-precision three-dimensional coordinates of all vertexes can be selected to be reconstructed, and high-precision three-dimensional coordinates of partial vertexes can also be reconstructed. High precision three-dimensional coordinates (x _r ,y _r ,z _r ) As shown in formulas twenty-five to twenty-seven:

twenty-five of the formula: x is x _r ＝f ₃ (x _l ,x _h ,QP _x )；

Formula twenty-six: y is _r ＝f ₃ (y _l ,y _h ,QP _y )；

Seventeenth formula: z _r ＝f ₃ (z _l ,z _h ,QP _z )；

f ₃ The function is a reconstruction function, the calculation process of the reconstruction function corresponds to the calculation process of the quantization function of the coding end, and various implementation modes exist. If f ₁ The function adopts the realization modes from the formula seven to the formula twelve, and the realization mode of the reconstruction function is shown as the formula twenty-eight to the formula thirty:

formula twenty-eight: x is x _r ＝x _l *QP _x +x _h ；

The formula twenty-nine: y is _r ＝y _l *QP _y +y _h ；

The formula thirty: z _r ＝z _l *QP _z +z _h ；

If f ₁ The function adopts the implementation modes from the formula thirteen to the formula eighteen, and the implementation mode of the reconstruction function is as shown in the formula thirty-one to the formula thirty-three:

formula thirty one: x is x _r ＝(x _l ＜＜log ₂ QP _x )|x _h ；

The formula thirty-two: y is _r ＝(y _l ＜＜log ₂ QP _y )|y _h ；

The formula thirty-three: z _r ＝(z _l ＜＜log ₂ QP _z )|z _h 。

The video-based three-dimensional grid geometric information decoding framework of the embodiment of the application is shown in fig. 12, and the overall decoding flow is as follows:

firstly, decomposing a code stream into a patch information sub-code stream, a bitmap sub-code stream and a geometric map sub-code stream, and respectively decoding; the geometric information of the low-precision grid can be reconstructed by using the low-precision part in the occupation map and the geometric map, and the geometric information of the high-precision grid can be reconstructed by using the high-precision part in the occupation map and the geometric map; finally, reconstructing the grid by using the reconstructed geometric information and the connection relation obtained by other encoding and decoding modes.

It should be noted that, the embodiment of the present application is an opposite-end method embodiment corresponding to the foregoing encoding method embodiment, the decoding process is an inverse process of encoding, and all the foregoing implementation manners of the encoding side are applicable to the decoding-end embodiment, so that the same technical effects can be achieved, which is not repeated herein.

As shown in fig. 13, an embodiment of the present application further provides a decoding apparatus 1300, including:

The third obtaining module 1301 is configured to decompose the obtained code stream of the target three-dimensional grid, and obtain a occupation map and a second geometric map that includes second precision geometric information and first precision geometric information;

a fourth obtaining module 1302, configured to obtain second precision geometric information and the first geometric diagram according to the second geometric diagram;

a fifth obtaining module 1303, configured to obtain first precision geometric information according to the first geometric diagram and the occupancy map;

a sixth obtaining module 1304, configured to perform dequantization according to the second precision geometric information and the first precision geometric information, to obtain geometric information of the target three-dimensional grid;

Optionally, the third obtaining module 1301 includes:

a tenth acquisition unit, configured to acquire a target subcode stream according to the acquired code stream, where the target subcode stream includes: a chip information subcode stream, a occupancy map subcode stream, and a geometry map subcode stream;

an eleventh acquisition unit, configured to acquire, according to the target subcode stream, a occupation map and a second geometric map that includes second precision geometric information and first precision geometric information.

Optionally, the fourth obtaining module 1302 includes:

a twelfth obtaining unit, configured to obtain, in the second geometric diagram, an arrangement value corresponding to each vertex in the second precision geometric information and a first geometric diagram corresponding to the first precision geometric information;

a thirteenth obtaining unit, configured to recover three dimensional coordinate values of the arrangement value corresponding to each vertex according to the arrangement sequence of the vertices, to obtain second precision geometric information;

Optionally, the twelfth obtaining unit is configured to implement one of the following:

recovering an arrangement value corresponding to each vertex in the second precision geometric information in a first area in the second geometric figure according to the pixel distribution position of the projection point corresponding to the vertex in the first precision geometric information;

and carrying out translation processing on pixels of the projection points in the first area in the second geometric figure to obtain an arrangement value corresponding to each vertex in the second precision geometric information.

Optionally, the performing a translation process on pixels of the projection points in the first area in the second geometric diagram to obtain an implementation manner of the arrangement value corresponding to each vertex in the second precision geometric information is:

scanning pixels of projection points of the first geometric figure row by row or column by column along a first direction to obtain a position index in a row or a column where each pixel in each row or column is located;

scanning pixels of the projection points in the first area in the second geometric figure row by row or column by column along a first direction, and numbering each pixel in each row or column according to a position index corresponding to the pixels of the projection points in the first geometric figure;

according to the scanning sequence, acquiring an arrangement value corresponding to each vertex in the second precision geometric information from a first area in the second geometric figure;

wherein the first direction is a horizontal direction or a vertical direction.

Optionally, the fifth obtaining module 1303 includes:

a fourteenth acquisition unit configured to acquire two-dimensional image information according to the first geometric figure and the occupancy map;

a fifteenth acquisition unit configured to acquire a two-dimensional slice according to the two-dimensional image information;

A sixteenth obtaining unit, configured to perform three-dimensional back projection on the two-dimensional slice according to slice information corresponding to the slice information subcode stream, to obtain a three-dimensional slice;

a seventeenth obtaining unit, configured to obtain the first precision geometric information according to the three-dimensional slice.

Optionally, the sixth acquisition module 1304 includes:

a first determining unit, configured to determine coordinates of each vertex in the first precision geometric information according to the first precision geometric information and a quantization parameter of each component;

and the second determining unit is used for determining the target three-dimensional grid according to the coordinates of each vertex in the first precision geometric information and the second precision geometric information.

Optionally, the apparatus further comprises:

a seventh acquisition module, configured to acquire a geometric figure of the supplemental points;

the first determining module is used for determining a first original piece corresponding to the third precision geometric information of the supplementary point and a second original piece corresponding to the fourth precision geometric information of the supplementary point according to the geometric diagram of the supplementary point;

the second determining module is used for determining information of the supplementary points according to the first original sheet and the second original sheet;

Optionally, the second determining unit is configured to:

and determining the target three-dimensional grid according to the information of the supplementary points, the second precision geometric information and the coordinates of each vertex in the first precision geometric information.

Optionally, the information of the supplemental point includes at least one of:

It should be noted that, the embodiment of the apparatus is an apparatus corresponding to the above method, and all implementation manners in the embodiment of the method are applicable to the embodiment of the apparatus, so that the same technical effects can be achieved, which is not described herein again.

Preferably, the embodiment of the present application further provides a decoding device, which includes a processor, a memory, and a program or an instruction stored in the memory and capable of running on the processor, where the program or the instruction is executed by the processor to implement each process of the decoding method embodiment described above, and the same technical effects can be achieved, so that repetition is avoided, and details are not repeated here.

The embodiment of the application also provides a readable storage medium, on which a program or an instruction is stored, which when executed by a processor, implements the respective processes of the decoding method embodiment described above, and can achieve the same technical effects, and in order to avoid repetition, the description is omitted here.

Wherein the computer readable storage medium is selected from Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk.

The embodiment of the application also provides decoding equipment, which comprises a processor and a communication interface, wherein the processor is used for decomposing the acquired code stream of the target three-dimensional grid to acquire a occupation map and a second geometric figure containing second precision geometric information and first precision geometric information; acquiring second precision geometric information and a first geometric figure according to the second geometric figure; acquiring first precision geometric information according to the first geometric figure and the occupancy map; performing inverse quantization according to the second precision geometric information and the first precision geometric information to obtain the geometric information of the target three-dimensional grid;

The decoding device embodiment corresponds to the decoding method embodiment, and each implementation process and implementation manner of the method embodiment can be applied to the decoding device embodiment, and the same technical effect can be achieved.

Specifically, the embodiment of the application also provides decoding equipment. Specifically, the structure of the decoding device is shown in fig. 9, and will not be described herein. Specifically, the decoding device of the embodiment of the present application further includes: instructions or programs stored in the memory and capable of running on the processor, which invokes the instructions or programs in the memory to execute the method executed by each module shown in fig. 13, achieve the same technical effects, and are not repeated here.

The embodiment of the application also provides a readable storage medium, on which a program or an instruction is stored, which when executed by a processor, implements each process of the decoding method embodiment described above, and can achieve the same technical effects, and in order to avoid repetition, the description is omitted here.

Wherein the processor is a processor in the decoding device described in the above embodiment. The readable storage medium includes computer readable storage medium such as computer readable memory ROM, random access memory RAM, magnetic or optical disk, etc.

Optionally, as shown in fig. 14, the embodiment of the present application further provides a communication device 1400, including a processor 1401 and a memory 1402, where the memory 1402 stores a program or instructions that can be executed on the processor 1401, for example, when the communication device 1400 is an encoding device, the program or instructions implement the steps of the foregoing embodiment of the encoding method when executed by the processor 1401, and achieve the same technical effects. When the communication device 1400 is a decoding device, the program or the instructions implement the steps of the decoding method embodiment described above when executed by the processor 1401, and the same technical effects can be achieved, so that repetition is avoided and detailed description is omitted.

The embodiment of the application further provides a chip, the chip comprises a processor and a communication interface, the communication interface is coupled with the processor, the processor is used for running a program or instructions, the processes of the above coding method or decoding method embodiment can be realized, the same technical effects can be achieved, and the repetition is avoided, and the description is omitted here.

It should be understood that the chips referred to in the embodiments of the present application may also be referred to as system-on-chip chips, or the like.

The embodiments of the present application further provide a computer program/program product stored in a storage medium, where the computer program/program product is executed by at least one processor to implement each process of the foregoing encoding method or decoding method embodiment, and achieve the same technical effects, and are not repeated herein.

The embodiment of the application also provides a communication system, which at least comprises: an encoding device operable to perform the steps of the encoding method as described above, and a decoding device operable to perform the steps of the decoding method as described above. And the same technical effects can be achieved, and in order to avoid repetition, the description is omitted here.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present application is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in an opposite order depending on the functions involved, e.g., the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a computer software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present application.

The embodiments of the present application have been described above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present application and the scope of the claims, which are to be protected by the present application.

Claims

1. A method of encoding, comprising:

the encoding end encodes the second geometric figure and the occupation map;

2. The method of claim 1, wherein the obtaining a second geometry map comprising second precision geometry information and first precision geometry information from the second precision geometry information and the first geometry map comprises:

3. The method according to claim 2, wherein the arranging the arrangement value corresponding to each vertex in the first area, and obtaining the second geometric figure including the second precision geometric information and the first precision geometric information includes one of:

the coding end arranges pixels of the arrangement value corresponding to each vertex in a first area according to the pixel distribution positions of projection points corresponding to the vertices in the first precision geometric information, and a second geometric figure containing second precision geometric information and the first precision geometric information is obtained;

the encoding end performs pixel arrangement of the projection points corresponding to each vertex in the second precision geometric information in a first area according to the pixel distribution positions of the projection points corresponding to the vertices in the first precision geometric information, performs translation processing on the pixels, and arranges the arrangement values of the vertices in the pixel positions of the projection points corresponding to the vertices in the first area after the translation processing to obtain a second geometric figure containing the second precision geometric information and the first precision geometric information.

4. The method of claim 3, wherein the performing a translation process on the pixels, arranging the arrangement values of the vertices at the pixel positions of the projection points corresponding to the vertices in the translated first region, and obtaining the second geometric figure including the second precision geometric information and the first precision geometric information, includes:

wherein the first direction is a horizontal direction or a vertical direction.

5. The method of claim 1, wherein the encoding side obtaining the first geometric map and the occupancy map of the first precision geometric information comprises:

The coding end performs three-dimensional slice division on the first precision geometric information;

the coding end carries out two-dimensional projection on the divided three-dimensional slices to obtain two-dimensional slices;

the encoding end packs the two-dimensional slices to obtain two-dimensional image information;

and the encoding end acquires a first geometric figure and a occupation map according to the two-dimensional image information.

6. The method according to claim 1, wherein the quantifying the geometric information of the target three-dimensional grid, and obtaining the first precision geometric information and the second precision geometric information, the method further comprises:

the encoding end obtains the information of the supplementary points based on the quantization of the geometric information of the target three-dimensional grid;

7. The method of claim 6, wherein the obtaining information of the supplemental points comprises:

and the encoding end determines the information of the supplementary points according to the geometric information of the target three-dimensional grid and the first precision geometric information.

8. The method according to claim 6 or 7, characterized in that the information of the supplementary points comprises at least one of the following:

9. The method of claim 1, wherein quantifying the geometric information of the target three-dimensional mesh to obtain the first precision geometric information and the second precision geometric information comprises:

10. A decoding method, comprising:

11. The method of claim 10, wherein decomposing the acquired code stream of the target three-dimensional grid to obtain the occupancy map and the second geometric map including the second precision geometric information and the first precision geometric information comprises:

12. The method of claim 10, wherein the obtaining second precision geometry information and the first geometry from the second geometry comprises:

13. The method of claim 12, wherein the obtaining, in the second geometric figure, the permutation value corresponding to each vertex in the second precision geometric information includes one of:

14. The method of claim 13, wherein performing a translation process on pixels of the projection points in the first region in the second geometric figure to obtain an arrangement value corresponding to each vertex in the second precise geometric information, includes:

wherein the first direction is a horizontal direction or a vertical direction.

15. The method of claim 10, wherein the decoding side obtaining the first precision geometric information according to the first geometric figure and the occupancy map comprises:

16. The method of claim 10, wherein said dequantizing said second precision geometry information and said first precision geometry information to obtain geometry information for a target three-dimensional grid comprises:

17. The method according to claim 16, wherein in case of decomposing the code stream of the acquired target three-dimensional grid, the method further comprises:

the decoding end acquires the geometric figure of the supplementary point;

The decoding end determines a first original slice corresponding to the third precision geometric information of the supplementary point and a second original slice corresponding to the fourth precision geometric information of the supplementary point according to the geometric diagram of the supplementary point;

18. The method of claim 17, wherein the determining the target three-dimensional mesh from the coordinates of each vertex in the first precision geometry information and the second precision geometry information comprises:

19. The method according to claim 17 or 18, wherein the information of the supplemental points comprises at least one of:

20. An encoding device, comprising:

21. The apparatus of claim 20, wherein the second acquisition module comprises:

22. An encoding device comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, implement the steps of the encoding method according to any one of claims 1 to 9.

23. A decoding apparatus, comprising:

24. A decoding device comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, implement the steps of the decoding method according to any one of claims 10 to 19.

25. A readable storage medium, characterized in that the readable storage medium has stored thereon a program or instructions which, when executed by a processor, implement the steps of the encoding method according to any one of claims 1 to 9 or the steps of the decoding method according to any one of claims 10 to 19.