CN107623858B - 3D video fast coding method based on adaptive segmentation skipping rule - Google Patents
3D video fast coding method based on adaptive segmentation skipping rule Download PDFInfo
- Publication number
- CN107623858B CN107623858B CN201710975608.XA CN201710975608A CN107623858B CN 107623858 B CN107623858 B CN 107623858B CN 201710975608 A CN201710975608 A CN 201710975608A CN 107623858 B CN107623858 B CN 107623858B
- Authority
- CN
- China
- Prior art keywords
- coding
- optimal
- texture
- criterion
- block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses a 3D video fast coding method based on a self-adaptive segmentation skipping rule, which has the problem of selectively skipping a computational SVDC value with higher complexity when a depth map sequence is coded, and belongs to the field of HEVC video coding. The method is characterized in that firstly, a region which does not cause synthesized viewpoint distortion in the depth map is judged according to a texture smoothness criterion and a depth zero-distortion criterion, then the zero-distortion region is skipped in advance based on an adaptive segmentation skipping rule before SVDC calculation is carried out on the depth map, the calculation process of the region is terminated, and finally, the optimal coding mode and the optimal segmentation skipping interval of each to-be-coded sequence are analyzed based on a motion vector method, so that the coding complexity is effectively reduced. The invention terminates the SVDC calculation process of each coding unit by the original coding method in advance, and can reduce 21.711% of coding time on average on the premise of keeping the coding quality unchanged.
Description
Technical Field
The invention relates to a video coding technology based on HEVC, in particular to a fast video coding method based on an adaptive segmentation skip rule in 3D (three-dimensional) video depth map coding.
Background
In recent years, with the rapid development of multimedia technology, three-dimensional video has received attention from academic and industrial fields, particularly from the movie industry and home entertainment industry. The 3D-HEVC standard supports coding of a Multi-view video plus Depth Map (MVD) video format, and a decoding end synthesizes a new virtual view by using a Depth Image based rendering (DBIR). Since the depth map is used for the synthesis of the virtual view, distortion of the coded depth map directly results in distortion of the virtual view. In order to consider the quality of the virtual viewpoint while coding the depth map, the depth map coding decision process based on the rate-distortion optimization method needs to consider the distortion of the virtual viewpoint while calculating the coding distortion of the depth map. Therefore, 3D-HEVC replaces Rate Distortion Optimization (RDO) in the original depth map coding mode decision process with synthesized view Distortion, a technique known as VSO.
At present, many scholars at home and abroad focus on how to reduce the complexity of VSO. Document [1] proposes a depth-distortion-allowed model with segmentation functionality, which builds a new rate-distortion model for mode decision and motion/disparity estimation by minimizing view synthesis distortion at a given bitrate. Furthermore, in order to characterize the view synthesis quality, document [2] considers both the distortion caused by video coding, the distortion caused by depth quantization and the inherent geometric distortion. However, these algorithms use mathematical models to measure the distortion of the synthesized viewpoint, and their results are approximate values and cannot accurately represent the distortion of the synthesized viewpoint. In order to accurately calculate the synthesized viewpoint distortion, an SVDC method is proposed to measure distortion variation in the synthesized view.
SVDC can be used to accurately measure the distortion of the VSO, which is defined as the difference between the distortions of the two synthetic viewpoints, the calculation process is shown in fig. 2. Wherein V represents a virtual viewpoint image frame synthesized from an uncoded original texture image frame T and an uncoded original depth image frame D, and in addition, two virtual viewpoint image frames synthesized from a coded texture image frame T 'with two partially coded depth image frames D' and D ", respectively, are denoted as V 'and V", but D' further contains depth information distorted in the current depth map coding block to be decided, on the basis of D "; and recording the sum of the squared differences calculated by V ' and V "with V as SSD ' and SSD", and finally calculating the difference between SSD ' and SSD ", to obtain the value of SVDC.
The introduction of SVDC inevitably brings huge coding complexity, and in order to accelerate the computation process of the SVDC model, 3D-HEVC adopts an ES method to terminate some unnecessary SVDC computation processes in advance. Current HTM-16.0 employs block-based and line-based ES decision modes to skip unnecessary coding blocks to reduce SVDC computation time. The core skip condition of the ES method is: if the current depth value is the same as the disparity vector calculated from the original depth value, the depth value will not cause distortion of the synthesized view, and the calculation of SVDC can be skipped directly.
However, when calculating SSD ", each decision candidate is encoded by the current block to be decided to obtain different prediction block and reconstruction block, so that the encoder needs to calculate SSD" once for each candidate, which still has high computational complexity for the encoding end. Therefore, although the ES method can reduce the encoding complexity to some extent, there is still room for improvement. The invention provides a fast video coding method based on an adaptive segmentation skip rule based on the HEVC standard, which terminates the SVDC calculation process of each coding unit of the original coding method in advance, and can reduce the coding time by 21.711% on average on the premise of keeping the coding quality unchanged.
Attached: reference to the literature
[1]Zhang Y,Kwong S,Hu SD,et al.Efficient multi-view depth codingOptimization based onallowable depth distortion in view synthesis.IEEETRANSACTIONS ON IMAGE PROCESSING,NOV 2014,23(11):4879-4892.
[2]Liu YW,Huang QM,Ma SW,et al.Joint video/depth rate allocation for3D video coding based on view synthesis distortion model.SIGNAL PROCESSING-IMAGE COMMUNICATION,SEP 2009,24(8):666-681.
[3]Ma SW,Wang SQ,Gao W.Low complexity adaptive view synthesisoptimization in HEVC based 3D video coding.IEEETRANSACTIONS ON MULTIMEDIA,JAN2014,16(1):266-271.
Disclosure of Invention
The invention aims to provide a View Synthesis Optimization (VSO) method based on an adaptive segmentation skip rule, aiming at the problem of high complexity of computing Synthesized View Distortion Change (SVDC) adopted by depth map coding in the current three-dimensional high efficiency video coding standard 3D-HEVC (3D extension of high efficiency video coding, 3D-HEVC).
In order to solve the technical problems, the technical scheme adopted by the invention is as follows:
A3D video fast coding method based on adaptive segmentation skip rule, first of all, distinguish the area that will not cause the distortion of the synthetic view point in the depth map with the texture smoothness criterion and the zero distortion criterion of the depth, then skip the above-mentioned zero distortion area in advance based on adaptive segmentation skip rule before the depth map carries on SVDC calculation, terminate the computational process of the area, finally analyze the optimum coding mode and interval that the optimum segmentation of every code sequence skips based on method of the motion vector, reduce the complexity of encoding effectively, the concrete step is as follows:
1.1, selecting an encoding block with NxN size and an encoding mode being any encoding mode to be decided;
1.2, sequentially detecting the data by using a block-based ES method and a line-based ES method, if the coding block meets the block-based ES condition or the line-based ES condition, terminating the SVDC process of the whole coding block in advance, and judging whether the next pixel line meets the block-based ES condition or the line-based ES condition;
and 1.3, if a certain pixel row does not meet the texture smoothness criterion and the depth zero distortion criterion, judging the texture smoothness criterion and the depth zero distortion criterion, if the texture smoothness criterion and the depth zero distortion criterion meet the judgment condition, setting the SVDC of the current region to be 0, and otherwise, calculating the SVDC value of the current region.
In the method for fast encoding 3D video based on adaptive segmentation skip rule provided by the present invention, the process of determining the texture smoothness criterion includes the following steps:
2.1, the texture smoothness criterion is determined by |i-li+1|≤T；
2.2, calculating a threshold value T in the texture smoothness criterion in the step 2.1, wherein the calculation formula is
Wherein li,jAnd li+1,jIs a horizontal line of pixel pairs, NhAnd NW-1Representing the size of each module, IntraXXBlockNum represents one of 35 intra coding modes.
In the 3D video fast encoding method based on adaptive segment skip rule provided by the present invention, the motion vector based method analyzes the process of the optimal encoding mode and the optimal segment skip interval of each sequence to be encoded, including the following steps:
3.1, observing the main moving direction of a main moving object in the current video sequence, and selecting 5 optimal modes from 35 intra-frame coding modes to be selected as candidate modes;
3.2, performing experiments on the 5 candidate modes selected in the step 3.1, and determining a final intra-frame coding mode by observing BDrate and PSNR values in experimental results;
3.3, the optimal segmentation interval is selected by performing multiple tests on the standard test sequence { balloon, Kendo, newsapper, Poznan _ Hall2, Poznan street }, and the size of the interval is generally selected to be {1,2,3,4,5}, and comparing the required coding time to determine which interval is the optimal interval.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a block diagram of an SVDC (synthesized View DistoreationChange) computing process;
FIG. 3 is a diagram of 35 intra coding modes;
FIG. 4 shows the selected optimal coding mode and the interval of optimal segment skipping for different standard test sequences;
fig. 5 is a motion vector analysis diagram of the sequence of basetball;
FIG. 6 is a bar graph illustrating the percent time savings of a standard test sequence under the method of the present invention.
Detailed Description
The invention is explained in further detail below with reference to the drawings.
Referring to fig. 1, a 3D video fast encoding method based on an adaptive segment skip rule includes:
1. selecting an encoding block with the size of NxN and an encoding mode of any one to-be-decided encoding mode;
2. sequentially detecting the ES based on the block and the ES based on the line, if the coding block meets the ES condition based on the block or the ES condition based on the line, the SVDC process of the whole coding block is terminated in advance, and whether the next pixel line meets the ES condition based on the block or the ES condition based on the line is judged;
3. if a certain pixel row does not meet the texture smoothness criterion and the depth zero distortion criterion, judging the texture smoothness criterion and the depth zero distortion criterion, if the texture smoothness criterion and the depth zero distortion criterion meet the judgment condition, setting the SVDC of the current region to be 0, otherwise, calculating the SVDC value of the current region, and the calculation process is shown in FIG. 2;
4. and finally, analyzing the optimal coding mode and the optimal segmentation skipping interval of each to-be-coded sequence by a motion vector-based method, wherein the to-be-coded mode refers to fig. 3, and the to-be-selected segmentation interval refers to fig. 4.
Further, the texture smoothness criterion determining process specifically includes:
1) using the brightness value between two adjacent pixels in the reconstructed texture map corresponding to the current depth map, i.e. horizontal pixel pair,/i,jAnd li+1,jTo define a texture smoothing criterion, such as the equation | li-li+1Is less than or equal to T;
2) the selection of the threshold value T in the step 1) is used for determining whether the current reconstructed texture pixel pair is in a texture smooth region, and the threshold value T needs to be set in a targeted manner in order to accurately find out the zero-distortion region by using a texture smooth criterion;
further, the threshold T in the texture smoothing criterion proposed by the present invention can be calculated after the texture map coding and before the depth map coding, such as formula
As shown, the calculation steps are as follows:
1) after a texture map is coded, the size N of all the texture maps that use a certain (usually intra-frame DC mode) prediction mode as their final coding mode is recordedw×NhThe coding block of (2);
2) recording the horizontal adjacent pixels, l in a certain coding block in the step 1)i,jAnd li+1,jCalculating a measure difference value, and then calculating the sum of absolute errors of the whole row;
3) calculating the absolute error sum of all pixel rows in the step 2) and averaging to obtain the average horizontal direction brightness difference value of the coding block;
4) calculating an average value of all the horizontal direction brightness difference values obtained in the step 3) in the texture map, and defining a threshold value T in a texture smoothing criterion by the average value;
5) the texture map updates the threshold value T once after each I frame is coded, so that all images in the same frame period adopt the same threshold value T;
further, in the 3D video fast encoding method based on the adaptive segmentation skip rule proposed by the present invention, the motion vector based method analyzes the process of the optimal encoding mode and the optimal segmentation skip interval of each to-be-encoded sequence, and includes the following steps:
1) referring to fig. 5, observing the main moving direction of the main moving object in the current video sequence, selecting 5 best modes from 35 intra-frame coding modes to be selected as candidate modes;
2) performing experiments on the 5 candidate modes selected in the step 1), and determining a final intra-frame coding mode by observing BDrate and PSNR values in experimental results;
3) the optimal segmentation interval is selected by performing multiple tests on the standard test sequence { balloon, Kendo, newsapper, Poznan _ Hall2, Poznan street }, usually with an interval size of {1,2,3,4,5}, see fig. 6, and determining which interval is the optimal interval by comparing the required coding times.
To test the performance of the proposed method, the method of the present invention was compared to the original method. The experimental platform used HTM16.0, the test sequences were balloon, Kendo, newsapper, Poznan _ Hall2 and Poznan street, and the specific test environment settings are shown in table 1.
Table 1 test environment setup
The original encoding method of the HTM and the viewpoint synthesis optimization method proposed herein were compared according to published 3DV international universal test standards, and the comparison of the encoding time results is shown in table 2. As can be seen from table 1, the encoding method proposed herein effectively reduces the average complexity, which is 21.711% on average.
TABLE 2 comparison of code times for different methods
In addition to the comparison of encoding times, rate-distortion performance was also compared. The results are shown in Table 3. As can be seen from table 3, the coding method proposed herein maintains substantially the same coding performance as the original coding method except that the BDBR of the synthetic view of the uno _ Dancer sequence is high.
Table 3 shows the comparison of the rate-distortion performance of the coding method with the original method
As can be seen from table 2, the coding method proposed herein maintains substantially the same coding performance as the original coding method except that the BDBR of the synthetic view of the balloon sequence is high.
Claims (3)
1. A3D video fast coding method based on an adaptive segmentation skip rule is characterized by comprising the following steps:
step 1.1, selecting an encoding block with NxN size and an encoding mode being any encoding mode to be decided;
step 1.2, detecting the encoding blocks in sequence by using a block-based ES method and a line-based ES method, if the encoding block meets the block-based ES condition or the line-based ES condition, terminating the SVDC process of the whole encoding block in advance, and judging whether the next pixel line meets the block-based ES condition or the line-based ES condition or not;
step 1.3, if a certain pixel row does not meet the texture smoothness criterion and the depth zero distortion criterion, setting the SVDC of the current region to 0 if the texture smoothness criterion and the depth zero distortion criterion meet the judgment condition, and otherwise, calculating the SVDC value of the current region;
step 1.4, analyzing the optimal coding mode and the interval skipped by the optimal subsection of each sequence to be coded by a method based on motion vectors;
the texture smoothness criterion determining process comprises the following steps:
step 2.1, the texture smoothness criterion is determined asi-li+1≤T；
Step 2.2, calculating a threshold value T in the texture smoothness criterion in the step 2.1, wherein the calculation formula is
Wherein li,jAnd li+1,jIs a horizontal line of pixel pairs, NhAnd NW-1Representing the size of each module, IntraXXBlockNum represents one of 35 intra coding modes.
2. The adaptive segment skip rule based 3D video fast encoding method as claimed in claim 1, wherein the step 2.2 is specifically:
1) after a texture map is coded, the size N of all the texture maps which adopt a prediction mode as the final coding mode is recordedw×NhThe coding block of (2);
2) recording the horizontal adjacent pixels, l in a certain coding block in the step 1)i,jAnd li+1,jCalculating a measure difference value, and then calculating the sum of absolute errors of the whole row;
3) calculating the absolute error sum of all pixel rows in the step 2) and averaging to obtain the average horizontal direction brightness difference value of the coding block;
4) calculating an average value of all the horizontal direction brightness difference values obtained in the step 3) in the texture map, and defining a threshold value T in a texture smoothing criterion by the average value;
5) the texture map updates the threshold T once after each I frame is encoded, and all images in the same frame period use the same threshold T.
3. The adaptive segment skip rule based 3D video fast encoding method as claimed in claim 2, wherein the motion vector based method analyzes a process of an optimal encoding mode and an optimal segment skip interval for each sequence to be encoded, comprising the steps of:
3.1, observing the moving direction of a moving object in the current video sequence, and selecting 5 optimal modes from the intra-frame coding modes to be selected as candidate modes;
3.2, performing experiments on the 5 candidate modes selected in the step 3.1, and determining a final intra-frame coding mode by observing BD rate and PSNR values in experimental results;
3.3, the optimal segmentation interval is selected by performing multiple tests on the standard test sequence { balloon, Kendo, newsapper, Poznan _ Hall2, Poznan street }, and the size of the interval is generally selected to be {1,2,3,4,5}, and comparing the required coding time to determine which interval is the optimal interval.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710975608.XA CN107623858B (en) | 2017-10-19 | 2017-10-19 | 3D video fast coding method based on adaptive segmentation skipping rule |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710975608.XA CN107623858B (en) | 2017-10-19 | 2017-10-19 | 3D video fast coding method based on adaptive segmentation skipping rule |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107623858A CN107623858A (en) | 2018-01-23 |
| CN107623858B true CN107623858B (en) | 2020-08-28 |
Family
ID=61092604
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710975608.XA Active CN107623858B (en) | 2017-10-19 | 2017-10-19 | 3D video fast coding method based on adaptive segmentation skipping rule |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107623858B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102752596A (en) * | 2012-07-05 | 2012-10-24 | 深圳广晟信源技术有限公司 | Rate distortion optimization method |
| CN103200404A (en) * | 2013-04-07 | 2013-07-10 | 成都博盛信息技术有限公司 | Encode mode rapid prediction method based on macro block movement liveness |
| CN104506871A (en) * | 2014-11-23 | 2015-04-08 | 北京工业大学 | Three-dimensional (3D) video fast coding method based on high efficiency video coding (HEVC) |
-
2017
- 2017-10-19 CN CN201710975608.XA patent/CN107623858B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102752596A (en) * | 2012-07-05 | 2012-10-24 | 深圳广晟信源技术有限公司 | Rate distortion optimization method |
| CN103200404A (en) * | 2013-04-07 | 2013-07-10 | 成都博盛信息技术有限公司 | Encode mode rapid prediction method based on macro block movement liveness |
| CN104506871A (en) * | 2014-11-23 | 2015-04-08 | 北京工业大学 | Three-dimensional (3D) video fast coding method based on high efficiency video coding (HEVC) |
Non-Patent Citations (2)
| Title |
|---|
| Low Complexity Adaptive View Synthesis Optimization in HEVC Based 3D Video Coding;Ma Siwei等;《IEEE TRANSACTIONS ON MULTIMEDIA》;20140131;第16卷(第1期);第266-271页 * |
| 基于纹理平衡度的视点合成失真优化快速算法;窦环等;《通信学报》;20160331;第37卷(第3期);第98-106页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107623858A (en) | 2018-01-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101710993B (en) | Block-based self-adaptive super-resolution video processing method and system | |
| US10965959B2 (en) | Multi-frame quality enhancement for compressed video | |
| CN108347616B (en) | Depth prediction method and device based on optional time domain motion vector prediction | |
| KR101275066B1 (en) | System and method to process motion vectors of video data | |
| CN109068142B (en) | 360-degree video intra-frame prediction rapid decision-making method, device, coder-decoder and medium | |
| CN105430415B (en) | Fast encoding method in a kind of 3D HEVC deep video frames | |
| CN101378504B (en) | Method for estimating block matching motion of H.264 encode | |
| CN107087200B (en) | Skip coding mode advanced decision method for high-efficiency video coding standard | |
| CN108174208B (en) | Efficient video coding method based on feature classification | |
| CN104378643A (en) | Intra-frame prediction mode selection method and system of 3D (3-dimension) video plus depth image | |
| CN109587503B (en) | 3D-HEVC depth map intra-frame coding mode fast decision method based on edge detection | |
| Zhang et al. | Fast intra mode decision and block matching for HEVC screen content compression | |
| CN103856781A (en) | Self-adaptation threshold value video streaming multi-texture-direction error concealment method | |
| CN108449599B (en) | Video coding and decoding method based on surface transmission transformation | |
| US9609361B2 (en) | Method for fast 3D video coding for HEVC | |
| KR20060111528A (en) | Detection of local visual space-time details in a video signal | |
| CN109660811A (en) | A kind of quick HEVC inter-frame encoding methods | |
| CN113079373A (en) | Video coding method based on HEVC-SCC | |
| CN106878754A (en) | A kind of 3D video depths image method for choosing frame inner forecast mode | |
| CN107623858B (en) | 3D video fast coding method based on adaptive segmentation skipping rule | |
| CN109547798A (en) | A kind of quick HEVC inter-frame mode selecting method | |
| WO2020227911A1 (en) | Method for accelerating coding/decoding of hevc video sequence | |
| CN107483936B (en) | A kind of light field video inter-prediction method based on macro pixel | |
| JP2005348008A (en) | Moving picture coding method, moving picture coder, moving picture coding program and computer-readable recording medium with record of the program | |
| Chen et al. | CNN-based fast HEVC quantization parameter mode decision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20221031 Address after: 100012 817, Floor 8, No. 101, Floor 3 to 8, Building 17, Rongchuang Road, Chaoyang District, Beijing Patentee after: Beijing Ge Lei Information Technology Co.,Ltd. Address before: 100124 No. 100 Chaoyang District Ping Tian Park, Beijing Patentee before: Beijing University of Technology |