CN102158710B - Depth view encoding rate distortion judgment method for virtual view quality - Google Patents

Depth view encoding rate distortion judgment method for virtual view quality Download PDF

Info

Publication number
CN102158710B
CN102158710B CN 201110140492 CN201110140492A CN102158710B CN 102158710 B CN102158710 B CN 102158710B CN 201110140492 CN201110140492 CN 201110140492 CN 201110140492 A CN201110140492 A CN 201110140492A CN 102158710 B CN102158710 B CN 102158710B
Authority
CN
China
Prior art keywords
present encoding
encoding piece
dis
distortion
rec
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN 201110140492
Other languages
Chinese (zh)
Other versions
CN102158710A (en
Inventor
元辉
刘琚
孙建德
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong University
Original Assignee
Shandong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong University filed Critical Shandong University
Priority to CN 201110140492 priority Critical patent/CN102158710B/en
Publication of CN102158710A publication Critical patent/CN102158710A/en
Application granted granted Critical
Publication of CN102158710B publication Critical patent/CN102158710B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The invention discloses a depth view encoding rate distortion judgment method for virtual view quality. The method comprises the steps of performing predication on a current encoding block to obtain a predicated block; computing the difference between the current encoding block and the predicated block, and performing discrete cosine transform, quantification and entropy coding on the difference to obtain the code rate of the current encoding block; converting pixel gray values of the current encoding block and the predicated block into parallax values; then computing the distortion of the current encoding block according to the converted parallax values; and finally computing the rate-distortion cost of the current encoding block according to the distortion and the code rate. The method better reflects the influence on the composite virtual view quality by the compression distortion of a depth view according to the influence on the composite virtual view quality by the compression distortion of the depth view, improves the encoding code rate of three-dimensional video, and can be applied to the coding standard of three-dimensional videos.

Description

A kind of depth map encoding rate distortion determination methods towards the virtual view quality
Technical field
The present invention relates to a kind of rate distortion judgment criterion that improves depth map encoding efficient, belong to the depth map encoding technical field in the 3 D stereo video coding standard.
Background technology
The 3 D stereo video is meant that as main video applications in future the user can enjoy real 3 D stereo video content through 3 D stereo video display device.The correlation technique of 3 D video, such as, the technology such as demonstration of the collection of 3 D stereo video, 3 D stereo video coding, 3 D stereo video are paid close attention to widely.In order to promote the standardization of 3 D stereo video technology, 2002, (Motion Picture Experts Group MPEG) proposed any viewpoint TV (Free View Television, notion FTV) in Motion Picture Experts Group.It can provide vividly real, interactively 3 D stereo audiovisual system.The user can watch the 3 D stereo video of this angle from different angles, makes the user have and incorporates the sense of reality in the video scene.FTV can be widely used in fields such as broadcast communication, amusement, education, medical treatment and video monitoring.In order to make the user can watch 3 D stereo video at any angle, FTV system service end uses the video camera array of having demarcated to obtain the video on certain viewpoint.And, utilize corrected video information to generate the virtual view of virtual view through the virtual view synthetic technology to the video correction on the different points of view.MPEG suggestion is at present specifically used based on the degree of depth-image (Depth-Image Based Rendering, virtual view synthetic technology DIBR).Depth information is generally represented through depth map.The main process that virtual view synthesizes is following:
1). confirm to want the relative position of virtual view in video camera array.
2). confirm to be used for the texture video of synthetic virtual view.
3). confirm step 2) the corresponding depth map of texture video.
4). according to step 2) with 3) and in texture video and depth map, adopt the DIBR technology, synthetic virtual view.
The standardization effort of FTV is divided into two stages to carry out.Phase I is 2006 to 2008 the expansion scheme-MVC H.264/AVC (Multi-View Video Coding) that formulated by JVT (Joint Video Team, joint video code sets).MVC can encode to many viewpoints texture video.But to finally realize the function of FTV system, also must encode depth information.The standardization formulation work of FTV has at present got into second stage, i.e. 3DVC (Three Dimensional Video Coding).3DVC mainly pays close attention to the expression and the coding of depth information, and the combined coding of texture video and depth information.Among the 3DVC, depth information is represented through depth map.
The leading indicator of weighing the 3DVC performance is the quality of synthetic virtual view, and the encoder bit rate of texture video, depth map.The quality of virtual view:
Usually (Peak Signal-to-Noise Ratio PSNR) weighs the quality of video to adopt Y-PSNR.The computing formula of PSNR is shown below,
PSNR = 10 × log ( 255 2 MSE )
(1) MSE representes the mean square error between original view and the synthetic virtual view in the formula, is used for weighing the distortion of virtual view, and the coding distortion of the coding distortion of texture video, depth map.
In practical application, the view of virtual view is non-existent, does not also promptly have original view.But,, at first adopt the existing texture video of un-encoded and the corresponding synthetic virtual view V of depth map thereof therefore for weighing the performance of 3DVC because 3DVC mainly pays close attention to coding efficiency Orig, the depth map that adopts the texture video of the reconstruction after process is encoded and the back of encoding to rebuild then synthesizes virtual view V Rec, at last through calculating V RecWith V OrigBetween MSE, and then obtain PSNR, to weigh the performance of 3DVC.
Fig. 1 has provided existing rate distortion criterion calculation flow chart H.264/AVC, in video encoding standard H.264/AVC, at first the present encoding piece is predicted, calculates Mean Square Error MSE between present encoding piece and the predict blocks then as distortion D H264Then the grey scale pixel value of present encoding piece is deducted the grey scale pixel value of predict blocks, obtain prediction difference, prediction difference is carried out discrete cosine transform, quantize, entropy coding obtains the encoder bit rate R of present encoding piece then H264, adopt the rate distortion costs J of computes present encoding piece at last H264: J H264=D H266+ λ R H264, wherein λ is a Lagrange multiplier.
H.264/AVC the rate distortion judgment criterion that adopts is not considered the influence of the compression artefacts of depth map to the quality of synthetic virtual view, and the efficient that therefore adopts rate distortion judgment criterion H.264/AVC that depth map is encoded is not high.Can know that by theory analysis the distortion of synthetic virtual view is directly by the parallax decision, and the quality of depth map can have influence on the accuracy of parallax.In the compression process of depth map, the distortion of depth map itself not necessarily can cause the inaccurate of parallax; But when the depth map entirely accurate, parallax also is an entirely accurate.
Summary of the invention
Do not consider of the influence of the compression artefacts of depth map to the quality of synthetic virtual view to the rate distortion judgment criterion that H.264/AVC adopts; The present invention proposes a kind of rate distortion determination methods towards the virtual view quality that is applicable to depth map encoding according to the influence to synthetic virtual view quality of the compression artefacts of depth map.
The present invention towards the depth map encoding rate distortion determination methods of virtual view quality is:
The present encoding piece is predicted, obtained predict blocks; Calculate the difference of present encoding piece and predict blocks, and difference is carried out discrete cosine transform, quantification and entropy coding, obtain the code check of present encoding piece; Convert the grey scale pixel value of present encoding piece and predict blocks into parallax value; Calculate the distortion of present encoding piece then according to the parallax value after the conversion; At last according to the distortion of gained and the rate distortion costs of code check calculating present encoding piece; Concrete steps are following:
(1) the present encoding piece of depth map is predicted that obtain the predict blocks of present encoding piece, the gray value of each pixel in the present encoding piece is used L iExpression, each grey scale pixel value in the predict blocks is used L P, iExpression, i ∈ 1 ... N}, N are the pixel quantity in the present encoding piece;
The present encoding piece of depth map is carried out forecast method to be adopted in the various frames of H.264/AVC standard code or inter-frame prediction method.
(2) with L iWith L P, iSubtract each other, obtain the difference of present encoding piece, and difference is carried out discrete cosine transform, quantification and entropy coding, confirm the code check R of present encoding piece;
(3) the entropy coding data are decoded, and the data of decoding are carried out inverse quantization and inverse discrete cosine transformation, rebuild difference signal;
(4) with the difference signal addition of present encoding piece and reconstruction, rebuild current block;
(5) convert each grey scale pixel value in the reconstruction current block of present encoding piece and step (4) gained into parallax value respectively according to following formula,
dis i = f · b · [ L i 255 · ( 1 Z near - 1 Z far ) + 1 Z far ] ,
dis rec , i = f · b · [ L rec , i 255 · ( 1 Z near - 1 Z far ) + 1 Z far ] ,
Dis wherein iThe pairing parallax value of gray value of i pixel in the expression present encoding piece, dis Rec, iThe pairing parallax value of gray value of i pixel in the current block, L are rebuild in expression Rec, iThe grey scale pixel value of current block is rebuild in expression, and f representes focus of camera, and b is the spacing between the adjacent camera, Z NearThe real depth value of the object point that expression is nearest apart from video camera, Z FarExpression is apart from the real depth value of video camera object point farthest;
(6) according to dis iWith dis Rec, iCalculate the distortion D of present encoding piece,
Figure BDA0000064445220000033
Wherein, work as dis iWith dis Rec, iWhen equating, δ iEqual 0, work as dis iWith dis Rec, iWhen unequal, δ iEqual L i-L Rec, i
(7) according to the code check R of the present encoding piece of step (2) gained and the distortion D of step (6) gained, and according to the rate distortion costs J of computes present encoding piece, J=D+ λ R, wherein λ is a Lagrange multiplier.Lagrange multiplier λ adopts the numerical value of H.264/AVC standard code.
The present invention confirms rate distortion according to the compression artefacts of depth map to the influence of synthetic virtual view quality, can better reflect the influence of the compression artefacts of depth map to synthetic virtual view quality, improves the efficient of 3 D stereo video coding; Only needing degree of depth grayvalue transition is corresponding parallax value, and according to the parallax value calculated distortion, calculates and simply be easy to realization, can not increase encoder complexity.
Description of drawings
Fig. 1 is a rate distortion criterion calculation flow chart H.264/AVC;
Fig. 2 is a depth map encoding rate distortion determination methods flow chart of the present invention;
Fig. 3 is the rate distortion curve comparison diagram that adopts respectively after method of the present invention and method are H.264/AVC encoded to depth map.
Embodiment
Depth map encoding rate distortion determination methods towards the virtual view quality of the present invention, as shown in Figure 2, specifically comprise the steps:
(1) the present encoding piece of depth map is predicted that obtain the predict blocks of present encoding piece, the gray value of each pixel in the present encoding piece is used L iExpression, each grey scale pixel value in the predict blocks is used L P, iExpression, i ∈ 1 ... N}, N are the pixel quantity in the present encoding piece.The present encoding piece of depth map is carried out forecast method to be adopted in the various frames of H.264/AVC standard code or inter-frame prediction method.
(2) with L iWith L P, iSubtract each other, obtain the difference of present encoding piece, and difference is carried out discrete cosine transform, quantize, entropy coding is confirmed the code check R of present encoding piece;
(3) the entropy coding data are decoded, and the data of decoding are carried out inverse quantization and inverse discrete cosine transformation, rebuild difference signal;
(4) with the difference signal addition of present encoding piece and reconstruction, rebuild current block;
(5) convert each grey scale pixel value in the predict blocks of present encoding piece and step 1 gained into parallax value respectively according to following formula,
dis i = f · b · [ L i 255 · ( 1 Z near - 1 Z far ) + 1 Z far ] ,
dis rec , i = f · b · [ L rec , i 255 · ( 1 Z near - 1 Z far ) + 1 Z far ] ,
Dis wherein iThe pairing parallax value of gray value of i pixel in the expression present encoding piece, dis Rec, iThe pairing parallax value of gray value of i pixel in the expression reconstructed block, L Rec, iThe grey scale pixel value of expression reconstructed block, f representes focus of camera, b is the spacing between the adjacent camera, Z NearThe real depth value of the object point that expression is nearest apart from video camera, Z FarExpression is apart from the real depth value of video camera object point farthest;
(6) according to dis iWith dis Rec, iCalculate the distortion D of present encoding piece,
D = Σ i = 1 N δ i 2 ,
Wherein, work as dis iWith dis Rec, iWhen equating, δ iEqual 0, work as dis iWith dis Rec, iWhen unequal, δ iEqual L i-L P, i
(7) according to the code check R of the present encoding piece of step 2 gained and the distortion D of step 6 gained, and according to the rate distortion costs J of computes present encoding piece,
J＝D+λ·R，
Wherein λ is a Lagrange multiplier.Lagrange multiplier λ adopts the numerical value of H.264/AVC standard code.
Effect of the present invention can further specify through experiment.
Experiment test under the different quantized parameters conditions, the encoder bit rate after adopting the present invention that depth map is encoded and the objective quality PSNR of synthetic virtual view.Fig. 3 has compared the rate distortion curve after adopting the present invention and existing method H.264/AVC to depth map encoding.Wherein Fig. 3 (a) is the experimental result that the depth map of 3 D video sequence B ookarrival is encoded; Fig. 3 (b) is the experimental result that the depth map of 3 D video sequence Kendo is encoded, and Fig. 3 (c) is the experimental result that the depth map of 3 D video sequence Lovebirdl is encoded.Visible by Fig. 3, and H.264/AVC compare, adopt the present invention to encode after, under the identical condition of the encoder bit rate of depth map, the objective quality of synthetic virtual view is higher, explains that the present invention has improved the code efficiency of depth map.As far as 3 D video sequence B ookarrival, the objective quality of synthetic virtual view on average increases 0.215dB; As far as 3 D video sequence Kendo, the objective quality of synthetic virtual view on average increases 0.237dB; As far as 3 D video sequence Lovebird1, the objective quality of synthetic virtual view on average increases 0.45dB.

Claims (3)

1. depth map encoding rate distortion determination methods towards the virtual view quality is characterized in that:
The present encoding piece is predicted, obtained predict blocks; Calculate the difference of present encoding piece and predict blocks, and difference is carried out discrete cosine transform, quantification and entropy coding, obtain the code check of present encoding piece; Convert the grey scale pixel value of present encoding piece and predict blocks into parallax value; Calculate the distortion of present encoding piece then according to the parallax value after the conversion; At last according to the distortion of gained and the rate distortion costs of code check calculating present encoding piece; Concrete steps are following:
(1) the present encoding piece of depth map is predicted, obtained the predict blocks of present encoding piece, the gray value of i pixel in the present encoding piece representes that with Li i grey scale pixel value in the predict blocks used L P, iExpression, i ∈ 1 ... N}, N are the pixel quantity in the present encoding piece;
(2) with L iWith L P, iSubtract each other, obtain the difference of present encoding piece, and difference is carried out discrete cosine transform, quantification and entropy coding, confirm the code check R of present encoding piece;
(3) the entropy coding data are decoded, and the data of decoding are carried out inverse quantization and inverse discrete cosine transformation, rebuild difference signal;
(4) with the difference signal addition of present encoding piece and reconstruction, rebuild current block;
(5) convert each grey scale pixel value in the reconstruction current block of present encoding piece and step (4) gained into parallax value respectively according to following formula,
dis i = f · b · [ L i 255 · ( 1 Z near - 1 Z far ) + 1 Z far ] ,
dis rec , i = f · b · [ L rec , i 255 · ( 1 Z near - 1 Z far ) + 1 Z far ] ,
Dis wherein iThe pairing parallax value of gray value of i pixel in the expression present encoding piece, dis Rec, iThe pairing parallax value of gray value of i pixel in the current block, L are rebuild in expression Rec, iI grey scale pixel value in the current block rebuild in expression, and f representes focus of camera, and b is the spacing between the adjacent camera, Z NearThe real depth value of the object point that expression is nearest apart from video camera, Z FarExpression is apart from the real depth value of video camera object point farthest;
(6) according to dis iWith dis Rec, iCalculate the distortion D of present encoding piece,
Figure FDA00001658040000013
Wherein, work as dis iWith dis Rec, iWhen equating, δ iEqual 0, work as dis iWith ds Rec, iWhen unequal, δ iEqual L i-L Rec, i
(7) according to the code check R of the present encoding piece of step (2) gained and the distortion D of step (6) gained, and according to the rate distortion costs J of computes present encoding piece, J=D+ λ R, wherein λ is a Lagrange multiplier.
2. the depth map encoding rate distortion determination methods towards the virtual view quality according to claim 1; It is characterized in that, in the said step (1) the present encoding piece of depth map is carried out forecast method and adopt in the various frames of H.264/AVC standard code or inter-frame prediction method.
3. the depth map encoding rate distortion determination methods towards the virtual view quality according to claim 1 is characterized in that the Lagrange multiplier λ in the said step (7) adopts the numerical value of H.264/AVC standard code.
CN 201110140492 2011-05-27 2011-05-27 Depth view encoding rate distortion judgment method for virtual view quality Expired - Fee Related CN102158710B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 201110140492 CN102158710B (en) 2011-05-27 2011-05-27 Depth view encoding rate distortion judgment method for virtual view quality

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 201110140492 CN102158710B (en) 2011-05-27 2011-05-27 Depth view encoding rate distortion judgment method for virtual view quality

Publications (2)

Publication Number Publication Date
CN102158710A CN102158710A (en) 2011-08-17
CN102158710B true CN102158710B (en) 2012-12-26

Family

ID=44439851

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 201110140492 Expired - Fee Related CN102158710B (en) 2011-05-27 2011-05-27 Depth view encoding rate distortion judgment method for virtual view quality

Country Status (1)

Country Link
CN (1) CN102158710B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013113134A1 (en) * 2012-02-02 2013-08-08 Nokia Corporation An apparatus, a method and a computer program for video coding and decoding
CN102595166B (en) * 2012-03-05 2014-03-05 山东大学 Lagrange factor calculation method applied for depth image encoding
CN103826135B (en) * 2013-12-24 2017-02-08 浙江大学 Three-dimensional video depth map coding method based on just distinguishable parallax error estimation
CN104506856B (en) * 2015-01-14 2017-03-22 山东大学 Method of estimating quality of virtual view applicable to 3D (Three-dimensional) video system

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101478677A (en) * 2008-12-24 2009-07-08 西安交通大学 Scalable multi-description video encoding structure design method based on code rate control
CN101888566A (en) * 2010-06-30 2010-11-17 清华大学 Estimation method of distortion performance of stereo video encoding rate
CN102065296A (en) * 2011-01-06 2011-05-18 宁波大学 Three-dimensional video coding method

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8477853B2 (en) * 2006-12-14 2013-07-02 Thomson Licensing Method and apparatus for encoding and/or decoding bit depth scalable video data using adaptive enhancement layer prediction

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101478677A (en) * 2008-12-24 2009-07-08 西安交通大学 Scalable multi-description video encoding structure design method based on code rate control
CN101888566A (en) * 2010-06-30 2010-11-17 清华大学 Estimation method of distortion performance of stereo video encoding rate
CN102065296A (en) * 2011-01-06 2011-05-18 宁波大学 Three-dimensional video coding method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
杨海涛，等.三维电视***中的视频-深度联合预测编码.《光学学报》.2009,第29卷(第12期), *

Also Published As

Publication number Publication date
CN102158710A (en) 2011-08-17

Similar Documents

Publication Publication Date Title
CN102209243B (en) Depth map intra prediction method based on linear model
CN101159875B (en) Double forecast video coding/decoding method and apparatus
CN101835056B (en) Allocation method for optimal code rates of texture video and depth map based on models
CN103329522B (en) For the method using dictionary encoding video
US10009611B2 (en) Visual quality measure for real-time video processing
CN102281446B (en) Visual-perception-characteristic-based quantification method in distributed video coding
CN102413353B (en) Method for allocating code rates of multi-view video and depth graph in stereo video encoding process
Yuan et al. Rate distortion optimized inter-view frame level bit allocation method for MV-HEVC
KR20120000485A (en) Apparatus and method for depth coding using prediction mode
CN107277550A (en) Multi-view signal codec
CN102625102B (en) H.264/scalable video coding medius-grain scalability (SVC MGS) coding-oriented rate distortion mode selection method
CN102158710B (en) Depth view encoding rate distortion judgment method for virtual view quality
CN107864380A (en) 3D HEVC fast intra-mode prediction decision-making techniques based on DCT
CN103546758A (en) Rapid depth map sequence interframe mode selection fractal coding method
CN102308583A (en) Apparatus and method for encoding and decoding multi-view image
CN102291582A (en) Distributed video encoding method based on motion compensation refinement
US20140340478A1 (en) Method and apparatus for depth video coding using endurable view synthesis distortion
CN101854555B (en) Video coding system based on prediction residual self-adaptation regulation
CN104159095A (en) Code rate control method for multi-view texture video and depth map coding
CN101309404A (en) Resolution descending video transcoding method and transcoding device thereof
CN106534855B (en) A kind of Lagrange factor calculation method towards SATD
CN102595166B (en) Lagrange factor calculation method applied for depth image encoding
CN101883283A (en) Control method for code rate of three-dimensional video based on SAQD domain
CN102333230A (en) Method for improving quality of synthetized virtual views in three-dimensional video system
CN102572440B (en) Multi-viewpoint video transmission method based on depth map and distributed video coding

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20121226

Termination date: 20160527