CN106791773A - A kind of novel view synthesis method based on depth image - Google Patents

Abstract

Texture maps at the reference view of left and right and depth map are carried out three-dimension varying by a kind of novel view synthesis method based on depth image；The edge of object in search left and right reference view depth map, three-dimension varying is carried out at new viewpoint by edge pixel, then erases corresponding depth pixel point at new viewpoint；Depth map to obtaining carries out medium filtering, and the image after after filtering is contrasted with the depth map obtained through three-dimension varying, marks the pixel of change；Pixel to being labeled carries out back projection, projects at original reference view, then the pixel value in initial reference texture maps is assigned in new viewpoint image, with labeled pixel point coordinates identical pixel；The occlusion area of the new viewpoint image to obtaining enters row interpolation again；Cavity to remaining is repaired, and obtains final new viewpoint image.The present invention effectively eliminates cavity and ghost image in new viewpoint image, and experiment effect is good, and the new viewpoint image of generation meets human eye viewing effect.

Description

A kind of novel view synthesis method based on depth image

Technical field

It is used for the present invention relates to the fields such as image procossing, numerical analysis, three-dimensional reconstruction, computer science, especially one kind The rendering intent based on depth map of binocular camera new viewpoint virtual image synthesis.

Background technology

The synthesis of binocular camera new viewpoint virtual image is that a kind of utilization is demarcated inside and outside existing visual point image and video camera Parameter, the technology being reconstructed to image at new viewpoint.Its main method is to combine camera calibration parameter, and has been regarded Corresponding texture maps are projected and re-projection by the depth information of dot image, construct the image at New Century Planned Textbook.It is new building During viewpoint, can there is a problem of it is more, it is such as new and into image occur crack, cavity, ghost image, regional occlusion and The phenomenons such as object incompleteness, the presence of these problems have impact on the quality of new viewpoint image, and the interaction to user is generated greatly Influence.

The content of the invention

In order to overcome problem, the experiment effect of the various influence picture qualities produced in existing new viewpoint image synthesizing procedure Deficiency really poor, that human eye viewing effect cannot be met, the problem for proposing a kind of various influence picture qualities of elimination of the invention, Experiment effect is good, effectively meet the novel view synthesis method based on depth image of human eye viewing effect.

The technical solution adopted in the present invention is as follows：

A kind of novel view synthesis method based on depth image, described visual point synthesizing method is comprised the following steps：

(1) three-dimension varying is carried out to the texture maps at the reference view of left and right and depth map, conversion process is as follows：

1.1) image is stored into each pixel in coordinate system, with reference to corresponding depth information, projects to world's seat In mark system：

P_Wi=(X_Wi, Y_Wi, Z_Wi)^T=(K_iR_i)^-1(λ_iP_i+K_iR_iC_i), (i=l, r)

Wherein P_Wi={ P_Wi=(X_Wi, Y_Wi, Z_Wi)^T| i=l, r } represent that the pixel of reference view image current position exists Three-dimensional coordinate in world coordinate system, l and r represent left and right, X respectively_WiAnd Y_WiTransverse and longitudinal coordinate respectively in world coordinate system,It is depth, d_iIt is the gray value of current position depth map, MinZ_iAnd MaxZ_iPoint Minimum and maximum gray value that Wei be in the corresponding depth image of present frame,Represent The internal reference matrix of video camera, f_xi={ f_xi∈ R | i=l, r } and f_yi={ f_yi∈ R | i=l, r } it is respectively left and right cameras x directions With the focal length on y directions, R is real number, u_0i={ u_0i∈ R | i=l, r } and v_0i={ v_0i∈ R | i=l, r } it is respectively left and right shooting Coordinate of the principal point of machine in image storage coordinate system, R_i={ R_i∈R^3×3| i=l, r } represent video camera spin matrix, R^3×3 Represent the real number field space of matrices of dimension 3 × 3, λ_i={ λ_i=F_i31X_Wi+F_i32Y_Wi+F_i33Z_Wi+F_i34| i=l, r } it is video camera Homogeneous scale factor,It is projection matrix, T_i={ T_i∈R^3×1| i=l, r } be The translation matrix of video camera, P_i={ P_i=(u_i, v_i, 1)^T| i=l, r } it is the homogeneous graph picture storage coordinate of reference view, u_i= {u_i∈ R | i=l, r } and v_i={ v_i∈ R | i=l, r } it is respectively horizontal, vertical seat of the current pixel point in image storage coordinate system Mark, C_iIt is the centre coordinate of video camera；

Same treatment is done to corresponding depth map：

D_Wi=(X_Wi, Y_Wi, Z_Wi) T=(K_iR_i)^-1(λ_iD_i+K_iR_iC_i), (i=l, r)

Wherein D_Wi={ D_Wi=(X_Wi, Y_Wi, Z_Wi)^T| i=l, r } represent the pixel of reference view depth map current position Three-dimensional coordinate in world coordinate system, D_i={ D_i=(u_0i, v_0i, 1)^T| i=l, r are the homogeneous graph picture of reference view depth map Storage coordinate；

1.2) by by projecting each point for obtaining, the inside and outside ginseng of virtual video camera at combining target new viewpoint for the first time Number, and depth information, in projecting to the plane of delineation coordinate system of new viewpoint again：

P_Newi=(K_NewiR_NewiP_Wi-K_NewiR_NewiC_Newi)/λ_Newi, (i=l, r)

Wherein P_Newi=P_NewI=(u_Newi, v_Newi, 1) | i=l, r } it is P_iThe coordinate of corresponding points, u at new viewpoint_Newi= {u_Newi∈ R | i=l, r } and v_Newi={ v_Newi∈ R | i=l, r be respectively current pixel point image storage coordinate system in Horizontal, ordinate, K_Newi、R_Newi、C_Newi、λ_NewiRepresent that the internal reference matrix of virtual video camera at new viewpoint, spin matrix, center are sat It is marked with and next scale factor；

Corresponding new viewpoint depth map is expressed as：

D_Newi=(K_NewiR_NewiD_Wi-K_NewiR_NewiC_Newi)/λ_Newi, (i=l, r)

Wherein D_Newi={ D_Newi=(u_Newi, v_Newi, 1) | i=l, r } it is D_iThe coordinate of corresponding points at new viewpoint；

(2) edge of object in the reference view depth map of left and right is searched for, the method in (1) is utilized, edge pixel is carried out Then three-dimension varying is erased corresponding depth pixel point at new viewpoint at new viewpoint：

D_{New_edgei}=0, (i=l, r)

Wherein D_{New_edgei}It is the target edges pixel after three-dimension varying；

(3) using 3 × 3 template, the depth map to obtaining carries out medium filtering, the crackle in removal depth image with it is thin Small leak, and the image after after filtering is contrasted with the depth map obtained through three-dimension varying, mark the picture of change Vegetarian refreshments：

I_Newi=S (M (D_{New_Imgi}), D_{New_Imgi}), (i=l, r)

Wherein I_NewiIt is the pixel being labeled, M is 3 × 3 medium filtering function, D_{New_Imgi}Be by step (1) and The depth map at new viewpoint obtained after step (2) treatment, S for comparing two images, and is marked to compare labeling function The pixel of change；

(4) crack produced because of three-dimension varying is repaired, mending course is as follows：

4.1) back projection is carried out to the pixel being labeled, is projected at original reference view：

P_INewi=W (I_Newi), (i=l, r)

Wherein, P_INewiRepresent in the new viewpoint image obtained by back projection, the pixel of cracks, W represents step (1) The three-dimension varying of description；

4.2) again by the pixel value in initial reference texture maps, it is assigned in new viewpoint image, with labeled pixel point coordinates Identical pixel；

(5) using the new viewpoint image by being obtained after step (4) treatment, the cavity to image occlusion area is inserted Value：

P_{In_Img}=IN (P_{New_Imgl}, P_{New_Imgr})

Wherein, P_{IN_Img}It is by the new viewpoint texture maps after interpolation, P_{New_Imgl}Represent by left reference picture by step (1) the new viewpoint texture maps obtained after all treatment of-(4), P_{New_Imgr}Represent by right reference picture by step (1)-(4) All treatment after obtain, the new viewpoint texture maps at same viewpoint, IN is interpolating function；

(6) the inpaint methods of Telea propositions in OpenCV built-in functions are called, the cavity to remaining is repaired, obtained Final new viewpoint image：

P_{New_Img}=inpaint (P_In__Img)

Wherein inpaint is OpenCV built-in functions, P_{New_Img}It is the final new viewpoint texture image for obtaining.

Further, in the step (2), find target edges pixel the step of it is as follows：

Wherein, P_ImgReference picture is represented, D represents the depth map of reference view, T_dIt is self-defined threshold value, experiments verify that, The 1/4 of maximum depth value typically is taken, when pixel meets above formula, represents that the point is ghost edge.

Further, the processing procedure of the comparing labeling function S in step (3) is comprised the following steps：

3.1) gray value of each pixel in the depth map before and after contrast medium filtering；

3.2) coordinate of the pixel that record gray value is differed.

Further, the step 4.2) in, the process for mending a split is as follows：

Using the mark point coordinates obtained in 4.1), the pixel value of mark point position in initial reference texture maps is chosen, assigned To corresponding mark pixel in new viewpoint image, the pixel of rest position keeps constant：

Wherein P_{New_Imgi}For at new viewpoint by the texture maps after crack repairing, P_ImgiIt is reference texture figure, P_{ONew_Imgi} It is the new viewpoint texture maps obtained after being processed by step (1) and step (2), mark represents whether current pixel point is marked Note, mark represents that current point is labeled point when being 1.

Further, in the step (5), the expression formula of IN interpolating functions is as follows：

WhereinIt is proportion parameter, T_{New_t}、T_{l_t}And T_{r_t}Respectively new viewpoint, The translation vector of left reference view and right reference view in camera coordinate system, O_i={ O_i(u, v) | i=l, r } represent figure It is 1 if in the presence of cavity as whetheing there is cavity at (u, v) place, expression formula is as follows：

Wherein Z_i={ Z_i(u, v) | i=l, r } it is depth value of the new viewpoint at (u, v) place, T_hIt is threshold value.

Beneficial effects of the present invention are embodied in：Ghost profile is eliminated present invention employs three-dimension varying method, is employed anti- To sciagraphy, the crack in new viewpoint depth map is effectively eliminated, employ left and right left and right viewpoint interpolation method, filled up blocked area Domain, and using the inpaint methods in OpenCV storehouses, remaining cavity has effectively been filled up, experiment effect is good, the new viewpoint of generation Image meets human eye viewing effect.

Specific embodiment

The present invention will be further described below.

A kind of novel view synthesis method based on depth image, described visual point synthesizing method is comprised the following steps：

(1) three-dimension varying is carried out to the texture maps at the reference view of left and right and depth map, conversion process is as follows：

1.1) image is stored into each pixel in coordinate system, with reference to corresponding depth information, projects to world's seat In mark system：

Wherein P_Wi={ P_Wi=(X_Wi, Y_Wi, Z_Wi)^T| i=l, r } represent that the pixel of reference view image current position exists Three-dimensional coordinate in world coordinate system, 1 and r represents left and right, X respectively_WiAnd Y_WiTransverse and longitudinal coordinate respectively in world coordinate system,It is depth, d_iIt is the gray value of current position depth map, MinZ_iAnd MaxZ_iPoint Minimum and maximum gray value that Wei be in the corresponding depth image of present frame,Represent The internal reference matrix of video camera, f_xi={ f_xi∈ R | i=l, r } and f_yi={ f_yi∈ R | i=l, r } it is respectively left and right cameras x directions With the focal length on y directions, R is real number, u_0i={ u_0i∈ R | i=l, r } and v_0i={ v_0i∈ R | i=l, r } it is respectively left and right shooting Coordinate of the principal point of machine in image storage coordinate system, R_i={ R_i∈R^3×3| i=l, r } represent video camera spin matrix, R^3×3 Represent the real number field space of matrices of dimension 3 × 3, λ_i={ λ_i=F_i31X_Wi+F_i32Y_Wi+F_i33Z_Wi+F_i34| i=l, r } it is video camera Homogeneous scale factor,It is projection matrix, T_i={ T_i∈R^3×1| i=l, r } It is the translation matrix of video camera, P_i={ P_i=(u_i, v_i, 1)^T| i=l, r } it is the homogeneous graph picture storage coordinate of reference view, u_i= {u_i∈ R | i=l, r } and v_i={ v_i∈ R | i=l, r } it is respectively horizontal, vertical seat of the current pixel point in image storage coordinate system Mark, C_iIt is the centre coordinate of video camera；

Same treatment is done to corresponding depth map：

D_Wi=(X_Wi, Y_Wi, Z_Wi) T=(K_iR_i)^-1(λ_iD_i+K_iR_iC_i), (i=l, r)

Wherein D_Wi={ D_Wi=(X_Wi, Y_Wi, Z_Wi)^T| i=l, r } represent the pixel of reference view depth map current position Three-dimensional coordinate in world coordinate system, D_i={ D_i∈(u_0i, v_0i, 1)^T| i=l, r } it is the homogeneous graph of reference view depth map As storage coordinate；

1.2) by by projecting each point for obtaining, the inside and outside ginseng of virtual video camera at combining target new viewpoint for the first time Number, and depth information, in projecting to the plane of delineation coordinate system of new viewpoint again：

P_Newi=(K_NewiR_NewiP_Wi-K_NewiR_NewiC_Newi)/λ_Newi, (i=l, r)

Wherein P_Newi={ P_Newi=(u_Newi, v_Newi, 1) | i=l, r } it is P_iThe coordinate of corresponding points, u at new viewpoint_Newi= {u_Newi∈ R | i=l, r } and v_Newi={ v_Newi∈ R | i=l, r be respectively current pixel point image storage coordinate system in Horizontal, ordinate, K_Newi、R_Newi、C_Newi、λ_NewiRepresent that the internal reference matrix of virtual video camera at new viewpoint, spin matrix, center are sat It is marked with and next scale factor；

Corresponding new viewpoint depth map is expressed as：

D_Newi=(K_NewiR_NewiD_Wi-K_NewiR_NewiC_Newi)/λ_Newi, (i=l, r)

Wherein D_Newi=D_Newi=(u_Newi, v_Newi, 1) | i=l, r } it is D_iThe coordinate of corresponding points at new viewpoint；

(2) edge of object in the reference view depth map of left and right is searched for, the method in (1) is utilized, edge pixel is carried out Then three-dimension varying is erased corresponding depth pixel point at new viewpoint at new viewpoint：

D_{New_edgei}=0, (i=l, r)

Wherein D_{New_edgei}It is the target edges pixel after three-dimension varying；

The process for finding target edges pixel is as follows：

Wherein, P_ImgReference picture is represented, D represents the depth map of reference view, T_dIt is self-defined threshold value, experiments verify that, The 1/4 of maximum depth value typically is taken, when pixel meets above formula, represents that the point is ghost edge；

(3) using 3 × 3 template, the depth map to obtaining carries out medium filtering, the crackle in removal depth image with it is thin Small leak, and the image after after filtering is contrasted with the depth map obtained through three-dimension varying, mark the picture of change Vegetarian refreshments：

I_Newi=S (M (D_{New_Imgi}), D_New__Imgi), (i=l, r)

Wherein I_NewiIt is the pixel being labeled, M is 3 × 3 medium filtering function, D_{New_Imgi}Be by step (1) and The depth map at new viewpoint obtained after step (2) treatment, S for comparing two images, and is marked to compare labeling function The pixel of change；

The processing procedure for comparing labeling function S is comprised the following steps：

3.1) gray value of each pixel in the depth map before and after contrast medium filtering；

3.2) coordinate of the pixel that record gray value is differed；

(4) crack produced because of three-dimension varying is repaired, mending course is as follows：

4.1) back projection is carried out to the pixel being labeled, is projected at original reference view：

PI_Newi=W (I_Newi), (i=l, r)

Wherein, P_INewiRepresent in the new viewpoint image obtained by back projection, the pixel of cracks, W represents step (1) The three-dimension varying of description；

4.2) again by the pixel value in initial reference texture maps, it is assigned in new viewpoint image, with labeled pixel point coordinates Identical pixel；

The process for mending a split is as follows：Using the mark point coordinates obtained in 4.1), the acceptance of the bid of initial reference texture maps is chosen The pixel value of note point position, is assigned to corresponding mark pixel in new viewpoint image, and the pixel of rest position keeps constant：

Wherein P_{New_Imgi}For at new viewpoint by the texture maps after crack repairing, P_ImgiIt is reference texture figure, P_{ONew_Imgi} It is the new viewpoint texture maps obtained after being processed by step (1) and step (2), mark represents whether current pixel point is marked Note, mark represents that current point is labeled point when being 1；

(5) using the new viewpoint image by being obtained after step (4) treatment, the cavity to image occlusion area is inserted Value：

P_{In_Img}=IN (P_{New_Imgl,}P_{New_Imgr})

Wherein, P_{In_Img}It is by the new viewpoint texture maps after interpolation, P_{New_Imgl}Represent by left reference picture by step (1) the new viewpoint texture maps obtained after all treatment of-(4), P_{New_Imgr}Represent by right reference picture by step (1)- (4) obtained after all treatment, the new viewpoint texture maps at same viewpoint, IN is interpolating function；

The expression formula of IN interpolating functions is as follows：

WhereinIt is proportion parameter, T_{New_t}、T_{l_t}And T_{r_t}Respectively new viewpoint, a left side The translation vector of reference view and right reference view in camera coordinate system, O_i={ O_i(u, v) | i=l, r } represent image Cavity is whether there is at (u, v) place, is 1 if in the presence of cavity, expression formula is as follows：

Wherein Z_i={ Z_i(u, v) | i=l, r } it is depth value of the new viewpoint at (u, v) place, T_hIt is threshold value；

(6) the inpaint methods of Telea propositions in OpenCV built-in functions are called, the cavity to remaining is repaired, obtained Final new viewpoint image：

P_{New_Img}=inpaint (P_{In_Img})

Wherein inpaint is OpenCV built-in functions, P_{New_Img}It is the final new viewpoint texture image for obtaining.

Claims (5)

1. a kind of novel view synthesis method based on depth image, it is characterised in that described visual point synthesizing method includes following Step：

(1) three-dimension varying is carried out to the texture maps at the reference view of left and right and depth map, conversion process is as follows：

1.1) image is stored into each pixel in coordinate system, with reference to corresponding depth information, projects to world coordinate system In：

P_Wi=(X_Wi, Y_Wi, Z_Wi)^T=(K_iR_i)^-1(λ_iP_i+K_iR_iC_i), (i=l, r)

Wherein P_Wi={ P_Wi=(X_Wi, Y_Wi, Z_Wi)^T| i=l, r } represent the pixel of reference view image current position in the world Three-dimensional coordinate in coordinate system, l and r represent left and right, X respectively_WiAnd Y_WiTransverse and longitudinal coordinate respectively in world coordinate system,It is depth, d_iIt is the gray value of current position depth map, MinZ_iAnd MaxZ_iPoint Minimum and maximum gray value that Wei be in the corresponding depth image of present frame,Represent The internal reference matrix of video camera, f_xi={ f_xi∈ R | i=l, r } and f_yi={ f_yi∈ R | i=l, r } it is respectively left and right cameras x directions With the focal length on y directions, R is real number, u_0i={ u_0i∈ R | i=l, r } and v_0i={ v_0i∈ R | i=l, r } it is respectively left and right shooting Coordinate of the principal point of machine in image storage coordinate system, R_i=R_i{∈R^3×3| i=l, r } represent video camera spin matrix, R^3×3 Represent the real number field space of matrices of dimension 3 × 3, λ_i=λ_i=F_i31X_Wi+F_i32Y_Wi+F_i33Z_Wi+F_i34| i=l, r } it is video camera Homogeneous scale factor,It is projection matrix, T_i={ T_i∈R^3×1| i=l, r are The translation matrix of video camera, P_i={ P_i(u_i, v_i, 1)^T| i=l, r } it is the homogeneous graph picture storage coordinate of reference view, u_i={ u_i ∈ R | i=l, r } and v_i={ v_i∈ R | i=l, r } horizontal, ordinate of the current pixel point in image storage coordinate system is respectively, C_iIt is the centre coordinate of video camera；

Same treatment is done to corresponding depth map：

D_Wi=(X_Wi, Y_Wi, Z_Wi)^T=(K_iR_i)^-1(λ_iD_i+K_iR_iC_i), (i=l, r)

Wherein D_Wi={ D_Wi=(X_Wi, Y_Wi, Z_Wi) T | i=l, r } represent that the pixel of reference view depth map current position is alive Three-dimensional coordinate in boundary's coordinate system, D_i={ D_i=(u_0i, v_0i, 1)^T| i=l, r } deposit for the homogeneous graph picture of reference view depth map Storage coordinate；

1.2) will be by projecting each point for obtaining for the first time, the inside and outside parameter of virtual video camera at combining target new viewpoint, And depth information, in projecting to the plane of delineation coordinate system of new viewpoint again：

P_Newi=(K_NewiR_NewiP_Wi-K_NewiR_NewiC_Newi)/λ_Newi, (i=l, r)

Wherein P_Newi={ P_Newi=(u_Newi, v_Newi, 1) | i=l, r } it is P_iThe coordinate of corresponding points, u at new viewpoint_Newi={ u_Newi ∈ R | i=l, r } and v_Newi={ v_Newi∈ R | i=l, r } it is respectively horizontal, vertical seat of the current pixel point in image storage coordinate system Mark, K_Newi、R_Newi、G_Newi、λ_NewiRepresent the internal reference matrix of virtual video camera at new viewpoint, spin matrix, centre coordinate and its Secondary scale factor；

Corresponding new viewpoint depth map is expressed as：

D_Newi=(K_NewiR_NewiD_Wi-K_NewiR_NewiC_Newi)/λ_Newi, (i=l, r)

Wherein D_Newi={ D_Newi=(u_Newi, v_Newi, 1) | i=l, r } it is D_iThe coordinate of corresponding points at new viewpoint；

(2) edge of object in the reference view depth map of left and right is searched for, the method in (1) is utilized, edge pixel is carried out into three-dimensional Transform at new viewpoint, then erase corresponding depth pixel point at new viewpoint：

D_{New_edgei}=0, (i=l, r)

Wherein D_{New_edgei}It is the target edges pixel after three-dimension varying；

(3) 3 × 3 template is utilized, the depth map to obtaining carries out medium filtering, crackle and tiny leakage in removal depth image Hole, and the image after after filtering is contrasted with the depth map obtained through three-dimension varying, mark the pixel of change：

I_Newi=S (M (D_{New_Imgi}), D_{New_Imgi}), (i=l, r)

Wherein I_NewiIt is the pixel being labeled, M is 3 × 3 medium filtering function, D_{New_Imgi}It is by step (1) and step (2) depth map at new viewpoint obtained after processing, S for comparing two images, and marks change to compare labeling function Pixel；

(4) crack produced because of three-dimension varying is repaired, mending course is as follows：

4.1) back projection is carried out to the pixel being labeled, is projected at original reference view：

P_INewi=W (I_Newi), (i=l, r)

Wherein, P_INewiRepresent in the new viewpoint image obtained by back projection, the pixel of cracks, W represents that step (1) is described Three-dimension varying；

4.2) again by the pixel value in initial reference texture maps, it is assigned in new viewpoint image, it is identical with labeled pixel point coordinates Pixel；

(5) using the new viewpoint image by being obtained after step (4) treatment, row interpolation is entered in the cavity to image occlusion area：

P_{In_Img}=IN(P_{New_Imgl},P_{New_Imgr})

Wherein, P_{In_Img}It is by the new viewpoint texture maps after interpolation, P_{New_Imgl}Represent by left reference picture by step (1) the new viewpoint texture maps obtained after all treatment of-(4), P_{New_Imgr}Represent by right reference picture by step (1)- (4) obtained after all treatment, the new viewpoint texture maps at same viewpoint, IN is interpolating function；

(6) the inpaint methods of Telea propositions in OpenCV built-in functions are called, the cavity to remaining is repaired, and obtains final New viewpoint image：

P_{New_Img}=inpaint(P_{In_Img})

Wherein inpaint is OpenCV built-in functions, P_{New_Img}It is the final new viewpoint texture image for obtaining.

2. a kind of novel view synthesis method based on depth image as claimed in claim 1, it is characterised in that：The step (2) in, the process for finding target edges pixel is as follows：

$\∀u,v\∈P_{\mathrm{Im}g},\underset{i=-1}{\overset{1}{\Σ}}\underset{j=-1}{\overset{1}{\Σ}}D(u+i,v+j)-9\×(u,v)>T_d$

Wherein, P_ImgReference picture is represented, D represents the depth map of reference view, T_dIt is self-defined threshold value, when pixel meets above formula When, represent that the point is ghost edge.

3. a kind of novel view synthesis method based on depth image as claimed in claim 1 or 2, it is characterised in that：The step Suddenly in (3), the processing procedure for comparing labeling function S is comprised the following steps：

3.1) gray value of each pixel in the depth map before and after contrast medium filtering；

3.2) coordinate of the pixel that record gray value is differed.

4. a kind of novel view synthesis method based on depth image as claimed in claim 1 or 2, it is characterised in that：The step It is rapid 4.2) in, the process for mending a split is as follows：

Using the mark point coordinates obtained in 4.1), the pixel value of mark point position in initial reference texture maps is chosen, be assigned to new Corresponding mark pixel in visual point image, the pixel of rest position keeps constant：

$P_{New\_\mathrm{Im}gi}=\left\{\begin{array}{cc}{P}_{\mathrm{Im}gi}\left(P_{INewi}\right),& mark=1\\ P_{ONew\_\mathrm{Im}gi},& mark=0\end{array}\right.,(i=l,r)$

Wherein P_{New_Imgi}For at new viewpoint by the texture maps after crack repairing, P_ImgiIt is reference texture figure, P_{ONew_Imgi}Be through The new viewpoint texture maps obtained after step (1) and step (2) treatment are crossed, mark represents whether current pixel point is labeled, Mark represents that current point is labeled point when being 1.

5. a kind of novel view synthesis method based on depth image as claimed in claim 1 or 2, it is characterised in that：The step Suddenly in (5), the expression formula of IN interpolating functions is as follows：

$P_{In\_\mathrm{Im}g}(u,v)=\left\{\begin{array}{ccc}(1-\mathrm{\α})P_{New\_\mathrm{Im}gl}(u_1,v_1)+{\mathrm{\αP}}_{New\_\mathrm{Im}gr}(u_2,v_2),& O_l(u,v)=0,& O_r(u,v)=0\\ P_{New\_\mathrm{Im}gl}(u_1,v_1),& O_l(u,v)=0,& O_r(u,v)=1\\ P_{New\_\mathrm{Im}gr}(u_2,v_2),& O_l(u,v)=1,& O_r(u,v)=0\\ 0,& O_l(u,v)=1,& O_r(u,v)=1\end{array}\right.$

WhereinIt is proportion parameter, T_{New_t}、T_{l_t}And T_{r_t}Respectively new viewpoint, left reference The translation vector of viewpoint and right reference view in camera coordinate system, O_i={ O_i(u, v) | i=l, r represent image (u, V) place whether there is cavity, is 1 if in the presence of cavity, and expression formula is as follows：

$O_l(u,v)=\left\{\begin{array}{cc}1& \left(Z_l\right(u,v)<T_h)\\ 0& \left(Z_l\right(u,v)>T_h)\end{array}\right.$

$O_r(u,v)=\left\{\begin{array}{cc}1& \left(Z_r\right(u,v)<T_h)\\ 0& \left(Z_r\right(u,v)>T_h)\end{array}\right.$

Wherein Z_i={ Z_i(u, v) | i=l, r } it is depth value of the new viewpoint at (u, v) place, T_hIt is threshold value.