CN102761764B - Upper sampling method used for depth picture of three-dimensional stereo video - Google Patents

Abstract

The invention relates to an upper sampling method used for a depth picture of a three-dimensional stereo video, belonging to the technical field of image processing. The method comprises the following steps of firstly training three mutually associated databases, namely a low resolution depth picture block library, a color picture block library and a high resolution depth picture block library; cutting the three databases into picture blocks when a new low resolution depth picture and corresponding color picture library are input, and then solving the sparse linear representation thereof in the low resolution depth picture library and the color picture library in a combined way; and reconstructing a high resolution depth picture on the high resolution depth picture library through the linear representation. According to the method provided by the invention, higher depth picture recovery quality can be obtained compared with the conventional method; the high resolution depth picture with a higher quality can be obtained; and particularly, more ideal effect for the acquired depth picture which has a very low quality can be realized by the method as validated by experiment.

Description

A kind of top sampling method of the degree of depth picture for 3 D stereo video

Technical field

The top sampling method that the present invention relates to a kind of degree of depth picture for 3 D stereo video, belongs to technical field of image processing.

Background technology

3 D stereo video has replaced traditional two-dimentional color TV in numerous application, because it can bring brand-new viewing experience to user: user can obtain more real three-dimensional scenic and experiences and experience, and can freely select the viewpoint of watching and switch between different points of view at any time.Compare with traditional two-dimensional video, the huge advantage of 3 D stereo video in visual effect and user interactivity, makes 3 D stereo video become the focus of research, is developed rapidly and popularizes.

A kind of conventional 3 D stereo video Display Technique is " color video+deep video " technology at present: Yong Yi road color video is expressed texture information, and the Yong Yi road deep video corresponding with it expressed longitudinal depth information of each pixel in color video.A frame in deep video is corresponding with the corresponding frame " pixel-pixel " in color video, and the greyscale video of the value of each pixel in 0～255 scope.

Because degree of depth picture is that " pixel-pixel " is corresponding with colour picture, degree of depth picture need to have the resolution identical with colour picture, but due to the restriction of collecting device, the composition algorithm etc. of degree of depth picture, the degree of depth picture collecting has lower resolution conventionally, and the depth camera based on reflection interval that for example obtains at present broad research just has the characteristic of low resolution.Therefore, the up-sampling of degree of depth picture is a very important technology in 3 D stereo technology.The effect of degree of depth picture is the depth information that provides each pixel, and therefore degree of depth picture has two requirements that can impact 3D vision effect: 1) object in degree of depth picture requires to have clearly edge; 2) interior of articles, part has identical depth value.In the process of degree of depth picture up-sampling, should ensure above-mentioned 2 points, so the good 3-D effect of guarantee as far as possible.

As a traditional problem, picture up-sampling has had the research history of decades.Because degree of depth picture can be regarded common greyish white picture as, existing picture top sampling method can be directly applied on degree of depth picture.But traditional picture top sampling method has been ignored the corresponding relation of deep video and color video, thereby can not obtain marginal value accurately when recovering edge, and destroy to a certain extent the characteristic of depth map locally consistent.Up-to-date research shows, utilizes high-resolution colour picture to instruct the up-sampling of degree of depth picture, can improve well the quality of up-sampling.And at present existing to utilize the top sampling method that colour picture coaches be not the algorithm based on study, there is further improved space in the quality of up-sampling therefore.Method based on study is exactly to learn a mapping from low resolution degree of depth picture to high-resolution degree of depth picture by a large amount of given datas, and the method based on study has better up-sampling quality than common method conventionally.

The method of machine learning has a variety of, wherein based on picture block the learning method of the rarefaction representation in a very large picture block storehouse be proved to be for plurality of picture quality recover problem all there is good effect, comprising sample problem last time of picture.The core concept of this method is: in the time having a picture block storehouse, a new picture block can be expressed as a linear combination of the picture block in this picture library; In the time that this picture block storehouse is enough large, in this linear expression, there is a large amount of neutral element (sparse), this picture block can add and represent by the picture block linearity of the only a few in picture block storehouse.Rarefaction representation characteristic based on picture, basic thought of the present invention is three corresponding picture block storehouses of first trained: a low resolution picture block storehouse, a cromogram valut and a high-resolution pictures piece storehouse, the corresponding picture block in three picture block storehouses is low resolution picture block and the colour picture piece of correspondence and the up-sampling result of its standard.Then, in the time inputting the colour picture piece of a new low resolution picture block and correspondence thereof, combine and solve its optimum rarefaction representation on picture block storehouse separately, and this is sparse in just obtaining the high-resolution pictures piece corresponding to low resolution picture block of new input, the i.e. result of its up-sampling on high-resolution pictures piece storehouse.Introduce particular content of the present invention below.

Summary of the invention

The object of the invention is to propose a kind of top sampling method of the degree of depth picture for 3 D stereo video, the low resolution degree of depth picture producing with reply many reasons, due to its higher Quality of recovery, be particularly suitable for tackling collection terminal and collect the situation of the degree of depth picture that resolution is very low.

The top sampling method of the degree of depth picture for 3 D stereo video that the present invention proposes, comprises the following steps:

(1) definition α is the up-sampling rate to 3 D stereo video depth map sheet, S _lfor the degree of depth picture of low resolution 3 D stereo video being divided into the size of block of pixels, S _hfor the high-resolution degree of depth picture that the degree of depth picture of low resolution 3 D stereo video is carried out obtaining after up-sampling being divided into the size of block of pixels and the colour picture of collection being divided into the size of block of pixels, S _h=S _l× α;

(2) extract respectively the characteristic value of the colour picture block of pixels of low resolution degree of depth picture block of pixels, high-resolution degree of depth picture block of pixels and collection, leaching process is as follows:

(2-1) the low resolution degree of depth picture of 3 D stereo video is used to bilinear interpolation (Bicubic) algorithm, to low resolution degree of depth picture, above sample rate α carries out up-sampling, obtains initial high resolution degree of depth picture H ^int, then by H ^intbe divided into size for S _h× S _hblock of pixels, be S from size _h× S _hblock of pixels in extract this block of pixels at single order, second order gradient totally four characteristic value L laterally, on vertical both direction _i;

(2-2) the high-resolution degree of depth picture that the degree of depth picture of low resolution 3 D stereo video is carried out obtaining after up-sampling is divided into size for S _h× S _hblock of pixels by calculating the characteristic value H of high-resolution degree of depth picture block of pixels _i: wherein for with equal-sized block of pixels, and the depth value of each pixel in block of pixels equals block of pixels in the mean value of each pixel depth value;

(2-3) edge detection algorithm for colour picture (Sober operator) gathering is carried out to edge extracting, obtain initial edge image C ^raw, remove edge image C ^rawin because object texture produce pseudo-object edge, obtain edge image edge image C is divided into size for S _h× S _hblock of pixels, the each block of pixels in C is as the characteristic value C of block of pixels of correspondence position in the colour picture gathering _i;

(3) set up three low resolution degree of depth picture block of pixels storehouse D that are mutually related _l, gather colour picture block of pixels storehouse D _cwith high-resolution degree of depth picture block of pixels storehouse D _h, process of establishing comprises the following steps:

(3-1) from Same Scene, gather low resolution degree of depth picture, high-resolution color picture and standard high-resolution degree of depth picture;

(3-2) respectively by the above-mentioned low resolution degree of depth picture collecting, high-resolution color picture and standard high-resolution degree of depth picture are partitioned into block of pixels, will be at low resolution degree of depth picture, high-resolution color picture is defined as a block of pixels group with the corresponding block of pixels in position in standard high-resolution degree of depth picture, obtain multiple block of pixels groups, from multiple block of pixels groups, randomly draw multiple block of pixels groups, use the Eigenvalue Extraction Method of above-mentioned steps (2), respectively to the low resolution degree of depth picture block of pixels in block of pixels group, the colour picture block of pixels of high-resolution degree of depth picture block of pixels and collection is carried out characteristic value extraction,

(3-3), by solving following formula, obtain three block of pixels storehouse D that are mutually related _l, D _cand D _h:

$\underset{D_l,D_c,D_h,{\mathrm{\α}}_i}{\min }\underset{i}{\mathrm{\Σ}}{\mathrm{\λ}}_l{\left|\right|D_l{\mathrm{\α}}_i-L_i\left|\right|}_2^2+{\mathrm{\λ}}_c{\left|\right|D_c{\mathrm{\α}}_i-C_i\left|\right|}_2^2+{\left|\right|D_h{\mathrm{\α}}_i-H_i\left|\right|}_2^2$

Wherein || α _i|| ₁≤ ε,

Wherein, L _ifor low resolution degree of depth picture block of pixels characteristic value, C _ifor high-resolution color picture block of pixels characteristic value, H _ifor high-resolution degree of depth picture block of pixels characteristic value, λ _l, λ _cfor the weight of respective items, value is to be greater than zero real number, α _ifor coefficient variation, ε is one and is greater than zero real number;

(4) according to the low resolution degree of depth picture block of pixels storehouse D that is mutually related obtained above _l, gather colour picture block of pixels storehouse D _cwith high-resolution degree of depth picture block of pixels storehouse D _h, the low resolution degree of depth picture of input is carried out to up-sampling, up-sampling process comprises the following steps:

(4-1) adopt bilinear interpolation (Bicubic) method to carry out up-sampling in the low resolution degree of depth picture of input, use associating colour picture bilateral filtering method to carry out filtering to the up-sampling picture obtaining, obtaining resolution is the high-resolution degree of depth picture H of target resolution ^b;

(4-2) Eigenvalue Extraction Method of use above-mentioned steps (2), respectively L in the characteristic value of the low resolution degree of depth picture block of pixels of extraction input _i', the characteristic value C of colour picture block of pixels of input _i', and the high-resolution degree of depth picture H of above-mentioned steps (4-1) ^bthe characteristic value of block of pixels

(4-3) use following mapping function, obtain and the low resolution degree of depth picture block of pixels L inputting _i' corresponding high-resolution degree of depth picture block of pixels H _i', mapping function is as follows:

H _i’=D _hα ^*

${\mathrm{\α}}^{*}=\underset{{\left|\right|\mathrm{\α}\left|\right|}_1{\≤\mathrm{\ϵ}}^{\′}}{\mathrm{agr}\min }\{{{\mathrm{\λ}}_l}^{\′}{\left|\right|D_l{\mathrm{\α}}^{\′}-L_i\left|\right|}_2^2+{{\mathrm{\λ}}_c}^{\′}{\left|\right|D_c{\mathrm{\α}}^{\′}-C_i\left|\right|}_2^2+{{\mathrm{\λ}}_r}^{\′}{\left|\right|D_h{\mathrm{\α}}^{\′}-H_i^b\left|\right|}_2^2\}$

Wherein, λ _l', λ _c' and λ _r' be the weight of respective items, value is to be greater than zero real number, and α ' is coefficient variation, and ε ' is one and is greater than zero real number.

The top sampling method of the degree of depth picture for 3 D stereo video that the present invention proposes, its advantage is:

The top sampling method of the degree of depth picture for 3 D stereo video that l, the present invention propose, in block of pixels storehouse, there is the characteristic of rarefaction representation in the block of pixels in use nature picture, the low resolution degree of depth picture of foundation from low resolution 3 D stereo video is to the mapping function of high-resolution degree of depth picture, can the high-quality up-sampling that carries out degree of depth picture, with respect to traditional degree of depth picture top sampling method, there is better up-sampling effect;

The top sampling method of the degree of depth picture for 3 D stereo video that 2, the present invention proposes, the guidance of colour picture in introducing 3 D stereo video to degree of depth picture up-sampling process, the mapping function of setting up has comprised high-resolution colour picture, thereby with respect to other the algorithm based on setting up mapping function class, can better utilize the degree of depth picture of 3 D stereo video to have the characteristic of colour picture corresponding to one " pixel-pixel ", reach better up-sampling effect;

The top sampling method of the degree of depth picture for 3 D stereo video that 3, the present invention proposes, be specially adapted to the situation that up-sampling rate is higher, through experimental results show that, when in the very high situation of up-sampling rate, the up-sampling quality of most of degree of depth picture top sampling method all significantly decreases, even and algorithm of the present invention is in the situation that up-sampling rate is very high, can ensure too higher up-sampling quality, this feature makes the inventive method have wider application.

Brief description of the drawings

Fig. 1 is the flow process of algorithm of the present invention and the schematic diagram of feature extraction.

Specific implementation method

The top sampling method of the degree of depth picture for 3 D stereo video that the present invention proposes, its FB(flow block) as shown in Figure 1, comprises the following steps:

(1) definition α is the up-sampling rate to 3 D stereo video depth map sheet, S _lin order the degree of depth picture of low resolution 3 D stereo video is divided into the size of block of pixels, (each low resolution degree of depth picture block size is S _l× S _l), S _hfor the high-resolution degree of depth picture that the degree of depth picture of low resolution 3 D stereo video is carried out obtaining after up-sampling being divided into the size of block of pixels and the colour picture of collection being divided into the size of block of pixels, S _h=S _l× α; For example, up-sampling rate α=2 represent the length of degree of depth picture and wide before 2 times that all become, and selection S _l=8, S _h=16, thus the degree of depth picture of low resolution is divided into 8 × 8 block of pixels, and high-resolution depth map and colour picture are divided into the block of pixels of 16x16;

The present invention need to probe into the relation between colour picture block of pixels and the high-resolution degree of depth picture block of pixels of low resolution degree of depth picture block of pixels, collection, in considering of algorithm robustness, the present invention does not explore three kinds of incidence relations between block of pixels itself, but extracts every kind of block of pixels some feature separately as expression;

(2) extract respectively the characteristic value of the colour picture block of pixels of low resolution degree of depth picture block of pixels, high-resolution degree of depth picture block of pixels and collection, leaching process is as follows:

(2-1) the low resolution degree of depth picture of 3 D stereo video is used to bilinear interpolation (Bicubic) algorithm, to low resolution degree of depth picture, above sample rate α carries out up-sampling, obtains initial high resolution degree of depth picture H ^int, then by H ^intbe divided into size for S _h× S _hblock of pixels, be S from size _h× S _hblock of pixels in extract this block of pixels at single order, second order gradient totally four characteristic value L laterally, on vertical both direction _i, for representing low resolution depth pixel tiles;

(2-2) the high-resolution degree of depth picture that the degree of depth picture of low resolution 3 D stereo video is carried out obtaining after up-sampling is divided into size for S _h× S _hblock of pixels by calculating the characteristic value Hi of high-resolution degree of depth picture block of pixels: wherein for with equal-sized block of pixels, and the depth value of each pixel in block of pixels equals block of pixels in the mean value of each pixel depth value;

(2-3) edge detection algorithm for colour picture (Sober operator) gathering is carried out to edge extracting, obtain initial edge image C ^raw, remove edge image C ^rawin because object texture produce pseudo-object edge, obtain edge image edge image C is divided into size for S _h× S _hblock of pixels, the each block of pixels in C is as the characteristic value C of block of pixels of correspondence position in the colour picture gathering _i;

Fig. 1 is the schematic diagram that above three characteristic values are extracted;

(3) set up three low resolution degree of depth picture block of pixels storehouse D that are mutually related _l, gather colour picture block of pixels storehouse D _cwith high-resolution degree of depth picture block of pixels storehouse D _h, in these three storehouses, corresponding item is the local corresponding low resolution degree of depth picture block collecting of Same Scene, the characteristic vector that the high-resolution degree of depth picture block of the high-resolution color picture block collecting and standard is extracted respectively.Process of establishing comprises the following steps:

(3-1) from Same Scene, gather low resolution degree of depth picture, high-resolution color picture and standard high-resolution degree of depth picture;

(3-2) respectively the above-mentioned low resolution degree of depth picture collecting, high-resolution color picture and standard high-resolution degree of depth picture are partitioned into block of pixels, the corresponding block of pixels in position in low resolution degree of depth picture, high-resolution color picture and standard high-resolution degree of depth picture is defined as to a block of pixels group, obtain multiple block of pixels groups, from multiple block of pixels groups, randomly draw multiple block of pixels groups, for example, gather 10 ⁵group block of pixels, the Eigenvalue Extraction Method of use above-mentioned steps (2), carries out characteristic value extraction to the colour picture block of pixels of low resolution degree of depth picture block of pixels, high-resolution degree of depth picture block of pixels and collection in block of pixels group respectively;

(3-3), by solving following formula, obtain three block of pixels storehouse D that are mutually related _l, D _cand D _h:

$\underset{D_l,D_c,D_h,{\mathrm{\α}}_i}{\min }\underset{i}{\mathrm{\Σ}}{\mathrm{\λ}}_l{\left|\right|D_l{\mathrm{\α}}_i-L_i\left|\right|}_2^2+{\mathrm{\λ}}_c{\left|\right|D_c{\mathrm{\α}}_i-C_i\left|\right|}_2^2+{\left|\right|D_h{\mathrm{\α}}_i-H_i\left|\right|}_2^2$

Wherein || α _i|| ₁≤ ε,

Wherein, L _ifor low resolution degree of depth picture block of pixels characteristic value, C _ifor high-resolution color picture block of pixels characteristic value, H _ifor high-resolution degree of depth picture block of pixels characteristic value, λ _l, λ _cfor the weight of respective items, value is to be greater than zero real number, for example, can choose λ _l=λ _c=1, α _ifor coefficient variation, ε is one and is greater than zero real number;

For example, final training result is that the size in three block of pixels storehouses is 1024 picture block, experiment showed, that 1024 picture block have had very high ability to express, can reach good up-sampling effect.

The method for solving of above-mentioned formula, is a classical optimization problem, has a large amount of methods to solve it, for example wire drawing algorithm (Lars algorithm).

(4) according to the low resolution degree of depth picture block of pixels storehouse D that is mutually related obtained above _l, gather colour picture block of pixels storehouse D _cwith high-resolution degree of depth picture block of pixels storehouse D _h, the low resolution degree of depth picture of input is carried out to up-sampling, up-sampling process comprises the following steps:

(4-1) adopt bilinear interpolation (Bicubic) method to carry out up-sampling in the low resolution degree of depth picture of input, use associating colour picture bilateral filtering method to carry out filtering to the up-sampling picture obtaining, obtaining resolution is the high-resolution degree of depth picture H of target resolution ^b;

(4-2) Eigenvalue Extraction Method of use above-mentioned steps (2), respectively the characteristic value L of the low resolution degree of depth picture block of pixels of extraction input _i', the characteristic value C of colour picture block of pixels of input _i', and the high-resolution degree of depth picture H of above-mentioned steps (4-1) ^bthe characteristic value of block of pixels

(4-3) use following mapping function, obtain and the low resolution degree of depth picture block of pixels L inputting _i' corresponding high-resolution degree of depth picture block of pixels H _i', mapping function is as follows:

H _i′=D _hα ^*

${\mathrm{\α}}^{*}=\underset{{\left|\right|\mathrm{\α}\left|\right|}_1\≤{\mathrm{\ϵ}}^{\′}}{\mathrm{agr}\min }\{{{\mathrm{\λ}}_l}^{\′}{\left|\right|D_l{\mathrm{\α}}^{\′}-L_i\left|\right|}_2^2+{{\mathrm{\λ}}_c}^{\′}{\left|\right|D_c{\mathrm{\α}}^{\′}-C_i\left|\right|}_2^2+{{\mathrm{\λ}}_r}^{\′}{\left|\right|D_h{\mathrm{\α}}^{\′}-H_i^b\left|\right|}_2^2\}$

Wherein, λ _l', λ _c' and λ _r' be the weight of respective items, value is to be greater than zero real number, for example, choose λ _l'=λ _c'=λ _h'=1, α ' is coefficient variation, and ε ' is one and is greater than zero real number.

Claims (1)

1. for a top sampling method for the degree of depth picture of 3 D stereo video, it is characterized in that the method comprises the following steps:

(1) definition α is the up-sampling rate to 3 D stereo video depth map sheet, S _lfor the degree of depth picture of low resolution 3 D stereo video being divided into the size of block of pixels, S _hfor the high-resolution degree of depth picture that the degree of depth picture of low resolution 3 D stereo video is carried out obtaining after up-sampling being divided into the size of block of pixels and the colour picture of collection being divided into the size of block of pixels, S _h=S _l× α;

(2) extract respectively the characteristic value of the colour picture block of pixels of low resolution degree of depth picture block of pixels, high-resolution degree of depth picture block of pixels and collection, leaching process is as follows:

(2-1) the low resolution degree of depth picture of 3 D stereo video is used to bilinear interpolation method, to low resolution degree of depth picture, above sample rate α carries out up-sampling, obtains initial high resolution degree of depth picture H ^int, then by H ^intbe divided into size for S _h× S _hblock of pixels, be S from size _h× S _hblock of pixels in extract this block of pixels in single order, second order gradient totally four characteristic values laterally, on vertical both direction;

(2-2) the high-resolution degree of depth picture that the degree of depth picture of low resolution 3 D stereo video is carried out obtaining after up-sampling is divided into size for S _h× S _hblock of pixels by calculating the characteristic value H of high-resolution degree of depth picture block of pixels _i: $H_i=H_i^{\mathrm{raw}}-\mathrm{mean}\left(H_i^{\mathrm{raw}}\right),$ Wherein for with equal-sized block of pixels, and the depth value of each pixel in block of pixels equals block of pixels in the mean value of each pixel depth value;

(2-3) colour picture gathering is carried out to edge extracting with edge detection algorithm, obtain initial edge image C ^raw, remove edge image C ^rawin because object texture produce pseudo-object edge, obtain edge image C=▽ C ^raw× ▽ H ^int, edge image C is divided into size for S _h× S _hblock of pixels, the each block of pixels in C is as the characteristic value C of block of pixels of correspondence position in the colour picture gathering _i;

(3) set up three low resolution degree of depth picture block of pixels storehouse D that are mutually related _l, gather colour picture block of pixels storehouse D _cwith high-resolution degree of depth picture block of pixels storehouse D _h, process of establishing comprises the following steps:

(3-1) from Same Scene, gather low resolution degree of depth picture, high-resolution color picture and standard high-resolution degree of depth picture;

(3-2) respectively the above-mentioned low resolution degree of depth picture collecting, colour picture and standard high-resolution degree of depth picture are partitioned into block of pixels, by three corresponding block of pixels D of position in low resolution degree of depth picture, colour picture and standard high-resolution degree of depth picture _l, D _c, D _hform a block of pixels group, obtain multiple block of pixels groups, from multiple block of pixels groups, randomly draw several block of pixels groups, the Eigenvalue Extraction Method that uses above-mentioned steps (2), carries out characteristic value extraction to the colour picture block of pixels of low resolution degree of depth picture block of pixels, high-resolution degree of depth picture block of pixels and collection in several block of pixels groups respectively;

(3-3), by solving following formula, obtain three block of pixels storehouse D that are mutually related _l, D _cand D _h:

$\underset{D_l,D_c,D_h,{\mathrm{\α}}_i}{\min }\underset{i}{\mathrm{\Σ}}{\mathrm{\λ}}_l{\left|\right|D_l{\mathrm{\α}}_i-L_i\left|\right|}_2^2+{\mathrm{\λ}}_c{\left|\right|D_c{\mathrm{\α}}_i-C_i\left|\right|}_2^2+{\left|\right|D_h{\mathrm{\α}}_i-H_i\left|\right|}_2^2$

Wherein || α _i|| ₁≤ ε,

Wherein, L _ifor low resolution degree of depth picture block of pixels characteristic value, C _ifor colour picture block of pixels characteristic value, H _ifor high-resolution degree of depth picture block of pixels characteristic value, λ _l, λ _cfor the weight of respective items, value is to be greater than zero real number, α _ifor coefficient variation, ε is one and is greater than zero real number;

(4) according to the low resolution degree of depth picture block of pixels storehouse D that is mutually related obtained above _l, gather colour picture block of pixels storehouse D _cwith high-resolution degree of depth picture block of pixels storehouse D _h, low resolution degree of depth picture and the colour picture of input are carried out to up-sampling, up-sampling process comprises the following steps:

(4-1) adopt bilinear interpolation method to carry out up-sampling in the low resolution degree of depth picture of input, use associating colour picture bilateral filtering method to carry out filtering to the up-sampling picture obtaining, obtaining resolution is the high-resolution degree of depth picture H of target resolution ^b;

(4-2) Eigenvalue Extraction Method of use above-mentioned steps (2), respectively the characteristic value L of the low resolution degree of depth picture block of pixels of extraction input _i', the characteristic value C of colour picture block of pixels of input _i', and the high-resolution degree of depth picture H of above-mentioned steps (4-1) ^bthe characteristic value of block of pixels

(4-3) use following mapping function, obtain and the low resolution degree of depth picture block of pixels L inputting _i' corresponding high-resolution degree of depth picture block of pixels H _i', mapping function is as follows:

H _i′＝D _hα ^*

${\mathrm{\α}}^{*}=\underset{{\left|\right|\mathrm{\α}\left|\right|}_1\≤{\mathrm{\ϵ}}^{\′}}{\arg \min }\{{{\mathrm{\λ}}_l}^{\′}{\left|\right|D_l{\mathrm{\α}}^{\′}-L_i\left|\right|}_2^2+{{\mathrm{\λ}}_c}^{\′}{\left|\right|D_c{\mathrm{\α}}^{\′}-C_i\left|\right|}_2^2+{{\mathrm{\λ}}_r}^{\′}{\left|\right|D_h{\mathrm{\α}}^{\′}-H_i^b\left|\right|}_2^2\}$

Wherein, λ _l', λ _c' and λ _r' be the weight of respective items, value is to be greater than zero real number, and α ' is coefficient variation, and ε ' is one and is greater than zero real number, D _l, D _c, D _hrespectively three corresponding block of pixels of position in low resolution degree of depth picture, colour picture and standard high-resolution degree of depth picture.