CN105139401A - Depth credibility assessment method for depth map - Google Patents

Abstract

The invention provides a depth credibility assessment method for a depth map. The method comprises the steps: obtaining at least two groups of first original image pairs in different scenes; generating a predictor according to the first original image pairs; obtaining an inputted second original image pair, wherein the second original image pair comprises a second original color map and a second original depth map, which are aligned with each other in one scene; extracting a second characteristic vector of the second original image pair; and predicting the depth credibility of the second original depth map according to the predictor and the second characteristic vector. The method can predict the confidence of scene depth information collected based on low-precision depth equipment.

Description

The appraisal procedure of the confidence level of the degree of depth in a kind of depth map

Technical field

The present invention relates to computer vision field, refer to the appraisal procedure of the confidence level of the degree of depth in depth map especially.

Background technology

At present, at computer vision field, obtain depth information of scene is accurately one of important research content always.Obtain depth information and mainly contain two kinds of methods: the method based on solid geometry theory and the method based on depth transducer.

Obtain depth information of scene based on solid geometry theory is the method that people compare concern all the time, and the method is employed the earliest, and corresponding algorithm is relatively ripe.But its time complexity is higher, what the effect of Depth Information Acquistion depended on scene texture to a great extent enriches degree, thus the considerable restraint range of application of this method.

Along with the development of hardware technology, the scene depth acquisition methods based on hardware sensor have also been obtained increasing concern.This method obtains depth information of scene from single visual angle, and the corresponding feature of obstructed overmatching obtains scene depth, and time complexity is low, and meanwhile, the degree of depth obtains the texture not relying on scene, so there is good robustness.Particularly in recent years, along with selling of low price depth transducer, scene depth is obtained accomplished good unification with price especially, promoted the application of the method greatly.But due to the character of low price depth transducer equipment self, it is than being easier to the impact being subject to noise, and then depth information of scene is caused to catch complete and precision is not high.

In order to the scene depth degree of confidence not high to precision is predicted, and then can provide tutorial message to other application, a lot of method produces.For ToF flying time technology camera, the people such as May and Swadzba calculate the degree of confidence of scene depth according to the amplitude that ToF camera depth transducer equipment obtains.But simple dependence amplitude is foundation is insufficient, and this method robustness is poor, and the degree of confidence of depth information of scene is easily labeled mistake.

The people such as Reynolds propose the degree of confidence of a kind of method prediction ToF camera.They use high accuracy depth sensor device, such as laser 3 d scanner, the depth information of scene gathered is as standard value, by calculating the depth information of scene and the difference of standard value that low side depth transducer obtains, and be translated into degree of confidence, extract the feature of the image that low side depth transducer equipment gathers simultaneously, use the fallout predictor that the method for training obtains, and then predict the degree of confidence of the depth information of scene that low end sensor obtains.This method needs the support of high precision apparatus, thus limits its application to a certain extent; Simultaneously their feature of proposing be not suitable for other depth transducer, such as Kinect (name that Microsoft formally issues body sense periphery peripheral hardware) equipment.

Summary of the invention

The technical problem to be solved in the present invention is, provides the appraisal procedure of the confidence level of the degree of depth in a kind of depth map, can predict the degree of confidence of the depth information of scene gathered based on low precision depth device.

In described depth map, the appraisal procedure of the confidence level of the degree of depth comprises:

Obtain at least two group first original images pair under different scene, described first original image is to the first original color figure and the first original depth-map that comprise alignment;

According to described first original image pair, generation forecast device; Described fallout predictor represents the mapping relations between proper vector and degree of depth confidence level extracted from image;

Obtain the second original image pair of input, described second original image is to the second original color figure alignd under comprising a scene and the second original depth-map;

Extract the second feature vector that described second original image is right;

According to described fallout predictor and described second feature vector, predict the confidence level of the degree of depth of described second original depth-map.

Described according to described first original image pair, the step of generation forecast device comprises:

According at least two group first original images pair described under each scene, calculate average color figure corresponding to scene described in each and average depth map;

According to the described mean depth figure of scene described in each, generate the degree of depth confidence level figure under scene described in each;

According to the described average color figure under scene described in each and described mean depth figure, extract the feature that pixel is associated with degree of depth confidence level, composition first eigenvector

According to described first eigenvector and the described degree of depth confidence level figure under scene described in each, utilize random forest method, the generation forecast device by training.

Described fallout predictor is:

Wherein, (i, j) represents the coordinate of pixel in described average color figure and described mean depth figure;

refer to the proper vector extracted from image;

F refers to the mapping relations between proper vector and degree of depth confidence level;

C _{i, j}refer to the degree of depth confidence level of pixel (i, j).

Described according to the described mean depth figure under scene described in each, the step generating the degree of depth confidence level figure under scene described in each comprises:

According to the described mean depth figure under scene described in each, calculate the variance depth map under scene described in each;

Utilize the Sigmoid tangent bend function after adjustment, the described variance depth map under scene described in each is converted to the degree of depth confidence level figure under scene described in each.

Described according to the described mean depth figure under scene described in each, the step calculating the variance depth map under scene described in each is carried out according to following formula:

$v_{i,j}=\frac{1}{{\Σ}_{m=1}^N{D}_{i,j}}{\Σ}_{m=1}^N{(D_{i,j}-D_{i,j}^{-})}^2,$

Wherein, m is that each organizes the sequence number of the first original depth-map of the first original image centering; The total quantity that the first original image that N gathers under referring to a scene is right, D _ijit is the pixel value of the first original depth-map; refer to the pixel value of mean depth figure; v _{i, j}represent the variance referring to that pixel (i, j) is corresponding.

Sigmoid tangent bend function after described utilization adjustment, the step of the degree of depth confidence level figure be converted under scene described in each by the described variance depth map under scene described in each is carried out according to following formula:

$c_{i,j}=2(1-\frac{1}{1+e^{-{\mathrm{\λv}}_{i,j}}}),$

Wherein, c _{i, j}refer to the confidence level of pixel (i, j); v _{i, j}refer to the variance of pixel (i, j);

E is natural constant, and λ is conversion parameter.

The step of at least two group first original images under described acquisition scene described in each comprises:

Gather the first original image pair under a scene;

When described first original image is to when comprising N frame the first original depth-map, according to described N frame first original depth-map, calculate the first mean depth figure; Average again after cumulative to the pixel value of all pixels in described N frame first mean depth figure, obtain the first pixel value D;

When described first original image is to when comprising N+1 frame the first original depth-map, according to described N+1 frame first original depth-map, calculate the second mean depth figure; Average again after cumulative to the pixel value of all pixels in described N+1 frame second mean depth figure, obtain the second pixel value D';

Judge that the absolute value whether little 5 of the difference between described first pixel value D and described second pixel value D' is in threshold value;

When the absolute value of the difference between described first pixel value D and described second pixel value D' is less than threshold value, stop gathering the first original image pair under this scenario.

Described first eigenvector comprise following one or more:

The radial distance of image slices vegetarian refreshments X (i, j)

The value of the following feature of image slices vegetarian refreshments X (i, j) and each described feature minimum value in the picture, maximal value, average and intermediate value;

Describedly to be characterized as:

The R red component of image slices vegetarian refreshments X (i, j), G green component, B blue component, D depth component;

The gradient information of pixel (i, j) in mean depth figure;

The marginal information of pixel (i, j) in mean depth figure;

The marginal information of pixel (i, j) in average color figure;

Pixel (i, j) carries out the filtered information in Gabor Garbo to average color figure;

Pixel (i, j) carries out the filtered information in Gabor Garbo to mean depth figure;

The texture information of pixel (i, j) in mean depth figure;

The texture information of pixel (i, j) in average color figure;

Pixel (i, j) arrives the distance of degree of depth hole in mean depth figure.

Described second feature vector comprise following one or more:

The radial distance of image slices vegetarian refreshments X (i, j)

The value of the following feature of image slices vegetarian refreshments X (i, j) and each described feature minimum value in the picture, maximal value, average and intermediate value;

Describedly to be characterized as:

The R red component of image slices vegetarian refreshments X (i, j), G green component, B blue component, D depth component;

The gradient information of pixel (i, j) in the second original depth-map;

The marginal information of pixel (i, j) in the second original depth-map;

The marginal information of pixel (i, j) in the second original color figure;

Pixel (i, j) carries out the filtered information in Gabor Garbo to the second original color figure;

Pixel (i, j) carries out the filtered information in Gabor Garbo to the second original depth-map;

The texture information of pixel (i, j) in the second original depth-map;

The texture information of pixel (i, j) in the second original color figure;

Pixel (i, j) arrives the distance of degree of depth hole in the second original depth-map.

The beneficial effect of technique scheme of the present invention is as follows:

The scene depth figure of the many groups of alignment that the present invention gathers with low accurate facility self and cromogram are for foundation, use random forest method, training fallout predictor, thus the depth image degree of confidence of the effectively low accurate facility acquisition of prediction, and then it can be used as input data to join in other application.

The depth maps of many groups of alignment that disclosure of the invention one utilizes low accurate facility (such as Kinect device) to gather and the method for cromogram predetermined depth figure confidence level.Its basic methods comprises:

First, the cromogram of many group alignment and depth map is used to calculate average color and the depth image of often organizing image;

Then, extract according to average color and depth map the feature be associated with degree of depth confidence level;

Meanwhile, utilize mean depth figure to calculate and often organize the right degree of depth variogram of image, use sigmoid function to be converted into degree of depth confidence level figure;

Then, utilize the feature composition characteristic of said extracted vector and their corresponding degree of depth confidence levels, use random forest method training fallout predictor.

Then, for the colour and the depth image pair that need reliability forecasting, usage forecastings device can the confidence level of effective predetermined depth.Degree of depth confidence level can as input market demand in more senior computer vision, augmented reality personage.

The method of the depth accuracy confidence level that assessment Kinect device of the present invention gathers, the image that this method uses Kinect device to gather is to the method with machine learning, do not need the support of high accuracy depth equipment, such as high-precision laser 3 d scanner, only need the colour after the alignment using Kinect device self to gather and depth image pair, just can judge the confidence level of Kinect depth accuracy, and extract corresponding feature, then machine learning method is utilized to train fallout predictor, when the colour newly gathered and depth image are to input, with regard to the degree of confidence of the degree of depth of its correspondence measurable.

Application scenarios of the present invention is below described.For Kinect device, a kind of method assessing the depth accuracy confidence level that Kinect device gathers, comprises the following steps:

Step 1, chooses the different scenes under indoor environment.Under each scene (visual angle of each scene is different), gather cromogram and the depth map of several alignment by Kinect device, composition multiple image pair.Here alignment parameters can use the parameter stored in Kinect device to carry out.The right quantity of image is as the criterion so that the mean depth figure tried to achieve under this quantity is stable.Mean depth image refers to that each the frame depth image for the multiple series of images centering gathered under a certain scene adds up, and then solves mean value.When image comprises N frame depth image to group, averaging of income depth image is D; When comprising N+1 frame, averaging of income depth image is D', if abs (D-D') <t (abs represents absolute value), (t is a certain threshold value), then think that image log amount is enough stable; Otherwise, continue to gather under this scene.

Step 2, for cromogram and the depth map of several alignment under each scene, calculates their average color figure and average depth map; Then calculate the variance depth map under this scene according to mean depth figure, computing method are as follows:

$v_{i,j}=\frac{1}{{\&Sigma;}_{m=1}^N{D}_{i,j}}{\&Sigma;}_{m=1}^N{(D_{i,j}-D_{i,j}^{-})}^2,$

Wherein, the colour that N gathers under referring to each scene and the right quantity of depth image, D refers to wherein a certain amplitude deepness image, refer to mean depth image.It is to be noted: gather in image, degree of depth pixel value be zero pixel be not considered at this.

Step 3, variance can reflect the stability of data, and variance little expression data are more stable, otherwise then data are unstable.In [2] step, pixel depth variance is large, represents that depth value obtains unstable, can affect data precision; Pixel depth variance is little, and represent that the degree of depth obtains stable, data precision is higher.Utilize the Sigmoid function after adjustment, degree of depth variogram is converted to the degree of depth confidence level figure of standard, Sigmoid function is as follows:

$c_{i,j}=2(1-\frac{1}{1+e^{-{\mathrm{\&lambda;v}}_{i,j}}}),$

Wherein, c _{i, j}, v _{i, j}refer to the confidence level (degree of confidence) that pixel (i, j) is corresponding and variance, e is natural constant, and λ can be set to: 0.17;

Step 4, according to the many groups average color figure calculated in step 2 and average depth map, extracts the feature that each pixel of image pair is associated with degree of depth confidence level, composition characteristic vector feature comprises:

(1) colour that comprises of pixel and depth information, i.e. R (redness), G (green), B (blueness), D (degree of depth) component of image slices vegetarian refreshments X (i, j) formula is as follows:

$X_{i,j}^{rgbd}=I_{i,j},D_{i,j},$

Wherein, i, j represent pixel coordinate, I _{i, j}=(R, G, B), represents the chromatic information of pixel (i, j); D _{i, j}=D, represents the depth information of pixel (i, j).

(2) radial distance of described pixel (i, j) refer to the Euclidean distance between current pixel point and image center.Because each pixel has RGBD tetra-kinds of information, RGB is stored in coloured image, and D is stored in depth image, and two kinds of image resolution ratios are consistent, and image center here can refer to any one of two kinds of images.Formula is as follows:

$X_{i,j}^{rdist}=\sqrt{{\left(p_{i-}{c}_i\right)}^2+{(p_j-c_j)}^2},$ ，

Wherein, (p _i, p _j) refer to the coordinate of pixel (i, j); (c _i, c _j) refer to the coordinate of image center.

(3) gradient information of described pixel (i, j) in depth image, comprises gradient in the x direction gradient in y-direction population gradient according to the intensity magnitude information that Robert Robert operator calculates and according to the edge direction theta information that Robert Robert operator calculates respectively according to following formulae discovery:

$X_{i,j}^{{\mathrm{grad}}_x}=D(i,j)-D(i+1,j),$

$X_{i,j}^{{\mathrm{grad}}_y}=D(i,j)-D(i,j+1),$

$X_{i,j}^{{\mathrm{grad}}_{whole}}=D(i,j)+D(i+1,j+1)-2D(i+1,j),$

$X_{i,j}^{{\mathrm{grad}}_{magitude}}=\sqrt{x_{grad}^2+y_{grad}^2},$

$X_{i,j}^{{\mathrm{grad}}_{theta}}={\tan }^{-1}\frac{y_{grad}}{x_{grad}},$

Wherein, $x_{grad}=D(i,j)-D(i+1,j+1),y_{grad}=D(i+1,j)-D(i,j+1),$ D (i, j) refers to the depth value that in depth image, pixel (i, j) is corresponding;

D (i+1, j+1) refers to the depth value of pixel (i+1, j+1); D (i+1, j) refers to the depth value of pixel (i+1, j); The depth value of D (i, j+1)

(4) marginal information of described pixel (i, j) in depth image with the marginal information of described pixel (i, j) in coloured image the pixel of fringe region is more easily by noise effect, and use the marginal information of Canny card Buddhist nun's operator extraction coloured image and depth image here, formula is as follows:

$X_{i,j}^{{\mathrm{Canny}}_{RGB}}={\mathrm{ICanny}}_{i,j},$

$X_{i,j}^{{\mathrm{Canny}}_{Depth}}={\mathrm{DCanny}}_{i,j},$

Wherein, ICanny _{i, j}and DCnny _{i, j}the marginal information of cromogram referring to respectively to use canny algorithm to extract at pixel (i, j) and the marginal information of depth map, use the canny convolution kernel computation bound information of 3 × 3,5 × 5 and 7 × 7 in this method.

(5) pixel (i, j) is at the Gabor Garbo filtering information of depth map with the Gabor Garbo filtering information in cromogram

In this method, parameter is 0 °, 45 °, 90 °, and the Gabor filter of 135 ° does filtering to coloured image and depth image respectively.Formula is as follows:

$X_{i,j}^{{\mathrm{Gabor}}_{RGB}}={\mathrm{IGabor}}_{i,j},$ $X_{i,j}^{{\mathrm{Gabor}}_{Depth}}={\mathrm{DGabor}}_{i,j},$

Wherein, IGabor _{i, j}and DGabor _{i, j}refer to respectively carry out the filtered result of Gabor to coloured image and depth image;

(6) pixel (i, j) is at the texture information of depth map and cromogram with lBP (local binary patterns) algorithm is used to extract texture information to coloured image and depth image respectively in this method.Formula is as follows:

$X_{i,j}^{{\mathrm{Texture}}_{RGB}}={\mathrm{ILBP}}_{i,j},$

$X_{i,j}^{{\mathrm{Texture}}_{Depth}}={\mathrm{DLBP}}_{i,j},$

Wherein, ILBP _{i, j}and DLBP _{i, j}refer to that LBP algorithm is applied to the result on coloured image and depth image respectively respectively;

(7) described pixel (i, j) is to the distance of degree of depth hole degree of depth hole refers to the region that in depth map, depth data cannot gather because of the reason such as light, noise, namely the null value region in depth image, represent that the depth information of scene cannot obtain with depth transducer, so the closer pixel in distance degree of depth cavity is more easily subject to the impact of noise.Formula is as follows:

$X_{i,j}^{dhole}=\sqrt{{(p_i-{\mathrm{ph}}_x)}^2+{(p_j-{\mathrm{ph}}_y)}^2},$

Wherein, (p _i, p _j) refer to the coordinate of pixel (i, j); (ph _x, ph _y) span is from the coordinate of the nearest degree of depth hole pixel of pixel (i, j).

(8) global characteristics.Because the minimum value in entire image feature, maximal value, average and intermediate value contain important information when stating picture characteristics.So in this method, above-mentioned several features (except the radial distance) minimum value in entire image, maximal value, average and intermediate value are all comprised in proper vector.Wherein, intermediate value is that in one group of data, size is arranged in middle numerical value.

$X_{i,j}^{global}=\left\{\begin{array}{c}min,\\ max,\\ mean,\\ median,\end{array}\right.,$

According to above-mentioned feature, each minimum value, maximal value, average and intermediate value comprise 26 dimensional features.Final proper vector comprises 131 dimensional features (26 features × 5+ radial distance).

According to front 8 kinds of features, except radial distance outward, have 26 dimensional features at each pixel (i, j) place, comprise R, G, B, D, 3 × 3 5 × 5 7 × 7 3 × 3 5 × 5 7 × 7 0 degree 45 degree 90 degree 135 degree 0 degree 45 degree 90 degree 135 degree on this basis, for the feature that all pixels calculate, the maximal values (26 dimension) can tried to achieve in this dimension all in each dimension, minimum value (26 dimension), average (26 dimension) and intermediate value (26 dimension), so total characteristic dimension totally 26 × 5+1=131 dimension.

About the explanation of profile maxima, minimum value, average and intermediate value, for maximal value, and be only characterized as example with R, G, B, D these four kinds:

Suppose that we have one group of average color figure and average depth map now, comprise four pixels altogether.

The RGBD of pixel 1 correspondence is respectively: 2,4,3,5;

The RGBD of pixel 2 correspondence is respectively: 5,7,8,4;

The RGBD of pixel 3 correspondence is respectively: 4,9,7,9;

The RGBD of pixel 4 correspondence is respectively: 6,3,2,8;

Maximal value then for first dimensional feature (i.e. the R value of each pixel) of this group image is: 6, corresponding two, three, the maximal value of four dimensional features is: 9,8,9, then final proper vector is: R, G, B, D, max (R), max (G), max (B), max (D);

Pixel 1:2,4,3,5,6,9,8,9;

Pixel 2:5,7,8,4,6,9,8,9;

Pixel 3:4,9,7,9,6,9,8,9;

Pixel 4:6,3,2,8,6,9,8,9;

For minimum value, average and intermediate value are adopted and are used the same method.

Step 5, by the degree of depth confidence level calculated in the proper vector extracted in step 4 and step 3, utilizes random forest method, training fallout predictor, and concrete correspondence is as follows:

Wherein, refer to the proper vector extracted from cromogram and depth map, f refers to the mapping relations between proper vector with corresponding degree of confidence.In the training process, use 100 to set and build random forest, the degree of depth of every tree is 25, and the precision simultaneously set is 0.0001.

That is, for the depth image often organizing alignment and the coloured image of Kinect device collection, corresponding mean depth figure and average cromogram can be calculated, and then the variance depth map of each group data can be obtained.For each pixel value v in variance depth image _{i, j}, be translated into degree of confidence c by sigmodel function _{i, j}.This degree of confidence correspond to picture point (i, j).Extract cromogram and feature relevant to degree of depth degree of confidence in depth map, composition characteristic vector, then uses the method based on study simultaneously, training fallout predictor.

Step 6, to the degree of depth and the coloured image pair of new input, extracting the proper vector identical with step 4, training the fallout predictor obtained, predetermined depth confidence level, end-of-job according to entering forest.Because the degree of depth that newly inputs and coloured image are to just one group, therefore, do not need to calculate average color figure and average depth map, directly extract from depth map and cromogram.

The invention provides a kind of based on the degree of depth method for evaluating confidence of low precision (such as: Kinect) equipment self with random forest, the degree of confidence of the depth information of scene that Kinect depth device gathers can be predicted.The scene depth figure of the many groups of alignment that the present invention gathers with Kinect device self and cromogram are for foundation, use random forest method, training fallout predictor, thus the depth image degree of confidence of effectively prediction Kinect device acquisition, and then it can be used as input data to join in other application.That is, by the data gathered, the method training fallout predictor of applied for machines study, the mapping relationship f namely in formula.For the coloured image and the depth image pair that need forecast confidence, extract identical proper vector, usage forecastings device, just can obtain the degree of confidence of image to each pixel.

That is, use above-mentioned feature, composition characteristic vector, simultaneously according to the standard confidence level calculated, use random forest method, train a fallout predictor.Because confidence level is continuous print, thus random forest uses Regression Model.For the colour and the depth image pair that need predetermined depth confidence level, extract identical proper vector, the fallout predictor obtained has been trained in application, obtains their degree of depth confidence level.The present invention does not need the support of high accuracy depth scanning device (as high-precision laser spatial digitizer etc.).

The above is the preferred embodiment of the present invention; it should be pointed out that for those skilled in the art, under the prerequisite not departing from principle of the present invention; can also make some improvements and modifications, these improvements and modifications also should be considered as protection scope of the present invention.

Accompanying drawing explanation

Fig. 1 is the schematic flow sheet of the appraisal procedure of the confidence level of the degree of depth in depth map of the present invention;

Fig. 2 is the principle schematic of the appraisal procedure of the confidence level of the degree of depth in depth map of the present invention.

Embodiment

For making the technical problem to be solved in the present invention, technical scheme and advantage clearly, be described in detail below in conjunction with the accompanying drawings and the specific embodiments.

As shown in Figure 1, in described depth map, the appraisal procedure of the confidence level of the degree of depth comprises:

Step 11, obtains at least two group first original images pair under different scene, and described first original image is to the first original color figure and the first original depth-map that comprise alignment; Scene can be indoor scene.Alignment is under depth image and coloured image are transformed into same visual angle.

Step 12, according to described first original image pair, generation forecast device; Described fallout predictor represents the mapping relations between proper vector and degree of depth confidence level extracted from image;

Step 13, obtains the second original image pair of input, and described second original image is to the second original color figure alignd under comprising a scene and the second original depth-map;

Step 14, extracts the second feature vector that described second original image is right;

Step 15, according to described fallout predictor and described second feature vector, predicts the confidence level of the degree of depth of described second original depth-map.

Wherein, step 12 comprises:

Step 121, according at least two group first original images pair described under each scene, calculates average color figure corresponding to scene described in each and average depth map;

Step 122, according to the described mean depth figure of scene described in each, generates the degree of depth confidence level figure under scene described in each;

Step 123, according to the described average color figure under scene described in each and described mean depth figure, extracts the feature that pixel is associated with degree of depth confidence level, composition first eigenvector

Step 124, according to described first eigenvector and the described degree of depth confidence level figure under scene described in each, utilize random forest method, the generation forecast device by training.

Described fallout predictor is:

Wherein, (i, j) represents the coordinate of pixel in described average color figure and described mean depth figure;

refer to the proper vector extracted from described average color figure and described mean depth figure;

F refers to the mapping relations between proper vector and degree of depth confidence level;

C _{i, j}refer to the degree of depth confidence level of pixel (i, j).

Wherein, step 122 comprises:

Step 1221, according to the described mean depth figure under scene described in each, calculates the variance depth map under scene described in each;

Step 1222, utilizes the Sigmoid tangent bend function after adjustment, the described variance depth map under scene described in each is converted to the degree of depth confidence level figure under scene described in each.

Step 1221 is carried out according to following formula:

$v_{i,j}=\frac{1}{{\&Sigma;}_{m=1}^N{D}_{i,j}}{\&Sigma;}_{m=1}^N{(D_{i,j}-D_{i,j}^{-})}^2,$

Wherein, m is that each organizes the sequence number of the first original depth-map of the first original image centering; The total quantity that the first original image that N gathers under referring to a scene is right, D _ijit is the pixel value of the first original depth-map; refer to the pixel value of mean depth figure; v _{i, j}represent the variance referring to that pixel (i, j) is corresponding.

Step 1222 is carried out according to following formula:

$c_{i,j}=2(1-\frac{1}{1+e^{-{\mathrm{\&lambda;v}}_{i,j}}}),$

Wherein, c _{i, j}refer to the confidence level of pixel (i, j); v _{i, j}refer to the variance of pixel (i, j);

E is natural constant, and λ is conversion parameter, and can control the size that variance is converted to confidence level, λ can be set to: 0.17.

In this step, Kinect device is used to gather many group colours and depth image pair at multiple visual angles of scene.For the image pair at each visual angle, can calculate an average color and depth image pair, composition average color and depth image are to group.In this, as foundation, calculate the variance depth information v often organizing image.

High variance yields correspond to low confidence, and vice versa.For each value v in variance depth information _{i, j}application sigmoid function, calculates the degree of confidence c of their correspondences _{i, j}.

Step 11 comprises:

Step 111, gathers the first original image pair under a scene;

Step 112, when described first original image is to when comprising N frame the first original depth-map, according to described N frame first original depth-map, calculates the first mean depth figure; Average again after cumulative to the pixel value of all pixels in described N frame first mean depth figure, obtain the first pixel value D;

Step 113, when described first original image is to when comprising N+1 frame the first original depth-map, according to described N+1 frame first original depth-map, calculates the second mean depth figure; Average again after cumulative to the pixel value of all pixels in described N+1 frame second mean depth figure, obtain the second pixel value D';

Step 114, judges whether the absolute value of the difference between described first pixel value D and described second pixel value D' is less than threshold value;

Step 115, when the absolute value of the difference between described first pixel value D and described second pixel value D' is less than threshold value, stops gathering the first original image pair under this scenario.

Wherein, described first eigenvector comprise following one or more:

The radial distance of image slices vegetarian refreshments X (i, j)

The value of the following feature of image slices vegetarian refreshments X (i, j) and each described feature minimum value in the picture, maximal value, average and intermediate value;

Describedly to be characterized as:

The R red component of image slices vegetarian refreshments X (i, j), G green component, B blue component, D depth component;

The gradient information of pixel (i, j) in mean depth figure;

The marginal information of pixel (i, j) in mean depth figure;

The marginal information of pixel (i, j) in average color figure;

Pixel (i, j) carries out the filtered information in Gabor Garbo to average color figure;

Pixel (i, j) carries out the filtered information in Gabor Garbo to mean depth figure;

The texture information of pixel (i, j) in mean depth figure;

The texture information of pixel (i, j) in average color figure;

Pixel (i, j) arrives the distance of degree of depth hole in mean depth figure.

Wherein, described second feature vector comprise following one or more:

The radial distance of image slices vegetarian refreshments X (i, j)

The value of the following feature of image slices vegetarian refreshments X (i, j) and each described feature minimum value in the picture, maximal value, average and intermediate value;

Describedly to be characterized as:

The R red component of image slices vegetarian refreshments X (i, j), G green component, B blue component, D depth component;

The gradient information of pixel (i, j) in the second original depth-map;

The marginal information of pixel (i, j) in the second original depth-map;

The marginal information of pixel (i, j) in the second original color figure;

Pixel (i, j) carries out the filtered information in Gabor Garbo to the second original color figure;

Pixel (i, j) carries out the filtered information in Gabor Garbo to the second original depth-map;

The texture information of pixel (i, j) in the second original depth-map;

The texture information of pixel (i, j) in the second original color figure;

Pixel (i, j) arrives the distance of degree of depth hole in the second original depth-map.

The method extracting first eigenvector is similar with the Measures compare extracting second feature vector.There is concrete description hereinafter.

Claims (9)

1. the appraisal procedure of the confidence level of the degree of depth in depth map, it is characterized in that, described method comprises:

Obtain at least two group first original images pair under different scene, described first original image is to the first original color figure and the first original depth-map that comprise alignment;

According to described first original image pair, generation forecast device; Described fallout predictor represents the mapping relations between proper vector and degree of depth confidence level extracted from image;

Obtain the second original image pair of input, described second original image is to the second original color figure alignd under comprising a scene and the second original depth-map;

Extract the second feature vector that described second original image is right;

According to described fallout predictor and described second feature vector, predict the confidence level of the degree of depth of described second original depth-map.

2. method according to claim 1, is characterized in that, described according to described first original image pair, the step of generation forecast device comprises:

According at least two group first original images pair described under each scene, calculate average color figure corresponding to scene described in each and average depth map;

According to the described mean depth figure of scene described in each, generate the degree of depth confidence level figure under scene described in each;

According to the described average color figure under scene described in each and described mean depth figure, extract the feature that pixel is associated with degree of depth confidence level, composition first eigenvector

According to described first eigenvector and the described degree of depth confidence level figure under scene described in each, utilize random forest method, the generation forecast device by training.

3. method according to claim 2, is characterized in that, described fallout predictor is:

Wherein, (i, j) represents the coordinate of pixel in described average color figure and described mean depth figure;

refer to the first eigenvector extracted from image;

F refers to the mapping relations between proper vector and degree of depth confidence level;

C _{i, j}refer to the degree of depth confidence level of pixel (i, j).

4. method according to claim 2, is characterized in that, described according to the described mean depth figure under scene described in each, the step generating the degree of depth confidence level figure under scene described in each comprises:

According to the described mean depth figure under scene described in each, calculate the variance depth map under scene described in each;

Utilize the Sigmoid tangent bend function after adjustment, the described variance depth map under scene described in each is converted to the degree of depth confidence level figure under scene described in each.

5. method according to claim 4, is characterized in that, described according to the described mean depth figure under scene described in each, the step calculating the variance depth map under scene described in each is carried out according to following formula:

$v_{i,j}=\frac{1}{{\&Sigma;}_{m=1}^N{D}_{i,j}}{\&Sigma;}_{m=1}^N{(D_{i,j}-D_{i,j}^{-})}^2,$

Wherein, m is that each organizes the sequence number of the first original depth-map of the first original image centering; The total quantity that the first original image that N gathers under referring to a scene is right, D _i,jit is the pixel value of the first original depth-map; refer to the pixel value of mean depth figure; v _{i, j}represent the variance referring to that pixel (i, j) is corresponding.

6. method according to claim 4, it is characterized in that, Sigmoid tangent bend function after described utilization adjustment, the step of the degree of depth confidence level figure be converted under scene described in each by the described variance depth map under scene described in each is carried out according to following formula:

$c_{i,j}=2(1-\frac{1}{1+e^{-{\mathrm{\&lambda;v}}_{i,j}}}),$

Wherein, c _{i, j}refer to the confidence level of pixel (i, j); v _{i, j}refer to the variance of pixel (i, j);

E is natural constant, and λ is conversion parameter.

7. method according to claim 1, is characterized in that, the step of at least two group first original images under described acquisition scene described in each comprises:

Gather the first original image pair under a scene;

When described first original image is to when comprising N frame the first original depth-map, according to described N frame first original depth-map, calculate the first mean depth figure; Average again after cumulative to the pixel value of all pixels in described N frame first mean depth figure, obtain the first pixel value D;

When described first original image is to when comprising N+1 frame the first original depth-map, according to described N+1 frame first original depth-map, calculate the second mean depth figure; Average again after cumulative to the pixel value of all pixels in described N+1 frame second mean depth figure, obtain the second pixel value D';

Judge whether the absolute value of the difference between described first pixel value D and described second pixel value D' is less than threshold value;

When the absolute value of the difference between described first pixel value D and described second pixel value D' is less than threshold value, stop gathering the first original image pair under this scenario.

8. method according to claim 2, is characterized in that, described first eigenvector comprise following one or more:

The radial distance of image slices vegetarian refreshments X (i, j)

The value of the following feature of image slices vegetarian refreshments X (i, j) and each described feature minimum value in the picture, maximal value, average and intermediate value;

Describedly to be characterized as:

The R red component of image slices vegetarian refreshments X (i, j), G green component, B blue component, D depth component;

The gradient information of pixel (i, j) in mean depth figure;

The marginal information of pixel (i, j) in mean depth figure;

The marginal information of pixel (i, j) in average color figure;

Pixel (i, j) carries out the filtered information in Gabor Garbo to average color figure;

Pixel (i, j) carries out the filtered information in Gabor Garbo to mean depth figure;

The texture information of pixel (i, j) in mean depth figure;

The texture information of pixel (i, j) in average color figure;

Pixel (i, j) arrives the distance of degree of depth hole in mean depth figure.

9. method according to claim 1, is characterized in that, described second feature vector comprise following one or more:

The radial distance of image slices vegetarian refreshments X (i, j)

The value of the following feature of image slices vegetarian refreshments X (i, j) and each described feature minimum value in the picture, maximal value, average and intermediate value;

Describedly to be characterized as:

The R red component of image slices vegetarian refreshments X (i, j), G green component, B blue component, D depth component;

The gradient information of pixel (i, j) in the second original depth-map;

The marginal information of pixel (i, j) in the second original depth-map;

The marginal information of pixel (i, j) in the second original color figure;

Pixel (i, j) carries out the filtered information in Gabor Garbo to the second original color figure;

Pixel (i, j) carries out the filtered information in Gabor Garbo to the second original depth-map;

The texture information of pixel (i, j) in the second original depth-map;

The texture information of pixel (i, j) in the second original color figure;

Pixel (i, j) arrives the distance of degree of depth hole in the second original depth-map.