CN103218771A - Parameter self-adaptation selecting method based on autoregressive model depth recovery - Google Patents
Parameter self-adaptation selecting method based on autoregressive model depth recovery Download PDFInfo
- Publication number
- CN103218771A CN103218771A CN2013100732429A CN201310073242A CN103218771A CN 103218771 A CN103218771 A CN 103218771A CN 2013100732429 A CN2013100732429 A CN 2013100732429A CN 201310073242 A CN201310073242 A CN 201310073242A CN 103218771 A CN103218771 A CN 103218771A
- Authority
- CN
- China
- Prior art keywords
- depth
- value
- adaptation
- self
- parameter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000011084 recovery Methods 0.000 title claims abstract description 26
- 230000015556 catabolic process Effects 0.000 claims description 24
- 238000006731 degradation reaction Methods 0.000 claims description 24
- 230000003044 adaptive effect Effects 0.000 claims description 14
- 238000001914 filtration Methods 0.000 claims description 9
- 238000005457 optimization Methods 0.000 claims description 9
- 238000006467 substitution reaction Methods 0.000 claims description 8
- 230000002146 bilateral effect Effects 0.000 claims description 4
- 230000011218 segmentation Effects 0.000 claims description 2
- 230000008034 disappearance Effects 0.000 description 6
- 239000007787 solid Substances 0.000 description 5
- 230000007812 deficiency Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000003716 rejuvenation Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 239000012141 concentrate Substances 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
Images
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a parameter self-adaptation selecting method based on autoregressive model depth recovery, and belongs to the field of computer vision. The parameter self-adaptation selecting method based on the autoregressive model depth recovery aims at achieving recovery of an optimal final depth image according to various characteristics of depth images and color images. According to the technical scheme, the parameter self-adaptation selecting method based on the autoregressive model depth recovery comprises the following steps: (1) specifically formulating a depth recovery problem to be an autoregressive model solving equation, (2) carrying out self-adaptation dereferencing on sigma, namely carrying out the self-adaptation dereferencing on the sigma through texture structures of the color images and an initial depth image, (3) carrying out self-adaptation dereferencing on tg, namely carrying out the self-adaptation dereferencing on the tg through gradient gc of the color images, finally calculating a final prediction coefficient a, and (4) substituting the prediction coefficient a to an energy equation E(f) for final solution. The parameter self-adaptation selecting method based on the autoregressive model depth recovery is mainly applied in self-adaptation selection of multiple parameters in a depth image recovery process.
Description
Technical field
The invention belongs to computer vision field, relate to the self-adaptation choosing method of a plurality of parameters in the depth map recuperation.Specifically, adopt autoregressive model, recovering contaminated (low resolution, the band noise, have random error and structural mistake) in the process of depth map, make the method that each parameter that relates to rejuvenation is chosen according to the picture characteristics self-adaptation, promptly based on the parameter adaptive choosing method of autoregressive model depth recovery.
Background technology
Along with the progress of imaging technique, conventional laser scanning broken through by the degree of depth camera that appears on the market in recent years and three-dimensional coupling is carried out the restriction of Depth Imaging, can obtain the degree of depth of Real-time and Dynamic three-dimensional scenic more easily.But, be subjected to the restriction of many conditions, for example, need to gather in real time, need to realize complete dynamic scene, and the interference that prevents bias light, depth map that degree of depth camera is taken out exists a lot of not enough: 1) resolution low 2) image is subjected to noise pollution serious 3) fathom influenced by bias light and inaccurate 4) exist at random or structurized depth value disappearance.We call the deficiency of these existence the depth map that contains degradation model.Because the existence of these degradation models makes that the application of degree of depth camera reality is not very extensive.So the method for seeking a kind of depth recovery is necessary.
Present stage, work both at home and abroad adopts the high-resolution cromogram through alignment to obtain high-quality depth map with the depth map that contains degradation model.Like this, edge that just in time can corresponding coloured image in the hypothesis degree of depth discontinuous place, and the zone of solid colour is contained under the situation of similar 3D (three-dimension) geometry, can utilize high-resolution chromatic information that the low resolution depth map is carried out super-resolution optimization.The topmost method of one class is to adopt wave filter to optimize depth map.These class methods can be handled low resolution simultaneously, have noise and contain the multiple mixing degradation model that depth value lacks.Wherein, a kind of is to utilize two-sided filter to realize super-resolution; Not by fuzzy,, be applied to depth map because two-sided filter can well keep the edge in filtering, can when enlarging resolution, carry out denoising image according to these characteristics.Recently, also have a kind of method to adopt non-local mean filtering to realize depth recovery.They can judge the characteristic of analog structure by non-local mean filtering, realize super-resolution and image denoising.
But, related to many in the said method and complicated parameter, and all be rule of thumb to be worth to set, parameter is not carried out self-adaptation and choose at the structure of image and characteristic.Different picture depths is recovered only to adopt same kind of parameter setting, will inevitably cause some image well not recover, and perhaps can not reach optimum solution.
Summary of the invention
The present invention is intended to overcome the deficiencies in the prior art, realization recovers optimum ultimate depth figure according to the characteristic of different depth maps and cromogram, and for this reason, the technical scheme that the present invention takes is, parameter adaptive choosing method based on the autoregressive model depth recovery comprises the following steps:
1) the depth recovery problem is expressed as the autoregressive model solving equation particularly:
Equation comprises two parts, wherein, and E
ARBe the autoregressive model item, D is the ultimate depth figure that wishes to get, and D (f), D (p) are respectively the depth value of 2 of f, p, | f-p| represents the space length between neighborhood territory pixel and the current pixel, and unit is a pixel, and S is the current pixel neighborhood of a point, and a is a predictive coefficient; E
DataBe data item, G is the depth information that contains degradation model that can be observed, and G (f) is the depth value of pixel f on G, and ζ is the set of significant depth information among the G, and E (f) is an energy function, and λ is a weighting factor, the weight between two of the balances;
Predictive coefficient a predicts that by known cromogram and the initial depth figure that obtains with the neighbor interpolation method constructive method is as follows: a=a
c* a
g, wherein, a
cBe the coefficient that prediction obtains on cromogram, a
gBe the coefficient that prediction obtains on initial depth figure, equation is as follows:
Wherein,
Wherein, for each p point in the neighborhood of current pixel f, P (I
f), P (I
p) be respectively with value of color I
f, I
pPrediction piece for the center; σ is control a
F, pThe parameter of size, B
fBe the bilateral filtering item, B
f(f, p) representative is from 2 B that calculate of f, p
fIn weights, τ
1, τ
2Be respectively the regulation and control parameter of spatial resolution and colour resolution, I (f), I (p) represent the value of color of 2 of f on the cromogram, p; D
gBe the fuzzy initial depth figure after the interpolation, D
g(f), D
g(p) be respectively the depth value of 2 of f, p, τ
gBe control a
gThe parameter of size;
In the above-mentioned equation, following parameter: τ is arranged
g, σ, τ
1, τ
2, λ, wherein, τ
1, τ
2, λ is different and be consistent along with image restored, is taken as definite value; τ
g, σ adopts different self-adaptation means to come value respectively, method sees 2), 3);
2) σ is carried out the self-adaptation value: texture structure and initial depth figure by cromogram carry out the self-adaptation value to σ;
3) to τ
gCarry out the self-adaptation value: by the gradient g of cromogram
cCome τ
gCarry out the self-adaptation value, last, calculate final predictive coefficient a;
4) predictive coefficient a substitution energy equation E (f) is finally found the solution.
To contain following degradation model respectively: 1) low resolution; 2) low resolution and contain much noise; 3) have random error; 4) depth map that has structural mistake can be tried to achieve final optimization D as a result as the observation depth map G substitution energy equation of needs optimization.
21) utilize the Sigmoid function that the relation between initial depth figure gradient and the σ value is carried out match:
When the pixel that will recover is in degree of depth smooth domain, adopt a big σ value to remove the depth structure that recovers current, promptly the σ value is big more, and weight is more little, and cromogram predicts that partly weight is more little; On the contrary, if pixel is in abundant depth texture zone, adopt a little σ value to come predetermined depth texture information better, the gradient information of initial depth figure can well reflect the degree of enriching of depth texture, so the gradient of employing initial depth figure goes to estimate the value of σ;
22) utilize the span realization self-adaptation of piecewise function to σ
The σ value that on behalf of us, the minimum value that the Sigmoid function can be got can get to peaked scope, according to equation (2), the predictive coefficient a of coloured image
cBe to be that the colored piece P at center predicts according to current pixel, so, the span of σ of having enriched degree direct influence of the texture structure of colored piece comes the span of σ is carried out self-adaptation according to the Grad structural segmentation function of the color texture structure of current pixel piece.
31) pass through g
cCome τ
gCarry out the self-adaptation value;
32) pass through two parameter τ
g, the self-adaptation of σ solves two predictive coefficient a
c, a
g, getting predictive coefficient to the end, equation is as follows:
a=a
c×a
g (4)
Wherein, a is final predictive coefficient.
Technical characterstic of the present invention and effect:
The inventive method is at the various depth maps that contain degradation model, adopt the depth recovery framework of autoregressive model, by necessary parameter in the rejuvenation is carried out the self-adaptation value, realized recovering optimum ultimate depth figure according to the characteristic of different depth maps and cromogram.Have following characteristics:
1, program is simple, is easy to realize.
2, adopt optimum at present autoregressive model framework that the depth map that contains degradation model is carried out depth recovery.
3, distinguish main influence parameter and the non-main parameter that influences.According to test, the non-main parameter that influences is adopted fixed value, the main parameter that influences is then adopted adaptive approach.Make that like this program execution speed is faster.
4,, not only realized value, and realized dynamic value the σ span according to picture structure for parameter σ.
5, for parameter τ
g, realized that employing color texture structure carries out the self-adaptation value to it.
6,, realized that at the different depth maps that contains degradation model automatic regulation and control parameter means are all arranged, rather than adopted the same value by the self-adaptation of parameter.Make at each special depth map that like this restoration result of an optimum all can be arranged.
Description of drawings
Fig. 1 is actual implementing procedure figure;
Fig. 2 is fuzzy initial depth figure;
Fig. 3 is a cromogram;
Fig. 4 is the depth map that contains degradation model, and is among the figure, upper left: structural disappearance; Lower-left: lack at random; Upper right: the band noise; Bottom right: low resolution;
Fig. 5 is the gradient map of initial depth figure;
Fig. 6 is the Sigmoid functional arrangement;
Fig. 7 is the gradient information figure of cromogram;
Fig. 8 be four kinds of degradation models finally find the solution depth map.Among the figure, restoration result: upper left: structural disappearance; Lower-left: lack at random; Upper right: the band noise; Bottom right: low resolution.
Embodiment
The present invention adopts autoregressive model that the depth map that contains degradation model is carried out depth recovery, by the cromogram of alignment and the initial fuzzy depth map of being constructed are predicted, come the depth map that contains degradation model (band noise, low resolution contain at random, structure disappearance) is recovered; In this process, at the structure of image and characteristic parameter is carried out self-adaptation and choose.Comprise the following steps:
Below in conjunction with embodiment and accompanying drawing the parameter adaptive that the present invention is based on the recovery of autoregressive model frame depth is made a detailed description.
The present invention adopts autoregressive model that the depth map that contains degradation model is carried out depth recovery, and utilize the onesize depth map that provides in the Middlebury data set and cromogram as experimental data, and the hypothesis degree of depth discontinuous place just in time can corresponding coloured image the edge, and similar 3D (three-dimension) geometry is contained in the zone of solid colour.
1) structure primary data and solving equation, and list the adaptive parameter of needs.
11) with MTD figure down-sampling, the mode with the neighbor interpolation method is upsampled to original resolution again, obtains fuzzy initial depth figure.We predict the depth map to containing degradation model (band noise, low resolution contain at random, structure disappearance) to recover by alignment cromogram and initial depth figure.
12) the depth recovery problem is expressed as the autoregressive model solving equation particularly:
Equation comprises two parts, wherein, and E
ARBe the autoregressive model item, D is the depth map of wishing to get, and D (f), D (p) are respectively the depth value of 2 of f, p, | f-p| represents the space length between a f and the some p, and unit is a pixel, and S is the current pixel neighborhood of a point, and a is a predictive coefficient; E
DataBe data item, G is the depth information that contains degradation model that can be observed, and G (f) is the depth value of pixel f on G, and ζ is the set of significant depth information among the G.E (f) is an energy function, and λ is a weighting factor, the weight between two of the balances.
13) predictive coefficient a predicts by known cromogram and the initial depth figure that obtains with the neighbor interpolation method.Constructive method is as follows: a=a
c* a
gWherein, a
cBe the coefficient that prediction obtains on cromogram, a
gBe the coefficient that prediction obtains on initial depth figure:
Wherein,
Wherein, for each p point in the neighborhood of current pixel f, P (I
f), P (I
p) be respectively with value of color I
f, I
pPrediction piece for the center; σ is control a
F, pThe parameter of size, B
fBe the bilateral filtering item of the gaussian kernel that has replaced the local filtering of non-average, B
f(f, p) representative is from 2 B that calculate of f, p
fIn weights, τ
1, τ
2Be respectively the regulation and control parameter of spatial resolution and colour resolution, I (f), I (p) represent the value of color of 2 of f on the cromogram, p; a
F, pValue big more, illustrate that the similarity of f, two points of p is high more, go to predict that with the p point the accurate probability of f point is very higher.D
gBe the fuzzy initial depth figure after the interpolation, D
g(f), D
g(p) be respectively the depth value of 2 of f, p, τ
gBe control a
gThe parameter of size.
14) in the above-mentioned equation, following parameter: τ is arranged
g, σ, τ
1, τ
2, λ.Wherein, τ
gBe controlled at the weight of predicting on the initial depth figure; The weight of predicting on the σ control cromogram; τ
1The size of control spatial resolution, τ
2The reversibility of control solving equation; Balance between λ balance data item and the autoregression item.By checking, τ
1, τ
2, λ is different and be consistent along with image restored, can be taken as definite value; τ
g, σ adopts different self-adaptation means to come value respectively, method sees 2), 3).
2) σ is carried out the self-adaptation value.
The weight of predicting on the σ control cromogram, and whether smoothly have very big related with actual grade zone that the pixel that will predict and its neighborhood are positioned.So we carry out the self-adaptation value by the texture structure and the initial depth figure of cromogram to σ.
21) utilize the Sigmoid function that the relation between initial depth figure gradient and the σ value is carried out match:
When the pixel that will recover is in degree of depth smooth domain, we adopt a big σ value (be that the σ value is big more, weight is more little, and cromogram predicts that partly weight is more little) to remove the depth structure that recovers current; On the contrary, if pixel is in abundant depth texture zone, we adopt a little σ value to come predetermined depth texture information better.The gradient information of initial depth figure can well reflect the degree of enriching of depth texture, so we adopt the gradient of initial depth figure to go to estimate the value of σ.The gradient method for solving of initial depth figure is as follows:
Wherein, D
gBe initial depth figure, grad represents image D
gAsk difference, grad
xBe to ask difference in the direction of horizontal ordinate, grad
yBe to ask difference in the direction of ordinate.
In view of the value of Grad and σ presents inverse relation, we adopt the Sigmoid function to go the relation of match between them, and equation is as follows:
Wherein, g
mBe the initial depth Grad of pixel, σ is according to g
mValue and value; The minimum value that on behalf of σ, min can get, the maximal value that on behalf of σ, max can get, max-min represents the span of σ; Phase control matched curve translational movement.
We extract the substantive test image at random out from Middlebury data set the inside and test, because depth information all is that non-ordinary light is sliding at most of location of pixels, most pixel gradient value is all concentrated and is distributed in a scope, and Phase is representing the Sigmoid average value of a function.Therefore, getting Phase is a definite value.
22) utilize the span realization self-adaptation of piecewise function to σ
The σ value that on behalf of us, the minimum value that the Sigmoid function can be got can get to peaked scope.According to equation (2), the predictive coefficient a of coloured image
cBe to be that the colored piece P at center predicts according to current pixel, so, the span of σ of having enriched degree direct influence of the texture structure of colored piece.The gradient information of cromogram can well reflect the degree of enriching of colored piece texture, so we adopt the gradient of cromogram to go to estimate the span of σ.The gradient method for solving of cromogram is as follows:
Wherein, g
cBe the Grad of the colored piece of current pixel, { B}, RGB represent redness respectively for R, G, and be green and blue, I in ζ representative color value space
kBe the pixel value of current color passage, k takes from the color space under the ζ.
Therefore, for each piece to be asked,, have the span of different σ according to its color texture structure.We fix other parameters σ are tested, and extract the colored piece of substantive test image once more from Middlebury data set the inside at random out and test, and obtain following result:
Wherein, the minimum value that on behalf of σ, min can get, the maximal value that on behalf of σ, max can get.So far, we have finished the adaptive prediction to parameter σ.
3) to τ
gCarry out the self-adaptation value.
31) though supposed in the discontinuous place of the degree of depth can corresponding coloured image the edge, and similar 3D (three-dimension) geometry is contained in the zone of solid colour, the texture of cromogram is more a lot of than the texture-rich of depth map.Predictive coefficient a
gIn originally level and smooth depth areas the texture structure similar to cromogram appears in order to prevent exactly, and τ
gThen control the weight of predicting on the initial depth figure.τ
gValue too small (being that weight becomes greatly) can cause image restored edge fog (the initial depth figure that is promptly blured influences); τ
gValue is excessive, and then the depth information of Hui Fuing can have the color texture that originally should not have.Because the gradient g of cromogram
cThe texture-rich degree that can reflect the current pixel piece is so pass through g
cCome τ
gCarry out the self-adaptation value, equation is as follows:
So far, realized that all need adaptive parameter.
32) pass through two parameter τ
g, the self-adaptation of σ solves two predictive coefficient a
ca
g, getting predictive coefficient to the end, equation is as follows:
a=a
c×a
g (9)
Wherein, a is final predictive coefficient.
4) predictive coefficient a substitution energy equation E (f) is finally found the solution.
We will contain following degradation model respectively: 1) low resolution; 2) low resolution and contain much noise; 3) have random error; 4) depth map that has structural mistake can be tried to achieve final optimization D as a result as the observation depth map G substitution energy equation of needs optimization.
The present invention adopts autoregressive model that the depth map that contains degradation model is carried out depth recovery (shown in the flow process of Fig. 1), and utilize the onesize depth map that provides in the Middlebury data set and cromogram as experimental data, and the hypothesis degree of depth discontinuous place just in time can corresponding coloured image the edge, and similar 3D (three-dimension) geometry is contained in the zone of solid colour, reaches embodiment in conjunction with the accompanying drawings and is described in detail as follows:
1) structure primary data and solving equation, and list the adaptive parameter of needs.
11) with MTD figure down-sampling, the mode with the neighbor interpolation method is upsampled to original resolution again, obtains fuzzy initial depth figure, as shown in Figure 2.We predict by alignment cromogram (as shown in Figure 3) and initial depth figure, and (band noise, low resolution contain at random, the structure disappearance, and depth map as shown in Figure 4) recovers to containing degradation model.
12) the depth recovery problem is expressed as the autoregressive model solving equation particularly:
Equation comprises two parts, wherein, and E
ARBe the autoregressive model item, D is the depth map of wishing to get, and D (f), D (p) are respectively the depth value of 2 of f, p, and S is the current pixel neighborhood of a point, and a is a predictive coefficient; E
DataBe data item, G is the depth information that contains degradation model that can be observed, and G (f) is the depth value of pixel f on G, and ζ is the set of significant depth information among the G.E (f) is an energy function, and λ is a weighting factor, the weight between two of the balances.
13) predictive coefficient a predicts by known cromogram and the initial depth figure that obtains with the neighbor interpolation method.Constructive method is as follows: a=a
c* a
gWherein, a
cBe the coefficient that prediction obtains on cromogram, a
gBe the coefficient that prediction obtains on initial depth figure:
Wherein,
Wherein, for each p point in the neighborhood of current pixel f, P (I
f), P (I
p) be respectively with value of color I
f, I
pPrediction piece for the center; σ is control a
F, pThe parameter of size, B
fBe the bilateral filtering item of the gaussian kernel that has replaced the local filtering of non-average, B
f(f, p) representative is from 2 B that calculate of f, p
fIn weights, τ
1, τ
2Be respectively the regulation and control parameter of spatial resolution and colour resolution, I (f), I (p) represent the value of color of 2 of f on the cromogram, p; a
F, pValue big more, illustrate that the similarity of f, two points of p is high more, go to predict that with the p point the accurate probability of f point is very higher.D
gBe the fuzzy depth map after the interpolation, D
g(f), D
g(p) be respectively the depth value of 2 of f, p, τ
gBe control a
gThe parameter of size.
14) in the above-mentioned equation, following parameter: τ is arranged
g, σ, τ
1, τ
2, λ.Wherein, τ
gBe controlled at the weight of predicting on the initial depth figure; The weight of predicting on the σ control cromogram; τ
1The size of control spatial resolution, τ
2The reversibility of control solving equation; Balance between λ balance data item and the autoregression item.By checking, τ
1, τ
2, λ is different and be consistent along with image restored, can be taken as definite value, we set τ here
1=3, τ
2=0.1, λ=0.01; τ
g, σ adopts different self-adaptation means to come value respectively, method sees 2), 3).
2) σ is carried out the self-adaptation value
The weight of predicting on the σ control cromogram, and whether smoothly have very big related with actual grade zone that the pixel that will predict and its neighborhood are positioned.So we carry out the self-adaptation value by the texture structure and the initial depth figure of cromogram to σ.
21) utilize the Sigmoid function that the relation between initial depth figure gradient and the σ value is carried out match:
When the pixel that will recover is in degree of depth smooth domain, we adopt a big σ value (be that the σ value is big more, weight is more little, and cromogram predicts that partly weight is more little) to remove the depth structure that recovers current; On the contrary, if pixel is in abundant depth texture zone, we adopt a little σ value to come predetermined depth texture information better.The gradient information of initial depth figure (as shown in Figure 5) can well reflect the degree of enriching of depth texture, so we adopt the gradient of initial depth figure to go to estimate the value of σ.The gradient method for solving of initial depth figure is as follows:
Wherein, D
gBe initial depth figure, grad represents image D
gAsk difference, grad
xBe to ask difference in the direction of horizontal ordinate, grad
yBe to ask difference in the direction of ordinate.
In view of the value of Grad and σ presents inverse relation, we adopt the Sigmoid function to go the relation of match between them, and equation is as follows:
Wherein, g
mBe the initial depth Grad of pixel, σ is according to g
mValue and value; The minimum value that on behalf of σ, min can get, the maximal value that on behalf of σ, max can get, max-min represents the span of σ; Phase control matched curve translational movement.The image of function as shown in Figure 6.
We extract 100 test pattern pieces out at random from Middlebury data set the inside, and construct initial depth figure, and ask for the gradient of each pixel on the initial depth figure, and the histogram that draws.Can draw according to histogrammic distribution, because depth information all is that non-ordinary light is sliding in most of pixel, most pixel gradient value all is distributed in below 1.5, and therefore, according to test, we are decided to be 1.5. with Phase
22) utilize the span realization self-adaptation of piecewise function to σ
The σ value that on behalf of us, the minimum value that the Sigmoid function can be got can get to peaked scope.According to equation (2), the predictive coefficient a of coloured image
cBe to be that the colored piece P at center predicts according to current pixel, so, the span of σ of having enriched degree direct influence of the texture structure of colored piece.The gradient information of cromogram (as shown in Figure 7) can well reflect the degree of enriching of colored piece texture, so we adopt the gradient of cromogram to go to estimate the span of σ.The gradient method for solving of cromogram is as follows:
Wherein, g
cBe the Grad of the colored piece of current pixel, { B}, RGB represent redness respectively for R, G, and be green and blue, I in ζ representative color value space
kBe the pixel value of current color passage, k takes from the color space under the ζ.
Therefore, for each piece to be asked,, have the span of different σ according to its color texture structure.So we fix other parameters σ is tested, extract the colored piece of 100 test patterns once more at random out from Middlebury data set the inside.By observation to the color gradient histogram distribution, Grad is distributed in 3 zones, the color gradient value that is most of piece is distributed between 5 to 8, so the small part point concentrates on less than 5 or greater than 8., we construct a discontinuous point decides σ at 5 and 8 piecewise functions value:
Wherein, g
cBe the Grad of the colored piece of current pixel, the minimum value that on behalf of σ, min can get, the maximal value that on behalf of σ, max can get.So far, we have finished the parameter adaptive prediction on cromogram.
3) to τ
gCarry out the self-adaptation value
31) though supposed in the discontinuous place of the degree of depth can corresponding coloured image the edge, and similar 3D (three-dimension) geometry is contained in the zone of solid colour, the texture of cromogram is more a lot of than the texture-rich of depth map.Predictive coefficient a
gIn originally level and smooth depth areas the texture structure similar to cromogram appears in order to prevent exactly, and τ
gThen control the weight of predicting on the initial depth figure.τ
gValue too small (being that weight becomes greatly) can cause image restored edge fog (the initial depth figure that is promptly blured influences); τ
gValue is excessive, and then the depth information of Hui Fuing can have the color texture that originally should not have.Because the gradient g of cromogram
cThe texture-rich degree that can reflect the current pixel piece is so pass through g
cCome τ
gCarry out the self-adaptation value, equation is as follows:
Wherein, work as g
cGreater than 8 o'clock, adopt a less τ
g=2.5 limit the unnecessary texture structure of generation; Work as g
cLess than 5 o'clock, adopt a bigger τ
g=4; Work as g
cIn the time of between being in 5 and 8, just get an intermediate value τ
g=3.
So far, realized that all need adaptive parameter.
32) pass through two parameter τ
g, the self-adaptation of σ solves two predictive coefficient a
ca
g, getting predictive coefficient to the end, equation is as follows:
a=a
c×a
g (9)
Wherein, a is final predictive coefficient.
4) predictive coefficient a substitution energy equation E (f) is finally found the solution.
We will contain following degradation model respectively: 1) low resolution; 2) low resolution and contain much noise; 3) have random error; 4) depth map that has structural mistake can be tried to achieve final optimization D as a result as the observation depth map G substitution energy equation of needs optimization.Four kinds of degradation models finally find the solution depth map as shown in Figure 8.
Claims (3)
1. the parameter adaptive choosing method based on the autoregressive model depth recovery is characterized in that, comprises the following steps:
1) the depth recovery problem is expressed as the autoregressive model solving equation particularly:
Equation comprises two parts, wherein, and E
ARBe the autoregressive model item, D is the ultimate depth figure that wishes to get, and D (f), D (p) are respectively the depth value of 2 of f, p, | f-p| represents the space length between neighborhood territory pixel and the current pixel, and S is the current pixel neighborhood of a point, and a is a predictive coefficient; E
DataBe data item, G is the depth information that contains degradation model that can be observed, and G (f) is the depth value of pixel f on G, and ζ is the set of significant depth information among the G, and E (f) is an energy function, and λ is a weighting factor, the weight between two of the balances;
Predictive coefficient a predicts that by known cromogram and the initial depth figure that obtains with the neighbor interpolation method constructive method is as follows: a=a
c* a
g, wherein, a
cBe the coefficient that prediction obtains on cromogram, a
gBe the coefficient that prediction obtains on initial depth figure, equation is as follows:
Wherein,
(2)
Wherein, for each p point in the neighborhood of current pixel f, P (I
f), P (I
p) be respectively with value of color I
f, I
pPrediction piece for the center; σ is control a
F, pThe parameter of size, B
fBe the bilateral filtering item, B
f(f, p) representative is from 2 B that calculate of f, p
fIn weights, τ
1, τ
2Be respectively the regulation and control parameter of spatial resolution and colour resolution, I (f), I (p) represent the value of color of 2 of f on the cromogram, p; D
gBe the fuzzy initial depth figure after the interpolation, D
g(f), D
g(p) be respectively the depth value of 2 of f, p, τ
gBe control a
gThe parameter of size;
In the above-mentioned equation, following parameter: τ is arranged
g, σ, τ
1, τ
2, λ, wherein, τ
1, τ
2, λ is different and be consistent along with image restored, is taken as definite value; τ
g, σ adopts different self-adaptation means to come value respectively, method sees 2), 3);
2) σ is carried out the self-adaptation value: texture structure and initial depth figure by cromogram carry out the self-adaptation value to σ;
3) to τ
gCarry out the self-adaptation value: by the gradient g of cromogram
cCome τ
gCarry out the self-adaptation value, last, calculate final predictive coefficient a;
4) predictive coefficient a substitution energy equation E (f) is finally found the solution.
To contain following degradation model respectively: 1) low resolution; 2) low resolution and contain much noise; 3) have random error; 4) depth map that has structural mistake can be tried to achieve final optimization D as a result as the observation depth map G substitution energy equation of needs optimization.
2. the parameter adaptive choosing method based on the autoregressive model depth recovery as claimed in claim 1 is characterized in that, σ is carried out the self-adaptation value be specially:
When the pixel that will recover is in degree of depth smooth domain, adopt a big σ value to remove the depth structure that recovers current, promptly the σ value is big more, and weight is more little, and cromogram predicts that partly weight is more little; On the contrary, if pixel is in abundant depth texture zone, adopt a little σ value to come predetermined depth texture information better, the gradient information of initial depth figure can well reflect the degree of enriching of depth texture, so the gradient of employing initial depth figure goes to estimate the value of σ;
22) utilize the span realization self-adaptation of piecewise function to σ
The σ value that on behalf of us, the minimum value that the Sigmoid function can be got can get to peaked scope, according to equation (2), the predictive coefficient a of coloured image
cBe to be that the colored piece P at center predicts according to current pixel, so, the span of σ of having enriched degree direct influence of the texture structure of colored piece comes the span of σ is carried out self-adaptation according to the Grad structural segmentation function of the color texture structure of current pixel piece.
3. the parameter adaptive choosing method based on the autoregressive model depth recovery as claimed in claim 1 is characterized in that, to τ
gCarrying out the self-adaptation value is specially
31) pass through g
cCome τ
gCarry out the self-adaptation value;
32) pass through two parameter τ
g, the self-adaptation of σ solves two predictive coefficient a
c, a
g, getting predictive coefficient to the end, equation is as follows:
a=a
c×a
g (4)
Wherein, a is final predictive coefficient.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201310073242.9A CN103218771B (en) | 2013-03-07 | 2013-03-07 | Based on the parameter adaptive choosing method of autoregressive model depth recovery |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201310073242.9A CN103218771B (en) | 2013-03-07 | 2013-03-07 | Based on the parameter adaptive choosing method of autoregressive model depth recovery |
Publications (2)
Publication Number | Publication Date |
---|---|
CN103218771A true CN103218771A (en) | 2013-07-24 |
CN103218771B CN103218771B (en) | 2015-09-23 |
Family
ID=48816526
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201310073242.9A Expired - Fee Related CN103218771B (en) | 2013-03-07 | 2013-03-07 | Based on the parameter adaptive choosing method of autoregressive model depth recovery |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN103218771B (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105261008A (en) * | 2015-09-15 | 2016-01-20 | 天津大学 | Method for estimating boundary protection depth video based on structure information |
CN105657402A (en) * | 2016-01-18 | 2016-06-08 | 深圳市未来媒体技术研究院 | Depth map recovery method |
CN106056622A (en) * | 2016-08-17 | 2016-10-26 | 大连理工大学 | Multi-view depth video recovery method based on Kinect camera |
CN106447714A (en) * | 2016-09-13 | 2017-02-22 | 大连理工大学 | Scene depth recovery method based on signal decomposition |
CN110197029A (en) * | 2019-05-29 | 2019-09-03 | 包头钢铁(集团)有限责任公司 | A kind of analysis method of analogue simulation material parameter |
CN117689663A (en) * | 2024-02-04 | 2024-03-12 | 电科科知(成都)科技集团有限公司 | Visual detection method and system for welding robot |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102663712A (en) * | 2012-04-16 | 2012-09-12 | 天津大学 | Depth calculation imaging method based on flight time TOF camera |
CN102722863A (en) * | 2012-04-16 | 2012-10-10 | 天津大学 | Super-resolution reconstruction method for depth map by adopting autoregressive model |
-
2013
- 2013-03-07 CN CN201310073242.9A patent/CN103218771B/en not_active Expired - Fee Related
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102663712A (en) * | 2012-04-16 | 2012-09-12 | 天津大学 | Depth calculation imaging method based on flight time TOF camera |
CN102722863A (en) * | 2012-04-16 | 2012-10-10 | 天津大学 | Super-resolution reconstruction method for depth map by adopting autoregressive model |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105261008A (en) * | 2015-09-15 | 2016-01-20 | 天津大学 | Method for estimating boundary protection depth video based on structure information |
CN105657402A (en) * | 2016-01-18 | 2016-06-08 | 深圳市未来媒体技术研究院 | Depth map recovery method |
CN106056622A (en) * | 2016-08-17 | 2016-10-26 | 大连理工大学 | Multi-view depth video recovery method based on Kinect camera |
CN106056622B (en) * | 2016-08-17 | 2018-11-06 | 大连理工大学 | A kind of multi-view depth video restored method based on Kinect cameras |
CN106447714A (en) * | 2016-09-13 | 2017-02-22 | 大连理工大学 | Scene depth recovery method based on signal decomposition |
CN106447714B (en) * | 2016-09-13 | 2019-04-09 | 大连理工大学 | A kind of scene depth restoration methods based on signal decomposition |
CN110197029A (en) * | 2019-05-29 | 2019-09-03 | 包头钢铁(集团)有限责任公司 | A kind of analysis method of analogue simulation material parameter |
CN117689663A (en) * | 2024-02-04 | 2024-03-12 | 电科科知(成都)科技集团有限公司 | Visual detection method and system for welding robot |
CN117689663B (en) * | 2024-02-04 | 2024-04-26 | 电科科知(成都)科技集团有限公司 | Visual detection method and system for welding robot |
Also Published As
Publication number | Publication date |
---|---|
CN103218771B (en) | 2015-09-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN102722863B (en) | Super-resolution reconstruction method for depth map by adopting autoregressive model | |
CN107154023B (en) | Based on the face super-resolution reconstruction method for generating confrontation network and sub-pix convolution | |
CN103218771A (en) | Parameter self-adaptation selecting method based on autoregressive model depth recovery | |
CN103514580B (en) | For obtaining the method and system of the super-resolution image that visual experience optimizes | |
CN109255831A (en) | The method that single-view face three-dimensional reconstruction and texture based on multi-task learning generate | |
CN107066916B (en) | Scene semantic segmentation method based on deconvolution neural network | |
CN109671023A (en) | A kind of secondary method for reconstructing of face image super-resolution | |
CN109272447A (en) | A kind of depth map super-resolution method | |
CN103093444A (en) | Image super-resolution reconstruction method based on self-similarity and structural information constraint | |
CN103455984A (en) | Method and device for acquiring Kinect depth image | |
CN101729791A (en) | Apparatus and method for image processing | |
CN103034983B (en) | A kind of defogging method capable based on anisotropic filtering | |
Shen et al. | Convolutional neural pyramid for image processing | |
CN103971354A (en) | Method for reconstructing low-resolution infrared image into high-resolution infrared image | |
CN103440662A (en) | Kinect depth image acquisition method and device | |
Ye et al. | Depth super-resolution with deep edge-inference network and edge-guided depth filling | |
CN112489050A (en) | Semi-supervised instance segmentation algorithm based on feature migration | |
CN114092774B (en) | RGB-T image significance detection system and detection method based on information flow fusion | |
CN105550989A (en) | Image super-resolution method based on nonlocal Gaussian process regression | |
CN111899295A (en) | Monocular scene depth prediction method based on deep learning | |
CN110415169B (en) | Depth map super-resolution reconstruction method, system and electronic equipment | |
CN105787932A (en) | Stereo matching method based on segmentation cross trees | |
CN104661013B (en) | A kind of virtual viewpoint rendering method based on spatial weighting | |
CN104159098B (en) | The translucent edge extracting method of time domain consistence of a kind of video | |
Liao et al. | Multi-scale mutual feature convolutional neural network for depth image denoise and enhancement |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20150923 |
|
CF01 | Termination of patent right due to non-payment of annual fee |