CN104484884A - Intrinsic image decomposition method based on multi-scale L0 sparse constraint - Google Patents

Abstract

The invention discloses an intrinsic image decomposition method based on multi-scale L0 sparse constraint. The method includes the following steps that reflectivity components (please see the symbol in the specification) are initialized, and a non-local zero paradigm sparse constraint model is used for obtaining a similarity measure WR of the reflectivity components among pixels; a similarity measure WS of the illumination components among the pixels is calculated; the reflectivity component R and the illumination component S of the layer are obtained by solving an optimization problem. Experimental results show that non-local zero paradigm sparse constraint can ensure global consistency of intrinsic image decomposition, by using the hierarchical iteration method, the efficiency problem of non-local zero paradigm sparse constraint expression is solved, and dependency on the chromaticity characteristics is reduced. According to standard data and tests, the accuracy of the results of the method is improved substantially, so that the intrinsic image decomposition method is beneficial for improving the results of intrinsic image decomposition.

Description

A kind of intrinsic image decomposition method based on multiple dimensioned L0 sparse constraint

Technical field

The present invention relates to image processing field, particularly relate to a kind of intrinsic image decomposition method based on multiple dimensioned L0 sparse constraint.

Background technology

The target that intrinsic image decomposes is rely on picture breakdown the part of material and rely on the part of illumination, i.e. reflectance component and illumination component.Barrow and Tenenbaum ^[1]first proposed the isolation of this picture, and reflectance component and illumination component are referred to as the essential characteristic of image.Because decompose each part of obtaining to represent a different physical element, so intrinsic image decomposes have application in a lot of problems of computer vision and computer graphics, such as: heavy painted ^[2], Iamge Segmentation and object identification ^[3]deng.

But this problem remains a challenging problem, because it has serious ill feature, namely provide an image, the number of unknown quantity is the twice of equation number.In order to address this problem, early stage researcher utilizes the plurality of pictures of Same Scene under different light to obtain good result to do eigen decomposition ^[4,5], this harsh condition limits actual application.In recent years, utilize single picture to carry out eigen decomposition and obtain more concern.

Solving the general method of this ill-conditioning problem is add some priori conditions, Retinex model ^[6,7]be use maximum priori conditions, its hypothesis is the change that illumination component causes is by a small margin, and significantly change is caused by reflectance component.But although this simply supposes to be difficult to ensure in real scene image directly perceived.So based on the method for training ^[8,9]be developed the variation characteristic obtaining reflectance component and illumination component and cause.But the result of training is relevant to training data, is difficult to the rule comprising all images.Shen ^[10]propose a kind of method, with the weighted sum of the local window of the neighborhood of pixels reflectance component as a pixel, thus eigen decomposition problem is converted into function minimization problem.

These methods are above all the priori restrictive conditions using local, and nearest research shows that the restriction of non-local priori can ensure the consistance of the overall situation and improve decomposition result greatly.Shen and Yeo proposes a kind of priori restriction of the overall situation ^[11], the sum of reflectance component color is little, and it can be used as the cost of the number of the different value of reflectance component in image.Zhao ^[12]propose another kind of non local priori restriction according to texture analysis, if the pixel of two non-neighbors has similar texture structure, they generally should have similar reflectance component.These methods all depend on chromatic value, and chromatic value sometimes and unreliable.Especially in natural image.Therefore, need badly and propose a kind ofly to rely on less overall restrictive condition to chromatic value.

Summary of the invention

The invention provides a kind of intrinsic image decomposition method based on multiple dimensioned L0 sparse constraint, non-local zero normal form sparse constraint adds in original intrinsic image decomposition method by the present invention, and a kind of method proposing hierarchical alterative solves efficiency, decrease the dependence to chromatic value simultaneously, described below:

Based on an intrinsic image decomposition method for multiple dimensioned L0 sparse constraint, said method comprising the steps of:

Initialization reflectance component

Non-local zero normal form sparse constraint model is utilized to obtain the measuring similarity W of reflectance component between pixel ^r;

Calculate the tolerance W of illumination component similarity between pixel ^s;

Reflectance component R and the illumination component S of this layer is obtained by solving optimization problem.

Describedly non-local zero normal form sparse constraint model is utilized to obtain the measuring similarity W of reflectance component between pixel ^rstep be specially:

1) by initialization reflectance component obtain feature vector, X _i;

2) the dictionary D that each pixel is corresponding is obtained _i;

3) X is obtained by orthogonal matching pursuit algorithm solution L0 minimization problem _iat dictionary D _iunder sparse expression factor alpha _i;

4) by weighing the similarity of different pixels point reflection rate component based on regularization residue, the measuring similarity W of reflectance component between pixel is obtained ^r.

The described step by solving reflectance component R and illumination component S that optimization problem obtains this layer is specially:

1) objective function optimized is built;

2) objective function is converted into standard double optimization, obtains the solution of closed form, judge whether to enter next iteration still as final solution.

The objective function of described optimization is specially:

$F\left(S\right)=\underset{i\↔j}{\mathrm{\Σ}}{w}_{\mathrm{ij}}^s{(S_i-S_j)}^2+\underset{i~j}{\mathrm{\Σ}}{w}_{\mathrm{ij}}^R{(\mathrm{\Δ}{I}_{\mathrm{ij}}-S_i+S_j)}^2+\underset{i\∈B}{\mathrm{\Σ}}{S}_i^2$

Wherein, S _i, S _jrepresent pixel i, the illumination component of j; represent pixel i, the similarity of the illumination component of j; represent pixel i, the similarity of j reflectance component; represent the set that neighbor is right, i ~ j represents the right set of non-conterminous pixel; Δ I _ij=I _i-I _jrepresent pixel i, the difference of j intensity; The set of pixel the brightest in B representative image.

The beneficial effect of technical scheme provided by the invention is: first use the colourity of image as initial reflectance component, re-use the similarity measure that zero normal form sparse constraint solves the reflectance component of this layer, then obtained reflectance component and the illumination component of this layer by optimization energy function, bring next iteration or Output rusults into.Experimental result shows, the technology that the present invention proposes can improve the precision of single image eigen decomposition significantly.

Accompanying drawing explanation

Fig. 1 is experimental result picture;

Fig. 2 is a kind of process flow diagram of the intrinsic image decomposition method based on multiple dimensioned L0 sparse constraint.

Embodiment

For making the object, technical solutions and advantages of the present invention clearly, below embodiment of the present invention is described further in detail.

The present invention proposes a kind of hierarchical alterative method of single image eigen decomposition.First use the colourity of image as initial reflectance component, re-use the similarity measure that zero normal form sparse constraint solves the reflectance component of this layer, then reflectance component and the illumination component of this layer is obtained by optimization energy function, bring next iteration or Output rusults into, concrete technical scheme comprises following content: set M as the number of plies, image pyramid is obtained, iteration following steps M time by up-sampling and down-sampling.

101: initialization reflectance component

That is, if input picture is I, the ratio of adjacent level is d, hierachy number is M, degree of rarefication be t and hybrid weight is λ.The image of initialization every layer the colourity of every layer following 102-104 step needs iteration M time.

Obtain the initial reflectance component of every layer

If ground floor, directly use chromatic value, now

If other layers, the weighted sum of the reflectance component up-sampling of chromatic value and last layer is used to obtain, now ${\hat{R}}^k=\mathrm{\λ}{C}^k+(1-\mathrm{\λ})R_{\↑}^{k-1},$ Wherein represent the up-sampling of reflectance component.

102: utilize non-local zero normal form sparse constraint model to obtain the tolerance W of reflectance component similarity between pixel ^r;

1) reflectance component of the first step is first utilized to obtain X _i;

By in the window of K*K around each pixel three dimensions couple together, obtain 3K ²the feature vector, X of dimension _i.Be set to this layer of pixel i characteristic of correspondence vector.

2) the dictionary D that each pixel is corresponding is obtained _i;

The dictionary that zero original normal form restricted coefficients of equation method needs is very large, and efficiency is very low, in order to reduce the dimension of dictionary, improves the efficiency of algorithm.Nonzero term in the sparse coefficient that this method uses upper strata to obtain obtains the dictionary of this layer by up-sampling and down-sampling.

If ground floor, by the feature vector, X of all pixels _iform matrix by row, be dictionary; Otherwise construct the dictionary D that this layer of pixel i is corresponding _istep as follows:

Determine the expression factor alpha that upper strata obtains _i-1the pixel set G of the down-sampling that the nonzero term in (being obtained by 102-103 step in last iteration) is corresponding, these pixels are relevant to the reflectance component of i;

Determine the pixel set that the up-sampling of pixel set G obtains, in pixel set G, the block of pixels of corresponding this layer of each pixel, remembers that the set of all pixel index of these block of pixels is k is the number of pixel in set.

By the X of these pixels _imatrix, i.e. dictionary is formed by row $D_i=\left[\begin{array}{cccc}{X}_{c_0}& X_{c_1}& ...& X_{c_k}\end{array}\right].$

3) by orthogonal matching pursuit algorithm (Orthogonal Matching Pursuit) ^[14]solution L0 minimization problem below obtains X _iat dictionary D _iunder sparse expression factor alpha _i.

$\mathrm{Min}{\left|\right|X_i-D_i{\mathrm{\α}}_i\left|\right|}_2^2,s.t.{\left|\right|{\mathrm{\α}}_i\left|\right|}_0\≤t$

Wherein, || α _i|| ₀represent L0 normal form, return the number of nonzero term, t controls the degree of rarefication of expressing, and s.t. represents " submitting to ".By the similarity weighing different pixels point reflection rate component based on regularization residue ^[13,15], the similarity of the reflectance component of pixel i and pixel j can be defined as:

$w_{\mathrm{ij}}^R=\left\{{\left|\right|X_i-D_i{\mathrm{\α}}_i^j\left|\right|}_2^2\right\}/\left\{{\left|\right|X_i\left|\right|}_2^2\right\}$

Wherein, D _i, X _i, α _iby obtaining above, be defined as follows:

representation vector in l element, α _il () in like manner, will own the similarity matrix that is combined into just obtain W ^r.

103: the tolerance W calculating illumination component similarity between pixel ^s;

Wherein, Y _i, Y _jthe intensity of pixel i, j respectively, be the variance of the intensity distributions of the neighborhood of pixel i, e is natural constant; represent the similarity of pixel i and pixel j illumination component, by all be combined into matrix and namely obtain W ^s.

104: R and S obtaining this layer by separating optimization problem.

A) because can represent by the difference of former figure and illumination component S that at reflectance component part R the eigen decomposition problem arises of single width figure can be then the optimization problem of an energy function by i.e. R=I-S:

$F\left(S\right)={\mathrm{\Σ}}_{i\↔j}{w}_{\mathrm{ij}}^s{(S_i-S_j)}^2+{\mathrm{\Σ}}_{i~j}{w}_{\mathrm{ij}}^R{(\mathrm{\Δ}{I}_{\mathrm{ij}}-S_i+S_j)}^2+{\mathrm{\Σ}}_{i\∈B}{S}_i^2$

Wherein F (S) is the objective function that will optimize, and the illumination component S that get is the S of F (S) when getting minimum value.

S _i, S _jrepresent pixel i, the illumination component of j. represent pixel i, the similarity of the illumination component of j. represent pixel i, the similarity of j reflectance component. represent the set that neighbor is right, i ~ j represents the right set of non-conterminous pixel.Δ I _ij=I _i-I _jrepresent pixel i, the difference of j intensity.The set of pixel the brightest in B representative image.

B) be translated into shape as the double optimization problem of standard, wherein A is positive semi-definite square formation, and S is unknown quantity, and b is vector, and c is constant term.

I) form calculating A, A is as follows:

A＝4L(W ^S*)+4L(W ^R*)+2B

Wherein $W^{S*}=\frac{1}{2}(W^s+W^{s^T}),W^{R*}=\frac{1}{2}(W^R+W^{R^T})$

L (W ^r*) be the W calculated ^r*laplacian matrix ^[16], it is matrix W ^stransposition, L (W ^s*) in like manner, in like manner.B is diagonal matrix, if S _i∈ B, B _ii=1, otherwise be 0.

Ii) b is calculated ^t.B is a vector, wherein

$b\left(i\right)=\underset{j}{\overset{N}{\mathrm{\Σ}}}{w}_{\mathrm{ji}}^R\mathrm{\Δ}{I}_{\mathrm{ji}}+\underset{j}{\overset{N}{\mathrm{\Σ}}}{w}_{\mathrm{ij}}^R\mathrm{\Δ}{I}_{\mathrm{ij}}$

Wherein N is pixel count, Δ I _jirepresent pixel j, the intensity difference of i.

Δ I _ijrepresent pixel i, the intensity difference of j. represent pixel i, the similarity of the reflectance component of j. represent pixel j, the similarity of the reflectance component of i.

Iii) c is calculated.C is a constant,

$c=\underset{i}{\overset{N}{\mathrm{\Σ}}}\underset{j}{\overset{N}{\mathrm{\Σ}}}{w}_{\mathrm{ij}}^R\mathrm{\Δ}{I}_{\mathrm{ij}}^2$

Every implication and ii) in identical.

C) A is positive semi-definite, and this double optimization problem can obtain the solution of closed form, judges whether to enter next iteration still as final solution.

In order to be embodied in image eigen decomposition process the rationality applying overall zero normal form sparse constraint, in order to verify the validity of this method, this method is at standard data set MIT ^[17]on test, MIT data set comprises the image of three classes and gives correct result (representing with GT), by the result of this method and the best traditional color Retinex method of ttxe precedence effect ^[18](representing with CR) and nearest another are applied with the work of overall sparse restriction equally ^[12](representing with CFS) contrasts.

List the Comparative result of a pictures of each class in FIG, carry out weighing result by local mean square deviation (LMSE).Can see that CFS and this method obtain good result on jaguar (panther) and green turtle (turtle), but the results contrast of CR is poor.In the test of cup 1 (cup1), only have the result of this method and GT closely, obvious this method is better than other method.

Table 1 is that the present invention uses the mode of local mean square deviation (LMSE) to carry out quantification contrast

	Box	Quilt	Deer	Dinosaur	Frog 1	Frog 2	The sun
								CR	0.013	0.011	0.041	0.035	0066	0.071	0.003
CFS	0.005	0.005	0.045	0.026	0.051	0.069	0.002
								This method	0.007	0.004	0.042	0.028	0.050	0.046	0.003

	Newspaper 1	Newspaper 2	Racoon	Squirrel fish	Tea in bag 1	Tea in bag 2
								CR	0.004	0.004	0.015	0.073	0.041	0.023
CFS	0.008	0.005	0.04	0.074	0.042	0.017
								This method	0.002	0.005	0.004	0.073	0.020	0.026

The result of this method 0.021 is better than 0.030 of CR and 0.025 of CFS, and in the insecure image of some colourities, this method still obtains reasonable result, and this just proves that this method is really less to the dependence of colourity, and can obtain better degree of accuracy.

List of references:

[1]H.Barrow and J.Tenenbaum,“Recovering intrinsic scene characteristics from images,”Computer Vision Systems,pp.3–26,1978.

[2]X.Liu,L.Wan,T.-T.Qu,S.Lin,C.-S.Leung,andP.-A.Heng,“Intrinsiic colorization,”ACMTrans.on Graph-ics(SIGGRAPH Asia),vol.27,no.5,pp.152:1–152:9,2008.

[3]M.Shao and Y.-H.Wang,“Recovering facial intrinsic imagesfromasingleinput,”LectureNotesinComputer Science,vol.5754,pp.82–91,2009.

[4]Y.Weiss,“Deriving intrinsic images from image se-quences,”in ICCV,2001.[5]I.Malioutov and R.Barzilay,“Minimum cut model for spoken lecture segmentation,”in ACL,2006.

[5]Y.Matsushita,S.Lin,S.B.Kang,andH.-Y.Shum,“Estimating intrinsic images from imagesequences with bised illumination,”in ECCV,2004.

[6]E.H.Land and J.J.McCann,“Lightness and retinex theory,”Journal of the Optical Societyof America,vol.61,no.1,pp.1–11,1978.

[7]B.K.P.Horn,“Robot vision,”MIT Press,1978.

[8]M.F.Tappen,W.T.Freeman,and E.H.Adelson,“Re-covering intrinsic images from asingle image,”IEEE Trans.on Pattern Analysis and Machine Intelligence,vol.27,no.9,pp.1459–1472,2005.

[9]M.F.Tappen,E.H.Adelson,andW.T.Freeman,“Esti-mating intrinsic component images usingnon-linear re-gression,”in CVPR,2006.

[10]J.Shen,X.Yang,Y.Jia,and X.Li,“Intrinsic images using optimization,”in CVPR,2011.

[11]L.Shen and C.Yeo,“Intrinsic images decomposition using a local and global sparserepresentation of re-flectance,”in CVPR,2011.

[12]Q.Zhao,P.Tan,Q.Dai,L.Shen,E.Wu,and S.Lin,“A closed-form solution to retinex withnon-local texture constraints,”IEEE Trans.on Pattern Analysis and Ma-chine Intelligence,vol.1,no.8,pp.1437–1444,2012.

[13]X.Wang,H.Li,and C.-E.Bichot,“A graph-cut ap-proach to image segmentation using anaffinity graph based on l0-sparse representation of features,”in ICIP,2013.

[14]Y.C.Pati,R.Rezaiifar,and P.S.Krishnaprasad,“Or-thogonal matching pursuit:recursivefunction approx-imation with applications to wavelet decomposition,”27th AsilomarConference on Signals,Systems and Computers,1993.

[15]B.Cheng,J.Yang,S.Yan,Y.Fu,and T.S.Huang,“Learning with l1-graph for imageanalysis,”IEEE Trans.onImageProcessing,vol.19,no.4,pp.858–866,2010.

[16]Chung,Fan(1997).Spectral Graph Theory.American Mathematical Society

[17]R.Grosse,M.K.Johnson,E.H.Adelson,and W.T.Freeman,“Ground truth dataset andbaseline evalua-tions for intrinsic image algorithms,”in ICCV,2009.

[18]G.D.Finlayson,S.D.Hordley,and M.Drew,“Remov-ing shadows from images usingretinex,”in Color and Imaging Conference,2002.

It will be appreciated by those skilled in the art that accompanying drawing is the schematic diagram of a preferred embodiment, the invention described above embodiment sequence number, just to describing, does not represent the quality of embodiment.

The foregoing is only preferred embodiment of the present invention, not in order to limit the present invention, within the spirit and principles in the present invention all, any amendment done, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.

Claims (4)

1., based on an intrinsic image decomposition method for multiple dimensioned L0 sparse constraint, it is characterized in that, said method comprising the steps of:

Initialization reflectance component

Non-local zero normal form sparse constraint model is utilized to obtain the measuring similarity W of reflectance component between pixel ^r;

Calculate the tolerance W of illumination component similarity between pixel ^s;

Reflectance component R and the illumination component S of this layer is obtained by solving optimization problem.

2. a kind of intrinsic image decomposition method based on multiple dimensioned L0 sparse constraint according to claim 1, is characterized in that, describedly utilizes non-local zero normal form sparse constraint model to obtain the measuring similarity W of reflectance component between pixel ^rstep be specially:

1) by initialization reflectance component obtain feature vector, X _i;

2) the dictionary D that each pixel is corresponding is obtained _i;

3) X is obtained by orthogonal matching pursuit algorithm solution L0 minimization problem _iat dictionary D _iunder sparse expression factor alpha _i;

4) by weighing the similarity of different pixels point reflection rate component based on regularization residue, the measuring similarity W of reflectance component between pixel is obtained ^r.

3. a kind of intrinsic image decomposition method based on multiple dimensioned L0 sparse constraint according to claim 1, is characterized in that, the described step by solving reflectance component R and illumination component S that optimization problem obtains this layer is specially:

1) objective function optimized is built;

2) objective function is converted into standard double optimization, obtains the solution of closed form, judge whether to enter next iteration still as final solution.

4. a kind of intrinsic image decomposition method based on multiple dimensioned L0 sparse constraint according to claim 3, it is characterized in that, the objective function of described optimization is specially:

$F\left(S\right)=\underset{i\↔j}{\mathrm{\Σ}}{w}_{\mathrm{ij}}^S{(S_i-S_j)}^2+\underset{i~j}{\mathrm{\Σ}}{w}_{\mathrm{ij}}^R{(\mathrm{\Δ}{I}_{\mathrm{ij}}-S_i+S_j)}^2+\underset{i\∈B}{\mathrm{\Σ}}{S}_i^2$

Wherein, S _i, S _jrepresent pixel i, the illumination component of j; represent pixel i, the similarity of the illumination component of j; represent pixel i, the similarity of j reflectance component; represent the set that neighbor is right, i ~ j represents the right set of non-conterminous pixel; Δ I _ij=I _i-I _jrepresent pixel i, the difference of j intensity; The set of pixel the brightest in B representative image.