CN105893649A - Optimal model based interactive image recolorating method - Google Patents

Abstract

The present invention discloses an optimal model based interactive image recolorating method. The method comprises nine steps of: converting a to-be-processed color image to perform manual line coloration; then separately performing matting based on global optimization and based on local optimization on the colorated image, so as to separately obtain a globally optimized matted image and a locally optimized matted image; performing binarization processing on the obtained globally optimized matted image and locally optimized matted image; combining binarized images by adopting an addition or subtraction policy according to a selected region attribute of line coloration; calculating a region that needs to be recolorated in the to-be-processed color image; converting an RGB color space of a colorated region to an HSV color space; adjusting a tone channel and a saturation channel in the colorated region in an interactive manner; converting the HSV color space of the image to the RGB color space; and finally, combining an image of which the region is reclorated with an original color image to obtain a recolorated image.

Description

A kind of interactive image based on optimal model is restained method

Technology neighborhood

The present invention relates to a kind of coloured image and restain method, especially a kind of interactive image based on optimal model is restained method.

Background technology

The image technology of restaining refers to that the color by changing image changes original style of image, reaches the processing procedure of certain specific color effect. the technology of restaining generally involves numerous images and the video processing technique such as the segmentation, cluster, estimation of image, huge economic benefit directly can be produced, the technology of restaining be derived from put into practice, direct application oriented technology, restaining technical research so current is a challenging hot subject of computer graphics and computer vision neighborhood, in video display, animation, image and science, industry and all multiple-project neighborhoods such as military have very wide application prospect. the traditional technology of restaining is by pure manual or complete under computer software is auxiliary, the time that this need of work manpower consumption is a large amount of, such as Photoshop have employed color replacement tool to realize restaining of image, but this instrument simply have employed aberration tolerance in constituency, is many times difficult to realization and chooses accurately. therefore domestic and international many scientific research personnel constantly explore new algorithm to improve efficiency and the effect of the technology of restaining in recent years. at present, color transfer method based on reference picture and the color method of diffusion of tinting based on lines being divided into that technology that image restains can be rough is realized. the image technology of restaining based on color transfer realizes color according to the similitude of pixel and Neighborhood Statistics feature and shifts, the color that is about to a secondary color reference image is transferred on target image, the effect of the method is strongly depend on the similarity degree of reference picture and target image, when the brightness conditions of reference picture and target image differs greatly, incorrect restain result can be produced. therefore, the result of choosing restaining of reference picture plays vital effect. the color method of diffusion of tinting based on lines is the lines of tinting roughly that utilize user to input, and color expansion is arrived entire image by the unique characteristics (as the similitude of color between pixel or texture) of combining image. comparatively speaking, processing form based on the color method of diffusion that lines are tinted more can meet user's demand, therefore present invention employs and be similar to the color method of diffusion of tinting based on lines, difference is directly to input the line color after restaining based on the color method of diffusion that lines are tinted by user, then different strategies is adopted to carry out color propagation, and the lines of tinting of the present invention (only containing foreground and background colour) are only used for choosing the region of restaining, accurately after selection area, be transformed into HSV color space and carry out interactive color adjustment, ensure that the COLOR COMPOSITION THROUGH DISTRIBUTION of original image color. image restains technical research and image is scratched figure, and picture editting, the technology such as image chroma adjustment are closely bound up. according to the distribution of propagating neighborhood territory pixel, the color method of diffusion of tinting based on lines can be divided into two kinds of methods that local neighborhood is propagated and overall neighborhood is propagated roughly.

Based on local neighborhood propagate method representative be the people's such as Levin work. The people such as Levin suppose that the color between adjoining pixel is level and smooth, propose an energy optimization model. When user inputs abundant tinting during lines, the method can produce the high-quality effect of restaining. Afterwards, the people such as people and Criminisi such as Yatziv propose based on geodesic curve apart from the method for restaining merging, the people such as the people such as Fattal introduce image edit method wavelet transformation technique, Bhat establish the energy optimization framework of gradient field for picture editting. Above-mentioned several method is the hypothesis of the spatial continuity based on image pixel, therefore just can not obtain desirable restaining effect for high-contrast texture image. For the effect that can obtain at high texture region and segment area, the people such as Farbman replace Euler apart from the compatibility of calculating between pixel by diffusion length. The people such as people and Sheng such as Qu utilize the texture information of image to be respectively used to the colorize of animation and natural image. The method of restaining of propagating based on local neighborhood can realize good user partial control, but when pixel is distant from the lines of tinting, the effect of restaining often just does not reach expected effect.

Recently, the method for propagating based on overall neighborhood has attracted numerous researchers' concern. The method of restaining of propagating based on overall neighborhood can realize overall color and propagate, even input the lines of tinting when distant when the pixel that need to restain from user, color also can well be propagated. Therefore the method has carried out well supplementing to the method for restaining of propagating based on local neighborhood. In recent years, many scholars expand research to the method for propagating based on overall neighborhood, and the people such as people and Lee such as Chen define the image of a non local neighborhood and scratch nomography. The people such as people and Chen such as Musialski propose the editing distance algorithm that the main territory of non local neighborhood keeps. Compared with the method for restaining of propagating based on local neighborhood, the method of restaining of propagating based on overall neighborhood can realize overall color propagation, therefore the tint lines quantity of user input can be reduced, but the method lacks local or directly selects to control, when two similar colors are tinted into different colours, color mixture can be caused and occur mistake.

Previously restained based on what color was propagated the following problem existing in technology: (1) artificially coloring lines need to all cover all colours in image, otherwise color can be existed to lose when restaining; (2) the color and vein information of restaining region is decrease; (3) need adjustment algorithm parameter to mate the region that need to restain, method is not directly perceived; (4) overall color transmission method lacks Partial controll, and local color transmission method needs more user to input lines; (6) for there is mirror-reflection and shadow of object part in picture, easily there is color bleeding phenomenon in the process of restaining. What the present invention proposed restains method, the first step is that user is painted, similar with Photoshop in replacement color instrument, paintbrush is divided into foreground and background colour, can increase, reduce selected zone, add the overall situation propagation, the local propagation attribute that in Photoshop, do not have simultaneously, improve previous overall color transmission method and lack Partial controll, and local color transmission method needs more user to input the restriction of lines. When second step is selected to restain region, owing to add overall and local propagation optimization scheme, user is only needed to input a small amount of prospect and background colour, except stingy figure region out, other field color remain unchanged automatically, the lines that so just do not need to tint as previously restaining method will all cover all colors, greatly reduce user's input lines. 3rd step is that the color adjustment that HSV is unified is carried out to stingy figure region out, ensure that the distribution of texture in region, also the picture that has mirror-reflection and shadow of object is suitable for simultaneously, whole adjustment process is easy to carry out man-machine interactively, convenient, simple, fast, in algorithm, do not need to adjust extra parameter.

Summary of the invention

The object of the invention is: provide a kind ofly effectively only needs the interactive colo r image based on optimal model of a small amount of lines to restain method, improve and existingly restain based on what color was propagated the following problem existing in technology: (1) artificially coloring lines need to all cover all colours in image, otherwise color can be existed when restaining to lose; (2) the color and vein information of restaining region is decrease; (3) need adjustment algorithm parameter to mate the region that need to restain, method is not directly perceived; (4) overall color transmission method lacks Partial controll, and local color transmission method needs more user to input lines; (5) when user inputs, line color is improper when need to adjust, and whole algorithm need to recalculate, and a large amount of time need to be consumed.

The interactive image based on optimal model designed by the present invention is restained method, comprises following nine steps:

(1) pending coloured image I is inputted_Org, then pending coloured image is carried out to artificial line color, obtains rendered image;

(2) rendered image is carried out respectively based on global optimization and the stingy figure based on suboptimization, obtain respectively the stingy figure image I of global optimization_AlphaGlobalWith local optimized stingy figure image I_AlphaLocal；

(3) the stingy figure image I to the global optimization obtaining_AlphaGlobalWith local optimized stingy figure image I_AlphaLocalCarry out binary conversion treatment, corresponding binary image is respectively I_SegGlobalAnd I_SegLocal；

(4) according to the constituency attribute of painted lines, take the strategy that adds or deduct, merge binary image I_SegGlobalAnd I_SegLocalFor I_Seg；

(5) according to binary image I_SegWith pending coloured image I_Org, calculate pending coloured image I_OrgIn region I need to be restained_Sel；

(6) by painted areas I_SelRGB color space conversion to hsv color space, obtain respectively tone passage H, saturation degree passage S and luminance channel V;

(7) user adopts interactively mode to adjust painted areas I_SelIn tone passage H, saturation degree passage S, namely realize selected areas and restain, obtain image I_recolor；

(8) by image I_recolorHsv color space transforming to rgb color space, obtain image I_RGBcolor；

(9) the image I after region is restained_RGBcolorAfter merging with coloured image I, obtain and restain image I_New。

2, the interactive image based on optimal model according to claim 1 is restained method, it is characterized in that: have as properties (attribute of input lines sees the following form) in the artificially coloring lines described in step (1): be divided into foreground white and background colour black from color, wherein foreground is propagated be divided into the overall situation to propagate and local propagation from color, is respectively and adds from area attribute and deduct.

Color	Color is propagated	Area attribute
			Foreground white	The overall situation is propagated or local propagation	Add or deduct
Background colour black	--	--

3, restain method according to the interactive image based on optimal model according to claim 1 or claim 2, it is characterized in that: carrying out respectively described in step (2) is divided into following three steps based on global optimization with based on the stingy drawing method of suboptimization:

(1) K neighborhood territory pixel of each pixel is first searched for; High-dimensional feature space used includes HSV color channel and space coordinates, and feature space F (i) is defined as follows:

F(i)＝(H(i)，S(i)，V(i)，γx(i)，γy(i))

In formula, H is the tone value of image HSV color space, S is saturation degree, V is brightness value, it is to adopt the overall situation to propagate or local propagation that γ state modulator color is propagated, γ value is 0 or ∞ (if γ value is 0, being that overall color is propagated, if γ value is ∞, for local color is propagated), adopts FLANN storehouse to realize searching K the neighborhood N of pixel i_i(KNN)；

(2) secondly build the optimal model of restaining, and consider that user inputs painted lines, build following optimization cost function:

J(U)＝λ(U-G)^TD_s(U-G)+U^TL_ColorU

In formula:

U is the stingy figure matrix solving;

D_sFor diagonal matrix, in this diagonal matrix, the value on the diagonal that the lines pixel of tinting is corresponding is 1, and the value on all the other diagonal is 0;

G corresponds to the value of the lines of tinting, and wherein prospect lines respective value is 1, and the value that background lines are corresponding is 0;

In formula, Section 1 ensure that to scratch the tint value of lines of figure image and user approaching as much as possible, Section 2 ensure that the similitude of pixel and its neighborhood, the pixel of having tinted is propagated by neighborhood, and parameter lambda is used for adjusting this balance of two, and in algorithm realization, λ is set to 1;

The Laplacian Matrix L of the coloured image of neighborhood_ColorBuild as follows:

$L_{\mathrm{Color}}=\underset{k|(i,j)\∈N_k}{\mathrm{\Σ}}({\mathrm{\δ}}_{\mathrm{ij}}-\frac{1}{K}(1+(I_i-{\mathrm{\μ}}_k){({\mathrm{\Σ}}_k+\frac{\mathrm{\ϵ}}{K}{I}_3)}^{-1}(I_j-{\mathrm{\μ}}_k)))$

In formula:

J and k is image pixel index value;

δ_ijKronecker function, if i and j is equal, then δ_ijBe 1, otherwise δ_ijBe 0;

μ_kWithMean value and the variance of K the non local neighborhood territory pixel of pixel k respectively;

K is the non local neighborhood number of pixel k;

∈ is that regularisation parameter value is 10^-6；

I₃For the chromatic value of input picture at HSV color space, saturation degree and brightness value;

Solve optimal model and obtain following sparse linear systems:

U＝(L_Color+λD_s)^-1λD_sG

In formula: L_Color+λD_sFor sparse matrix solves, algorithms selection is solved by Gauss-Seidel iteration method;

(3) finally, the foreground of input lines is divided into overall foreground and local foreground according to color propagation property, utilize overall foreground and background colour to carry out scratching figure based on overall image, i.e. during neighborhood territory pixel search, γ value is 0; Utilize local foreground look and background colour to carry out scratching figure based on local image, i.e. during neighborhood territory pixel search, γ value is ∞.

4, the interactive image based on optimal model according to claim 1 is restained method, it is characterized in that: the stingy figure image I to the global optimization obtaining described in step (3)_AlphaGlobalWith local optimized stingy figure image I_AlphaLocalCarry out binary conversion treatment as follows:

$I_{\mathrm{SegGlobal}}=\left\{\begin{array}{ccc}0& \mathrm{if}& I_{\mathrm{AlphaGlobal}}=0;\\ 1& \mathrm{if}& I_{\mathrm{AlphaGlobal}>0.}\end{array}\right.$

$I_{\mathrm{SegLocal}}=\left\{\begin{array}{ccc}0& \mathrm{if}& I_{\mathrm{AlphaLocal}}=0;\\ 1& \mathrm{if}& I_{\mathrm{AlphaLocal}>0.}\end{array}\right.$

5, the interactive image based on optimal model according to claim 1 is restained method, it is characterized in that: in the merging binary image I described in step (4)_SegGlobalAnd I_SegLocalFor I_SegAs follows:

I_Seg1＝S_Global×I_SegGlobal+S_Local×I_SegLocal

$I_{\mathrm{Seg}}\left(i\right)=\left\{\begin{array}{cc}{I}_{\mathrm{Seg}1}\left(i\right)& \mathrm{if}{I}_{\mathrm{seg}1}\left(i\right)\≥0\\ 0& \mathrm{otherelse}.\end{array}\right.$

S in formula_GlobalAnd S_LocalCorrespondence is chosen attribute respectively, and add if, then numerical value is 1, deducts if, and then numerical value is-1.

6, the interactive image based on optimal model according to claim 1 is restained method, it is characterized in that: in the pending coloured image I of the calculating described in step (5)_OrgIn region I need to be restained_SelAs follows:

I_Sel＝I_Org×I_Seg

7, the interactive image based on optimal model according to claim 1 is restained method, it is characterized in that: as follows by the conversion method of RGB color space conversion to hsv color space described in step (6):

We mark Max is the maximum in (R, G, B), and Min is the minimum of a value in (R, G, B),

$H=\left\{\begin{array}{ccc}\mathrm{undefined},& \mathrm{if}& \mathrm{MAX}=\mathrm{MIN}\\ 60\&times;\frac{G-B}{\mathrm{MAX}-\mathrm{MIN}}+0,& \mathrm{if}& \mathrm{MAX}=\mathrm{R\; and\; G}\&GreaterEqual;B\\ 60\&times;\frac{G-B}{\mathrm{MAX}-\mathrm{MIN}}+360,& \mathrm{if}& \mathrm{MAX}=\mathrm{R\; and\; G}<B\\ 60\&times;\frac{B-R}{\mathrm{MAX}-\mathrm{MIN}}+120,& \mathrm{if}& \mathrm{MAX}=G\\ 60\&times;\frac{R-G}{\mathrm{MAX}-\mathrm{MIN}}+240,& \mathrm{if}& \mathrm{MAX}=B\end{array}\right.$

$S=\left\{\begin{array}{cc}0,& \mathrm{if\; MAX}=0\\ 1-\frac{\mathrm{MIN}}{\mathrm{MAX}},& \mathrm{otherwise}\end{array}\right.$

V＝MAX

R in formula, G, B represents respectively the red, green, blue color value of image, and H is the tone value of image, and S is the saturation degree of image, and V is the brightness of image.

8, the interactive image based on optimal model according to claim 1 is restained method, it is characterized in that: described in step (8) by hsv color space transforming to the conversion method of RGB color space by following formula:

$H_i=\begin{array}{ccc}\left[\frac{H}{60}\right]& \mathrm{mod}& 6\end{array}$

$f=\frac{H}{60}-H_i$

p＝V(1-S)

q＝V(1-fS)

t＝V(1-(1-f)S)

R＝V，G＝t，B＝p if H_i＝0

R＝q，G＝V，B＝p if H_i＝1

R＝p，G＝V，B＝t if H_i＝2

R＝p，G＝q，B＝V if H_i＝3

R＝t，G＝p，B＝V if H_i＝4

R＝V，G＝p，B＝q if H_i＝5

9, the interactive image based on optimal model according to claim 1 is restained method, it is characterized in that: the image I after the merging described in step (9) is restained_RGBcolorWith coloured image I_OrgFor I_NewAs follows:

I_New＝I_Sel×I_RGBcolor+(1-I_Sel)×I_Org

The beneficial effect that the interactive image based on optimal model designed by the present invention is restained method is:

What 1, the present invention proposed restains method, the first step is that user is painted, similar with Photoshop in replacement color instrument, paintbrush is divided into foreground and background colour, selected zone be can increase, reduce, the overall situation propagation, the local propagation attribute that in Photoshop, do not have add simultaneously. When second step is selected to restain region, owing to add overall and local propagation optimization scheme, user is only needed to input a small amount of prospect and background colour, except stingy figure region out, other field color remain unchanged automatically, so just do not need all colours all to cover, 3rd step is that hsv color adjustment is carried out to stingy figure region out, ensure that the distribution of texture in region, be also suitable for the picture that has mirror-reflection and shadow of object simultaneously, adjust color convenient, simple, fast.

2, the image that this method provides is restained technology, only need to input a small amount of lines of tinting (only containing foreground and background colour), can extract accurately and restain region, realize the effective separated of foreground area and background area, high-quality restain effect can be produced, for layman provides the easy replacement method of color of image intuitively.

3, the effect that technology is restained in the efficiency that the image that this method the proposes technology of restaining improves CAD, minimizing designer's working time, improvement, supports that the batch of image is restained. This system can be extracted the region that user need to restain accurately, adopts interactively mode, makes designer carry out alternately, designing high-quality artistic effect with software well.

Accompanying drawing explanation

Fig. 1 is the schematic flow sheet of embodiment 1;

Fig. 2 is that the present invention restains image effect schematic diagram in process;

Detailed description of the invention

Below by embodiment, the invention will be further described by reference to the accompanying drawings.

Below in conjunction with accompanying drawing, a kind of method of restaining of the interactive image based on optimal model of the present invention is described in detail by embodiment: the present embodiment is implemented under taking technical solution of the present invention as prerequisite; combine detailed embodiment and process, but protection scope of the present invention is not limited to following embodiment. Fig. 1 is the schematic flow sheet of embodiment 1, comprise nine steps altogether, Fig. 2 is that the present invention restains image effect schematic diagram in process, Fig. 2 depicts from artificially coloring is carried out to pending image, wherein white lines represent foreground area, black lines represents background area, and (G+) of lines side standard represents overall color propagation and area attribute is selected to add, and (L-) expression local color propagation and regional choice are chosen as and deduct. The overall situation proposing in the present invention and local color are propagated and are made user use still less mutual, realize the selection of restaining region fast, and user adopt the interactively mode color of adjustment region fast, meets user's custom.

Embodiment 1:

As shown in Figure 1, the interactive image based on optimal model described by the present embodiment is restained method, comprises following nine steps:

(10) pending coloured image I is inputted_Org, then pending coloured image is carried out to artificial line color, obtains rendered image;

(11) rendered image is carried out respectively based on global optimization and the stingy figure based on suboptimization, obtain respectively the stingy figure image I of global optimization_AlphaGlobalWith local optimized stingy figure image I_AlphaLocal；

(12) the stingy figure image I to the global optimization obtaining_AlphaGlobalWith local optimized stingy figure image I_AlphaLocalCarry out binary conversion treatment, corresponding binary image is respectively I_SegGlobalAnd I_SegLocal；

(13) according to the constituency attribute of painted lines, take the strategy that adds or deduct, merge binary image I_SegGlobalAnd I_SegLocalFor I_Seg；

(14) according to binary image I_SegWith pending coloured image I_Org, calculate pending coloured image I_OrgIn region I need to be restained_Sel；

(15) by painted areas I_SelRGB color space conversion to hsv color space, obtain respectively tone passage H, saturation degree passage S and luminance channel V;

(16) user adopts interactively mode to adjust painted areas I_SelIn tone passage H, saturation degree passage S, namely realize selected areas and restain, obtain image I_recolor；

(17) by image I_recolorHsv color space transforming to rgb color space, obtain image I_RGBcolor；

(18) the image I after region is restained_RGBcolorAfter merging with coloured image I, obtain and restain image I_New。

2, the interactive image based on optimal model according to claim 1 is restained method, it is characterized in that: have as properties (attribute of input lines sees the following form) in the artificially coloring lines described in step (1): be divided into foreground white and background colour black from color, wherein foreground is propagated be divided into the overall situation to propagate and local propagation from color, is respectively and adds from area attribute and deduct.

Color	Color is propagated	Area attribute
			Foreground white	The overall situation is propagated or local propagation	Add or deduct
Background colour black	--	--

3, restain method according to the interactive image based on optimal model according to claim 1 or claim 2, it is characterized in that: carrying out respectively described in step (2) is divided into following three steps based on global optimization with based on the stingy drawing method of suboptimization:

(1) K neighborhood territory pixel of each pixel is first searched for; High-dimensional feature space used includes HSV color channel and space coordinates, and feature space F (i) is defined as follows:

F(i)＝(H(i)，S(i)，V(i)，γx(i)，γy(i))

In formula, H is the tone value of image HSV color space, S is saturation degree, V is brightness value, it is to adopt the overall situation to propagate or local propagation that γ state modulator color is propagated, γ value is 0 or ∞ (if γ value is 0, being that overall color is propagated, if γ value is ∞, for local color is propagated), adopts FLANN storehouse to realize searching K the neighborhood N of pixel i_i(KNN)；

(2) secondly build the optimal model of restaining, and consider that user inputs painted lines, build following optimization cost function:

J(U)＝λ(U-G)^TD_s(U-G)+U^TL_ColorU

In formula:

U is the stingy figure matrix solving;

D_sFor diagonal matrix, in this diagonal matrix, the value on the diagonal that the lines pixel of tinting is corresponding is 1, and the value on all the other diagonal is 0;

G corresponds to the value of the lines of tinting, and wherein prospect lines respective value is 1, and the value that background lines are corresponding is 0;

In formula, Section 1 ensure that to scratch the tint value of lines of figure image and user approaching as much as possible, Section 2 ensure that the similitude of pixel and its neighborhood, the pixel of having tinted is propagated by neighborhood, and parameter lambda is used for adjusting this balance of two, and in algorithm realization, λ is set to 1;

The Laplacian Matrix L of the coloured image of neighborhood_ColorBuild as follows:

$L_{\mathrm{Color}}=\underset{k|(i,j)\&Element;N_k}{\mathrm{\&Sigma;}}({\mathrm{\&delta;}}_{\mathrm{ij}}-\frac{1}{K}(1+(I_i-{\mathrm{\&mu;}}_k){({\mathrm{\&Sigma;}}_k+\frac{\mathrm{\&epsiv;}}{K}{I}_3)}^{-1}(I_j-{\mathrm{\&mu;}}_k)))$

In formula:

J and k is image pixel index value;

δ_ijKronecker function, if i and j is equal, then δ_ijBe 1, otherwise δ_ijBe 0;

μ_kWithMean value and the variance of K the non local neighborhood territory pixel of pixel k respectively;

K is the non local neighborhood number of pixel k;

∈ is that regularisation parameter value is 10^-6；

I₃For the chromatic value of input picture at HSV color space, saturation degree and brightness value;

Solve optimal model and obtain following sparse linear systems:

U＝(L_Color+λD_s)^-1λD_sG

In formula: L_Color+λD_sFor sparse matrix solves, algorithms selection is solved by Gauss-Seidel iteration method;

(3) finally, the foreground of input lines is divided into overall foreground and local foreground according to color propagation property, utilize overall foreground and background colour to carry out scratching figure based on overall image, i.e. during neighborhood territory pixel search, γ value is 0; Utilize local foreground look and background colour to carry out scratching figure based on local image, i.e. during neighborhood territory pixel search, γ value is ∞.

4, the interactive image based on optimal model according to claim 1 is restained method, it is characterized in that: the stingy figure image I to the global optimization obtaining described in step (3)_AlphaGlobalWith local optimized stingy figure image I_AlplhaLocalCarry out binary conversion treatment as follows:

$I_{\mathrm{SegGlobal}}=\left\{\begin{array}{ccc}0& \mathrm{if}& I_{\mathrm{AlphaGlobal}}=0;\\ 1& \mathrm{if}& I_{\mathrm{AlphaGlobal}>0.}\end{array}\right.$

$I_{\mathrm{SegLocal}}=\left\{\begin{array}{ccc}0& \mathrm{if}& I_{\mathrm{AlphaLocal}}=0;\\ 1& \mathrm{if}& I_{\mathrm{AlphaLocal}>0.}\end{array}\right.$

5, the interactive image based on optimal model according to claim 1 is restained method, it is characterized in that: in the merging binary image I described in step (4)_SegGlobalAnd I_SegLocalFor I_SegAs follows:

I_Seg1＝S_Global×I_SegGlobal+S_Local×I_SegLocal

$I_{\mathrm{Seg}}\left(i\right)=\left\{\begin{array}{cc}{I}_{\mathrm{Seg}1}\left(i\right)& \mathrm{if}{I}_{\mathrm{seg}1}\left(i\right)\&GreaterEqual;0\\ 0& \mathrm{otherelse}.\end{array}\right.$

S in formula_GlobalAnd S_LocalCorrespondence is chosen attribute respectively, and add if, then numerical value is 1, deducts if, and then numerical value is-1.

6, the interactive image based on optimal model according to claim 1 is restained method, it is characterized in that: in the pending coloured image I of the calculating described in step (5)_OrgIn region I need to be restained_SelAs follows:

I_Sel＝I_Org×I_Seg

7, the interactive image based on optimal model according to claim 1 is restained method, it is characterized in that: as follows by the conversion method of RGB color space conversion to hsv color space described in step (6):

We mark Max is the maximum in (R, G, B), and Min is the minimum of a value in (R, G, B),

$H=\left\{\begin{array}{ccc}\mathrm{undefined},& \mathrm{if}& \mathrm{MAX}=\mathrm{MIN}\\ 60\&times;\frac{G-B}{\mathrm{MAX}-\mathrm{MIN}}+0,& \mathrm{if}& \mathrm{MAX}=\mathrm{R\; and\; G}\&GreaterEqual;B\\ 60\&times;\frac{G-B}{\mathrm{MAX}-\mathrm{MIN}}+360,& \mathrm{if}& \mathrm{MAX}=\mathrm{R\; and\; G}<B\\ 60\&times;\frac{B-R}{\mathrm{MAX}-\mathrm{MIN}}+120,& \mathrm{if}& \mathrm{MAX}=G\\ 60\&times;\frac{R-G}{\mathrm{MAX}-\mathrm{MIN}}+240,& \mathrm{if}& \mathrm{MAX}=B\end{array}\right.$

$S=\left\{\begin{array}{cc}0,& \mathrm{if\; MAX}=0\\ 1-\frac{\mathrm{MIN}}{\mathrm{MAX}},& \mathrm{otherwise}\end{array}\right.$

V＝MAX

R in formula, G, B represents respectively the red, green, blue color value of image, and H is the tone value of image, and S is the saturation degree of image, and V is the brightness of image.

8, the interactive image based on optimal model according to claim 1 is restained method, it is characterized in that: described in step (8) by hsv color space transforming to the conversion method of RGB color space by following formula:

$H_i=\begin{array}{ccc}\left[\frac{H}{60}\right]& \mathrm{mod}& 6\end{array}$

$f=\frac{H}{60}-H_i$

p＝V(1-S)

q＝V(1-fS)

t＝V(1-(1-f)S)

R＝V，G＝t，B＝p if H_i＝0

R＝q，G＝V，B＝p if H_i＝1

R＝p，G＝V，B＝t if H_i＝2

R＝p，G＝q，B＝V if H_i＝3

R＝t，G＝p，B＝V if H_i＝4

R＝V，G＝p，B＝q if H_i＝5

9, the interactive image based on optimal model according to claim 1 is restained method, it is characterized in that: the image I after the merging described in step (9) is restained_RGBcolorWith coloured image I_OrgFor I_NewAs follows:

I_New＝I_Sel×I_RGBcolor+(1-I_Sel)×I_Org

Here the concrete derivation of Local Linear Model is provided:

Algorithm of the present invention is based on following hypothesis: similar neighborhood in feature space, has similar alpha value after scratching figure. Based on this hypothesis, the neighborhood that gives each pixel meets following color linear model:

$U_i={\mathrm{\&alpha;}}^c{I}_i^c+\mathrm{\&beta;},\&ForAll;i\&Element;N_i---\left(1\right)$

In formula, (α, β) is linear coefficient, and I is input picture (adopting HSV color space), and C is color channel. N_iBe the neighborhood of pixel i, U is the alpha value after scratching figure. It should be noted that this method have employed the spatial neighborhood based on high dimensional feature. In order to find the neighborhood of pixel i, a very crucial step is exactly the suitable feature space of design, and the feature space that the present invention proposes includes HSV color channel and space coordinates, and feature space F (i) is defined as follows:

F(i)＝(H(i)，S(i)，V(i)，γx(i)，γy(i)) (2)

In formula, H is the tone value of image HSV color space, S is saturation degree, V is brightness value, it is to adopt the overall situation to propagate or local propagation that γ state modulator color is propagated, γ value is 0 or ∞ (if γ value is 0, being that overall color is propagated, if γ value is ∞, for local color is propagated), adopts FLANN storehouse to realize searching K the neighborhood N of pixel i_i(KNN)；

By the color linear model based on neighborhood, be defined as follows cost function:

$J(U,\mathrm{\&alpha;},\mathrm{\&beta;})=\underset{k\&Element;I}{\mathrm{\&Sigma;}}\left(\underset{i\&Element;N_j}{\mathrm{\&Sigma;}}({({\mathrm{\&alpha;}}_k{I}_i+{\mathrm{\&beta;}}_k-U_i)}^2+\mathrm{\&epsiv;}\underset{c}{\mathrm{\&Sigma;}}{\mathrm{\&alpha;}}_k^{c^2})\right)$

In formula, ∈ is regularisation parameter, prevents α_kThat gets is too large, and (proposition is ∈=10 in algorithm to increase numerical stability^-6)。

Linear coefficient (α, β) can by solve formula (3) obtain minimize cost function obtain

$J\left(U\right)=\underset{\mathrm{\&alpha;},\mathrm{\&beta;}}{\min }J(U,\mathrm{\&alpha;},\mathrm{\&beta;})$

In formula, energy optimization model can be regarded variable (α as_k，β_k) quadratic equation, can by formula ask first-order partial derivative obtain, solve and can obtain:

$J\left(U\right)=\underset{\mathrm{\&alpha;},\mathrm{\&beta;}}{\min }J(U,\mathrm{\&alpha;},\mathrm{\&beta;})=U^T{L}_{\mathrm{Color}}U$

The wherein Laplacian Matrix L of the coloured image of neighborhood_ColorBuild as follows:

$L_{\mathrm{Color}}=\underset{k|(i,j)\&Element;N_k}{\mathrm{\&Sigma;}}({\mathrm{\&delta;}}_{\mathrm{ij}}-\frac{1}{K}(1+(I_i-{\mathrm{\&mu;}}_k){({\mathrm{\&Sigma;}}_k+\frac{\mathrm{\&epsiv;}}{K}{I}_3)}^{-1}(I_j-{\mathrm{\&mu;}}_k)))---\left(3\right)$

In formula, δ_ijKronecker function, if i and j is equal, then δ_ijBe 1, otherwise δ_ijBe 0; μ_kWithMean value and the variance of K the non local neighborhood territory pixel of pixel k respectively. K is the neighborhood number of pixel k.

Color linear model by merging based on neighborhood and the restriction of tinting of user's input, can obtain following energy optimization equation:

J(U)＝λ(U-G)^TD_s(U-G)+U^TL_ColorU (4)

D_sFor diagonal matrix, in this diagonal matrix, the value on the diagonal that the lines pixel of tinting is corresponding is 1, and the value on all the other diagonal is 0. G corresponds to the value of the lines of tinting, and wherein prospect lines respective value is 1, and the value that background lines are corresponding is 0. In formula, Section 1 ensure that to scratch the tint value of lines of figure image and user approaching as much as possible, and Section 2 ensure that the similitude of pixel and its neighborhood, and the pixel of having tinted is propagated by neighborhood. Parameter lambda is used for adjusting this balance of two, and in algorithm realization, λ is set to 1. J (U) in formula (4) scratches the quadratic equation of figure matrix U, and therefore the global minimum of J (U) can be zero to solve and obtain by the first-order partial derivative to stingy figure matrix U. Following sparse linear systems can be obtained:

U＝(L_Color+λD_s)^-1λD_sG (5)

L in formula (5)_Color+λD_sFor sparse matrix solves, solve sparse linear matrix equation and apply widely, how to improve the speed that solves sparse linear matrix equation and the use that reduces internal memory, become the focus of academia and engineering circles research. Solve sparse matrix direct method and iterative method can be divided into. Comparatively speaking, the complexity of iterative method algorithm and memory consumption ratio direct method solve low, and iterative method is conventionally than being easier to parallel computation, and therefore algorithms selection of the present invention is solved by Gauss-Seidel iteration method.

What the present invention proposed restains method, the first step is that user is painted, similar with Photoshop in replacement color instrument, paintbrush is divided into foreground and background colour, selected zone be can increase, reduce, the overall situation propagation, the local propagation attribute that in Photoshop, do not have add simultaneously. When second step is selected to restain region, owing to add overall and local propagation optimization scheme, user is only needed to input a small amount of prospect and background colour, except stingy figure region out, other field color remain unchanged automatically, and so just do not need all colours all to cover, the 3rd step is that hsv color adjustment is carried out to stingy figure region out, ensure that the distribution of texture in region, adjust color convenient, simple, fast.

Claims (9)

1. the interactive image based on optimal model is restained a method, it is characterized in that: comprise following nine steps:

(1) pending coloured image I is inputted_Org, then pending coloured image is carried out to artificial line color, obtains rendered image;

(2) rendered image is carried out respectively based on global optimization and scratching based on suboptimizationFigure, obtain respectively the stingy figure image I of global optimization_AlphaGlobalWith local optimized stingy figure image I_AlphaLocal；

(3) the stingy figure image I to the global optimization obtaining_AlphaGlobalWith local optimized stingy figure image I_AlphaLocalCarry out binary conversion treatment, corresponding binary image is respectively I_SegGlobalAnd I_SegLocal；

(4) according to the constituency attribute of painted lines, take the strategy that adds or deduct, merge binary image I_SegGlobalAnd I_SegLocalFor I_Seg；

(5) according to binary image I_SegWith pending coloured image I_Org, calculate pending coloured image I_OrgIn region I need to be restained_Sel；

(6) by painted areas I_SelRGB color space conversion to hsv color space, obtain respectively tone passage H, saturation degree passage S and luminance channel V;

(7) user adopts interactively mode to adjust painted areas I_SelIn tone passage H, saturation degree passage S, namely realize selected areas and restain, obtain image I_recolor；

(8) by image I_recolorHsv color space transforming to rgb color space, obtain image I_RGBcolor；

(9) the image I after region is restained_RGBcolorAfter merging with coloured image I, obtain and restain image I_New。

2. the interactive image based on optimal model according to claim 1 is restained method, it is characterized in that: the artificially coloring lines described in step (1) have as properties (input lines attribute seeFollowing table): be divided into foreground white and background colour black from color, wherein foreground is propagated be divided into the overall situation to propagate and local propagation from color, is respectively and adds from area attribute and deduct.

Color Color is propagated Area attribute Foreground white The overall situation is propagated or local propagation Add or deduct Background colour black -- --

3. restain method according to the interactive image based on optimal model according to claim 1 or claim 2, it is characterized in that: carrying out respectively based on global optimization and scratching based on suboptimization described in step (2)FigureMethod is divided into following three steps:

(1) K neighborhood territory pixel of each pixel is first searched for; High-dimensional feature space used includes HSV color channel and space coordinates, and feature space F (i) is defined as follows:

F(i)＝(H(i)，S(i)，V(i)，γx(i)，γy(i))

In formula, H is the tone value of image HSV color space, S is saturation degree, V is brightness value, it is to adopt the overall situation to propagate or local propagation that γ state modulator color is propagated, γ value is 0 or ∞ (if γ value is 0, being that overall color is propagated, if γ value is ∞, for local color is propagated), adopts FLANN storehouse to realize searching K the neighborhood N of pixel i_i(KNN)；

(2) secondly build the optimal model of restaining, and consider that user inputs painted lines, build following optimization cost function:

J(U)＝λ(U-G)^TD_S(U-G)+U^TL_ColorU

In formula:

U is scratching of solvingFigureMatrix;

D_sFor diagonal matrix, in this diagonal matrix, the value on the diagonal that the lines pixel of tinting is corresponding is 1, and the value on all the other diagonal is 0;

G corresponds to the value of the lines of tinting, and wherein prospect lines respective value is 1, and the value that background lines are corresponding is 0;

In formula, Section 1 ensure that to scratch the tint value of lines of figure image and user approaching as much as possible, Section 2 ensure that the similitude of pixel and its neighborhood, the pixel of having tinted is propagated by neighborhood, and parameter lambda is used for adjusting this balance of two, and in algorithm realization, λ is set to 1;

The Laplacian Matrix L of the coloured image of neighborhood_ColorBuild as follows:

In formula:

J and k is image pixel index value;

δ_ijKronecker function, if i and j is equal, then δ_ijBe 1, otherwise δ_ijBe 0;

μ_kWithMean value and the variance of K the non local neighborhood territory pixel of pixel k respectively;

K is the non local neighborhood number of pixel k;

∈ is that regularisation parameter value is 10^-6；

I₃For the chromatic value of input picture at HSV color space, saturation degree and brightness value;

Solve optimal model and obtain following sparse linear systems:

U＝(L_Color+λD_s)^-1λD_sG

In formula: L_Color+λD_sFor sparse matrix solves, algorithms selection is solved by Gauss-Seidel iteration method;

(3) finally, the foreground of input lines is divided into overall foreground and local foreground according to color propagation property, utilizes overall foreground and background colour to carry out scratching based on overall imageFigure, i.e. during neighborhood territory pixel search, γ value is 0; Local foreground look and background colour is utilized to carry out scratching based on local imageFigure, i.e. during neighborhood territory pixel search, γ value is ∞.

4. the interactive image based on optimal model according to claim 1 is restained method, it is characterized in that: the stingy figure image I to the global optimization obtaining described in step (3)_AlphaGlobalWith local optimized stingy figure image I_AlphaLocalCarry out binary conversion treatment as follows:

。

5. the interactive image based on optimal model according to claim 1 is restained method, it is characterized in that: in the merging binary image I described in step (4)_SegGlobalAnd I_SegLocalFor I_SegAs follows:

I_Segl＝S_Global×I_SegGlobal+S_Local×I_SegLocal

S in formula_GlobalAnd S_LocalCorrespondence is chosen attribute respectively, and add if, then numerical value is 1, deducts if, and then numerical value is-1.

6. the interactive image based on optimal model according to claim 1 is restained method, it is characterized in that: in the pending coloured image I of the calculating described in step (5)_OrgIn region I need to be restained_SelAs follows:

I_Sel＝I_Org×I_seg。

7. the interactive image based on optimal model according to claim 1 is restained method, it is characterized in that: as follows by the conversion method of RGB color space conversion to hsv color space described in step (6):

We mark Max is the maximum in (R, G, B), and Min is the minimum of a value in (R, G, B),

V＝MAX

R in formula, G, B represents respectively the red, green, blue color value of image, and H is the tone value of image, and S is the saturation degree of image, and V is the brightness of image.

8. the interactive image based on optimal model according to claim 1 is restained method, it is characterized in that: described in step (8) by hsv color space transforming to the conversion method of RGB color space by following formula:

p＝V(1-S)

q＝V(1-fS)

t＝V(1-(1-f)S)

R＝V，G＝t，B＝p if H_i＝0

R＝q，G＝V，B＝p if H_i＝1

R＝p，G＝V，B＝t if H_i＝2

R＝p，G＝q，B＝V if H_i＝3

R＝t，G＝p，B＝V if H_i＝4

R＝V，G＝p，B＝q if H_i＝5 。

9. the interactive image based on optimal model according to claim 1 is restained method, it is characterized in that: the image I after the merging described in step (9) is restained_RGBcolorWith coloured image I_OrgFor I_NewAs follows:

I_New＝I_Sel×I_RGBcolor+(1-I_Sel)×I_Org。