CN105761292A - Image rendering method based on color shift and correction - Google Patents

Abstract

The invention discloses an image rendering method based on color shift and correction. The image rendering method comprises the steps of carrying out space conversion and obtaining component information of brightness and color at any pixel point of a target gray scale image; reading out any one pixel point t in the target gray scale image; selecting a pixel point s in a source color image by using a non-local approach, with a window determined by the pixel point s having similar variance features and discrete Fourier transform features to those of a window determined by the pixel point t; carrying out initial rendering for a color value of the pixel t according to a color value of the pixel s; establishing a coupling total variation model, solving the model to obtain an optimal color value, taking the solved optimal value as a corrected color value, and keeping a brightness value unchanged; and repeating the above steps, correcting each pixel point t of the target gray scale image, and outputting the corrected image. The invention can well overcome the problem of inconsistent color space after rendering, and achieves automatic, quick and accurate rendering.

Description

A kind of image rendering methods based on color transfer and correction

Technical field

The present invention relates to a kind of image rendering methods based on color transfer and correction, belong to the technical field of image processing.

Background technology

In information transmission and exchange, image is important communications media.In human visual system, color is one of key factor embodying visual cognition.Owing to the sensitivity of half-tone information be can not show a candle to colour information by human eye, it is desirable to gray level image is become coloured image to strengthen visual effect, namely image colorant (imagecolorization) technology, is also referred to as gray level image colorization.Image colorant is the computer assisted procedures that color adds to still image or video sequence.Having a wide range of applications in video display, medical treatment, space probation and the field such as industry, scientific research at present, the research of image rendering methods simultaneously is always up image procossing and one, applied mathematics field is active, challenging research topic.

The image rendering methods that view-based access control model was cognitive in recent years can be divided mainly into two classes: a kind of be based on local color expansion color method；Another kind be based on color transfer color method.At present; in the painted technique study shifted based on color, owing to, in target gray image, the coloring treatment at each pixel place is independent; therefore would generally there is the problem that color is inconsistent in this traditional color method, and utilizes what partial differential equation modeled color method to be still far from perfect.How to allow color keep Space Consistency (spatialconsistency) in transfer process, be always up color and shift a great problem of painted technique study.And, the image rendering methods based on color transfer not yet has unified method.

Summary of the invention

The technical problem to be solved is in that to overcome the deficiencies in the prior art, a kind of image rendering methods based on color transfer and correction is provided, it is independent for solving prior art each pixel place coloring treatment in image colorant process, and cause that color cannot keep the problem of Space Consistency in transfer process, to realize the accurately and quickly painted of image.

The present invention specifically solves above-mentioned technical problem by the following technical solutions:

A kind of image rendering methods based on color transfer and correction, the method is based on source coloured image and target gray image, including step:

Step 1, space transforming: select YCbCr space, obtain the component information of any pixel point brightness and colourity in target gray image by the conversion of rgb space to YCbCr space, i.e. (Y, Cb, Cr)；

Any one pixel t (Y in step 2, reading target gray image_t,Cb_t,Cr_t), centered by this pixel t, open the window of L × L, it is determined that the Variance feature of this pixel place window and discrete Fourier transform feature；Utilize non local method, selected pixels point s (Y in the coloured image of source_s,Cb_s,Cr_s), it is desirable to the Variance feature of the window determined by this pixel s and discrete Fourier transform feature are close with pixel t determined window Variance feature in target gray image and discrete Fourier transform feature；According to the chromatic value (Cb of pixel s in this source coloured image_s,Cr_s) to the chromatic value (Cb of pixel t in target gray image_t,Cr_t) carry out tentatively painted, repeat this step until each pixel of preliminary painted target gray image；

Step 3, according to the chromatic value of pixel t in the tentatively painted target gray image obtained and the original brightness value Y of this point_t, set up coupling total variation model；Solving coupling total variation model and obtain the optimal solution of pixel t chromatic value, using the optimal solution of chromatic value tried to achieve as revised chromatic value, brightness value remains unchanged；

Step 4, repeating said steps 3, the image output each pixel t of target gray image being modified and being formed by revised pixel, what namely complete target gray image is painted.

Further, as a preferred technical solution of the present invention, described step 1 also includes selected YCbCr space is normalized.

Further, as a preferred technical solution of the present invention, described step 2 selected pixels point s specifically includes:

The sampling set being made up of several pixels is chosen in the coloured image of source；

Calculate Variance feature and the discrete Fourier transform feature of the determined window of pixel in described sampling set, and choose wherein close with pixel t determined window Variance feature and discrete Fourier transform feature pixel as feasible solution；

Select any one solution as target solution from several feasible solutions, and extract this pixel chromatic value corresponding to target solution.

Further, as a preferred technical solution of the present invention, described step 3 adopts ADM Algorithm for Solving coupling total variation model.

Further, as a preferred technical solution of the present invention, described ADM algorithm specifically includes: coupling total variation model is changed into the convex programming problem of the linear convex set contrained of standard；Use ADM algorithm to carry out alternating iteration and solve acquisition optimal solution.

The present invention adopts technique scheme, can produce following technique effect:

(1), the image rendering methods based on color transfer and correction provided by the present invention, mainly it is respectively established at preliminary tinting stage and color correct, at preliminary tinting stage, the pixel that target gray image is close with source coloured image feature is found out by non local method, and the chromatic value at this pixel place is passed to gray image, thus realize tentatively painted to gray level image.In the color correct stage, set up the total variation model of a coupling, carry out regional area color diffusion, revise the color at the inconsistent pixel place of color in preliminary coloring process, with obtain the rgb space coordinate of each pixel and complete painted after image output.Select to start to image colorant end of output from image space, when ensureing that color is not crossed the border, it is achieved regional area color spreads, and reaches the quick and precisely correction purpose at color inconsistent some place.The present invention can overcome the problem that painted rear color space is inconsistent preferably, it is achieved automatic, quickly accurately painted to target gray image, and acquired results is better than the existing image rendering methods based on color transfer at present.

(2) and the method for the present invention in model solution process, first model uses the Lagrangian method of augmentation be converted into the convex programming problem of linear convex set contrained of standard, then with alternating direction multiplier ADM Algorithm for Solving.Compared to traditional stains algorithm, this convergence and painted precision are obtained for and are effectively improved.Meanwhile, this algorithm has stronger robustness.

Accompanying drawing explanation

Fig. 1 is the schematic diagram based on color transfer and the image rendering methods of correction of the present invention.

Detailed description of the invention

Below in conjunction with Figure of description, embodiments of the present invention are described.

As it is shown in figure 1, the present invention devises a kind of image rendering methods based on color transfer and correction, the method is based on source coloured image and target gray image, and the method mainly includes four parts: 1. change between color；2. tentatively painted；3. the inconsistent point in space is carried out color correct；4. coloured image output.Selecting to start to image colorant end of output from image space, when ensureing that color is not crossed the border, it is achieved regional area color spreads, and reaches the quick and precisely correction purpose at color inconsistent some place, each step is respectively as following:

Step 1, color space are changed；Select part at color space, select suitable color space YCbCr space, by the conversion of rgb space to YCbCr space, it is thus achieved that the component information of the some brightness of image any pixel and colourity, i.e. (Y, Cb, Cr)；The present invention select can the YCbCr space of phenogram image brightness, chromaticity preferably, improve coloration efficiency.For the color space matrix extracted, it is necessary to be normalized, Cb (or Cr) value of each chrominance channel is held between 16/255～240/255.

Step 2, requirement by designed image rendering methods, next to that the color of preliminary coloured part moves process, its principle is to read any one pixel t (Y in target gray image_t,Cb_t,Cr_t), centered by this pixel t, open the window of L × L, it is determined that the Variance feature of this pixel place window and discrete Fourier transform feature；Utilize non local method, selected pixels point s (Y in the coloured image of source_s,Cb_s,Cr_s), the Variance feature of the window that requirement is determined by this pixel s and discrete Fourier transform feature, close with pixel t determined window Variance feature and discrete Fourier transform feature in target gray image, this close degree can arrange corresponding numerical range according to painted required precision, if range of error is [-0.05,0.05], but the invention is not restricted to this scope, other numerical rangies may be equally applied in this method.Thus according to the chromatic value (Cb of pixel s in this source coloured image_s,Cr_s) to the chromatic value (Cb of pixel t in target gray image_t,Cr_t) carry out tentatively painted, repeat this step until each pixel t of preliminary painted target gray image.

The process of above-mentioned selected pixels point s is: first choose the sampling set being made up of several pixels in the coloured image of source；Then, calculate Variance feature and the discrete Fourier transform feature of the determined window of pixel in described sampling set, and choose wherein close with pixel t determined window Variance feature and discrete Fourier transform feature pixel as feasible solution；Select any one solution as target solution from several feasible solutions, and extract this pixel chromatic value corresponding to target solution.

In the inventive method, the specific implementation process of preliminary tinting steps is as follows: coloured image S coordinate under YCbCr color space in source is (Y_S,Cb_S,Cr_S), target gray image T coordinate under YCbCr color space is (Y_T,Cb_T,Cr_T), wherein Cb_T,Cr_T=0.For any one pixel t in target gray image T, utilize non local method, find out so that Y_TWith Y_SFeature closest to time, the corresponding pixel s in the coloured image S of source, and by the chromatic value V of pixel s₀=(Cb₀,Cr₀) as the chromatic value of pixel t, can realize tentatively painted.Specifically, for improving coloration efficiency, source coloured image is carried out the stress and strain model of suitable size, n on grid is put as sub sampling set D (S_n), the pixel set of target gray image is D (T).For pixel t ∈ D (T) and s ∈ D (S_n), they coordinates in the picture be respectively t (x, y) and s (x', y'), region (i.e. window) brightness value of the L × L size centered by t, s respectively:

P_L(t)={ Y_T(x+k,y+l)},P_L(s)={ Y_S(x'+k,y'+l)}

(1.1)

Wherein, $k,l=-\frac{L}{2},...,\frac{L}{2}.$

For all pixels in the two region, choose two main characteristics of image: Variance feature, can picture engraving texture features preferably；Discrete Fourier transform feature (DFT), can picture engraving architectural characteristic preferably.The model of color transfer is as follows:

f₁(t,L),f₁(s, L) is pixel t and the Variance feature of pixel s respectively:

f₁(t, L)=Var (P_L(t)),f₁(s, L)=Var (P_L(s))(1.2)

f₂(t, L, ξ), f₂(s, L, ξ) is DFT (discrete Fourier transform) feature of pixel t and pixel s respectively:

f₂(t, L, ξ)=| DFT (P_L(t),ξ)|,f₂(s, L, ξ)=| DFT (P_L(s),ξ)|(1.3)

Wherein ξ=(ξ₁,ξ₂) represent frequency.

For pixel t ∈ D (T) and s ∈ D (S_n), the index of correlation under two kinds of features of definition respectively:

d₁(t, s, L)=| f₁(t,L)-f₁(s,L)|(1.4)

$d_2(t,s,L)=\underset{\mathrm{\ξ}}{\Σ}|f_2(t,L,\mathrm{\ξ})-f_1(s,L,\mathrm{\ξ})|---\left(1.5\right)$

F in formula₁Represent the variance of certain area pixel set, f₂Represent the DFT value of certain area pixel set.d₁And d₂Represent the variance in region centered by pixel t and s and the difference of DFT value respectively.For any pixel t, it is necessary to try to achieve d₁And d₂S coordinate figure corresponding time minimum.For making solving result more accurately solve, introduce the less variable h > 0 of numerical value, and be converted into index number problem, solve this index maximum time s coordinate figure.

$s_1=\underset{s\∈D\left(S_n\right)}{\mathrm{argmax}}\left\{\exp (-\frac{|d_1(t,s,L){|}^2}{h^2})\right\},s_2=\underset{s\∈D\left(S_n\right)}{\mathrm{argmax}}\left\{\exp (-\frac{|d_2(t,s,L){|}^2}{h^2})\right\}---\left(1.6\right)$

By solving above-mentioned two formulas, it is able to obtain under two kinds of features, the chromatic value that pixel t is corresponding:

$V_0^1\left(t\right)=\{{\mathrm{Cb}}_S\left(s_1\right),{\mathrm{Cr}}_S\left(s_1\right)\},V_0^2\left(t\right)=\{{\mathrm{Cb}}_S\left(s_2\right),{\mathrm{Cr}}_S\left(s_2\right)\}$

And then obtain YCbCr space coordinate figure corresponding for pixel t, use two kinds of features respectively target gray image to be carried out tentatively painted:

$T_0^1\left(t\right)=\{Y_T\left(t\right),{\mathrm{Cb}}_S\left(s_1\right),{\mathrm{Cr}}_S\left(s_1\right)\},T_0^2\left(t\right)=\{Y_T\left(t\right),{\mathrm{Cb}}_S\left(s_2\right),{\mathrm{Cr}}_S\left(s_2\right)\}$

WithSolving result be likely to have multiple, the design proposes arbitrarily to select one of them feasible solution as the target solution based on Variance feature and DFT feature respectively.

Step 3, mainly the inconsistent point in space is carried out color correct, i.e. the chromatic value of pixel t and the original brightness value Y of this point in the tentatively painted target gray image obtained_t, set up coupling total variation model；Revise part adopt color diffusion thought, it is proposed to coupling total variation model as follows:

$\underset{V\∈\mathrm{\Λ}}{min}\left\{{\∫}_{\mathrm{\Ω}}g\left(\right|{\mathrm{\ΔG}}_{\mathrm{\σ}}*Y_0\left|\right)\right|DV|+\underset{i=1}{\overset{2}{\Σ}}\frac{{\mathrm{\α}}_i}{2}{{\∫}_{\mathrm{\Ω}}|V-{V_0}^i|}^{2}dx\}---\left(2.1\right)$

Wherein, $\mathrm{\Λ}:=\left\{\begin{array}{c}V\left(x\right)=(Cb\left(x\right),Cr\left(x\right))|V\∈g-BV(\mathrm{\Ω};R^2),\\ a\≤Cb\left(x\right),Cr\left(x\right)\≤bfora.e.x\∈\mathrm{\Ω}\end{array}\right\},$

Described Ω is bounded image area, a, and b is constant, and a=16/255, b=240/255.

In this model, preceding paragraph is coupling total variation regularization term, and consequent is initially painted fidelity item.Wherein: α₁,α₂> 0 is weight parameter, and g is monotonic decreasing function, is set toT ∈ R, k are set as 5000.D is gradient operator, and Δ is Laplace operator.It is defined below respectively:

$DV:=({\∂}_{x}V,{\∂}_{y}V);\mathrm{\Δ}Y:={\∂}_{xx}Y+{\∂}_{yy}Y.$

G_σFor gaussian kernel, this model adopts σ=1.Y₀Brightness value for target gray image.|ΔG_σ*Y₀| can effectively portray the structural information in target image.|ΔG_σ*Y₀| representing color boundary time bigger, now corresponding g value is less, and this coupling total variation item can effectively prevent color from occurring more zone phenomenon in diffusion process.In fidelity item,For, in preliminary tinting stage, adopting the chromatic value that two kinds of different characteristics are tried to achieve.This model can when ensureing that color is not crossed the border, it is achieved regional area color spreads, thus reaching the automatic correction to preliminary painted rear color.

The optimal solution V of pixel t chromatic value is obtained, once optimal solution V determines, in conjunction with brightness value Y through solving coupling total variation model₀, the color at each pixel t place can be obtained, the chromatic value of preliminary painted rear pixel t is modified by the chromatic value of described optimal solution, brightness value remains unchanged, and then can complete the automatically painted of target image.When ensureing that color is not crossed the border, it is achieved regional area color spreads, and reaches the purpose quick and precisely revised of the inconsistent point in space.

For above-mentioned model, present invention preferably uses following derivation algorithm:

1. by created symbol Γ ≡ V=(Cb, Cr) | a≤Cb, Cr≤b}, and auxiliary variableAnd W, model is changed into the convex programming problem of the linear convex set contrained of following standard:

$\underset{\stackrel{\→}{d},V,W}{\arg \min }\left\{{\∫}_{\mathrm{\Ω}}g\left(\right|{\mathrm{\ΔG}}_{\mathrm{\σ}}*Y_0\left|\right)\right|\stackrel{\→}{d}\left(x\right)|dx+\underset{i=1}{\overset{2}{\Σ}}\frac{{\mathrm{\α}}_i}{2}||W-{V_0}^i|{|}_2^2\}---\left(2.2\right)$

$s.t\▿V=\stackrel{\→}{d},V=WandW\∈\mathrm{\Γ}.$

2. use change of direction Multiplier Algorithm, i.e. ADM algorithm, introduce LagrangianObtain following Augmented Lagrangian Functions:

$\begin{array}{c}l(\stackrel{\&RightArrow;}{d},W,V,{\stackrel{\&RightArrow;}{\mathrm{\&lambda;}}}_1,{\mathrm{\&lambda;}}_2)\\ {\&Integral;}_{\mathrm{\&Omega;}}g\left(\right|{\mathrm{\&Delta;G}}_{\mathrm{\&sigma;}}*Y_0\left|\right)\left|\stackrel{\&RightArrow;}{d}\left(x\right)\right|dx+<{\stackrel{\&RightArrow;}{\mathrm{\&lambda;}}}_1,\&dtri;V-\stackrel{\&RightArrow;}{d}>+\frac{{\mathrm{\&mu;}}_1}{2}\left|\right|\stackrel{\&RightArrow;}{d}-\&dtri;V|{|}_2^2\\ +x_{\mathrm{\&Gamma;}}\left(W\right)+\underset{i=1}{\overset{2}{\&Sigma;}}\frac{{\mathrm{\&alpha;}}_i}{2}\left|\right|W-{V_0}^i|{|}_2^2+<{\mathrm{\&lambda;}}_2,V-W>\\ +\frac{{\mathrm{\&mu;}}_2}{2}\left|\right|V-W|{|}_2^2\end{array}---\left(2.3\right)$

Wherein, $x_{\mathrm{\&Gamma;}}\left(V\right):=\left\{\begin{array}{cc}0,& V\&Element;\mathrm{\&Gamma;},\\ +\mathrm{\&infin;},& otherwise.\end{array}\right.$

Algorithm flow is as follows:

AssumeIt is any real number with λ, μ > 0, then ordered series of numbersCalculated by ADM algorithm and converge toWhereinIt it is the optimal solution of object function (2.1).

The present invention adopts ADMM algorithm in model solution process, can effectively overcome the shortcoming that tradition gradient descent method speed is slow and is easily trapped into local minimum.Compared to traditional stains algorithm, this convergence and painted precision are obtained for and are effectively improved.Meanwhile, this algorithm has higher robustness, but the invention is not restricted to this kind of derivation algorithm.

Step 4, repetition step 3, be modified each pixel t of target gray image, the image output formed by revised pixel.Last image output portion, utilizes YCbCr to rgb color space inverse conversion, obtain the rgb space coordinate of each pixel t in target gray image and complete painted after image output.

To sum up, the image rendering methods that must shift based on color and revise proposed by the invention, selects to start to image colorant end of output from image space, when ensureing that color is not crossed the border, realize the diffusion of regional area color, reach the quick and precisely correction purpose at color inconsistent some place.Overcoming the problem that painted rear color space is inconsistent preferably, it is achieved automatic, quickly accurately painted to target gray image, acquired results is better than the existing image rendering methods based on color transfer at present.

Above in conjunction with accompanying drawing, embodiments of the present invention are explained in detail, but the present invention is not limited to above-mentioned embodiment, in the ken that those of ordinary skill in the art possess, it is also possible to make a variety of changes under the premise without departing from present inventive concept.

Claims (5)

1., based on an image rendering methods for color transfer and correction, the method is based on source coloured image and target gray image, it is characterised in that include step:

Step 1, space transforming: select YCbCr space, obtain the component information of any pixel point brightness and colourity in target gray image by the conversion of rgb space to YCbCr space, i.e. (Y, Cb, Cr)；

Any one pixel t (Y in step 2, reading target gray image_t,Cb_t,Cr_t), centered by this pixel t, open the window of L × L, it is determined that the Variance feature of this pixel place window and discrete Fourier transform feature；Utilize non local method, selected pixels point s (Y in the coloured image of source_s,Cb_s,Cr_s), it is desirable to the Variance feature of the window determined by this pixel s and discrete Fourier transform feature are close with pixel t determined window Variance feature in target gray image and discrete Fourier transform feature；According to the chromatic value (Cb of pixel s in this source coloured image_s,Cr_s) to the chromatic value (Cb of pixel t in target gray image_t,Cr_t) carry out tentatively painted, repeat this step until each pixel of preliminary painted target gray image；

Step 3, according to the chromatic value of pixel t in the tentatively painted target gray image obtained and the original brightness value Y of this point_t, set up coupling total variation model；Solving coupling total variation model and obtain the optimal solution of pixel t chromatic value, using the optimal solution of chromatic value tried to achieve as revised chromatic value, brightness value remains unchanged；

Step 4, repeating said steps 3, the image output each pixel t of target gray image being modified and being formed by revised pixel, what namely complete target gray image is painted.

2. according to claim 1 based on the image rendering methods of color transfer and correction, it is characterised in that described step 1 also includes selected YCbCr space is normalized.

3. according to claim 1 based on the image rendering methods of color transfer and correction, it is characterised in that described step 2 selected pixels point s specifically includes:

The sampling set being made up of several pixels is chosen in the coloured image of source；

Calculate Variance feature and the discrete Fourier transform feature of the determined window of pixel in described sampling set, and choose wherein close with pixel t determined window Variance feature and discrete Fourier transform feature pixel as feasible solution；

Select any one solution as target solution from several feasible solutions, and extract this pixel chromatic value corresponding to target solution.

4. according to claim 1 based on the image rendering methods of color transfer and correction, it is characterised in that described step 3 adopts ADM Algorithm for Solving coupling total variation model.

5. according to claim 4 based on the image rendering methods of color transfer and correction, it is characterised in that described ADM algorithm specifically includes: the convex programming problem that total variation model is changed into the linear convex set contrained of standard will be coupled；Use ADM algorithm to carry out alternating iteration and solve acquisition optimal solution.