CN102592268A - Method for segmenting foreground image - Google Patents

Abstract

The invention discloses a method for segmenting a foreground image. The method comprises the following steps of: obtaining a conspicuity map of an original image by using a central peripheral histogram algorithm; carrying out threshold segmentation on the conspicuity map so as to obtain a rectangle R including a conspicuity object; using an image outside the rectangle R region as a background region; initializing a GrabCut algorithm; and iteratively operating the GrabCut algorithm so as to execute foreground segmentation of the original image. Compared with the prior art, the method disclosed by the invention can be used for increasing the foreground segmentation efficiency.

Description

A kind of method of cutting apart foreground image

Technical field

The present invention relates to image processing techniques, particularly relate to a kind of method of cutting apart foreground image.

Background technology

Salient region detects and foreground segmentation is two fundamental operations in the Computer Image Processing.Wherein, salient region detects and refers to the salient region of from picture, judging image, and notices the pith of image.Foreground segmentation refers to that to let computing machine from a width of cloth picture, which judged be foreground object, and which is a background object, and therefrom is partitioned into the crucial object of interested prospect.Though people's vision system can be judged salient region and foreground object at an easy rate, computing machine is not having to be difficult to possess this understandability under artificial the help.If can let computing machine independently accomplish foreground segmentation work apace; To be convenient to further to image analyze, discern, follow the tracks of, understanding, compressed encoding etc.; And the accuracy of extracting the result will directly influence the validity of follow-up work; How quickly and efficiently interested target to be split from complicated background, have crucial meaning.

People have carried out a large amount of research on salient region detects, summed up the algorithm of a lot of maturations, mainly contain HC, and RC, LC, CA and FT scheduling algorithm, these algorithms can access effect conspicuousness figure (saliency map) preferably to a certain extent.And image segmentation algorithm can roughly fall into 5 types at present; The border algorithm; Clustering algorithm, zone algorithm is cut apart the partitioning algorithm of blending algorithm and specific area; At the foreground segmentation technical elements, mainly contain method, based on the method for border (Edge-based) with based on the method in zone (Region-based) based on pixel (Pixel-based).Method based on pixel requires the user to come appointment prospect or background in single Pixel-level, so workload is very huge.Method based on the border allows the boundary mapping curve of user around foreground object, then this curve is carried out segmentation optimization, but the necessary careful curve plotting of user still needs a large amount of user interactions.Method based on the zone allows the user to specify some loose informations, and uses optimized Algorithm to extract actual foreground object border.

Summary of the invention

Technical matters to be solved by this invention is, a kind of method of cutting apart foreground image is provided, and reduces the requirement of operation to user interactions, improves foreground segmentation efficient.

Technical problem of the present invention solves by the following technical programs:

A kind of method of cutting apart foreground image is characterized in that, may further comprise the steps:

1) use the central peripheral histogramming algorithm to obtain the conspicuousness figure of original image;

2) said conspicuousness figure is carried out Threshold Segmentation, obtain comprising the conspicuousness object at interior rectangle R;

3) use the overseas image of said rectangle Zone R zone as a setting, initialization Grabcut algorithm, iteration operation GrabCut algorithm is carried out the foreground segmentation to original image.

Compared with prior art; The present invention has utilized the conspicuousness of image to cut apart the relevance with foreground segmentation; The initializes GrabCut algorithm that utilizes conspicuousness to cut apart has saved the user draws rectangle frame initialization GrabCut algorithm in target image step, in whole cutting procedure, can realize user's zero input; Automatically accomplish all foreground segmentation actions through computer class, improved the efficient of foreground segmentation.

Preferably, said step 2) may further comprise the steps: utilize predetermined gray threshold that conspicuousness figure is carried out binaryzation and obtain binary map; Binary map is carried out twice or opening operation repeatedly; UNICOM maximum in the image behind the measuring and calculating opening operation is regional, selects the rectangle R of certain size and coordinate position, and this UNICOM zone is included in this rectangle R just.

Said gray threshold is the average gray of conspicuousness figure.

Preferably, also comprise interactive editor's step: the partial pixel of original image is made as prospect or background according to user's input instruction.This preferred version allows the user to revise cutting apart, and remedies the weak point of cutting apart automatically.

Description of drawings

Fig. 1 is the process flow diagram of the specific embodiment of the invention.

Embodiment

Contrast accompanying drawing below and combine preferred embodiment that the present invention is carried out detailed elaboration.

One, the existing ripe image processing techniques that the present invention relates to

In order to help the understanding to technical scheme of the present invention, hereinafter at first describes the image processing techniques of maturation involved in the present invention:

(1) conspicuousness object inspection technology

The conspicuousness figure of image adopts center-surround algorithm (central peripheral histogramming algorithm) to calculate: at first add up the grey level histogram of three inner Color Channels of two rectangles, R ⁱBe the grey level histogram of image in the center rectangular area, Grey level histogram for image in the surround rectangular area.Calculate histogrammic fitting degree in center zone and the surround zone based on formula (1)

${\mathrm{\χ}}^2(R,R_S)=\frac{1}{2}\mathrm{\Σ}\frac{{(R^i-R_S^i)}^2}{R^i+R_S^i}---\left(1\right)$

$f_h(x,I)\∝{\mathrm{\Σ}}_{\left[x^{\′}\right|x\∈R^{*}\left(x^{\′}\right)]}{\mathrm{\ω}}_{{\mathrm{xx}}^{\′}}{\mathrm{\χ}}^2(R^{*}\left(x^{\′}\right),R_S^{*}\left(x^{\′}\right))---\left(2\right)$

${\mathrm{\ω}}_{{\mathrm{xx}}^{\′}}=\exp (-0.5{\mathrm{\σ}}_{x^{\′}}^{-2}{\left|\right|x{-x}^{\′}\left|\right|}^2)---\left(3\right)$

The eigenwert of confirming the x pixel according to formula (2) is R wherein ^*(x '),

Represent with x ' to be the center rectangle and the peripheral rectangle at center respectively, all can become R x ' expression ^*(x '),

Center and this R ^*(x '),

The pixel that can comprise pixel x, wherein ω _Xx' be Gauss's attenuation function suc as formula shown in (3), ‖ x-x ' ‖ is the Euclidean distance of pixel x apart from center pixel x ', K is a normaliztion constant.

(2) GrabCut foreground segmentation algorithm

The GrabCut algorithm is on the basis of GraphCut algorithm, to improve, and wherein the GraphCut algorithm is described below:

Image is regarded as a figure G={V, ε }, V is all nodes, ε is the limit that connects adjacent node.Image segmentation can be used as one two meta-tag problem, and each i ∈ V has a unique x _i{ prospect is 1 to ∈, and background is that 0} is corresponding with it.All x _iSet X can obtain through minimizing Gibbs energy E (X):

$E\left(X\right)=\underset{i\∈V}{\mathrm{\Σ}}{E}_1\left(x_i\right)+\mathrm{\λ}\underset{(i,j)\∈E}{\mathrm{\Σ}}{E}_2(x_i,x_j)$

Same, according to the curve that the user draws, we have foreground node collection F and background Node B, unknown node collection U.At first use the K-Mean method with F, the node clustering of B calculates the average color of each type,

Represent the average color set of all prospect classes, background classes is

Calculate the minor increment of each node i to each prospect class $d_i^B=\mathrm{Min}\left|\right|C\left(i\right)-K_n^F\left|\right|,$ Leave with corresponding back pitch $d_i^B=\mathrm{Min}\left|\right|C\left(i\right)-K_n^B\left|\right|,$ Defined formula:

$\left\{\begin{array}{ccc}{E}_1(x_i=1)=0& E_1(x_i=0)=\∞& \∀i\∈F\\ E_1(x_i=1)=\∞& E_1(x_i=0)=0& \∀i\∈B\\ E_1(x_i=1)=\frac{d_i^F}{d_i^F+d_i^B}& E_1(x_i=0)=\frac{d_i^B}{d_i^F+d_i^B}& \∀i\∈U\end{array}\right.$

It is consistent that preceding two groups of equalities guarantee that definition and user import, the 3rd group of equality mean with the color of preceding background mutually recency determining the mark of unknown point.

E ₂Be defined as a function relevant with gradient:

E ₂(x _i，x _j)＝|x _i-x _j|·g(C _ij)

$g\left(\mathrm{\ϵ}\right)=\frac{1}{\mathrm{\ϵ}+1}$ C _ij＝‖C(i)-C(j)‖ ²

E ₂Effect be to reduce between the close pixel of color, the possibility that exists mark to change is even it only occurs on the border.

At last, with E ₁And E ₂As the weights of figure, figure is cut apart, in prospect set or background set, just obtain the result of foreground extraction to the node division of zone of ignorance.

The GrabCut algorithm improves on the basis of GraphCut: (Gaussian Mixture Model GMM) replaces histogram, and gray level image is expanded to coloured image to utilize gauss hybrid models.

In the GrabCut algorithm, use the GMM model to set up the color image data model.Each GMM can regard the covariance of a K dimension as.Handle GMM for ease, in optimizing process, introduce vectorial k=(k ₁..., k _n..., k _N) as the independent GMM parameter of each pixel, and k _n∈ 1,2 ..., K}, the opacity α on the respective pixel point _n=0 or 1.

The Gibbs energy function is written as:

E(α，k，θ，z)＝U(α，k，θ，z)+V(α，z)

In the formula, α is an opacity, α ∈ 1, and 0}, 0 is background, 1 is foreground target; Z is a gradation of image value array,

z＝(z ₁，…，z _n，…，z _N)。

Introduce GMM color data model, its data item may be defined as:

$U(\mathrm{\α},k,\mathrm{\θ},z)=\underset{n}{\mathrm{\Σ}}D({\mathrm{\α}}_n,k_n,\mathrm{\θ},z_n)$

In the formula, D (α _n, k _n, θ, z _nLogp (the z of)=- _n| α _n, k _n, θ)-logn (α _n, k _n),

P () is that gaussian probability distributes, and π () is hybrid weight coefficient (accumulation and be constant).So have:

$D({\mathrm{\α}}_n,k_n,\mathrm{\θ},z_n)=-\log \mathrm{\π}({\mathrm{\α}}_n,k_n)+\frac{1}{2}\log \det ({\mathrm{\α}}_n,k_n)+\frac{1}{2}{[z_n-\mathrm{\μ}({\mathrm{\α}}_n,k_n)]}^T\mathrm{\Σ}({\mathrm{\α}}_n,k_n)[z_n-\mathrm{\μ}({\mathrm{\α}}_n,k_n)]$

The parameter of model is just confirmed as like this:

θ＝{π(α，k)，μ(α，k)，∑(α，k)，k＝1，2，…，K}

Level and smooth of coloured image does,

$V(\mathrm{\α},z)=\mathrm{\γ}\underset{(m,n)\∈c}{\mathrm{\Σ}}[{\mathrm{\α}}_m\≠{\mathrm{\α}}_n]\exp (-\mathrm{\β}{\left|\right|z_m-z_n\left|\right|}^2)$

Wherein, constant β confirms through following formula: β=(2＜(z _m-z _n) ²＞) ^-1, the β that obtains through such formula guarantees exponential term suitably conversion between high low value in the following formula.

Two, a specific embodiment of the present invention

The foreground segmentation method of present embodiment may further comprise the steps: use the central peripheral histogramming algorithm to obtain the conspicuousness figure of original image; Said conspicuousness figure is carried out Threshold Segmentation, obtain comprising the conspicuousness object at interior rectangle R; Use the overseas image of said rectangle Zone R zone as a setting, initialization GrabCut algorithm, iteration operation GrabCut algorithm is carried out the foreground segmentation to original image.

Hereinafter is an example with the concrete processing procedure to original image A, and technical scheme of the present invention is further produced:

1) input original image A is all possible as the center rectangle R of center-surround histogramming algorithm and the central point of peripheral rectangle Rs among the scan image A, calculates the grey level histogram R of image in corresponding R, the Rs zone ⁱAnd R ⁱ _s, the length and width of present embodiment rectangle R are 1/5 of image A length and width, the length and width of Rs are got 4/3 times of R length and width; Calculate the eigenwert of each central point according to aforementioned formula (1)～(3).Calculate histogrammic fitting degree in center zone and the surround zone according to formula (2); Confirm the conspicuousness value of x pixel according to formula (2).For satisfying R, the conspicuousness value of the pixel at Rs center directly is changed to 0.

2) choosing certain threshold value, conspicuousness figure is carried out the binary map that binaryzation obtains is C, and the threshold value of here choosing is preferably the average gray of conspicuousness figure B.Binary map C is carried out i.e. twice opening operation of twice corrosion expansion, thereby reduce the interference of isolated noise.UNICOM maximum among the measuring and calculating C is regional, selects the rectangle frame of certain size and relevant position, and this UNICOM zone is included in this rectangle frame just.If this rectangle frame is R.

3) begin to use the GrabCut algorithm to cut apart, may further comprise the steps original image A:

A) the rectangle R initialization GrabCut algorithm that obtains use rectangle 2): the zone that original image rectangle R is outer is regarded as background T _BCome initialization ternary diagram T (figure that appointment prospect, background and definite zone obtain on original image is exactly ternary diagram T), prospect is made as sky, promptly

Do not confirm regional T _UGet the supplementary set of background

B) for all pixel n ∈ T _g, make opacity α _n=0; N ∈ T _Uα is arranged _n=1.

C) use α respectively _n=0 and α _n=1 two set comes the GMM model of initialization prospect and background.

4) iteration minimizes, and may further comprise the steps:

D) try to achieve T _UIn the pairing GMM parameter of each pixel n k _n, k _{N=arg min}D _n(α, k _n, θ, z _n), (this formula representes to make D _n{ α, k _n, θ, z _n) k when getting minimum value _nValue, down with).

E) from data Z, obtain GMM parameter θ, and θ=argminU (α, k, θ, z).

F) obtain initial segmentation with least energy and revise ternary diagram T.

G) according to f) obtain cut apart ternary diagram T, begin iteration from step d) and repeat, up to E (α, k, θ, z) convergence, promptly (θ z) stops iteration when very little with relatively changing last time for α, k for E after the iteration.

5) if the user is satisfied inadequately to the division of individual pixel, can use the interactive editor to come to specify by force individual point to be prospect, individual point is a background, even individual pixel opacity α _n=0 (background) or α _n=1 (prospect) upgraded ternary diagram T, execution in step f accordingly according to user's correction).So far display foreground is cut apart completion.

Above content is to combine concrete preferred implementation to the further explain that the present invention did, and can not assert that practical implementation of the present invention is confined to these explanations.For person of ordinary skill in the field of the present invention, do not breaking away under the prerequisite of the present invention design, can also make some being equal to substitute or obvious modification, and performance or purposes are identical, all should be regarded as belonging to protection scope of the present invention.

Claims (4)

1. a method of cutting apart foreground image is characterized in that, may further comprise the steps:

1) use the central peripheral histogramming algorithm to obtain the conspicuousness figure of original image;

2) said conspicuousness figure is carried out Threshold Segmentation, obtain comprising the conspicuousness object at interior rectangle R;

3) use the overseas image of said rectangle Zone R zone as a setting, initialization GrabCut algorithm, iteration operation GrabCut algorithm is carried out the foreground segmentation to original image.

2. the method for cutting apart foreground image according to claim 1 is characterized in that: said step 2) may further comprise the steps: utilize predetermined gray threshold that said conspicuousness figure is carried out binaryzation and obtain binary map; Binary map is carried out twice or opening operation repeatedly; UNICOM maximum in the image behind the measuring and calculating opening operation is regional, selects the rectangle R of certain size and coordinate position, and this UNICOM zone is included in this rectangle R just.

3. the method for cutting apart foreground image according to claim 1 is characterized in that: said gray threshold is the average gray of said conspicuousness figure.

4. the method for cutting apart foreground image according to claim 1 is characterized in that, also comprises interactive editor's step: according to user's input instruction the partial pixel of original image is made as prospect or background.

Images