A kind of more example interactive image segmentation methods
Technical field
The invention belongs to image segmentation and integration technology fields, and in particular to a kind of more example interactive image segmentation sides
Method.
Background technique
Image segmentation is to extract in region significant in image by certain feature (such as edge, texture etc.),
Judge whether there is interested target in image;Image segmentation be generally based on similitude between pixel, pixel mutability or
The progress such as space length.Similitude between so-called pixel is exactly that the pixel in some region has similar feature, such as
Element value is close, texture is mutually same;Pixel mutability is marginal portion in the picture, can there is the mutation of pixel value;Space away from
From the foundation that can also be used as segmentation, a possibility that usual closer pixel of space length is divided into a classification, is larger.
There are many ways to image segmentation, is divided into semi-automatic segmentation and automatic segmentation according to whether the artificial process for participating in segmentation is needed.
Since target image is different and the similitude of background and object pixel, automatic segmentation is difficult will have high-level semantic in image
Object complete extraction comes out;Semi-automatic segmentation is that user's interaction is added on the basis of automatic segmentation as algorithm to input, and makes figure
Object as in high-level semantic can be completely extracted.
Interactive image segmentation passes through using some foreground points and the background dot marked in advance in the picture, can be effectively
Reduce the uncertainty of segmentation, therefore is widely used in picture editting field.The interaction based on area marking occurred in recent years
Formula dividing method, it is only necessary to the classification of partial pixel point is marked in image to be split.It is most popular in image segmentation algorithm at present
Based on figure hugger opinion interactive image segmentation method GrabCut just belong to this kind of methods, it by mark one include
The rectangle frame of prospect distinguishes foreground and background: the pixel of outer rectangular frame is background, and the pixel within rectangle frame has relatively probably
Rate is prospect.The major defect of GrabCut method is: can only divide the image for possessing single foreground target, it is desirable that prospect and back
The distribution of color of scene element meets mixed Gauss model, and the distribution of color of the two is required to have larger difference, to preceding, background pair
The borderline region segmentation effect not stronger than degree is poor.The method of another kind of prevalence is linear restriction Spectral Clustering, and this method will
Markup information is encoded to linear homogeneous equality constraint and is added in classical spectral clustering image segmentation frame, to obtain interactive mode
Image segmentation result;This method do not depend on before, the mixed Gauss model of background color distribution, therefore almost can apply to all fields
The image of scape, but the disadvantage is that: calculate it is more time-consuming, can not apply in real time, cannot by the space smoothing information coding of pixel at
Corresponding constraint, low for markup information utilization rate, needing largely to mark pixel could accurate segmentation result.It is based on
Target image to be split is converted to a graph structure by the interactive image segmentation method of figure hugger opinion, by will be with Solve problems
It is converted to and minimizes the problem of energy solves to realize that image segmentation, common method have GraphCut, GrabCut and spectrum poly-
Class method etc., but there are problems that two in the interactive image segmentation algorithm discussed currently based on figure hugger: the example 1. extracted
Several limitations;2. partitioning algorithm speed is slower, Real-time segmentation cannot achieve.
The Chinese patent of Patent No. CN 102360494A proposes a kind of more foreground target interactive image segmentation methods,
Different from the interactive image segmentation based on region, this method is not necessarily to assign weight to each side in figure, but at each
A discriminant analysis method is introduced in the local window field of pixel, and pixel is mapped directly into classification mark by its feature vector
Label;This method causes disadvantage is that the class label to each pixel of the local window of image carries out estimation error
Calculation amount is too big, and there may be similar or identical pixel in local window, causes to compute repeatedly.Patent No. CN
The Chinese patent of 107730528A proposes a kind of interactive image segmentation and fusion method based on Grabcut algorithm, the party
Method combines Grabcut algorithm and watershed algorithm solves the problems, such as the close segmentation inaccuracy of background before Grabcut, but the party
The defect of method is to be limited only to preceding background segment.
Summary of the invention
In view of above-mentioned, the present invention provides a kind of more example interactive image segmentation methods, only need to mark a small amount of
Pixel, energy Fast Segmentation random color distribution possess the target image of multiple examples.
A kind of more example interactive image segmentation methods, include the following steps:
(1) pass through each example in the artificial uncalibrated image of interactive mode, each foreground target and background in image are equal
Example is corresponded to, therefore example quantity K is foreground target number n plus background number 1 i.e. K=n+1;
(2) Gaussian modeling is carried out to each example according to the colouring information for having demarcated pixel;
(3) using EM (Expectation-Maximization, expectation maximization) algorithm to parameterμk、σk、xkWith
ykIt is iterated update;Wherein: μkAnd σkThe mean value and mean square deviation of respectively k-th example Gaussian statistics model, xkAnd ykRespectively
For the transverse and longitudinal coordinate of k-th of example central point in the picture,For the weight of k-th of example joint classification model, k is natural number
And 1≤k≤K;
(4) pixel p is not demarcated for any in image, according to parameter μ k and σkIt calculates pixel p and belongs to each example
Posterior probability;
(5) according to parameter xkAnd ykPixel p is calculated at a distance from each example central point;
(6) the joint classification model for combining to obtain each example by weight according to step (3) and the result of (4) is as follows;
uk(p)=λ Φk(p)+(1-λ)·dk(p)
Wherein: uk(p) belong to the joint classification probability of k-th of example, Φ for pixel pk(p) belong to kth for pixel p
The posterior probability of a example, dk(p) for pixel p with k-th of example central point at a distance from, λ for preset weight coefficient (preferably
There is preferable segmentation effect for all pictures when λ takes 0.8);
(7) according to parameter finally determining after iteration convergenceμk、σk、xkAnd ykRecalculate the joint classification of each example
Model, and then determine by following classifier the affiliated example of pixel p, and traverse all in image do not demarcate pixel according to this
Point;
Further, the concrete methods of realizing of the step (1) are as follows: for different examples, manually utilize different face
Color demarcates a small amount of pixel in each example region.
Further, parameter μ in step (3) iterative processkAnd σkCorresponding initial valueWithPass through following public affairs
Formula is calculated:
Wherein: CiThe color value of pixel is demarcated for i-th in k-th of example region, i is natural number and 1≤i≤nk,
nkFor the number for having demarcated pixel in k-th of example region.
Further, parameter x in step (3) iterative processkAnd ykCorresponding initial valueWithPass through following formula
It is calculated:
Wherein: xiAnd yiThe transverse and longitudinal coordinate of pixel in the picture, i have been demarcated i-th in respectively k-th of example region
For natural number and 1≤i≤nk, nkFor the number for having demarcated pixel in k-th of example region.
Further, parameter in step (3) iterative processCorresponding initial valueIt is calculated by the following formula
It arrives:
Wherein: nkFor the number for having demarcated pixel in k-th of example region.
Further, use EM algorithm to parameter in the step (3)μk、σk、xkAnd ykIt is iterated update, specifically
Based on following update formula:
Wherein:For the intermediate variable during iteration j,WithCorrespond to jth time and+1 iteration of jth
In the processWithCorrespond to the μ in+1 iterative process of jthkAnd σk,WithCorrespond to iteration j
X in the processkAnd yk,WithCorrespond to the x in+1 iterative process of jthkAnd yk,Correspond to iteration j
U in the processk(p), CpFor the color value of pixel p, nsFor the number for not demarcating pixel in image, Ω is not mark in image
Determine the set of pixel.
Further, the posterior probability that pixel p belongs to each example is calculated by the following formula in the step (4);
Wherein: CpFor the color value of pixel p.
Further, pixel p is calculated by the following formula at a distance from each example central point in the step (5);
Wherein: x and y is respectively the transverse and longitudinal coordinate of pixel p in the picture.
More example Gaussian modelings the present invention is based on colouring information be based on a kind of following color space, including
YCbCr color space, RGB color and hsv color space;More example Gaussian modelings are based on mixed Gauss model
More instance models, wherein the number of example is specified by user, and most example numbers are limited by calculator memory.Distance in the present invention
What formula was selected is Euclidean distance and the exponent conversion for carrying out e, combines range formula and more example mixed Gaussians are built
Mould information carries out classification judgement, and wherein range formula and more example mixed Gauss models this two are respectively provided with weight, and weight is logical
It crosses iterative calculation and obtains optimum.
The method of the present invention when solving the problems, such as more example interactive image segmentations, is not only sufficiently examined compared with the prior art
The colouring information of image is considered, and has also contemplated the location information between image, by the color and the position that comprehensively utilize image
It sets to realize high performance image segmentation.
Detailed description of the invention
Fig. 1 is the flow diagram of the more example interactive image segmentation methods of the present invention.
Fig. 2 is that the more examples of the present invention use interactive mode manually calibrated image.
Fig. 3 is the result images after the more example segmentations of the present invention.
Specific embodiment
In order to more specifically describe the present invention, with reference to the accompanying drawing and specific embodiment is to technical solution of the present invention
It is described in detail.
Interactive mode in the present embodiment using pixel will be based on mixing in conjunction with more examples as user's interactive mode
The method of Gauss model and range formula is closed to realize technical solution of the present invention, as shown in Figure 1, specific implementation step is as follows:
Step 1: user passes through multiple examples (including multiple foreground targets and background) in interaction uncalibrated image, this implementation
Using the interactive mode of pixel as interactive mode in example, when initial in the pixel uncalibrated image of user's different colours
Multiple foreground targets and background, as shown in Figure 2.
Step 2: selecting initial training data using the interactive information that user provides.Specific operation process in the present embodiment
Initial stage selects the pixel for being labeled as background as primary data to learn to be carried on the back for the training data of background model
Scape statistical model;The training data of different target model each for prospect, selection are labeled as the foreground target of a certain color
Pixel learn to obtain the statistical model of the multiple targets of prospect as primary data.
Step 3: the model that is learnt using training data obtained in step 2 simultaneously combines range formula, is combined
Disaggregated model, and then classified using joint classification model to the non-mark pixel in entire image, the following detailed description of connection
Close disaggregated model and its learning process.
It is similar to data statistics model that the present embodiment calculates each pixel based on the statistical sorter of Gauss model
Property, it is assumed that C is the color value of some pixel (x, y) not marked, then posterior probability calculation formula is as follows:
Wherein: μkAnd σkThe mean value and mean square deviation of k-th of Gauss model are respectively indicated, K indicates that foreground target number n adds background
Number 1, i.e. K=n+1, Φk(C|μk,σk) it is the probability that certain pixel (x, y) belongs to k-th of example.
Assuming that x, y are the coordinate position of pixel, then the calculation formula of distance is as follows:
Wherein: xkAnd ykRespectively represent central point cross, the ordinate of kth class example.
Before application formula (1) and formula (2), need to determine the color value of each pixel, each Gaussian mode first
The centre coordinate point of the mean value and mean square deviation of type and every a kind of example.
Specifically, the color value calculation formula of each pixel is as follows:
C=(R*30+G*59+B*11+50)/100 (3)
Wherein: R, G, B respectively represent the RGB triple channel value of pixel.
Specifically, the mean value of each Gauss model and mean square deviation calculation formula are as follows:
Specifically, the centre coordinate point calculation formula of every a kind of example is as follows:
Wherein: nkIndicate that the pixel that classification is k marks number, xikAnd yikRespectively indicate some pixel that classification is k
Cross, the ordinate of point.
Joint classification model is based on above-mentioned formula (1) and formula (2) and to give two formula certain weight combination
It forms, the joint classification new probability formula for belonging to kth class is as follows:
uk(x,y,C|xk,yk,μk,σk)=λ Φk(C|μk,σk)+(1-λ)·dk(x,y|xk,yk), k=1,2 ..., K (6)
Wherein: λ is specific gravity shared by colouring information in model, if its value is larger, then it represents that the foundation of classification depends on
The probability of colouring information is larger, and the parameter is by manually adjusting to obtain optimal solution;It was proved that for all when λ takes 0.8
Picture has preferable segmentation effect.
It combines and assigns the joint classification new probability formula of all categories to different weights, obtain joint classification model,
That is:
Wherein:Indicate the weight of every a kind of joint classification model, initialization formula is as follows:
Determination for above-mentioned parameter can be obtained using expectation maximization EM algorithm, particularly may be divided into following four step
It is rapid:
Step 3.1: initializing all parameters;It enablesPass through formula (4), formula (5) and formula
(8) all parameters have been initialized.
Step 3.2: estimation parameterPosterior probability, formula is as follows:
Wherein: nsIt indicates not marking number a little, calculates the probability density value not marked a little under the model with formula (6)
uk(xi,yi,Ci|Θ)。
Step 3.3: according to the estimated value of step 3.2, updating all parameters.
Update weight:
Update mean value:
Update variance:
Update centre coordinate:
Step 3.4: the condition of convergence, constantly iterative step 3.2 and step 3.3, repeat to update five values above, until
Parameters variation is not significant, is embodied in | Θ-Θ ' | < ε, Θ ' they are updated parameter, and ε value takes 10-5.It is calculated by using EM
Method respectively obtains the polytypic optimal joint disaggregated model parameter not marked a little, finally formula (14) is combined to can be obtained to not
More Exemplary classes devices of pixel are marked, classification standard is as follows:
Step 4: using classifier obtained in step 3, do not mark pixel to each and classify, and obtain containing only
Foreground target as a result, as shown in Figure 3.If user is dissatisfied to classification results, it can add or mark again being trained, return
Step 2 and step 3 obtain new results and feed back to user;Otherwise, current results figure is saved under specified path is PNG format
Image.
The above-mentioned description to embodiment is for that can understand and apply the invention convenient for those skilled in the art.
Person skilled in the art obviously easily can make various modifications to above-described embodiment, and described herein general
Principle is applied in other embodiments without having to go through creative labor.Therefore, the present invention is not limited to the above embodiments, ability
Field technique personnel announcement according to the present invention, the improvement made for the present invention and modification all should be in protection scope of the present invention
Within.