CN102592309A - Modeling method of nonlinear three-dimensional face - Google Patents

Abstract

The invention relates to a modeling method of a nonlinear three-dimensional face, which has better reconstruction effect, and comprises the following steps: (1) respectively selecting three-dimensional samples and two-dimensional samples from the exiting face database as training sample sets and carrying out the standardized operation on the samples; pairing the two-dimensional training sample set and the three-dimensional training sample set during the training stage according to the identity information, so that the samples in the two groups of sample sets mutually correspond according to the identity information; (2) training respective projection matrixes based on the two groups of sample sets, so that samples with different dimensionalities have maximum relevance after being projected; (3) during the reconstruction stage, carrying out standardized treatment on an inputted two-dimensional face image, projecting in a subspace of the two-dimensional training sample set, selecting three-dimensional samples with high relevance according to the relevance distance, constructing a three-dimensional face model based on the selected three-dimensional samples and matching the three-dimensional face model with the inputted image to realize the reconstruction of a three-dimensional face sample.

Description

A kind of modeling method of non-linear three-dimensional human face

Technical field

The invention belongs to the technical field of Flame Image Process, relate to a kind of modeling method of non-linear three-dimensional human face particularly.

Background technology

Three-dimensional facial reconstruction is the work of rebuilding its three-dimensional face data according to the input two-dimension human face image.Because three-dimensional face model has wide application prospect in fields such as computer game, man-machine interactions, thus three-dimensional facial reconstruction to have become be the most active research focus of computer graphics, computer vision.Since the Parke method representation people face that used a computer first in 1972, the modeling of people's face has just obtained paying close attention to widely.Through 30 years of development, the three-dimensional face modeling method has had significant progress.The human face model building that people such as Vetter proposed based on deformation model in 1999, this method have been tested the full automation of people's face modeling for the first time, and can be rebuild persona certa's three-dimensional face model by a width of cloth facial image.Because this model is to be based upon on the corresponding three-dimensional prototype people face data of Pixel-level, and has considered factors such as human face posture, illumination in the model, so this model can generate the three-dimensional face of the height sense of reality.The basis of deformation model is the thought of linear combination, and promptly a class object can be represented with the linear combination of this class object base vector.Modeling method based on deformation model is to realize the coupling between deformation model and the input picture through the combination parameter of regulating model.And people's face is to be nested in the central non-linearity manifold of higher dimensional space.Must ignore the details of human face structure based on the reconstruction algorithm of linear theory, be difficult to reach and rebuild effect preferably.

Summary of the invention

Technology of the present invention is dealt with problems and is: overcome the deficiency of prior art, a kind of modeling method of rebuilding the non-linear three-dimensional human face of better effects if is provided.

Technical solution of the present invention is: the modeling method of this non-linear three-dimensional human face; May further comprise the steps: (1) is chosen three-dimensional samples and two dimensional sample respectively from existing face database; As training sample set, and to these samples enforcement normalization operations; In the training stage, two-dimentional training sample set and three-dimensional training sample set are matched according to identity information, make that the sample in these two groups of sample sets is corresponding each other according to identity information; (2) be grounding projection matrix separately with these two groups corresponding sample sets, make to have maximum correlativity after the sample projection of different dimensions; (3) at phase of regeneration; Two-dimension human face image for input; At first, carry out projection in the subspace of two-dimentional training sample set then, choose the three-dimensional samples high with its degree of correlation according to the correlativity distance to its processing of standardizing; Make up the three-dimensional face deformation model based on these selected three-dimensional samples, and itself and input picture mated realize the reconstruction of three-dimensional face sample.

Because this method is based on canonical correlation analysis and deformation model, so rebuild better effects if.

Embodiment

The modeling method of this non-linear three-dimensional human face may further comprise the steps: (1) is chosen three-dimensional samples and two dimensional sample respectively from existing face database, as training sample set; In the training stage, two-dimentional training sample set and three-dimensional training sample set are matched according to identity information, make that the sample in these two groups of sample sets is corresponding each other according to identity information; (2) be grounding projection matrix separately with these two groups corresponding sample sets, make to have maximum correlativity after the sample projection of different dimensions; (3) at phase of regeneration, at first with the processing (it is carried out yardstick alignment according to the distance between eyes, carry out the displacement alignment according to the position of prenasale, and specification turns to the sample of 192*192 size) of standardizing of the two dimensional sample of two-dimentional training sample set; Two-dimension human face image for input; At first with its processing of standardizing; Carry out projection in the subspace of normalized two-dimentional training sample set then; Choose the three-dimensional samples high according to correlativity distance, make up the three-dimensional face deformation model, and itself and input picture mated realize the reconstruction of three-dimensional face sample based on these selected three-dimensional samples with its degree of correlation.

Preferably, step (1) comprises step by step following: three-dimensional samples collection and two dimensional sample collection are chosen in (1.1) from existing face database; (1.2) three-dimensional samples collection and two dimensional sample collection are corresponding according to identity information; (1.3) three-dimensional samples and two dimensional sample are standardized.

Further; Said step (1.3) comprising: three-dimensional samples is standardized: use alternant way to obtain the position between the three-dimensional samples pupil; Calculate the interpupillary distance of all three-dimensional samples; Mean value with these distances is that standard is carried out dimensional variation to all three-dimensional samples, is that standard is carried out the translation alignment to all three-dimensional samples with the place between the eyebrows point; Two dimensional sample is standardized: use alternant way to obtain the position between the two dimensional sample pupil; Calculate the interpupillary distance of all two dimensional samples; Mean value with these distances is that standard is carried out dimensional variation to all two dimensional samples, is that standard is carried out the translation alignment to all two dimensional samples with the place between the eyebrows point.

Preferably, step (2) comprises step by step following:

(2.1) mean vector of calculating two dimensional sample and three-dimensional samples;

(2.2) all samples are deducted each self-corresponding mean vector;

(2.3) with the relevance formula be objective function, its maximizing optimized that when objective function maximizes, obtain two dimensional sample and three-dimensional samples characteristic of correspondence vector, wherein relevance formula is:

$\mathrm{\ρ}=\frac{E\left[W_x^T{\mathrm{XY}}^T{M}_y\right]}{\sqrt{E\left[W_x^T{\mathrm{XY}}^T{W}_x\right]E\left[W_y^T{\mathrm{XY}}^T{W}_y\right]}},$ Wherein X represents the two dimensional sample collection, and Y represents the three-dimensional samples collection, and the projection base vector of X is W _x, the projection base vector of Y is W _y

(2.4) common dimension based on the two makes up projection subspace separately, is making the vector after the projection have maximum correlativity under the constraint of optimization aim.

In training process, at first these two groups of samples are mapped according to identity information one by one, so just formed the training set of a coupling.Can be calculated at phase of regeneration and the maximally related three-dimensional face sample set of input picture, according to the mapping relations of the 2D-3D that learns.Make X represent two dimensional sample, Y represents three-dimensional samples.

Canonical correlation analysis is a kind of statistical method of correlationship between two groups of variablees of research, and the problem that it solved is how to seek two groups of corresponding base vectors, makes the correlativity of these variablees between the projection on the corresponding base vector by maximization simultaneously.This paper uses this method to set up the relation between two dimensional image and the three-dimensional samples, and we can obtain the neighbor relationships between two dimensional sample and the three-dimensional samples based on this formula.Make that (X Y) is the variable set of two correspondences.Wherein X represents the two dimensional sample collection, and Y represents the three-dimensional samples collection.The projection base vector of supposing X is W _x, the projection base vector of Y is W _yX and Y are at base vector W _xAnd W _yOn linear projection do

With

According to the definition of relative coefficient, we can learn and want With

Correlativity by computes:

$\mathrm{\ρ}=\frac{E\left[W_x^T{\mathrm{XY}}^T{M}_y\right]}{\sqrt{E\left[W_x^T{\mathrm{XY}}^T{W}_x\right]E\left[W_y^T{\mathrm{XY}}^T{W}_y\right]}}$

That is:

$\mathrm{\ρ}=\frac{W_x^{T}E\left[{\mathrm{XY}}^T\right]W_y}{\sqrt{W_x^{T}E\left[{\mathrm{XY}}^T\right]W_x{W}_y^{T}E\left[{\mathrm{XY}}^T\right]W_y}}$

Definition C according to the covariance formula _Xx=E [XX ^T], C _Yy=E [YY ^T], C _Xy=E [XY ^T], following formula can be rewritten as

$\mathrm{\ρ}=\frac{w_x^T{C}_{\mathrm{xy}}{w}_y}{\sqrt{w_x^T{C}_{\mathrm{xx}}{w}_x{w}_y^T{C}_{\mathrm{yy}}{w}_y}}$

W _xAnd W _yFind the solution and can draw through the maximization following formula, just obtain through finding the solution following eigenvalue problem.

$C_{\mathrm{xx}}^{-1}{C}_{\mathrm{xy}}{C}_{\mathrm{yy}}^{-1}{C}_{\mathrm{yx}}{W}_x={\mathrm{\ρ}}^2{W}_x$

$C_{\mathrm{yy}}^{-1}{C}_{\mathrm{yx}}{C}_{\mathrm{yy}}^{-1}{C}_{\mathrm{xy}}{W}_y={\mathrm{\ρ}}^2{W}_y$

In training refers to, at first original sample is deducted average.

When calculating and input picture X _InDuring the most close three-dimensional samples, we calculate with

The three-dimensional samples Y that correlativity is maximum _C, the sample calculation formula does

$c=\frac{x\·y}{\left|\right|x\left|\right|\left|\right|y\left|\right|}$

The three-dimensional samples that is chosen should be greater than a certain threshold value with the correlativity of input two dimensional sample, and this threshold value is confirmed through the three-sigma criterion.

Preferably, step (3) comprises step by step following:

(3.1) input two-dimension human face image;

(3.2) this facial image of standardizing;

(3.3) carry out projection in the subspace of two dimensional sample;

(3.4) correlativity of the projection of calculation procedure (3.3) and three-dimensional samples projection vector;

(3.5) confirm the scope of selected three-dimensional samples according to the variance of all distances;

(3.6) make up deformation model based on selected three-dimensional samples;

(3.7) deformation model and the input picture that make up are mated;

(3.8) use particle cluster algorithm to carry out model optimization, realize the reconstruction of three-dimensional face sample.

The alignment of input picture:

Make by hand the pupil position of the method mark of demarcating, calculate the distance between the pupil, then according to interpupillary distance to input picture processings of standardizing, carry out translation according to the place between the eyebrows position then and align.

Three-dimensional samples is selected:

The purpose that sample is selected is to select and the representation space of the high sample of input picture correlativity as input picture.We use the distance after fitness tolerance two dimensional sample and the three-dimensional samples projection.

Based on the subspace of selected these samples of three-dimensional face sample structure, use PCA (principal component analysis (PCA), Principal Component Analysis) method to calculate the base vector in this space.The base vector that selection meets the demands is as the representation model of this subspace, part.Select n base vector before the expression and with all base vectors and ratio greater than 99%

Three-dimensional facial reconstruction: the process that just is to use the base vector of this subspace input picture to be carried out reconstruct.To carry out three-dimensional modeling for given facial image, will regulate the combination parameter of base vector exactly, make the three-dimensional face after the reconstruct minimum with the error of input facial image at the projected image of identical viewpoint.If the quadratic sum of gray scale difference of using the image corresponding pixel points, then will be tried to achieve model group as the error of two images and closed parameter and make following formula minimum:

E _I＝∑ _x，y||I _input(x，y)-I _model(x，y)|| ²

I wherein _InputBe given facial image, I _ModelBe that three-dimensional model people face is at the observable facial image of certain viewpoint.

The above; It only is preferred embodiment of the present invention; Be not that the present invention is done any pro forma restriction, every foundation technical spirit of the present invention all still belongs to the protection domain of technical scheme of the present invention to any simple modification, equivalent variations and modification that above embodiment did.

Claims (5)

1. the modeling method of a non-linear three-dimensional human face is characterized in that, may further comprise the steps:

(1) from existing face database, choose three-dimensional samples and two dimensional sample respectively, as training sample set, and to these samples enforcement normalization operations; In the training stage, two-dimentional training sample set and three-dimensional training sample set are matched according to identity information, make that the sample in these two groups of sample sets is corresponding each other according to identity information;

(2) be grounding projection matrix separately with these two groups corresponding sample sets, make to have maximum correlativity after the sample projection of different dimensions;

(3) at phase of regeneration; Two-dimension human face image for input; At first, carry out projection in the subspace of two-dimentional training sample set then, choose the three-dimensional samples high with its degree of correlation according to the correlativity distance to its processing of standardizing; Make up the three-dimensional face deformation model based on these selected three-dimensional samples, and itself and input picture mated realize the reconstruction of three-dimensional face sample.

2. method according to claim 1 is characterized in that, step (1) comprises step by step following:

(1.1) from existing face database, choose three-dimensional samples collection and two dimensional sample collection;

(1.2) three-dimensional samples collection and two dimensional sample collection are corresponding according to identity information;

(1.3) three-dimensional samples and two dimensional sample are carried out normalization operation.

3. method according to claim 2; It is characterized in that; Said step (1.3) comprising: three-dimensional samples is standardized: use alternant way to obtain the position between the three-dimensional samples pupil; Calculating the interpupillary distance of all three-dimensional samples, is that standard is carried out the yardstick alignment to all three-dimensional samples with the mean value of these distances, is that standard is carried out the translation alignment to all three-dimensional samples with the place between the eyebrows point; Two dimensional sample is standardized: use alternant way to obtain the position between the two dimensional sample pupil; Calculate the interpupillary distance of all two dimensional samples; Mean value with these distances is that standard is carried out the yardstick alignment to all two dimensional samples, is that standard is carried out the translation alignment to all two dimensional samples with the place between the eyebrows point.

4. method according to claim 3 is characterized in that, step (2) comprises step by step following:

(2.1) mean vector of calculating two dimensional sample and three-dimensional samples;

(2.2) all samples are deducted each self-corresponding mean vector;

(2.3) with the relevance formula be objective function, its maximizing optimized that when objective function maximizes, obtain two dimensional sample and three-dimensional samples characteristic of correspondence vector, wherein relevance formula is:

$\mathrm{\ρ}=\frac{E\left[W_x^T{\mathrm{XY}}^T{M}_y\right]}{\sqrt{E\left[W_x^T{\mathrm{XY}}^T{W}_x\right]E\left[W_y^T{\mathrm{XY}}^T{W}_y\right]}},$ Wherein X represents the two dimensional sample collection, and Y represents the three-dimensional samples collection, and the projection base vector of X is W _x, the projection base vector of Y is W _y

(2.4) making the vector after the projection have maximum correlativity under the constraint of optimization aim.

5. method according to claim 4 is characterized in that, step (3) comprises step by step following:

(3.1) input two-dimension human face image;

(3.2) this facial image of standardizing;

(3.3) carry out projection in the subspace of two dimensional sample;

(3.4) correlativity of the projection of calculation procedure (3.3) and three-dimensional samples projection vector;

(3.5) confirm the scope of selected three-dimensional samples according to the variance of all distances;

(3.6) make up deformation model based on selected three-dimensional samples;

(3.7) deformation model and the input picture that make up are mated;

(3.8) use particle cluster algorithm to carry out model optimization, realize the reconstruction of three-dimensional face sample.