CN102592309A - Modeling method of nonlinear three-dimensional face - Google Patents
Modeling method of nonlinear three-dimensional face Download PDFInfo
- Publication number
- CN102592309A CN102592309A CN2011104402240A CN201110440224A CN102592309A CN 102592309 A CN102592309 A CN 102592309A CN 2011104402240 A CN2011104402240 A CN 2011104402240A CN 201110440224 A CN201110440224 A CN 201110440224A CN 102592309 A CN102592309 A CN 102592309A
- Authority
- CN
- China
- Prior art keywords
- dimensional
- samples
- sample
- projection
- dimensional samples
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Processing Or Creating Images (AREA)
Abstract
The invention relates to a modeling method of a nonlinear three-dimensional face, which has better reconstruction effect, and comprises the following steps: (1) respectively selecting three-dimensional samples and two-dimensional samples from the exiting face database as training sample sets and carrying out the standardized operation on the samples; pairing the two-dimensional training sample set and the three-dimensional training sample set during the training stage according to the identity information, so that the samples in the two groups of sample sets mutually correspond according to the identity information; (2) training respective projection matrixes based on the two groups of sample sets, so that samples with different dimensionalities have maximum relevance after being projected; (3) during the reconstruction stage, carrying out standardized treatment on an inputted two-dimensional face image, projecting in a subspace of the two-dimensional training sample set, selecting three-dimensional samples with high relevance according to the relevance distance, constructing a three-dimensional face model based on the selected three-dimensional samples and matching the three-dimensional face model with the inputted image to realize the reconstruction of a three-dimensional face sample.
Description
Technical field
The invention belongs to the technical field of Flame Image Process, relate to a kind of modeling method of non-linear three-dimensional human face particularly.
Background technology
Three-dimensional facial reconstruction is the work of rebuilding its three-dimensional face data according to the input two-dimension human face image.Because three-dimensional face model has wide application prospect in fields such as computer game, man-machine interactions, thus three-dimensional facial reconstruction to have become be the most active research focus of computer graphics, computer vision.Since the Parke method representation people face that used a computer first in 1972, the modeling of people's face has just obtained paying close attention to widely.Through 30 years of development, the three-dimensional face modeling method has had significant progress.The human face model building that people such as Vetter proposed based on deformation model in 1999, this method have been tested the full automation of people's face modeling for the first time, and can be rebuild persona certa's three-dimensional face model by a width of cloth facial image.Because this model is to be based upon on the corresponding three-dimensional prototype people face data of Pixel-level, and has considered factors such as human face posture, illumination in the model, so this model can generate the three-dimensional face of the height sense of reality.The basis of deformation model is the thought of linear combination, and promptly a class object can be represented with the linear combination of this class object base vector.Modeling method based on deformation model is to realize the coupling between deformation model and the input picture through the combination parameter of regulating model.And people's face is to be nested in the central non-linearity manifold of higher dimensional space.Must ignore the details of human face structure based on the reconstruction algorithm of linear theory, be difficult to reach and rebuild effect preferably.
Summary of the invention
Technology of the present invention is dealt with problems and is: overcome the deficiency of prior art, a kind of modeling method of rebuilding the non-linear three-dimensional human face of better effects if is provided.
Technical solution of the present invention is: the modeling method of this non-linear three-dimensional human face; May further comprise the steps: (1) is chosen three-dimensional samples and two dimensional sample respectively from existing face database; As training sample set, and to these samples enforcement normalization operations; In the training stage, two-dimentional training sample set and three-dimensional training sample set are matched according to identity information, make that the sample in these two groups of sample sets is corresponding each other according to identity information; (2) be grounding projection matrix separately with these two groups corresponding sample sets, make to have maximum correlativity after the sample projection of different dimensions; (3) at phase of regeneration; Two-dimension human face image for input; At first, carry out projection in the subspace of two-dimentional training sample set then, choose the three-dimensional samples high with its degree of correlation according to the correlativity distance to its processing of standardizing; Make up the three-dimensional face deformation model based on these selected three-dimensional samples, and itself and input picture mated realize the reconstruction of three-dimensional face sample.
Because this method is based on canonical correlation analysis and deformation model, so rebuild better effects if.
Embodiment
The modeling method of this non-linear three-dimensional human face may further comprise the steps: (1) is chosen three-dimensional samples and two dimensional sample respectively from existing face database, as training sample set; In the training stage, two-dimentional training sample set and three-dimensional training sample set are matched according to identity information, make that the sample in these two groups of sample sets is corresponding each other according to identity information; (2) be grounding projection matrix separately with these two groups corresponding sample sets, make to have maximum correlativity after the sample projection of different dimensions; (3) at phase of regeneration, at first with the processing (it is carried out yardstick alignment according to the distance between eyes, carry out the displacement alignment according to the position of prenasale, and specification turns to the sample of 192*192 size) of standardizing of the two dimensional sample of two-dimentional training sample set; Two-dimension human face image for input; At first with its processing of standardizing; Carry out projection in the subspace of normalized two-dimentional training sample set then; Choose the three-dimensional samples high according to correlativity distance, make up the three-dimensional face deformation model, and itself and input picture mated realize the reconstruction of three-dimensional face sample based on these selected three-dimensional samples with its degree of correlation.
Preferably, step (1) comprises step by step following: three-dimensional samples collection and two dimensional sample collection are chosen in (1.1) from existing face database; (1.2) three-dimensional samples collection and two dimensional sample collection are corresponding according to identity information; (1.3) three-dimensional samples and two dimensional sample are standardized.
Further; Said step (1.3) comprising: three-dimensional samples is standardized: use alternant way to obtain the position between the three-dimensional samples pupil; Calculate the interpupillary distance of all three-dimensional samples; Mean value with these distances is that standard is carried out dimensional variation to all three-dimensional samples, is that standard is carried out the translation alignment to all three-dimensional samples with the place between the eyebrows point; Two dimensional sample is standardized: use alternant way to obtain the position between the two dimensional sample pupil; Calculate the interpupillary distance of all two dimensional samples; Mean value with these distances is that standard is carried out dimensional variation to all two dimensional samples, is that standard is carried out the translation alignment to all two dimensional samples with the place between the eyebrows point.
Preferably, step (2) comprises step by step following:
(2.1) mean vector of calculating two dimensional sample and three-dimensional samples;
(2.2) all samples are deducted each self-corresponding mean vector;
(2.3) with the relevance formula be objective function, its maximizing optimized that when objective function maximizes, obtain two dimensional sample and three-dimensional samples characteristic of correspondence vector, wherein relevance formula is:
(2.4) common dimension based on the two makes up projection subspace separately, is making the vector after the projection have maximum correlativity under the constraint of optimization aim.
In training process, at first these two groups of samples are mapped according to identity information one by one, so just formed the training set of a coupling.Can be calculated at phase of regeneration and the maximally related three-dimensional face sample set of input picture, according to the mapping relations of the 2D-3D that learns.Make X represent two dimensional sample, Y represents three-dimensional samples.
Canonical correlation analysis is a kind of statistical method of correlationship between two groups of variablees of research, and the problem that it solved is how to seek two groups of corresponding base vectors, makes the correlativity of these variablees between the projection on the corresponding base vector by maximization simultaneously.This paper uses this method to set up the relation between two dimensional image and the three-dimensional samples, and we can obtain the neighbor relationships between two dimensional sample and the three-dimensional samples based on this formula.Make that (X Y) is the variable set of two correspondences.Wherein X represents the two dimensional sample collection, and Y represents the three-dimensional samples collection.The projection base vector of supposing X is W
x, the projection base vector of Y is W
yX and Y are at base vector W
xAnd W
yOn linear projection do
With
According to the definition of relative coefficient, we can learn and want
With
Correlativity by computes:
That is:
Definition C according to the covariance formula
Xx=E [XX
T], C
Yy=E [YY
T], C
Xy=E [XY
T], following formula can be rewritten as
W
xAnd W
yFind the solution and can draw through the maximization following formula, just obtain through finding the solution following eigenvalue problem.
In training refers to, at first original sample is deducted average.
When calculating and input picture X
InDuring the most close three-dimensional samples, we calculate with
The three-dimensional samples Y that correlativity is maximum
C, the sample calculation formula does
The three-dimensional samples that is chosen should be greater than a certain threshold value with the correlativity of input two dimensional sample, and this threshold value is confirmed through the three-sigma criterion.
Preferably, step (3) comprises step by step following:
(3.1) input two-dimension human face image;
(3.2) this facial image of standardizing;
(3.3) carry out projection in the subspace of two dimensional sample;
(3.4) correlativity of the projection of calculation procedure (3.3) and three-dimensional samples projection vector;
(3.5) confirm the scope of selected three-dimensional samples according to the variance of all distances;
(3.6) make up deformation model based on selected three-dimensional samples;
(3.7) deformation model and the input picture that make up are mated;
(3.8) use particle cluster algorithm to carry out model optimization, realize the reconstruction of three-dimensional face sample.
The alignment of input picture:
Make by hand the pupil position of the method mark of demarcating, calculate the distance between the pupil, then according to interpupillary distance to input picture processings of standardizing, carry out translation according to the place between the eyebrows position then and align.
Three-dimensional samples is selected:
The purpose that sample is selected is to select and the representation space of the high sample of input picture correlativity as input picture.We use the distance after fitness tolerance two dimensional sample and the three-dimensional samples projection.
Based on the subspace of selected these samples of three-dimensional face sample structure, use PCA (principal component analysis (PCA), Principal Component Analysis) method to calculate the base vector in this space.The base vector that selection meets the demands is as the representation model of this subspace, part.Select n base vector before the expression and with all base vectors and ratio greater than 99%
Three-dimensional facial reconstruction: the process that just is to use the base vector of this subspace input picture to be carried out reconstruct.To carry out three-dimensional modeling for given facial image, will regulate the combination parameter of base vector exactly, make the three-dimensional face after the reconstruct minimum with the error of input facial image at the projected image of identical viewpoint.If the quadratic sum of gray scale difference of using the image corresponding pixel points, then will be tried to achieve model group as the error of two images and closed parameter and make following formula minimum:
E
I=∑
x,y||I
input(x,y)-I
model(x,y)||
2
I wherein
InputBe given facial image, I
ModelBe that three-dimensional model people face is at the observable facial image of certain viewpoint.
The above; It only is preferred embodiment of the present invention; Be not that the present invention is done any pro forma restriction, every foundation technical spirit of the present invention all still belongs to the protection domain of technical scheme of the present invention to any simple modification, equivalent variations and modification that above embodiment did.
Claims (5)
1. the modeling method of a non-linear three-dimensional human face is characterized in that, may further comprise the steps:
(1) from existing face database, choose three-dimensional samples and two dimensional sample respectively, as training sample set, and to these samples enforcement normalization operations; In the training stage, two-dimentional training sample set and three-dimensional training sample set are matched according to identity information, make that the sample in these two groups of sample sets is corresponding each other according to identity information;
(2) be grounding projection matrix separately with these two groups corresponding sample sets, make to have maximum correlativity after the sample projection of different dimensions;
(3) at phase of regeneration; Two-dimension human face image for input; At first, carry out projection in the subspace of two-dimentional training sample set then, choose the three-dimensional samples high with its degree of correlation according to the correlativity distance to its processing of standardizing; Make up the three-dimensional face deformation model based on these selected three-dimensional samples, and itself and input picture mated realize the reconstruction of three-dimensional face sample.
2. method according to claim 1 is characterized in that, step (1) comprises step by step following:
(1.1) from existing face database, choose three-dimensional samples collection and two dimensional sample collection;
(1.2) three-dimensional samples collection and two dimensional sample collection are corresponding according to identity information;
(1.3) three-dimensional samples and two dimensional sample are carried out normalization operation.
3. method according to claim 2; It is characterized in that; Said step (1.3) comprising: three-dimensional samples is standardized: use alternant way to obtain the position between the three-dimensional samples pupil; Calculating the interpupillary distance of all three-dimensional samples, is that standard is carried out the yardstick alignment to all three-dimensional samples with the mean value of these distances, is that standard is carried out the translation alignment to all three-dimensional samples with the place between the eyebrows point; Two dimensional sample is standardized: use alternant way to obtain the position between the two dimensional sample pupil; Calculate the interpupillary distance of all two dimensional samples; Mean value with these distances is that standard is carried out the yardstick alignment to all two dimensional samples, is that standard is carried out the translation alignment to all two dimensional samples with the place between the eyebrows point.
4. method according to claim 3 is characterized in that, step (2) comprises step by step following:
(2.1) mean vector of calculating two dimensional sample and three-dimensional samples;
(2.2) all samples are deducted each self-corresponding mean vector;
(2.3) with the relevance formula be objective function, its maximizing optimized that when objective function maximizes, obtain two dimensional sample and three-dimensional samples characteristic of correspondence vector, wherein relevance formula is:
(2.4) making the vector after the projection have maximum correlativity under the constraint of optimization aim.
5. method according to claim 4 is characterized in that, step (3) comprises step by step following:
(3.1) input two-dimension human face image;
(3.2) this facial image of standardizing;
(3.3) carry out projection in the subspace of two dimensional sample;
(3.4) correlativity of the projection of calculation procedure (3.3) and three-dimensional samples projection vector;
(3.5) confirm the scope of selected three-dimensional samples according to the variance of all distances;
(3.6) make up deformation model based on selected three-dimensional samples;
(3.7) deformation model and the input picture that make up are mated;
(3.8) use particle cluster algorithm to carry out model optimization, realize the reconstruction of three-dimensional face sample.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201110440224.0A CN102592309B (en) | 2011-12-26 | 2011-12-26 | Modeling method of nonlinear three-dimensional face |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201110440224.0A CN102592309B (en) | 2011-12-26 | 2011-12-26 | Modeling method of nonlinear three-dimensional face |
Publications (2)
Publication Number | Publication Date |
---|---|
CN102592309A true CN102592309A (en) | 2012-07-18 |
CN102592309B CN102592309B (en) | 2014-05-07 |
Family
ID=46480893
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201110440224.0A Expired - Fee Related CN102592309B (en) | 2011-12-26 | 2011-12-26 | Modeling method of nonlinear three-dimensional face |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN102592309B (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103310440A (en) * | 2013-05-16 | 2013-09-18 | 北京师范大学 | Canonical correlation analysis-based skull identity authentication method |
CN106469465A (en) * | 2016-08-31 | 2017-03-01 | 深圳市唯特视科技有限公司 | A kind of three-dimensional facial reconstruction method based on gray scale and depth information |
CN109377544A (en) * | 2018-11-30 | 2019-02-22 | 腾讯科技(深圳)有限公司 | A kind of face three-dimensional image generating method, device and readable medium |
US11188616B2 (en) | 2020-02-25 | 2021-11-30 | International Business Machines Corporation | Multi-linear dynamical model reduction |
TWI788630B (en) * | 2019-02-26 | 2023-01-01 | 大陸商騰訊科技(深圳)有限公司 | Method, device, computer equipment, and storage medium for generating 3d face model |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1818977A (en) * | 2006-03-16 | 2006-08-16 | 上海交通大学 | Fast human-face model re-construction by one front picture |
WO2008144825A1 (en) * | 2007-06-01 | 2008-12-04 | National Ict Australia Limited | Face recognition |
CN101866497A (en) * | 2010-06-18 | 2010-10-20 | 北京交通大学 | Binocular stereo vision based intelligent three-dimensional human face rebuilding method and system |
CN101877146A (en) * | 2010-07-15 | 2010-11-03 | 北京工业大学 | Method for extending three-dimensional face database |
CN101923721A (en) * | 2010-08-31 | 2010-12-22 | 汉王科技股份有限公司 | Non-illumination face image reconstruction method and system |
-
2011
- 2011-12-26 CN CN201110440224.0A patent/CN102592309B/en not_active Expired - Fee Related
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1818977A (en) * | 2006-03-16 | 2006-08-16 | 上海交通大学 | Fast human-face model re-construction by one front picture |
WO2008144825A1 (en) * | 2007-06-01 | 2008-12-04 | National Ict Australia Limited | Face recognition |
CN101866497A (en) * | 2010-06-18 | 2010-10-20 | 北京交通大学 | Binocular stereo vision based intelligent three-dimensional human face rebuilding method and system |
CN101877146A (en) * | 2010-07-15 | 2010-11-03 | 北京工业大学 | Method for extending three-dimensional face database |
CN101923721A (en) * | 2010-08-31 | 2010-12-22 | 汉王科技股份有限公司 | Non-illumination face image reconstruction method and system |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103310440A (en) * | 2013-05-16 | 2013-09-18 | 北京师范大学 | Canonical correlation analysis-based skull identity authentication method |
CN106469465A (en) * | 2016-08-31 | 2017-03-01 | 深圳市唯特视科技有限公司 | A kind of three-dimensional facial reconstruction method based on gray scale and depth information |
CN109377544A (en) * | 2018-11-30 | 2019-02-22 | 腾讯科技(深圳)有限公司 | A kind of face three-dimensional image generating method, device and readable medium |
CN109377544B (en) * | 2018-11-30 | 2022-12-23 | 腾讯科技(深圳)有限公司 | Human face three-dimensional image generation method and device and readable medium |
TWI788630B (en) * | 2019-02-26 | 2023-01-01 | 大陸商騰訊科技(深圳)有限公司 | Method, device, computer equipment, and storage medium for generating 3d face model |
US11188616B2 (en) | 2020-02-25 | 2021-11-30 | International Business Machines Corporation | Multi-linear dynamical model reduction |
Also Published As
Publication number | Publication date |
---|---|
CN102592309B (en) | 2014-05-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Cherian et al. | Riemannian dictionary learning and sparse coding for positive definite matrices | |
Sharma et al. | Pose invariant virtual classifiers from single training image using novel hybrid-eigenfaces | |
Sun et al. | Depth estimation of face images using the nonlinear least-squares model | |
Zhou et al. | A bayesian mixture model for multi-view face alignment | |
CN102592309B (en) | Modeling method of nonlinear three-dimensional face | |
CN104700076A (en) | Face image virtual sample generating method | |
Sung et al. | Pose-Robust Facial Expression Recognition Using View-Based 2D $+ $ 3D AAM | |
Yin et al. | Sparse representation over discriminative dictionary for stereo matching | |
Lee et al. | A SfM-based 3D face reconstruction method robust to self-occlusion by using a shape conversion matrix | |
CN108564619B (en) | Realistic three-dimensional face reconstruction method based on two photos | |
Efraty et al. | Profile-based 3D-aided face recognition | |
CN107292299A (en) | Side face identification method based on kernel specification correlation analysis | |
CN103745209A (en) | Human face identification method and system | |
Deng et al. | A multi-scale three-dimensional face recognition approach with sparse representation-based classifier and fusion of local covariance descriptors | |
Corman et al. | Continuous matching via vector field flow | |
CN105740838A (en) | Recognition method in allusion to facial images with different dimensions | |
Yang et al. | Robust template matching with large angle localization | |
CN114283265A (en) | Unsupervised face correcting method based on 3D rotation modeling | |
CN103325104A (en) | Facial image super-resolution reestablishing method based on iterative sparse representation | |
CN113469091A (en) | Face recognition method, training method, electronic device and storage medium | |
CN108022308A (en) | A kind of facial alignment schemes based on three-dimensional face model fitting | |
Zhao et al. | Three-dimensional face reconstruction of static images and computer standardization issues | |
Fooprateepsiri et al. | A general framework for face reconstruction using single still image based on 2D-to-3D transformation kernel | |
Li et al. | Learning weighted sparse representation of encoded facial normal information for expression-robust 3D face recognition | |
CN115359508A (en) | Performing complex optimization tasks with increased efficiency by expert neuron optimization |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20140507 Termination date: 20211226 |
|
CF01 | Termination of patent right due to non-payment of annual fee |