CN104574432B - Three-dimensional face reconstruction method and three-dimensional face reconstruction system for automatic multi-view-angle face auto-shooting image - Google Patents
Three-dimensional face reconstruction method and three-dimensional face reconstruction system for automatic multi-view-angle face auto-shooting image Download PDFInfo
- Publication number
- CN104574432B CN104574432B CN201510080860.5A CN201510080860A CN104574432B CN 104574432 B CN104574432 B CN 104574432B CN 201510080860 A CN201510080860 A CN 201510080860A CN 104574432 B CN104574432 B CN 104574432B
- Authority
- CN
- China
- Prior art keywords
- dimensional
- model
- image
- mark point
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/143—Segmentation; Edge detection involving probabilistic approaches, e.g. Markov random field [MRF] modelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Probability & Statistics with Applications (AREA)
- Software Systems (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a three-dimensional face reconstruction method for an automatic multi-view-angle face auto-shooting image. The three-dimensional face reconstruction method comprises the following steps: automatically positioning mark points of a multi-view-angle face image of the same person; establishing a target function according to the positioned mark points and mark points corresponding to a reference face model to solve camera parameters; designing a reconstruction target function; and converting a three-dimensional face reconstruction problem into a multi-label image partitioning problem under a Markov random field, and solving by using a multi-label image partitioning algorithm. The method can be used for reconstructing a thick and precision three-dimensional face model and does not depend on an outer database, so that full-automatic face reconstruction can be realized and manual interaction does not need to be carried out by users.
Description
Technical field
The present invention relates to computer vision field, more particularly to a kind of three-dimensional face of automatic multi-view-angle face auto heterodyne image
Method for reconstructing and system.
Background technology
Human face rebuilding is one of important research direction of three-dimensional reconstruction, is had in fields such as video display, game, three-dimensional face identifications
The prospect of being widely applied, is ground by fields such as computer graphics, computer vision, machine vision, CADs
The attention of the person of studying carefully.From the angle of data acquisition, three-dimensional facial reconstruction is broadly divided into initiative range measurement equipment and imaging and passive imaging sets
It is standby.Initiative range measurement equipment such as laser scanner, can scan and obtain the accurate three-dimensional information of stationary body, but its price is high
Expensive, sweep time is long, sweep limits is limited, is difficult to requirement of real-time application higher；Relative, depth camera can
Real-time Collection dynamic object, but the depth map resolution ratio of its correspondence generation is low, precision is low, noise is big.Imaging and passive imaging equipment makes
With it is most common be video camera, because equipment is simply cheap, and existing a large amount of two-dimension human face images at present, therefore from many
The method for recovering three-dimensional face structure in the two-dimension human face image of visual angle obtains extensive concern.Because facial image texture is sparse, because
This needs to solve ambiguity problem present in Feature Points Matching process.
Document【Y.Lin,G.Medioni,and J.Choi.Accurate 3d face reconstruction from
weakly calibrated wide baseline images with profile contours.In Computer
Vision and Pattern Recognition(CVPR),2010IEEE Conference on,pages 1490–
1497.IEEE,2010.】Under the conditions of proposing a kind of weak demarcation, the various visual angles facial reconstruction method based on baseline wide.The method is defeated
Enter the facial image under five different attitudes (0 degree of positive face, positive and negative 45 degree, positive and negative 90 degree of side faces),.By finding any three phases
Under adjacent visual angle stabilization matching point estimation video camera relative position relation, in conjunction with various visual angles colour consistency, flatness and
Side facial contour information respectively set up the object function based on voxel and solve by level, vertical direction under cylindrical-coordinate system.
But, in practical application, the automatic side profile for obtaining cannot often meet the requirement of reconstruction precision；On the other hand, from experiment
Result sees that the faceform that the method is rebuild exists compared with large deformation under some visual angles.Because facial image textural characteristics are dilute
Dredge, the method for such feature based Point matching will fail when can not find corresponding points.
Document【H.Han and A.K.Jain.3d face texture modeling from uncalibrated
frontal and profile images.In Biometrics:Theory,Applications and Systems
(BTAS),2012 IEEE Fifth International Conference on,pages 223–230.IEEE,2012.】
Propose a kind of three-dimensional facial reconstruction method based on two images (such as 0 degree positive face and 90 degree of side faces).The algorithm is based primarily upon
Three-dimensional deformation model (3DMM), and combine face mark point：Deformation and parametric texture are estimated first with front face mark point；
Further Modifying model is carried out using side mark point.The algorithm also depends on the side face mark point of hand labeled；
Meanwhile, reconstruct the faceform for coming and there is a certain degree of deformation under some visual angles (such as 45 degree).In addition, such is based on
The facial reconstruction method of 3DMM needs to combine a three-dimensional face database for having alignd, and reconstructed results are linear by face database
Superposition is obtained, therefore such method depends on the prior data bank for having alignd, and lacks the ability of description three-dimensional face details.
The deficiency of the existing facial reconstruction method based on multi-view image is mainly：1) due to facial image textural characteristics
Sparse particularity, the method based on traditional characteristic Point matching is not applied in actual applications, it is difficult to use traditional feature based
The method of Point matching obtains dense three-dimensional data；2) cumbersome man-machine interactively is needed；3) exterior three dimensional face number is depended on
The abundant degree of database is depended on according to the accuracy in storehouse, and reconstructed results.
The content of the invention
It is an object of the invention to overcome the above-mentioned deficiency in the presence of prior art, there is provided a kind of automatic multi-view-angle face
The three-dimensional facial reconstruction method and system of auto heterodyne image, it can reconstruct dense, fine three-dimensional face model, while the reconstruction
Method does not rely on external data base, is capable of achieving full-automatic human face rebuilding, does not require that user carries out man-machine interactively.
In order to realize foregoing invention purpose, the technical solution adopted by the present invention is：A kind of automatic multi-view-angle face is autodyned and is schemed
The three-dimensional facial reconstruction method of picture, comprises the following steps：
Step one, the facial image to multiple visual angles of same person are automatically positioned mark point；
The mark point in mark point and referenced human face model that step 2, basis are oriented sets up object function and solves shooting
Machine parameter Pi；Wherein described object function isXi is the mark positioned on i-th facial image Ii
Note point mi={ x1, x2 ..., xn }, Xi for the mark point Mi={ X1, X2 ..., Xn }, mi and Mi on referenced human face model one by one
Correspondence, parameter Pi represents three-dimensional point to the projective transformation matrix of correspondence image, and n is mark point number；
Step 3, sets up and rebuilds object function and optimize, and solving the object function using multi-tag image segmentation algorithm obtains
To three-dimensional face model.
Preferably, the object function after optimization is E=Edata+Ecolor+Esmooth；Wherein, data item isD represents that reference model is transformed into the reference model in two dimensional image space, and X is face mould to be solved
Type, i is two dimensional image each pixel spatially, XiRepresent the corresponding depth of pixel i, D on model to be estimatediExpression refers to mould
The corresponding depth of pixel i in type；Various visual angles solid colour isWherein, (k1,k2)
Represent different visual angles pair, PkThat is corresponding projection matrixes of visual angle k,Represent the three-dimensional point in the corresponding three dimensions of pixel i；
Depth smooth isWherein, N (i) is the neighborhood collection where pixel i, and Xi, Xj are represented respectively
The corresponding depth of pixel i, j.
Preferably, used in the step one based on the method or the method based on local optimum for returning to multiple visual angles
Facial image is automatically positioned mark point, and the mark point is including the interior tail of the eye, nose, the corners of the mouth, profile etc..
Preferably, optimization object function E should meet following condition：A) three-dimensional face model and referenced human face model of output
Approximately；B) colour consistency is met between the three-dimensional face model and the facial image of various visual angles of output；C) the three-dimensional people of output
Meet change in depth flatness in face model local neighborhood.
Preferably, the mark point on the referenced human face model is marked in advance.
The present invention also provides a kind of three-dimensional facial reconstruction system of automatic multi-view-angle face auto heterodyne image, including：
Locating mark points module, the facial image for the multiple visual angles to same person is automatically positioned mark point；
Camera parameters estimation module, for being set up with the mark point on referenced human face model according to the mark point oriented
Object function solves camera parameters P；Wherein described object function isXi is i-th face
On image Ii position mark point mi={ x1, x2 ..., xn }, Xi be referenced human face model on mark point Mi=X1,
X2 ..., Xn }, mi and Mi is corresponded, and parameter Pi represents three-dimensional point to the projective transformation matrix of correspondence image, and n is mark point
Number；
Optimization Solution module, for rebuilding objective function optimization, the target being solved using multi-tag image segmentation algorithm
Function obtains three-dimensional face model.
Preferably, the reconstruction object function is E=Edata+Ecolor+Esmooth；Wherein, data item isD represents that reference model is transformed into the reference model in two dimensional image space, and X is face mould to be solved
Type, i is two dimensional image each pixel spatially, XiRepresent the corresponding depth of pixel i, D on model to be estimatediExpression refers to mould
The corresponding depth of pixel i in type；Various visual angles solid colour isWherein, (k1,k2)
Represent different visual angles pair, PkThat is corresponding projection matrixes of visual angle k,Represent the three-dimensional point in the corresponding three dimensions of pixel i；
Depth smooth isWherein, N (i) is the neighborhood collection where pixel i, and Xi, Xj are represented respectively
The corresponding depth of pixel i, j.
Preferably, the locating mark points module is used based on the method or the method based on local optimum for returning to multiple
The facial image at visual angle is automatically positioned mark point, and the mark point is including the interior tail of the eye, nose, the corners of the mouth, profile etc..
Preferably, the Optimization Solution module optimization object function E should meet following condition：A) the three-dimensional face mould of output
Type is approximate with referenced human face model；B) solid colour is met between the three-dimensional face model and the facial image of various visual angles of output
Property；C) change in depth flatness is met in the three-dimensional face model local neighborhood of output.
Preferably, the mark point on the referenced human face model is marked in advance.
Compared with prior art, beneficial effects of the present invention：
The inventive method estimates relative between different visual angles facial image by introducing single canonical reference faceform
Position relationship and face solution space to be reconstructed, and three-dimensional facial reconstruction problem is converted into markov random file (MRF) framework
In multi-tag image segmentation problem, accurate three-dimensional face model can be reconstructed by the method, base can be further used for
In three-dimensional face identification system.By introducing referenced human face model, can effectively eliminate present in Feature Points Matching process
Ambiguity, while reducing solution space, increasing the calculating performance of algorithm, the reconstructed results efficiently quickly stablized can be reconstructed
Dense fine three-dimensional face model.Furthermore, the method does not rely on the external data base of alignment, and any one refers to face mould
Type may be used to computing, and the recovery of facial detail is obtained by the optimized algorithm of Pixel-level.In addition, the method can realize it is complete from
Dynamic human face rebuilding, does not require that user carries out man-machine interactively, and the three-dimensional face model of reconstruction can rotate to other visual angles.
Brief description of the drawings：
Fig. 1 is the method flow diagram in the embodiment of the present invention 1.
Fig. 2 is the system block diagram in the embodiment of the present invention 2.
Specific embodiment
With reference to specific embodiment, the present invention is described in further detail.But this should not be interpreted as the present invention
The scope of above-mentioned theme is only limitted to following embodiment, and all technologies realized based on present invention belong to model of the invention
Enclose.
Various visual angles facial image three-dimensional reconstruction problem is difficult dense to obtain with the method for traditional feature based Point matching
Three-dimensional data, this openness is determined by the texture of two-dimension human face image essence.How the problem to be solved in the present invention is
Facial image is not demarcated from several obtain dense full face three-dimensional face model.Inventor's research finds the face figure of different visual angles
As relative position relation is to determine between (coming from same person), and difference shows the nuance on local geometric.Cause
This, any face can be obtained from certain with reference to face conversion.The present invention is estimated by introducing a canonical reference faceform
Relative position relation and face solution space to be reconstructed between meter different visual angles facial image, and three-dimensional facial reconstruction problem is turned
The multi-tag image segmentation problem in markov random file (MRF) framework is turned to, corresponding function includes data item, various visual angles
Solid colour and depth smooth.The inventive method can reconstruct accurate three-dimensional face model, can be further used for
Based on three-dimensional face identification system.Illustrate below in conjunction with the accompanying drawings.
The three-dimensional facial reconstruction method flow chart of automatic multi-view-angle face auto heterodyne image as shown in Figure 1, including following step
Suddenly：
Step one, the facial image to multiple visual angles of same person are automatically positioned mark point.
Specifically, using based on method (such as Gauss-Newton deformation model method Gauss-Newton Deformable for returning
Part Model etc.) or method (such as restricted partial model Constrained Local Model) based on local optimum
Facial image to multiple visual angles is automatically positioned mark point, the mark point including the interior tail of the eye, nose, the corners of the mouth, profile etc.,
Can be other conventional mark points.
The mark point in mark point and referenced human face model that step 2, basis are oriented sets up object function and solves shooting
Machine parameter P；Wherein described object function isXi is the mark positioned on i-th facial image Ii
Note point mi={ x1, x2 ..., xn }, Xi for the mark point Mi={ X1, X2 ..., Xn }, mi and Mi on referenced human face model one by one
Correspondence, parameter Pi represents three-dimensional point to the projective transformation matrix of correspondence image, and n is mark point number.
For the mark point mi={ x1, x2 ..., xn } of the facial image Ii positioning of i-th input, wherein n is mark point
Number.Assuming that the mark point mi={ x1, x2 ..., xn } of positioning with reference to the mark point Mi on face (coming from same people) model
={ X1, X2 ..., Xn } is corresponded, and is solved by following energy function and is estimated correspondence camera parameters P：
Parameter P={ Pi } to be estimated represents three-dimensional point to the projective transformation of correspondence image, can be by based on least square
Levenberg-Marquardt algorithm optimizations obtain.Mark point on the referenced human face model is marked in advance.Estimate
Parameter P is to carry out camera calibration, and it is the important step of three-dimensional reconstruction, many using based on mark in early stage three-dimensional reconstruction
The camera marking method of earnest carries out European reconstruction, with going deep into for research, is rebuild including the Modern three-dimensional including the present invention
Self-calibration is carried out by calculating the intrinsic parameter of video camera more than technology, these are existing mature technology, no longer describe in detail.If not complete
Self-calibration into video camera cannot carry out three-dimensional reconstruction, behind can use estimation in energy function (optimize after function E)
Camera parameters.Solved function E=Edata+Ecolor+EsmoothSo as to recover three-dimensional face model.
Step 3, to rebuilding objective function optimization, three-dimensional facial reconstruction problem is converted under markov random file
Multi-tag image segmentation problem, solves the object function and obtains three-dimensional face model using multi-tag image segmentation algorithm.Many marks
Sign image segmentation algorithm mature, including alpha-expansion, alpha-swap, BP algorithm etc..It should be noted that
Above-mentioned each specific algorithm used in the present invention is existing mature technology.
Object function after optimization is E=Edata+Ecolor+Esmooth；Data item isD represents ginseng
Examine model conversion to the reference model in two dimensional image space, X is faceform to be solved, i is two dimensional image spatially every
Individual pixel, XiRepresent the corresponding depth of pixel i, D on model to be estimatediRepresent the corresponding depth of pixel i on reference model；
Various visual angles solid colour isWherein, (k1,k2) represent different visual angles
It is right, PkI.e. the corresponding projection matrixes of visual angle k, correspond to parameter Pi,Represent the three-dimensional point in the corresponding three dimensions of pixel i；
Depth smooth isWherein, N (i) is the neighborhood collection where pixel i, Xi, Xj
Represent the corresponding depth of pixel i, j.
Optimization object function E should meet following condition in the present invention：A) three-dimensional face model of output is near with reference model
Seemingly；B) colour consistency is met between the three-dimensional face model and the facial image of various visual angles of output；C) three-dimensional face of output
Change in depth flatness is met in model local neighborhood.
Specifically, be to determine due to relative position relation between different facial images, and difference shows local
Difference geometrically.The faceform for being needed can be converted from any face.The present invention is by the change between different faces
The process of changing is converted to the multi-tag image segmentation problem under markov random file.Based on this, correspondence majorized function E should meet with
Lower condition：1. output three-dimensional face model is approximate with referenced human face model；2. the face of three-dimensional face model and various visual angles is exported
Colour consistency is met between image；3. change in depth flatness is met in output three-dimensional face model local neighborhood.Finally make
Object function reconstruction is solved with multi-tag image segmentation algorithm obtain three-dimensional face model.
Because referenced human face model is present in three dimensions, it is necessary first to be transformed into two dimensional image space, with convenient
Solved using correspondence multi-tag image segmentation algorithm.The spatial transformation of referenced human face model can be realized by two ways：1. will
Three-dimensional face model projects to cylindrical coordinate system, projection value depth representing, then periphery is launched just to obtain two-dimensional space
Under depth map；2. threedimensional model is projected into a coordinate system for defining, wherein two-dimensional depth figure vertical direction and space
Coordinate system y-axis is identical；Horizontal direction is defined as the angle of current three-dimensional point and z-axis；Respective value is the depth value of current three-dimensional point.
Assuming that S is the three-dimensional solution space being transformed on two dimensional image, correspondence x-y axles are that the image after corresponding conversion is empty
Between, z-axis correspondence depth direction.That is, solution space is in a contiguous range centered on referenced human face model.Separately
On the one hand, S can regard a spatial surrounding box for rectangular-shape as, and wherein plane where front face is x-y direction planes, front
Facial orientation z directions.With the plane cutting parallel to x-y directions it is N etc. by spatial surrounding box according to the difference of modeling accuracy
Part, N is bigger to represent that modeling accuracy is higher.Therefore, the N number of possible depth value of each pixel correspondence, by setting up target letter
Number E is that each pixel solves optimal depth value, then converts it to three dimensions, just obtains final three-dimensional face mould
Type.
The object function is defined as：E=Edata+Ecolor+Esmooth；Wherein,Represent data
, D represents the reference model for being transformed into two dimensional image space, and X is faceform to be solved, i be two dimensional image spatially
Each pixel, then XiRepresent the corresponding depth of pixel i, D on model to be estimatediRepresent the corresponding depths of pixel i on reference model
Degree.It is approximate with referenced human face model that this represents that three-dimensional face model to be optimized should try one's best.
Various visual angles solid colour is defined asWherein, (k1,k2) represent different
Visual angle pair, PkThat is corresponding projection matrixes of visual angle k,Represent the three-dimensional point in the corresponding three dimensions of pixel i.This will
Seek same three-dimensional pointProjected color on different visual angles image should be consistent.
Section 3 is depth smooth, is defined asWherein, where N (i) is pixel i
Neighborhood collection.This requires that the corresponding depth of two neighboring pixel should keep smooth variation on image space.
The present invention is by introducing the relative position between a canonical reference faceform estimation different visual angles facial image
Relation and face solution space to be reconstructed, and three-dimensional facial reconstruction problem is converted into markov random file (MRF) framework
Multi-tag image segmentation problem, corresponding function includes data item, various visual angles solid colour item and depth smooth.The present invention
Method can reconstruct accurate three-dimensional face model, can be further used for based on three-dimensional face identification system.The present invention is logical
Introducing referenced human face model is crossed, ambiguity present in Feature Points Matching process can be effectively eliminated, while reducing solution space, increasing
The calculating performance of computation system, the reconstructed results efficiently quickly stablized can reconstruct dense fine three-dimensional face model.Again
Person, the method does not rely on external data base, and any one referenced human face model may be used to computing, the recovery of facial detail
Obtained by the optimized algorithm of Pixel-level.In addition, the method can realize full-automatic human face rebuilding, do not require that user carries out people's industry and traffic
Mutually, the three-dimensional face model of reconstruction can rotate to other visual angles.
Based on same inventive concept, with reference to Fig. 2, the embodiment of the present invention also provides a kind of automatic multi-view-angle face auto heterodyne image
Three-dimensional facial reconstruction system, including locating mark points module, camera parameters estimation module and Optimization Solution module.
The locating mark points module is used to be automatically positioned mark point to the facial image at multiple visual angles of same person.Institute
State mark point and referenced human face model that camera parameters estimation module is used to be oriented according to the locating mark points module
Mark point set up object function and solve camera parameters Pi；Wherein described object function is
Xi is the mark point mi={ x1, x2 ..., xn } positioned on i-th facial image Ii, and Xi is the mark point on referenced human face model
Mi={ X1, X2 ..., Xn }, mi and Mi are corresponded, and parameter Pi represents three-dimensional point to the projective transformation of correspondence image, and n is mark
Point number；
The Optimization Solution module is used to, to rebuilding objective function optimization, three-dimensional facial reconstruction problem is converted into Ma Erke
Multi-tag image segmentation problem under husband's random field, solves the object function and obtains three-dimensional people using multi-tag image segmentation algorithm
Face model.
The present invention is by introducing the relative position between a canonical reference faceform estimation different visual angles facial image
Relation and face solution space to be reconstructed, and three-dimensional facial reconstruction problem is converted into markov random file (MRF) framework
Multi-tag image segmentation problem, corresponding function includes data item, various visual angles solid colour item and depth smooth.The present invention
Method can reconstruct accurate three-dimensional face model, can be further used for based on three-dimensional face identification system.The present invention is logical
Introducing referenced human face model is crossed, ambiguity present in Feature Points Matching process can be effectively eliminated, while reducing solution space, increasing
The calculating performance of computation system, the reconstructed results efficiently quickly stablized can reconstruct dense fine three-dimensional face model.Again
Person, the method does not rely on external data base, and any one referenced human face model may be used to computing, the recovery of facial detail
Obtained by the optimized algorithm of Pixel-level.In addition, the method can realize full-automatic human face rebuilding, do not require that user carries out people's industry and traffic
Mutually, the three-dimensional face model of reconstruction can rotate to other visual angles.
Specifically, the mark point on the referenced human face model is marked in advance.Object function after optimization is E=
Edata+Ecolor+Esmooth；Wherein, data item isD represents that referenced human face model is transformed into two dimensional image
The reference model in space, X is faceform to be solved, and i is two dimensional image each pixel spatially, XiRepresent mould to be estimated
The corresponding depth of pixel i, D in typeiRepresent the corresponding depth of pixel i on referenced human face model；Various visual angles solid colour isWherein, (k1,k2) represent different visual angles pair, PkThat is the corresponding projection squares of visual angle k
Battle array,Represent the three-dimensional point in the corresponding three dimensions of pixel i；Depth smooth isIts
In, N (i) is the neighborhood collection where pixel i, and Xi, Xj represent the corresponding depth of pixel i, j.
The locating mark points module is using the method based on recurrence or the method based on local optimum to multiple visual angles
Facial image is automatically positioned mark point, and the mark point is including the interior tail of the eye, nose, the corners of the mouth, profile etc..
The Optimization Solution module optimization object function E should meet following condition：A) three-dimensional face model of output and ginseng
Examine model approximate；B) colour consistency is met between the three-dimensional face model and the facial image of various visual angles of output；C) export
Change in depth flatness is met in three-dimensional face model local neighborhood.The embodiment is based on same with embodiment of the method shown in Fig. 1
Design, something in common refer to the corresponding description in preceding method embodiment, no longer describe in detail herein.
The present invention is proposed a kind of based on single reference face mould for the deficiency of current various visual angles face image method
The facial reconstruction method of type.(wherein mark point has been for various visual angles facial image and referenced human face model for the object of the method input
Know), it is output as the corresponding three-dimensional face model of facial image of input.The inventive method, can by introducing referenced human face model
Effectively to eliminate ambiguity present in Feature Points Matching process, while reducing solution space, increasing the calculating performance of algorithm, efficiently
The quick reconstructed results stablized；Furthermore, the method does not rely on external data base, and any one can be used with reference to face
In computing, the recovery of facial detail is obtained by the optimized algorithm of Pixel-level；In addition, the method can realize full-automatic face weight
Build, do not require that user carries out man-machine interactively.The inventive method can reconstruct dense, fine three-dimensional face model；Three for rebuilding
Dimension module can rotate to other visual angles, can be used for face identification system.
Specific embodiment of the invention has been described in detail above in conjunction with accompanying drawing, but the present invention is not restricted to
Implementation method is stated, in the case of the spirit and scope for not departing from claims hereof, those skilled in the art can make
Go out various modifications or remodeling.
Claims (8)
1. a kind of three-dimensional facial reconstruction method of automatic multi-view-angle face auto heterodyne image, it is characterised in that comprise the following steps：
Step one, the facial image to multiple visual angles of same person carry out automatic locating mark points；
The mark point in mark point and referenced human face model that step 2, basis are oriented sets up object function and solves video camera ginseng
Number P；Wherein described object function isxiIt is i-th facial image IiThe mark point m of upper positioningi
={ x1,x2,…,xn, XiIt is the mark point M on referenced human face modeli={ X1,X2,…,Xn, miWith MiCorrespond, parameter Pi
Represent three-dimensional point to image IiProjective transformation matrix, n be mark point number；
Step 3, sets up and rebuilds object function E=Edata+Ecolor+Esmooth, to rebuilding objective function optimization, use multi-tag figure
Three-dimensional face model is obtained as partitioning algorithm solves the reconstruction object function；
Rebuild object function and be defined as E=Edata+Ecolor+Esmooth；
Wherein, data item isI is two dimensional image each pixel spatially,Represent mould to be estimated
The corresponding depth of pixel i, D in typeiRepresent the corresponding depth of pixel i on reference model；
Various visual angles solid colour is
Wherein, (k1,k2) represent different visual angles pair, PkThat is the corresponding projections of visual angle k
Matrix,Represent the three-dimensional point in the corresponding three dimensions of pixel i on model to be estimated；
Depth smooth isWherein, N (i) is the neighbour where pixel i on model to be estimated
Domain collection,Pixel i on model to be estimated, the corresponding depth of j are represented respectively.
2. the three-dimensional facial reconstruction method of automatic multi-view-angle face auto heterodyne image according to claim 1, it is characterised in that
Using automatically fixed to the facial image at each visual angle based on the method or the method based on local optimum for returning in the step one
Position mark point, the mark point includes the interior tail of the eye, nose, the corners of the mouth, profile.
3. the three-dimensional facial reconstruction method of automatic multi-view-angle face auto heterodyne image according to claim 2, it is characterised in that
Optimization object function E should meet following condition：A) three-dimensional face model of output is approximate with referenced human face model；B) the three of output
Meet colour consistency between dimension faceform and the facial image of various visual angles；C) in the three-dimensional face model local neighborhood of output
Meet change in depth flatness.
4. the three-dimensional facial reconstruction method of automatic multi-view-angle face auto heterodyne image according to claim 2, it is characterised in that
Mark point on the referenced human face model is marked in advance.
5. a kind of three-dimensional facial reconstruction system of automatic multi-view-angle face auto heterodyne image, it is characterised in that including：
Locating mark points module, the facial image for the multiple visual angles to same person is automatically positioned mark point；
Camera parameters estimation module, for setting up target with the mark point on referenced human face model according to the mark point oriented
Function solves camera parameters P；Wherein described object function isxiIt is i-th facial image
IiThe mark point m of upper positioningi={ x1,x2,…,xn, XiIt is the mark point M on referenced human face modeli={ X1,X2,…,Xn, mi
With MiCorrespond, parameter PiRepresent three-dimensional point to image IiProjective transformation matrix, n be mark point number；
Optimization Solution module, for rebuilding object function E=Edata+Ecolor+EsmoothOptimization, uses multi-tag image segmentation
The Algorithm for Solving reconstruction object function obtains three-dimensional face model；
Reconstruction object function is E=Edata+Ecolor+Esmooth；
Wherein, data item isI is two dimensional image each pixel spatially,Represent mould to be estimated
The corresponding depth of pixel i, D in typeiRepresent the corresponding depth of pixel i on reference model；
Various visual angles solid colour is
Wherein, (k1,k2) represent different visual angles pair, PkThat is the corresponding projections of visual angle k
Matrix,Represent the three-dimensional point in the corresponding three dimensions of pixel i on model to be estimated；
Depth smooth isWherein, N (i) is the neighbour where pixel i on model to be estimated
Domain collection,Pixel i on model to be estimated, the corresponding depth of j are represented respectively.
6. the three-dimensional facial reconstruction system of automatic multi-view-angle face auto heterodyne image according to claim 5, it is characterised in that
The locating mark points module is using the facial image of method or the method based on local optimum to each visual angle for being based on recurrence
Mark point is automatically positioned, the mark point includes the interior tail of the eye, nose, the corners of the mouth, profile.
7. the three-dimensional facial reconstruction system of automatic multi-view-angle face auto heterodyne image according to claim 6, it is characterised in that
The Optimization Solution module optimization object function E should meet following condition：A) three-dimensional face model of output and reference face mould
Type is approximate；B) colour consistency is met between the three-dimensional face model and the facial image of various visual angles of output；C) three-dimensional of output
Change in depth flatness is met in faceform's local neighborhood.
8. the three-dimensional facial reconstruction system of automatic multi-view-angle face auto heterodyne image according to claim 6, it is characterised in that
Mark point on the referenced human face model is marked in advance.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510080860.5A CN104574432B (en) | 2015-02-15 | 2015-02-15 | Three-dimensional face reconstruction method and three-dimensional face reconstruction system for automatic multi-view-angle face auto-shooting image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510080860.5A CN104574432B (en) | 2015-02-15 | 2015-02-15 | Three-dimensional face reconstruction method and three-dimensional face reconstruction system for automatic multi-view-angle face auto-shooting image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104574432A CN104574432A (en) | 2015-04-29 |
| CN104574432B true CN104574432B (en) | 2017-05-24 |
Family
ID=53090404
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510080860.5A Active CN104574432B (en) | 2015-02-15 | 2015-02-15 | Three-dimensional face reconstruction method and three-dimensional face reconstruction system for automatic multi-view-angle face auto-shooting image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104574432B (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105761243A (en) * | 2016-01-28 | 2016-07-13 | 四川川大智胜软件股份有限公司 | Three-dimensional full face photographing system based on structured light projection and photographing method thereof |
| CN105809734B (en) * | 2016-03-10 | 2018-12-25 | 杭州师范大学 | A kind of mechanical model three-dimensional modeling method based on multi-angle of view interactive mode |
| CN107194964B (en) * | 2017-05-24 | 2020-10-09 | 电子科技大学 | VR social contact system based on real-time human body three-dimensional reconstruction and method thereof |
| CN107730519A (en) * | 2017-09-11 | 2018-02-23 | 广东技术师范学院 | A kind of method and system of face two dimensional image to face three-dimensional reconstruction |
| CN109961477A (en) * | 2017-12-25 | 2019-07-02 | 深圳超多维科技有限公司 | A kind of space-location method, device and equipment |
| CN108596965B (en) * | 2018-03-16 | 2021-06-04 | 天津大学 | Light field image depth estimation method |
| CN108510583B (en) * | 2018-04-03 | 2019-10-11 | 北京华捷艾米科技有限公司 | The generation method of facial image and the generating means of facial image |
| CN110378994B (en) * | 2018-04-12 | 2021-05-28 | Oppo广东移动通信有限公司 | Face modeling method and related product |
| CN109523633B (en) * | 2018-09-30 | 2023-06-02 | 先临三维科技股份有限公司 | Model scanning method, device, equipment, storage medium and processor |
| CN109887076B (en) * | 2019-02-25 | 2021-02-12 | 清华大学 | Method and device for establishing three-dimensional model of human face according to visual angle change |
| CN109978989B (en) * | 2019-02-26 | 2023-08-01 | 腾讯科技（深圳）有限公司 | Three-dimensional face model generation method, three-dimensional face model generation device, computer equipment and storage medium |
| CN117974867B (en) * | 2024-04-01 | 2024-06-21 | 哈尔滨工业大学（威海） | Monocular face avatar generation method based on Gaussian point rendering |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101593365A (en) * | 2009-06-19 | 2009-12-02 | 电子科技大学 | A kind of method of adjustment of universal three-dimensional human face model |
| CN102609977A (en) * | 2012-01-12 | 2012-07-25 | 浙江大学 | Depth integration and curved-surface evolution based multi-viewpoint three-dimensional reconstruction method |
| CN103065289A (en) * | 2013-01-22 | 2013-04-24 | 清华大学 | Four-ocular video camera front face reconstruction method based on binocular stereo vision |
| CN103198523A (en) * | 2013-04-26 | 2013-07-10 | 清华大学 | Three-dimensional non-rigid body reconstruction method and system based on multiple depth maps |
| CN103914874A (en) * | 2014-04-08 | 2014-07-09 | 中山大学 | Compact SFM three-dimensional reconstruction method without feature extraction |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7184071B2 (en) * | 2002-08-23 | 2007-02-27 | University Of Maryland | Method of three-dimensional object reconstruction from a video sequence using a generic model |
-
2015
- 2015-02-15 CN CN201510080860.5A patent/CN104574432B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101593365A (en) * | 2009-06-19 | 2009-12-02 | 电子科技大学 | A kind of method of adjustment of universal three-dimensional human face model |
| CN102609977A (en) * | 2012-01-12 | 2012-07-25 | 浙江大学 | Depth integration and curved-surface evolution based multi-viewpoint three-dimensional reconstruction method |
| CN103065289A (en) * | 2013-01-22 | 2013-04-24 | 清华大学 | Four-ocular video camera front face reconstruction method based on binocular stereo vision |
| CN103198523A (en) * | 2013-04-26 | 2013-07-10 | 清华大学 | Three-dimensional non-rigid body reconstruction method and system based on multiple depth maps |
| CN103914874A (en) * | 2014-04-08 | 2014-07-09 | 中山大学 | Compact SFM three-dimensional reconstruction method without feature extraction |
Non-Patent Citations (3)
| Title |
|---|
| Dense Variational Reconstruction of Non-Rigid Surfaces from Monocular Video;Ravi Garg et al;《2013 IEEE Conference on Computer Vision and pattern Recognition》;20130630;第1272-1279页 * |
| 从多张非标定图像重建三维人脸;丁滨等;《计算机辅助设计与图形学学报》;20100228;第22卷(第2期);第211页左栏第15-17行，右栏第9-14行 * |
| 改进的三维人脸稠密对齐方法;陆涛等;《电子科技》;20120731;第25卷(第7期);第15-17页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104574432A (en) | 2015-04-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104574432B (en) | Three-dimensional face reconstruction method and three-dimensional face reconstruction system for automatic multi-view-angle face auto-shooting image | |
| CN109272537B (en) | Panoramic point cloud registration method based on structured light | |
| CN103106688B (en) | Based on the indoor method for reconstructing three-dimensional scene of double-deck method for registering | |
| CN107679537B (en) | A kind of texture-free spatial target posture algorithm for estimating based on profile point ORB characteristic matching | |
| Chauve et al. | Robust piecewise-planar 3D reconstruction and completion from large-scale unstructured point data | |
| CN101916454B (en) | Method for reconstructing high-resolution human face based on grid deformation and continuous optimization | |
| CN110728671B (en) | Dense reconstruction method of texture-free scene based on vision | |
| US10846844B1 (en) | Collaborative disparity decomposition | |
| CN109903327A (en) | A kind of object dimension measurement method of sparse cloud | |
| CN112001926B (en) | RGBD multi-camera calibration method, system and application based on multi-dimensional semantic mapping | |
| CN106709947A (en) | RGBD camera-based three-dimensional human body rapid modeling system | |
| CN101339669A (en) | Three-dimensional human face modelling approach based on front side image | |
| CN108898665A (en) | Three-dimensional facial reconstruction method, device, equipment and computer readable storage medium | |
| CN102999942A (en) | Three-dimensional face reconstruction method | |
| CN101404091A (en) | Three-dimensional human face reconstruction method and system based on two-step shape modeling | |
| CN103854301A (en) | 3D reconstruction method of visible shell in complex background | |
| CN105184857A (en) | Scale factor determination method in monocular vision reconstruction based on dot structured optical ranging | |
| CN111462302B (en) | Multi-view human body dynamic three-dimensional reconstruction method and system based on depth coding network | |
| CN109345570B (en) | Multi-channel three-dimensional color point cloud registration method based on geometric shape | |
| Liu et al. | Creating simplified 3D models with high quality textures | |
| CN103945209B (en) | A kind of DIBR method based on piecemeal projection | |
| TWI731604B (en) | Three-dimensional point cloud data processing method | |
| CN111127622B (en) | Three-dimensional point cloud outlier rejection method based on image segmentation | |
| CN116878524A (en) | Dynamic SLAM dense map construction method based on pyramid L-K optical flow and multi-view geometric constraint | |
| Neverova et al. | 2 1/2 D scene reconstruction of indoor scenes from single RGB-D images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |