Summary of the invention
In order to overcome the deficiency of the facial face monumented point of existing colourful attitude three-dimensional face localization method, the invention provides a kind of facial face monumented point of colourful attitude three-dimensional face localization method of multi-modal information combination to the attitude robust difference.This method adopts multi-modal information in conjunction with carrying out the facial face monumented point of three-dimensional face location, by the concavo-convex clearly demarcated advantage of face mark region in the clear and three-dimensional face model of face structure outline in the two dimensional image, can in advance faceform's attitude be estimated and compensate, thereby attitude is had stronger robustness.
The technical solution adopted for the present invention to solve the technical problems: a kind of facial face monumented point of colourful attitude three-dimensional face localization method of multi-modal information combination is characterized in comprising the steps:
(a) people's face 2 d texture image is carried out feature point detection, to carrying out accurate emulation by head-up position around the image deformation distortion that the camera light direction of principal axis change to produce, the inclination conversion degree is by the t sub sampling of orientation with every width of cloth people face 2 d texture image, emulating image according to
Carry out the φ rotation, adopt standard deviation in the x direction
Gaussian function image is carried out convolution operation, c=0.8; In the formula, φ is the camera longitude angle, and θ is the camera angle of latitude;
The deformation variable is regulated by camera longitude angle φ and camera angle of latitude θ;
Adopt the constant matching process of similarity that all emulating images are compared, extract the most representative face feature description point in the unique point that extracts;
(b) with detected face characteristic point data U={u in people's face 2 d texture image
i∈ R
2: i=1,2 ..., N}, R
2Expression 2-D data space is mapped in the three-dimensional face model according to corresponding relation; In the two dimensional image in the non-face zone eigenwert of detected unique point corresponding point in the three-dimensional face data be 0, reject non-face region point in promptly non-face zone from the three-dimensional face data; To three-dimensional face model data set P{p
i, i=1,2 ..., each the summit p among the N}
i, calculate by maximum curvature
And minimum curvature
The structrual description information crestal line l of the three-dimensional face face concavo-convex variation in zone of expression
RidgeWith valley line l
Valley:
Part sudden change point set as people's face face; In the formula, k
Thresh1, k
Thresh2Be respectively the thresholding curvature of regional area, calculate, obtain the crestal line l of people's face face regional location by genetic algorithm
RidgeWith valley line l
Valley
(c) Step1: respectively to different attitude three-dimensional face data set M
fAnd M
NfCatastrophe point set P
F|mark| fAnd P
G|mark NfIn each monumented point p
I, F|mark f, p
J, F|markn f, set up shape coding, and compare M
fAnd M
NfUnique point to restriction relation
Step2: the point that R is set up calculates P to constraint
F|mark fAnd P
F|mark NfMatching error D (the p of each point
I, F|mark f, p
J, F|mark Nf), count deviation threshold τ;
Step3: remove P
F|mark fAnd P
F|mark NfMiddle error amount is set up new set P greater than the point of deviation threshold τ
F|mark f' and P
F|mark Nf', change Step2, less than given error, iteration finishes as if adjacent twice threshold difference;
Step4:P
F|mark f' and P
F|mark Nf' promptly be the face monumented point in the corresponding human face data model.
The invention has the beneficial effects as follows: owing to adopt multi-modal information in conjunction with carrying out the facial face monumented point of three-dimensional face location, by the concavo-convex clearly demarcated advantage of face mark region in the clear and three-dimensional face model of face structure outline in the two dimensional image, can in advance faceform's attitude be estimated and compensate, thereby attitude there is stronger robustness, the average locating accuracy of face monumented point to the front three-dimensional face model reaches 98.5%, attitude is changed less faceform, and locating accuracy brings up to 91.6% by 88.3% of background technology; For attitude variation faceform greatly, the present invention has significant superiority, and locating accuracy brings up to 73.5% by 57.5% of background technology, is higher than the locating accuracy of background technology far away.
Below in conjunction with embodiment the present invention is elaborated.
Embodiment
The facial face monumented point of the colourful attitude three-dimensional face localization method of the multi-modal information combination of the present invention at first adopts affine constant Affine-SIFT method to carry out feature point detection to the two-dimension human face texture image; Then, utilize mapping relations that detected unique point in the two-dimensional space is projected to three dimensions, each summit to the three-dimensional face model data centralization, the structrual description information crestal line and the valley line of the three-dimensional face face concavo-convex variation in zone that calculating is represented by maximum, minimum curvature are as the part sudden change point set of people's face face; At last, set up different attitude human face characteristic points to restriction relation, carry out matching optimization with least square method, matching result is carried out statistical study, draw a certain deviate, make most points all in this deviate, as the big point of threshold value filtering deviation, set up new set with this deviate, think that the point that is filtered off is non-face monumented point, less than given error amount, the point in set this moment is determined to be people's face face monumented point up to the difference of adjacent twice threshold.Because the present invention adopts the thinking of multi-modal information combination to carry out the facial face monumented point of three-dimensional face location, by the easy concavo-convex clearly demarcated advantage of face mark region in the clear and three-dimensional face model of face structure outline in the two dimensional image, can in advance faceform's attitude be estimated and compensate, thereby attitude there is stronger robustness, has realized location the facial face monumented point of colourful attitude three-dimensional face.
1, based on the feature point detection of 2 d texture image.
People's face can carry out the motion of 6DOF in three dimensions, simultaneously the facial image that different cameral gets access in the practical application is owing to the axial variation of camera light produces image fault, thereby makes the detection of human face characteristic point be faced with bigger difficulty.ASIFT (Affine Scale Invariant Feature Transform) is keeping SIFT method premium properties, as being changed, rotation, scale, translation, brightness maintains the invariance, visual angle change, noise are also kept on the basis of stability to a certain degree affine variation is had stronger insensitivity.The ASIFT method is at first carried out accurate emulation to changing the image fault that produces by the camera light direction of principal axis, then, adopts the step identical with SIFT to carry out feature point detection.ASIFT can carry out emulation to three parameters, and promptly yardstick, camera longitude angle and camera angle of latitude carry out normalization to rotation and translation parameters simultaneously, therefore affined transformation are had unchangeability.
ASIFT method specific implementation step is as follows:
Step1: to carrying out emulation by head-up position around the affine deformation that the camera light direction of principal axis may cause, this deformation is determined by two parameters with every width of cloth figure: camera longitude angle φ and angle of latitude θ.Image is via according to inclination conversion degree correlation parameter
Carry out the φ rotation.For digital picture, the inclination conversion degree is finished by the t sub sampling of an orientation, and this process need at first uses one on the x direction have standard deviation
Gaussian function come image is carried out convolution operation, choose c=0.8.
Step2: these rotations and inclination conversion degree variable are regulated by a series of limited latitudes and longitude angle, and the sampling step assurance emulating image of these parameters and other view similaritys by camera longitude angle φ and angle of latitude θ generation are stronger.
Step3: all emulating images compare with the constant matching process SIFT of similarity.
ASIFT has unchangeability to affined transformation, simultaneously because it has kept the invariant feature of SIFT to yardstick, rotation, translation itself, so ASIFT comparatively the different attitude facial images of the same individuality of robust ground detection are right to a unique point that is complementary.But most of unique points of being extracted by ASIFT do not have practical significance at people's face face aspect describing, and describe monumented point for the needed of paramount importance face of recognition of face, and most unique points are redundancy feature.In order to carry out human face analysis effectively, must from the unique point that extracts, extract the most representative face feature description point.
2, the local sudden change of face is described.
Two-dimensional image data U={u in the face database
i∈ R
2: i=1,2 ..., N} and 3D grid data V={v
j=(x
j, y
j, z
j) ∈ R
3: j=1,2 ..., there are mapping relations in M}, is designated as Ψ: u
i→ v
j,
R
2, R
3Expression two dimension respectively, three-dimensional data space.The unique point of two dimensional image is shone upon to three dimensions, in the two dimensional image in the non-face zone eigenwert of detected unique point corresponding point in the three-dimensional face data be 0, be non-face zone, non-facial zone point can be rejected from the three-dimensional face data automatically thus.
With P{p
i, i=1,2 ..., N} represents the three-dimensional face data acquisition, to each summit p of this data centralization
i, its minimum and maximum curvature is expressed as respectively
With
The important information of curved-surface structure can be described, and the concavo-convex variation that crestal line on the three-dimension curved surface of being calculated by the minimax flexometer and valley line can be described people's face face zone preferably, respectively with l
RidgeAnd l
ValleyExpression, its calculating principle is:
K wherein
Thresh1, k
Thresh2Be respectively the thresholding curvature of regional area, its value can be calculated by genetic algorithm.By the calculating principle of above valley line and crestal line, can obtain the valley line and the crestal line of people's face face regional location respectively.Suppose to be mapped to three-dimensional facial zone feature point set by
Expression is satisfied the point of crestal line and valley line according to formula (1) and (2) calculating, and is kept corresponding crestal line and valley line in this set.Make feature point set P
FInterior crestal line and valley line are labeled as P respectively
F|ridge, P
F|valley, and merge into P with two
F|mark, the part sudden change point set of expression people face face.
3, iterative constrained face monumented point is located.
In order to gather P from the local catastrophe point of face
F|markIn determine final monumented point, need set up the restriction relation between different attitude people's face data centralization face monumented points, and adopt the mode of iteration optimization to revise, reach Optimum Matching.Algorithm thought is: set up different attitude human face characteristic points to restriction relation, carry out matching optimization with least square method then, matching result is carried out statistical study, draw a certain deviate τ, make most points all in this deviate, as the big point of threshold value filtering deviation, set up new set P with this deviate
F|mark', think that the point that is filtered off is non-face monumented point, less than given error amount, will gather P up to the difference of adjacent twice threshold at this moment
F|mark' in point determine to be people's face face monumented point.
Algorithmic procedure is as follows:
Step1: suppose different attitude three-dimensional face data set M
f, respectively to M
fAnd M
NfCatastrophe point set P
F|mark fAnd P
F|mark NfIn each monumented point p
I, F|mark f, p
J, F|mark Nf, set up shape coding, and compare M
fAnd M
NfUnique point to restriction relation
Step2: the point that R is set up calculates P to constraint
F|mark fAnd P
F|mark NfMatching error D (the p of each point
I, F|mark f, p
J, F|mark Nf), statistics draws deviation threshold τ;
Step3: remove P
F|mark fAnd P
F|mark NfMiddle error amount is set up new set P greater than the point of threshold tau
F|mark f' and P
F|mark Nf', change Step2, less than given error, iteration finishes as if adjacent twice threshold difference;
Step4: note P at this moment
F|mark f' and P
F|mark Nf' be the face monumented point in the corresponding human face data model.