CN102081733A - Multi-modal information combined pose-varied three-dimensional human face five-sense organ marking point positioning method - Google Patents

Abstract

The invention discloses a multi-modal information combined pose-varied three-dimensional human face five-sense organ marking point positioning method, which is used for solving the technical problem that the pose robustness of the traditional pose-varied three-dimensional human face five-sense organ marking point positioning method is poor. The invention adopts the technical scheme that: three-dimensional human face five-sense organ marking points are positioned through multi-modal information combination; and by means of the advantages of clear five-sense organ structure outlines in a two-dimensional image and obvious concave and convex five-sense organ marking areas in a three-dimensional human face model, the pose of the human face model is not required to be estimated and compensated previously, so that the robustness to pose is relatively strong. The average positioning accuracy to the five-sense organ marking points of the three-dimensional human face model is up to 98.5 percent; to the human face model with relatively low pose variation, the positioning accuracy is improved to 91.6 percent from 88.3 percent in the background technology; and to the human face model with relatively high pose variation, the method has obvious superiority and the positioning accuracy is improved to 73.5 percent from 57.5 percent in the background technology.

Description

The facial face monumented point of the colourful attitude three-dimensional face localization method of multi-modal information combination

Technical field

The present invention relates to the facial face monumented point of a kind of colourful attitude three-dimensional face localization method, the particularly a kind of facial face monumented point of colourful attitude three-dimensional face localization method of multi-modal information combination.

Background technology

Colourful attitude people's face face monumented point extraction is the gordian technique in the modeling of people's face, expression analysis and the recognition of face, and the facial face feature extraction of precise and high efficiency can lay a good foundation for above-mentioned application.

Document " Guangpeng Zhang; Yunhong Wang.A 3D facial feature point localization method based on statistical shape model.in Proceeding of ICASSP2007, pp.249-252. " discloses the facial face monumented point of a kind of colourful attitude three-dimensional face localization method.This method changes by detecting facial curve form, and sets up face distribution statistics model people's face face monumented point is positioned.At first, adopt curve form index feature that people's face surf zone is carried out coarse segmentation, be partitioned into the face zone; Next adopts facial face statistical shape model that divided area is carried out positioning feature point, satisfy statistical shape model in the face zone and cut apart the face unique point that rule and the nearest point of distance areas central point are final location, thereby realize the extraction of attitude invariant features point.But there is following problem in this method: at first, the curve form index in this method is subjected to the influence of curved surface noise spot bigger, and locating accuracy has only 88.3%; Secondly, adopt the face statistical shape model that attitude is had bigger dependence, though can overcome the influence of part attitude, for attitude variation faceform greatly, locating accuracy only reaches 57.5%.

Summary of the invention

In order to overcome the deficiency of the facial face monumented point of existing colourful attitude three-dimensional face localization method, the invention provides a kind of facial face monumented point of colourful attitude three-dimensional face localization method of multi-modal information combination to the attitude robust difference.This method adopts multi-modal information in conjunction with carrying out the facial face monumented point of three-dimensional face location, by the concavo-convex clearly demarcated advantage of face mark region in the clear and three-dimensional face model of face structure outline in the two dimensional image, can in advance faceform's attitude be estimated and compensate, thereby attitude is had stronger robustness.

The technical solution adopted for the present invention to solve the technical problems: a kind of facial face monumented point of colourful attitude three-dimensional face localization method of multi-modal information combination is characterized in comprising the steps:

(a) people's face 2 d texture image is carried out feature point detection, to carrying out accurate emulation by head-up position around the image deformation distortion that the camera light direction of principal axis change to produce, the inclination conversion degree is by the t sub sampling of orientation with every width of cloth people face 2 d texture image, emulating image according to

Carry out the φ rotation, adopt standard deviation in the x direction

Gaussian function image is carried out convolution operation, c=0.8; In the formula, φ is the camera longitude angle, and θ is the camera angle of latitude;

The deformation variable is regulated by camera longitude angle φ and camera angle of latitude θ;

Adopt the constant matching process of similarity that all emulating images are compared, extract the most representative face feature description point in the unique point that extracts;

(b) with detected face characteristic point data U={u in people's face 2 d texture image _i∈ R ²: i=1,2 ..., N}, R ²Expression 2-D data space is mapped in the three-dimensional face model according to corresponding relation; In the two dimensional image in the non-face zone eigenwert of detected unique point corresponding point in the three-dimensional face data be 0, reject non-face region point in promptly non-face zone from the three-dimensional face data; To three-dimensional face model data set P{p _i, i=1,2 ..., each the summit p among the N} _i, calculate by maximum curvature

And minimum curvature

The structrual description information crestal line l of the three-dimensional face face concavo-convex variation in zone of expression _RidgeWith valley line l _Valley:

$l_{\mathrm{ridge}}=\{p_i,\mathrm{if}(k_{1_{\mathrm{pi}}}>k_{\mathrm{thresh}1})|p_i\∈P\}---\left(1\right)$

$l_{\mathrm{valley}}=\{p_i,\mathrm{if}(k_{2_{\mathrm{pi}}}>k_{\mathrm{thresh}2})|p_i\∈P\}---\left(2\right)$

Part sudden change point set as people's face face; In the formula, k _Thresh1, k _Thresh2Be respectively the thresholding curvature of regional area, calculate, obtain the crestal line l of people's face face regional location by genetic algorithm _RidgeWith valley line l _Valley

(c) Step1: respectively to different attitude three-dimensional face data set M _fAnd M _NfCatastrophe point set P _F|mark| ^fAnd P _G|mark ^NfIn each monumented point p _{I, F|mark} ^f, p _{J, F|markn} ^f, set up shape coding, and compare M _fAnd M _NfUnique point to restriction relation

Step2: the point that R is set up calculates P to constraint _F|mark ^fAnd P _F|mark ^NfMatching error D (the p of each point _{I, F|mark} ^f, p _{J, F|mark} ^Nf), count deviation threshold τ;

Step3: remove P _F|mark ^fAnd P _F|mark ^NfMiddle error amount is set up new set P greater than the point of deviation threshold τ _F|mark ^f' and P _F|mark ^Nf', change Step2, less than given error, iteration finishes as if adjacent twice threshold difference;

Step4:P _F|mark ^f' and P _F|mark ^Nf' promptly be the face monumented point in the corresponding human face data model.

The invention has the beneficial effects as follows: owing to adopt multi-modal information in conjunction with carrying out the facial face monumented point of three-dimensional face location, by the concavo-convex clearly demarcated advantage of face mark region in the clear and three-dimensional face model of face structure outline in the two dimensional image, can in advance faceform's attitude be estimated and compensate, thereby attitude there is stronger robustness, the average locating accuracy of face monumented point to the front three-dimensional face model reaches 98.5%, attitude is changed less faceform, and locating accuracy brings up to 91.6% by 88.3% of background technology; For attitude variation faceform greatly, the present invention has significant superiority, and locating accuracy brings up to 73.5% by 57.5% of background technology, is higher than the locating accuracy of background technology far away.

Below in conjunction with embodiment the present invention is elaborated.

Embodiment

The facial face monumented point of the colourful attitude three-dimensional face localization method of the multi-modal information combination of the present invention at first adopts affine constant Affine-SIFT method to carry out feature point detection to the two-dimension human face texture image; Then, utilize mapping relations that detected unique point in the two-dimensional space is projected to three dimensions, each summit to the three-dimensional face model data centralization, the structrual description information crestal line and the valley line of the three-dimensional face face concavo-convex variation in zone that calculating is represented by maximum, minimum curvature are as the part sudden change point set of people's face face; At last, set up different attitude human face characteristic points to restriction relation, carry out matching optimization with least square method, matching result is carried out statistical study, draw a certain deviate, make most points all in this deviate, as the big point of threshold value filtering deviation, set up new set with this deviate, think that the point that is filtered off is non-face monumented point, less than given error amount, the point in set this moment is determined to be people's face face monumented point up to the difference of adjacent twice threshold.Because the present invention adopts the thinking of multi-modal information combination to carry out the facial face monumented point of three-dimensional face location, by the easy concavo-convex clearly demarcated advantage of face mark region in the clear and three-dimensional face model of face structure outline in the two dimensional image, can in advance faceform's attitude be estimated and compensate, thereby attitude there is stronger robustness, has realized location the facial face monumented point of colourful attitude three-dimensional face.

1, based on the feature point detection of 2 d texture image.

People's face can carry out the motion of 6DOF in three dimensions, simultaneously the facial image that different cameral gets access in the practical application is owing to the axial variation of camera light produces image fault, thereby makes the detection of human face characteristic point be faced with bigger difficulty.ASIFT (Affine Scale Invariant Feature Transform) is keeping SIFT method premium properties, as being changed, rotation, scale, translation, brightness maintains the invariance, visual angle change, noise are also kept on the basis of stability to a certain degree affine variation is had stronger insensitivity.The ASIFT method is at first carried out accurate emulation to changing the image fault that produces by the camera light direction of principal axis, then, adopts the step identical with SIFT to carry out feature point detection.ASIFT can carry out emulation to three parameters, and promptly yardstick, camera longitude angle and camera angle of latitude carry out normalization to rotation and translation parameters simultaneously, therefore affined transformation are had unchangeability.

ASIFT method specific implementation step is as follows:

Step1: to carrying out emulation by head-up position around the affine deformation that the camera light direction of principal axis may cause, this deformation is determined by two parameters with every width of cloth figure: camera longitude angle φ and angle of latitude θ.Image is via according to inclination conversion degree correlation parameter

Carry out the φ rotation.For digital picture, the inclination conversion degree is finished by the t sub sampling of an orientation, and this process need at first uses one on the x direction have standard deviation

Gaussian function come image is carried out convolution operation, choose c=0.8.

Step2: these rotations and inclination conversion degree variable are regulated by a series of limited latitudes and longitude angle, and the sampling step assurance emulating image of these parameters and other view similaritys by camera longitude angle φ and angle of latitude θ generation are stronger.

Step3: all emulating images compare with the constant matching process SIFT of similarity.

ASIFT has unchangeability to affined transformation, simultaneously because it has kept the invariant feature of SIFT to yardstick, rotation, translation itself, so ASIFT comparatively the different attitude facial images of the same individuality of robust ground detection are right to a unique point that is complementary.But most of unique points of being extracted by ASIFT do not have practical significance at people's face face aspect describing, and describe monumented point for the needed of paramount importance face of recognition of face, and most unique points are redundancy feature.In order to carry out human face analysis effectively, must from the unique point that extracts, extract the most representative face feature description point.

2, the local sudden change of face is described.

Two-dimensional image data U={u in the face database _i∈ R ²: i=1,2 ..., N} and 3D grid data V={v _j=(x _j, y _j, z _j) ∈ R ³: j=1,2 ..., there are mapping relations in M}, is designated as Ψ: u _i→ v _j,

R ², R ³Expression two dimension respectively, three-dimensional data space.The unique point of two dimensional image is shone upon to three dimensions, in the two dimensional image in the non-face zone eigenwert of detected unique point corresponding point in the three-dimensional face data be 0, be non-face zone, non-facial zone point can be rejected from the three-dimensional face data automatically thus.

With P{p _i, i=1,2 ..., N} represents the three-dimensional face data acquisition, to each summit p of this data centralization _i, its minimum and maximum curvature is expressed as respectively

With

The important information of curved-surface structure can be described, and the concavo-convex variation that crestal line on the three-dimension curved surface of being calculated by the minimax flexometer and valley line can be described people's face face zone preferably, respectively with l _RidgeAnd l _ValleyExpression, its calculating principle is:

$l_{\mathrm{ridge}}=\{p_i,\mathrm{if}(k_{1_{\mathrm{pi}}}>k_{\mathrm{thresh}1})|p_i\∈P\}---\left(1\right)$

$l_{\mathrm{valley}}=\{p_i,\mathrm{if}(k_{2_{\mathrm{pi}}}>k_{\mathrm{thresh}2})|p_i\∈P\}---\left(2\right)$

K wherein _Thresh1, k _Thresh2Be respectively the thresholding curvature of regional area, its value can be calculated by genetic algorithm.By the calculating principle of above valley line and crestal line, can obtain the valley line and the crestal line of people's face face regional location respectively.Suppose to be mapped to three-dimensional facial zone feature point set by

Expression is satisfied the point of crestal line and valley line according to formula (1) and (2) calculating, and is kept corresponding crestal line and valley line in this set.Make feature point set P _FInterior crestal line and valley line are labeled as P respectively _F|ridge, P _F|valley, and merge into P with two _F|mark, the part sudden change point set of expression people face face.

3, iterative constrained face monumented point is located.

In order to gather P from the local catastrophe point of face _F|markIn determine final monumented point, need set up the restriction relation between different attitude people's face data centralization face monumented points, and adopt the mode of iteration optimization to revise, reach Optimum Matching.Algorithm thought is: set up different attitude human face characteristic points to restriction relation, carry out matching optimization with least square method then, matching result is carried out statistical study, draw a certain deviate τ, make most points all in this deviate, as the big point of threshold value filtering deviation, set up new set P with this deviate _F|mark', think that the point that is filtered off is non-face monumented point, less than given error amount, will gather P up to the difference of adjacent twice threshold at this moment _F|mark' in point determine to be people's face face monumented point.

Algorithmic procedure is as follows:

Step1: suppose different attitude three-dimensional face data set M _f, respectively to M _fAnd M _NfCatastrophe point set P _F|mark ^fAnd P _F|mark ^NfIn each monumented point p _{I, F|mark} ^f, p _{J, F|mark} ^Nf, set up shape coding, and compare M _fAnd M _NfUnique point to restriction relation

Step2: the point that R is set up calculates P to constraint _F|mark ^fAnd P _F|mark ^NfMatching error D (the p of each point _{I, F|mark} ^f, p _{J, F|mark} ^Nf), statistics draws deviation threshold τ;

Step3: remove P _F|mark ^fAnd P _F|mark ^NfMiddle error amount is set up new set P greater than the point of threshold tau _F|mark ^f' and P _F|mark ^Nf', change Step2, less than given error, iteration finishes as if adjacent twice threshold difference;

Step4: note P at this moment _F|mark ^f' and P _F|mark ^Nf' be the face monumented point in the corresponding human face data model.

Claims (1)

1. the facial face monumented point of the colourful attitude three-dimensional face localization method of a multi-modal information combination is characterized in that comprising the steps:

(a) people's face 2 d texture image is carried out feature point detection, to carrying out accurate emulation by head-up position around the image deformation distortion that the camera light direction of principal axis change to produce, the inclination conversion degree is by the t sub sampling of orientation with every width of cloth people face 2 d texture image, emulating image according to

Carry out the φ rotation, adopt standard deviation in the x direction Gaussian function image is carried out convolution operation, c=0.8; In the formula, φ is the camera longitude angle, and θ is the camera angle of latitude;

The deformation variable is regulated by camera longitude angle φ and camera angle of latitude θ;

Adopt the constant matching process of similarity that all emulating images are compared, extract the most representative face feature description point in the unique point that extracts;

(b) with detected face characteristic point data U={u in people's face 2 d texture image _i∈ R ²: i=1,2 ..., N}, R ²Expression 2-D data space is mapped in the three-dimensional face model according to corresponding relation; In the two dimensional image in the non-face zone eigenwert of detected unique point corresponding point in the three-dimensional face data be 0, reject non-face region point in promptly non-face zone from the three-dimensional face data; To three-dimensional face model data set P{p _i, i=1,2 ..., each the summit p among the N} _i, calculate by maximum curvature

And minimum curvature

The structrual description information crestal line l of the three-dimensional face face concavo-convex variation in zone of expression _RidgeWith valley line l _Valley:

$l_{\mathrm{ridge}}=\{p_i,\mathrm{if}(k_{1_{\mathrm{pi}}}>k_{\mathrm{thresh}1})|p_i\∈P\}---\left(1\right)$

$l_{\mathrm{valley}}=\{p_i,\mathrm{if}(k_{2_{\mathrm{pi}}}>k_{\mathrm{thresh}2})|p_i\∈P\}---\left(2\right)$

Part sudden change point set as people's face face; In the formula, k _Thresh1, k _Thresh2Be respectively the thresholding curvature of regional area, calculate, obtain the crestal line l of people's face face regional location by genetic algorithm _RidgeWith valley line l _Valley

(c) Step1: respectively to different attitude three-dimensional face data set M _fAnd M _NfCatastrophe point set P _F|mark ^fAnd P _F|mark ^NfIn each monumented point p _IF|mark ^f, p _{J, F|mark} ^Nf, set up shape coding, and compare M _fAnd M _NfUnique point to restriction relation

Step2: the point that R is set up calculates P to constraint _F|mark ^fAnd P _F|mark ^NfMatching error D (the p of each point _{I, F|mark} ^f, p _{J, F|mark} ^Nf), count deviation threshold τ;

Step3: remove P _F|mark ^fAnd P _F|mark ^NfMiddle error amount is set up new set P greater than the point of deviation threshold τ _F|mark ^f' and P _F|markn ^f', change Step2, less than given error, iteration finishes as if adjacent twice threshold difference;

Step4:P _F|mark ^f' and P _F|markn ^f' promptly be the face monumented point in the corresponding human face data model.