GB2393065A - Method and system using a data-driven model for monocular face tracking - Google Patents

Method and system using a data-driven model for monocular face tracking Download PDF

Info

Publication number
GB2393065A
GB2393065A GB0328400A GB0328400A GB2393065A GB 2393065 A GB2393065 A GB 2393065A GB 0328400 A GB0328400 A GB 0328400A GB 0328400 A GB0328400 A GB 0328400A GB 2393065 A GB2393065 A GB 2393065A
Authority
GB
United Kingdom
Prior art keywords
model
data
face
monocular
driven
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
GB0328400A
Other versions
GB0328400D0 (en
GB2393065B (en
Inventor
Radek Grzeszczuk
Jean-Yves Bouguet
Salih Gokturk
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Intel Corp
Original Assignee
Intel Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Intel Corp filed Critical Intel Corp
Publication of GB0328400D0 publication Critical patent/GB0328400D0/en
Publication of GB2393065A publication Critical patent/GB2393065A/en
Application granted granted Critical
Publication of GB2393065B publication Critical patent/GB2393065B/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/251—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/239—Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/189—Recording image signals; Reproducing recorded image signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N2013/0074—Stereoscopic image analysis
    • H04N2013/0081—Depth or disparity estimation from stereoscopic image signals

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • Software Systems (AREA)
  • Geometry (AREA)
  • Computer Graphics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Image Analysis (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

A method and system using a data-driven model for monocular face tracking are disclosed, which provide a versatile system for tracking three-dimensional (3D) images, e.g., a face, using a single camera. For one method, stereo data based on input image sequences is obtained. A 3D model is built using the obtained stereo data. A monocular image sequence is tracked using the built 3D model. Principal Component Analysis (PCA) can be applied to the stereo data to learn, e.g., possible facial deformations, and to build a data-driven 3D model ("3D face model"). The 3D face model can be used to approximate a generic shape (e.g., facial pose) as a linear combination of shape basis vectors based on the PCA analysis.

Description

GB 2393065 A continuation (74) Agent and/or Address for Service: Harrison
Goddard Foote 40-43 Chancery Lane, LONDON, WC2A 1JA, United Kingdom
GB0328400A 2001-05-09 2002-05-02 Method and system using a data-driven model for monocular face tracking Expired - Fee Related GB2393065B (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US09/852,398 US9400921B2 (en) 2001-05-09 2001-05-09 Method and system using a data-driven model for monocular face tracking
PCT/US2002/014014 WO2002091305A2 (en) 2001-05-09 2002-05-02 Method and system, using a data-driven model for monocular face tracking

Publications (3)

Publication Number Publication Date
GB0328400D0 GB0328400D0 (en) 2004-01-14
GB2393065A true GB2393065A (en) 2004-03-17
GB2393065B GB2393065B (en) 2005-04-20

Family

ID=25313204

Family Applications (1)

Application Number Title Priority Date Filing Date
GB0328400A Expired - Fee Related GB2393065B (en) 2001-05-09 2002-05-02 Method and system using a data-driven model for monocular face tracking

Country Status (7)

Country Link
US (1) US9400921B2 (en)
KR (1) KR100571115B1 (en)
CN (1) CN1294541C (en)
AU (1) AU2002303611A1 (en)
GB (1) GB2393065B (en)
HK (1) HK1062067A1 (en)
WO (1) WO2002091305A2 (en)

Families Citing this family (41)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE10025922A1 (en) * 2000-05-27 2001-12-13 Robert Massen Automatic photogrammetric digitization of bodies and objects
US9400921B2 (en) * 2001-05-09 2016-07-26 Intel Corporation Method and system using a data-driven model for monocular face tracking
GB2382289B (en) * 2001-09-28 2005-07-06 Canon Kk Method and apparatus for generating models of individuals
GB2389289B (en) * 2002-04-30 2005-09-14 Canon Kk Method and apparatus for generating models of individuals
DE10235657A1 (en) * 2002-08-02 2004-02-12 Leica Microsystems Heidelberg Gmbh Process, arrangement and software for optimizing the image quality of moving objects taken with a microscope
NZ539632A (en) * 2002-10-22 2008-01-31 Artoolworks Tracking a surface in a 3-dimensional scene using natural visual features of the surface
JP4210926B2 (en) * 2004-01-16 2009-01-21 株式会社デンソー Occupant protection system
KR101238608B1 (en) * 2004-07-30 2013-02-28 익스트림 리얼리티 엘티디. A system and method for 3D space-dimension based image processing
JP4928451B2 (en) * 2004-07-30 2012-05-09 ユークリッド・ディスカバリーズ・エルエルシー Apparatus and method for processing video data
EP1982306A1 (en) * 2006-02-07 2008-10-22 France Télécom Method of tracking the position of the head in real time in a video image stream
US8026931B2 (en) 2006-03-16 2011-09-27 Microsoft Corporation Digital video effects
CN100449567C (en) * 2006-11-02 2009-01-07 中山大学 2-D main-element human-face analysis and identifying method based on relativity in block
CN100423020C (en) * 2006-12-15 2008-10-01 中山大学 Human face identifying method based on structural principal element analysis
ES2402957T3 (en) 2007-03-05 2013-05-10 Seeing Machines Pty Ltd Effective and accurate tracking of three-dimensional objects
KR100896065B1 (en) * 2007-12-17 2009-05-07 한국전자통신연구원 Method for producing 3d facial animation
JP5239396B2 (en) * 2008-02-28 2013-07-17 セイコーエプソン株式会社 Image output method, image output apparatus, and image output program
US8525871B2 (en) * 2008-08-08 2013-09-03 Adobe Systems Incorporated Content-aware wide-angle images
US8538072B2 (en) * 2008-08-27 2013-09-17 Imprivata, Inc. Systems and methods for operator detection
CN101425183B (en) * 2008-11-13 2012-04-25 上海交通大学 Deformable body three-dimensional tracking method based on second order cone programing
US8624962B2 (en) * 2009-02-02 2014-01-07 Ydreams—Informatica, S.A. Ydreams Systems and methods for simulating three-dimensional virtual interactions from two-dimensional camera images
JP2011039801A (en) * 2009-08-12 2011-02-24 Hitachi Ltd Apparatus and method for processing image
CN101763636B (en) * 2009-09-23 2012-07-04 中国科学院自动化研究所 Method for tracing position and pose of 3D human face in video sequence
US8937592B2 (en) * 2010-05-20 2015-01-20 Samsung Electronics Co., Ltd. Rendition of 3D content on a handheld device
US8259994B1 (en) * 2010-09-14 2012-09-04 Google Inc. Using image and laser constraints to obtain consistent and improved pose estimates in vehicle pose databases
CN102122239B (en) * 2011-03-21 2013-03-20 日照市活点网络科技有限公司 Method for processing 3D images of Internet of things
CN103814384B (en) * 2011-06-09 2017-08-18 香港科技大学 Tracking based on image
US20130271451A1 (en) * 2011-08-09 2013-10-17 Xiaofeng Tong Parameterized 3d face generation
US9123144B2 (en) * 2011-11-11 2015-09-01 Microsoft Technology Licensing, Llc Computing 3D shape parameters for face animation
US9129147B1 (en) * 2012-05-22 2015-09-08 Image Metrics Limited Optimal automatic capture of facial movements and expressions in video sequences
KR101311600B1 (en) * 2012-10-26 2013-09-26 동국대학교 산학협력단 Medical position tracking apparatus
US9406135B2 (en) 2012-10-29 2016-08-02 Samsung Electronics Co., Ltd. Device and method for estimating head pose
FR2998402B1 (en) * 2012-11-20 2014-11-14 Morpho METHOD FOR GENERATING A FACE MODEL IN THREE DIMENSIONS
EP2824913A1 (en) * 2013-07-09 2015-01-14 Alcatel Lucent A method for generating an immersive video of a plurality of persons
US20160070952A1 (en) * 2014-09-05 2016-03-10 Samsung Electronics Co., Ltd. Method and apparatus for facial recognition
GB201419441D0 (en) * 2014-10-31 2014-12-17 Microsoft Corp Modifying video call data
US9747573B2 (en) 2015-03-23 2017-08-29 Avatar Merger Sub II, LLC Emotion recognition for workforce analytics
US10441604B2 (en) * 2016-02-09 2019-10-15 Albireo Ab Cholestyramine pellets and methods for preparation thereof
US11736756B2 (en) * 2016-02-10 2023-08-22 Nitin Vats Producing realistic body movement using body images
EP3324254A1 (en) * 2016-11-17 2018-05-23 Siemens Aktiengesellschaft Device and method for determining the parameters of a control device
EP3593323B1 (en) * 2017-06-07 2020-08-05 Google LLC High speed, high-fidelity face tracking
CN109145758A (en) * 2018-07-25 2019-01-04 武汉恩智电子科技有限公司 A kind of recognizer of the face based on video monitoring

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0807902A2 (en) * 1996-05-16 1997-11-19 Cyberclass Limited Method and apparatus for generating moving characters
US6163322A (en) * 1998-01-19 2000-12-19 Taarna Studios Inc. Method and apparatus for providing real-time animation utilizing a database of postures

Family Cites Families (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1188948A (en) * 1996-12-27 1998-07-29 大宇电子株式会社 Method and apparatus for encoding facial movement
US6014625A (en) * 1996-12-30 2000-01-11 Daewoo Electronics Co., Ltd Method and apparatus for producing lip-movement parameters in a three-dimensional-lip-model
JP3055666B2 (en) * 1997-03-11 2000-06-26 株式会社エイ・ティ・アール知能映像通信研究所 3D image creation device
US6047078A (en) * 1997-10-03 2000-04-04 Digital Equipment Corporation Method for extracting a three-dimensional model using appearance-based constrained structure from motion
US6381346B1 (en) * 1997-12-01 2002-04-30 Wheeling Jesuit University Three-dimensional face identification system
US6272231B1 (en) * 1998-11-06 2001-08-07 Eyematic Interfaces, Inc. Wavelet-based facial motion capture for avatar animation
US6301370B1 (en) * 1998-04-13 2001-10-09 Eyematic Interfaces, Inc. Face recognition from video images
US6614428B1 (en) * 1998-06-08 2003-09-02 Microsoft Corporation Compression of animated geometry using a hierarchical level of detail coder
US6072496A (en) * 1998-06-08 2000-06-06 Microsoft Corporation Method and system for capturing and representing 3D geometry, color and shading of facial expressions and other animated objects
US6198485B1 (en) * 1998-07-29 2001-03-06 Intel Corporation Method and apparatus for three-dimensional input entry
IT1315446B1 (en) * 1998-10-02 2003-02-11 Cselt Centro Studi Lab Telecom PROCEDURE FOR THE CREATION OF THREE-DIMENSIONAL FACIAL MODELS TO START FROM FACE IMAGES.
JP2000293687A (en) * 1999-02-02 2000-10-20 Minolta Co Ltd Three-dimensional shape data processor and three- dimensional shape data processing method
US6200139B1 (en) * 1999-02-26 2001-03-13 Intel Corporation Operator training system
EP1039417B1 (en) * 1999-03-19 2006-12-20 Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V. Method and device for the processing of images based on morphable models
JP3639476B2 (en) * 1999-10-06 2005-04-20 シャープ株式会社 Image processing apparatus, image processing method, and recording medium recording image processing program
JP2001142380A (en) * 1999-11-12 2001-05-25 Sony Corp Device and method for producing hologram and hologram
EP1956557A1 (en) * 2000-03-08 2008-08-13 Cyberextruder.com, Inc. Generating a 3-D representation form
US6807290B2 (en) * 2000-03-09 2004-10-19 Microsoft Corporation Rapid computer modeling of faces for animation
JP2001331799A (en) * 2000-03-16 2001-11-30 Toshiba Corp Image processor and image processing method
IT1320002B1 (en) * 2000-03-31 2003-11-12 Cselt Centro Studi Lab Telecom PROCEDURE FOR THE ANIMATION OF A SYNTHESIZED VOLTOHUMAN MODEL DRIVEN BY AN AUDIO SIGNAL.
KR20020022504A (en) * 2000-09-20 2002-03-27 박종만 System and method for 3D animation authoring with motion control, facial animation, lip synchronizing and lip synchronized voice
US6850872B1 (en) * 2000-08-30 2005-02-01 Microsoft Corporation Facial image processing methods and systems
US6950104B1 (en) * 2000-08-30 2005-09-27 Microsoft Corporation Methods and systems for animating facial features, and methods and systems for expression transformation
US7127081B1 (en) * 2000-10-12 2006-10-24 Momentum Bilgisayar, Yazilim, Danismanlik, Ticaret, A.S. Method for tracking motion of a face
US6975750B2 (en) * 2000-12-01 2005-12-13 Microsoft Corp. System and method for face recognition using synthesized training images
US9400921B2 (en) 2001-05-09 2016-07-26 Intel Corporation Method and system using a data-driven model for monocular face tracking
US7027054B1 (en) * 2002-08-14 2006-04-11 Avaworks, Incorporated Do-it-yourself photo realistic talking head creation system and method
US6919892B1 (en) * 2002-08-14 2005-07-19 Avaworks, Incorporated Photo realistic talking head creation system and method
WO2008156437A1 (en) * 2006-04-10 2008-12-24 Avaworks Incorporated Do-it-yourself photo realistic talking head creation system and method
US20110115798A1 (en) * 2007-05-10 2011-05-19 Nayar Shree K Methods and systems for creating speech-enabled avatars

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0807902A2 (en) * 1996-05-16 1997-11-19 Cyberclass Limited Method and apparatus for generating moving characters
US6163322A (en) * 1998-01-19 2000-12-19 Taarna Studios Inc. Method and apparatus for providing real-time animation utilizing a database of postures

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Guenter, Brian et al; 'Making Faces'; Computer Graphics; Siggraph 98 Conference Proceedings, Orlando, FL, 19-24 July 1998, Computer Graphics Proceedings, Siggraph, New York, NY: ACM, US, 19 July 1998, pp55-66; XP002205956; ISBN 0-89791-999-8 *
Kouadio C et al; 'Real-time facial animation based upon a bank of 3D facial expressions'; Computer Animation 98 Proceedings, Philadelphia, PA, 8-10 June 1998, Los Alamitos, CA, IEEE Comput. Soc, pp128-136; XP010285090; ISBN 0-8186-8541-7 *

Also Published As

Publication number Publication date
WO2002091305A2 (en) 2002-11-14
GB0328400D0 (en) 2004-01-14
AU2002303611A1 (en) 2002-11-18
US20030012408A1 (en) 2003-01-16
HK1062067A1 (en) 2004-10-15
GB2393065B (en) 2005-04-20
CN1509456A (en) 2004-06-30
US9400921B2 (en) 2016-07-26
WO2002091305A3 (en) 2003-09-18
KR20040034606A (en) 2004-04-28
CN1294541C (en) 2007-01-10
KR100571115B1 (en) 2006-04-13

Similar Documents

Publication Publication Date Title
GB2393065A (en) Method and system using a data-driven model for monocular face tracking
Azarbayejani et al. Visually controlled graphics
Pearson Developments in model-based video coding
US20090190003A1 (en) Vision-based augmented reality system using invisible marker
Ohya et al. Real-time reproduction of 3D human images in virtual space teleconferencing
EP0813040A3 (en) Precision spatial mapping with combined video and infrared signals
ES2146765T3 (en) PROCEDURE FOR GENERATION OF SYNTHESIS IMAGE.
CN101743567A (en) Virtual interactive presence systems and methods
WO1999015945A3 (en) Generating three-dimensional models of objects defined by two-dimensional image data
CN105279750A (en) Equipment display guiding system based on IR-UWB and image moment
Gonzalez-Franco et al. Movebox: Democratizing mocap for the microsoft rocketbox avatar library
Kancherla et al. A novel virtual reality tool for teaching dynamic 3D anatomy
MX9800247A (en) Autostereoscopic image acquisition method and system.
KR20170044318A (en) Method for collaboration using head mounted display
JP2534617B2 (en) Real-time recognition and synthesis method of human image
Knoop et al. Fusion of 2D and 3D sensor data for articulated body tracking
Ho et al. An analytic solution for the pose determination of human faces from a monocular image
CN111947650A (en) Fusion positioning system and method based on optical tracking and inertial tracking
JP5731462B2 (en) Video communication system and video communication method
Kawai et al. A support system for visually impaired persons to understand three-dimensional visual information using acoustic interface
Varona et al. Hands and face tracking for VR applications
KR20180123248A (en) Apparatus of motion recognition figure for generating motion data of virtual space character and method for generating motion data
Zhang et al. Euclidean structure from uncalibrated images using fuzzy domain knowledge: Application to facial images synthesis
Fontana et al. A cross-modal electronic travel aid device
Tao et al. Tracking facial features using probabilistic network

Legal Events

Date Code Title Description
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 1062067

Country of ref document: HK

REG Reference to a national code

Ref country code: HK

Ref legal event code: GR

Ref document number: 1062067

Country of ref document: HK

PCNP Patent ceased through non-payment of renewal fee

Effective date: 20100502