CA2285158C - Methode et appareillage pour animer un modele synthetise d'un visage humain au moyen d'un signal audio - Google Patents

Methode et appareillage pour animer un modele synthetise d'un visage humain au moyen d'un signal audio Download PDF

Info

Publication number
CA2285158C
CA2285158C CA002285158A CA2285158A CA2285158C CA 2285158 C CA2285158 C CA 2285158C CA 002285158 A CA002285158 A CA 002285158A CA 2285158 A CA2285158 A CA 2285158A CA 2285158 C CA2285158 C CA 2285158C
Authority
CA
Canada
Prior art keywords
visemes
model
parameters
macroparameters
lip
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CA002285158A
Other languages
English (en)
Other versions
CA2285158A1 (fr
Inventor
Claudio Lande
Mauro Quaglia
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Telecom Italia SpA
Original Assignee
Telecom Italia Lab SpA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Telecom Italia Lab SpA filed Critical Telecom Italia Lab SpA
Publication of CA2285158A1 publication Critical patent/CA2285158A1/fr
Application granted granted Critical
Publication of CA2285158C publication Critical patent/CA2285158C/fr
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/001—Model-based coding, e.g. wire frame
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/06—Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids
    • G10L21/10—Transforming into visible information
    • G10L2021/105—Synthesis of the lips movements from speech, e.g. for talking heads

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Processing Or Creating Images (AREA)

Abstract

Procédé et appareil pour l'animation, entraînée par un signal audio, d'un modèle de visage humain de synthèse, permettant l'animation d'un modèle quelconque conforme à la norme ISO/IEC 14496 (« norme MPEG-4 »). Les phonèmes concernés sont dérivés du signal audio, et les visèmes correspondants sont identifiés au sein d'un ensemble comprenant à la fois des visèmes définis par la norme et des visèmes typiques de la langue. Les visèmes sont divisés en macroparamètres qui définissent la forme et les positions de la bouche et de la mâchoire du modèle et qui sont associés à des valeurs indiquant une différence par rapport à une position neutre. Ces macroparamètres sont ensuite transformés en paramètres d'animation faciale conformes à la norme, dont les valeurs définissent la déformation à appliquer au modèle afin d'obtenir l'animation.
CA002285158A 1998-10-07 1999-10-06 Methode et appareillage pour animer un modele synthetise d'un visage humain au moyen d'un signal audio Expired - Fee Related CA2285158C (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
ITTO98A0000842 1998-10-07
IT1998TO000842A IT1314671B1 (it) 1998-10-07 1998-10-07 Procedimento e apparecchiatura per l'animazione di un modellosintetizzato di volto umano pilotata da un segnale audio.

Publications (2)

Publication Number Publication Date
CA2285158A1 CA2285158A1 (fr) 2000-04-07
CA2285158C true CA2285158C (fr) 2006-04-11

Family

ID=11417087

Family Applications (1)

Application Number Title Priority Date Filing Date
CA002285158A Expired - Fee Related CA2285158C (fr) 1998-10-07 1999-10-06 Methode et appareillage pour animer un modele synthetise d'un visage humain au moyen d'un signal audio

Country Status (6)

Country Link
US (1) US6665643B1 (fr)
EP (1) EP0993197B1 (fr)
JP (1) JP3215823B2 (fr)
CA (1) CA2285158C (fr)
DE (1) DE69941942D1 (fr)
IT (1) IT1314671B1 (fr)

Families Citing this family (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6826540B1 (en) 1999-12-29 2004-11-30 Virtual Personalities, Inc. Virtual human interface for conducting surveys
US7080473B2 (en) * 2000-05-24 2006-07-25 Virtual Video Uk Ltd. Novelty animated device with synchronized audio output, and method for achieving synchronized audio output therein
US6661418B1 (en) 2001-01-22 2003-12-09 Digital Animations Limited Character animation system
US20020120643A1 (en) * 2001-02-28 2002-08-29 Ibm Corporation Audio-visual data collection system
US20020140718A1 (en) * 2001-03-29 2002-10-03 Philips Electronics North America Corporation Method of providing sign language animation to a monitor and process therefor
US7343082B2 (en) * 2001-09-12 2008-03-11 Ryshco Media Inc. Universal guide track
US20030058932A1 (en) * 2001-09-24 2003-03-27 Koninklijke Philips Electronics N.V. Viseme based video coding
US7076430B1 (en) * 2002-05-16 2006-07-11 At&T Corp. System and method of providing conversational visual prosody for talking heads
ITTO20020724A1 (it) * 2002-08-14 2004-02-15 Telecom Italia Lab Spa Procedimento e sistema per la trasmissione di messaggi su
US20050049005A1 (en) * 2003-08-29 2005-03-03 Ken Young Mobile telephone with enhanced display visualization
WO2005031701A2 (fr) * 2003-09-29 2005-04-07 Siemens Aktiengesellschaft Production automatique de representations graphiques en plusieurs dimensions d'elements de langage des signes
US8965771B2 (en) * 2003-12-08 2015-02-24 Kurzweil Ainetworks, Inc. Use of avatar with event processing
US20080228497A1 (en) * 2005-07-11 2008-09-18 Koninklijke Philips Electronics, N.V. Method For Communication and Communication Device
US7567251B2 (en) * 2006-01-10 2009-07-28 Sony Corporation Techniques for creating facial animation using a face mesh
US8224652B2 (en) * 2008-09-26 2012-07-17 Microsoft Corporation Speech and text driven HMM-based body animation synthesis
KR101541907B1 (ko) * 2008-10-14 2015-08-03 삼성전자 주식회사 음성 기반 얼굴 캐릭터 형성 장치 및 방법
CN101436312B (zh) * 2008-12-03 2011-04-06 腾讯科技（深圳）有限公司 一种生成视频动画的方法及装置
JP5178607B2 (ja) * 2009-03-31 2013-04-10 株式会社バンダイナムコゲームス プログラム、情報記憶媒体、口形状制御方法及び口形状制御装置
BRPI0904540B1 (pt) * 2009-11-27 2021-01-26 Samsung Eletrônica Da Amazônia Ltda método para animar rostos/cabeças/personagens virtuais via processamento de voz
US8594993B2 (en) 2011-04-04 2013-11-26 Microsoft Corporation Frame mapping approach for cross-lingual voice transformation
US20120276504A1 (en) * 2011-04-29 2012-11-01 Microsoft Corporation Talking Teacher Visualization for Language Learning
TW201301148A (zh) * 2011-06-21 2013-01-01 Hon Hai Prec Ind Co Ltd 網頁瀏覽控制系統及方法
US8655152B2 (en) 2012-01-31 2014-02-18 Golden Monkey Entertainment Method and system of presenting foreign films in a native language
CN102609969B (zh) * 2012-02-17 2013-08-07 上海交通大学 基于汉语文本驱动的人脸语音同步动画的处理方法
US20150279364A1 (en) * 2014-03-29 2015-10-01 Ajay Krishnan Mouth-Phoneme Model for Computerized Lip Reading
US10839825B2 (en) * 2017-03-03 2020-11-17 The Governing Council Of The University Of Toronto System and method for animated lip synchronization
US10910001B2 (en) * 2017-12-25 2021-02-02 Casio Computer Co., Ltd. Voice recognition device, robot, voice recognition method, and storage medium
GB201804807D0 (en) * 2018-03-26 2018-05-09 Orbital Media And Advertising Ltd Interaactive systems and methods
US10699705B2 (en) * 2018-06-22 2020-06-30 Adobe Inc. Using machine-learning models to determine movements of a mouth corresponding to live speech
CN111970540B (zh) * 2020-08-19 2021-05-04 王磊 基于远程互动和云计算的媒体数据处理方法及大数据平台
CN117877509B (zh) * 2024-03-13 2024-06-04 亚信科技（中国）有限公司 一种数字人实时交互方法及装置、电子设备、存储介质

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB8528143D0 (en) * 1985-11-14 1985-12-18 British Telecomm Image encoding & synthesis
US6122616A (en) * 1993-01-21 2000-09-19 Apple Computer, Inc. Method and apparatus for diphone aliasing
US5608839A (en) * 1994-03-18 1997-03-04 Lucent Technologies Inc. Sound-synchronized video system
US6330023B1 (en) * 1994-03-18 2001-12-11 American Telephone And Telegraph Corporation Video signal processing systems and methods utilizing automated speech analysis
US5657426A (en) * 1994-06-10 1997-08-12 Digital Equipment Corporation Method and apparatus for producing audio-visual synthetic speech
MX9504648A (es) * 1994-11-07 1997-02-28 At & T Corp Metodo y aparato para el procesamiento de imagenes, asistido por acustica.
AU2167097A (en) * 1996-03-26 1997-10-17 British Telecommunications Public Limited Company Image synthesis
US5818463A (en) * 1997-02-13 1998-10-06 Rockwell Science Center, Inc. Data compression for animated three dimensional objects
US6208356B1 (en) * 1997-03-24 2001-03-27 British Telecommunications Public Limited Company Image synthesis
US6154222A (en) * 1997-03-27 2000-11-28 At&T Corp Method for defining animation parameters for an animation definition interface
US5995119A (en) * 1997-06-06 1999-11-30 At&T Corp. Method for generating photo-realistic animated characters
US6177928B1 (en) * 1997-08-22 2001-01-23 At&T Corp. Flexible synchronization framework for multimedia streams having inserted time stamp
US6112177A (en) * 1997-11-07 2000-08-29 At&T Corp. Coarticulation method for audio-visual text-to-speech synthesis
US6250928B1 (en) * 1998-06-22 2001-06-26 Massachusetts Institute Of Technology Talking facial display method and apparatus

Also Published As

Publication number Publication date
CA2285158A1 (fr) 2000-04-07
ITTO980842A1 (it) 2000-04-07
US6665643B1 (en) 2003-12-16
IT1314671B1 (it) 2002-12-31
DE69941942D1 (de) 2010-03-11
EP0993197B1 (fr) 2010-01-20
JP2000113216A (ja) 2000-04-21
EP0993197A2 (fr) 2000-04-12
JP3215823B2 (ja) 2001-10-09
EP0993197A3 (fr) 2002-03-27

Similar Documents

Publication Publication Date Title
CA2285158C (fr) Methode et appareillage pour animer un modele synthetise d'un visage humain au moyen d'un signal audio
CN113192161B (zh) 一种虚拟人形象视频生成方法、***、装置及存储介质
CN113194348B (zh) 一种虚拟人讲课视频生成方法、***、装置及存储介质
US7145606B2 (en) Post-synchronizing an information stream including lip objects replacement
Cosatto et al. Lifelike talking faces for interactive services
US20100082345A1 (en) Speech and text driven hmm-based body animation synthesis
KR102098734B1 (ko) 대화 상대의 외형을 반영한 수어 영상 제공 방법, 장치 및 단말
CN110880198A (zh) 动画生成方法和装置
US6839672B1 (en) Integration of talking heads and text-to-speech synthesizers for visual TTS
Albrecht et al. " May I talk to you?:-)"-facial animation from text
CN114793300A (zh) 一种基于生成对抗网络的虚拟视频客服机器人合成方法和***
KR20090040014A (ko) 텍스트 분석 기반의 입 모양 동기화 장치 및 방법
CN114155321B (zh) 一种基于自监督和混合密度网络的人脸动画生成方法
CN115529500A (zh) 动态影像的生成方法和装置
EP0970467B1 (fr) Procede de synthese vocale
CN115170702A (zh) 数字人面部生成方法和装置、计算机装置和存储介质
EP0982684A1 (fr) Dispositif de generation d'images en mouvement et dispositif d'apprentissage via reseau de controle d'images
Beautemps et al. Telma: Telephony for the hearing-impaired people. from models to user tests
CN113763924B (zh) 声学深度学习模型训练方法、语音生成方法及设备
Faruquie et al. Translingual visual speech synthesis
Sato et al. HMM-based photo-realistic talking face synthesis using facial expression parameter mapping with deep neural networks
Fagel Merging methods of speech visualization
CN117750060A (zh) 一种基于多模态ai手语生成***、方法
Anitha et al. NextGen Dynamic Video Generator using AI
Kawano et al. Facial and head movements of a sign interpreter and their application to Japanese sign animation

Legal Events

Date Code Title Description
EEER Examination request
MKLA Lapsed

Effective date: 20171006

MKLA Lapsed

Effective date: 20171006