CA2285158C - Methode et appareillage pour animer un modele synthetise d'un visage humain au moyen d'un signal audio - Google Patents
Methode et appareillage pour animer un modele synthetise d'un visage humain au moyen d'un signal audio Download PDFInfo
- Publication number
- CA2285158C CA2285158C CA002285158A CA2285158A CA2285158C CA 2285158 C CA2285158 C CA 2285158C CA 002285158 A CA002285158 A CA 002285158A CA 2285158 A CA2285158 A CA 2285158A CA 2285158 C CA2285158 C CA 2285158C
- Authority
- CA
- Canada
- Prior art keywords
- visemes
- model
- parameters
- macroparameters
- lip
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 18
- 230000005236 sound signal Effects 0.000 title claims abstract description 15
- 230000007935 neutral effect Effects 0.000 claims abstract description 9
- 230000001815 facial effect Effects 0.000 claims description 40
- 238000006243 chemical reaction Methods 0.000 claims description 8
- 238000006073 displacement reaction Methods 0.000 claims description 4
- HEMJJKBWTPKOJG-UHFFFAOYSA-N Gemfibrozil Chemical compound CC1=CC=C(C)C(OCCCC(C)(C)C(O)=O)=C1 HEMJJKBWTPKOJG-UHFFFAOYSA-N 0.000 claims 1
- YAFQFNOUYXZVPZ-UHFFFAOYSA-N liproxstatin-1 Chemical compound ClC1=CC=CC(CNC=2C3(CCNCC3)NC3=CC=CC=C3N=2)=C1 YAFQFNOUYXZVPZ-UHFFFAOYSA-N 0.000 description 9
- 230000015572 biosynthetic process Effects 0.000 description 6
- 238000003786 synthesis reaction Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 4
- 101150077696 lip-1 gene Proteins 0.000 description 4
- 239000002131 composite material Substances 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000003780 insertion Methods 0.000 description 2
- 230000037431 insertion Effects 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 206010048865 Hypoacusis Diseases 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000002902 bimodal effect Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000009365 direct transmission Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 101150078951 mai-2 gene Proteins 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 238000013518 transcription Methods 0.000 description 1
- 230000035897 transcription Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/001—Model-based coding, e.g. wire frame
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/06—Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids
- G10L21/10—Transforming into visible information
- G10L2021/105—Synthesis of the lips movements from speech, e.g. for talking heads
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Processing Or Creating Images (AREA)
Abstract
Procédé et appareil pour l'animation, entraînée par un signal audio, d'un modèle de visage humain de synthèse, permettant l'animation d'un modèle quelconque conforme à la norme ISO/IEC 14496 (« norme MPEG-4 »). Les phonèmes concernés sont dérivés du signal audio, et les visèmes correspondants sont identifiés au sein d'un ensemble comprenant à la fois des visèmes définis par la norme et des visèmes typiques de la langue. Les visèmes sont divisés en macroparamètres qui définissent la forme et les positions de la bouche et de la mâchoire du modèle et qui sont associés à des valeurs indiquant une différence par rapport à une position neutre. Ces macroparamètres sont ensuite transformés en paramètres d'animation faciale conformes à la norme, dont les valeurs définissent la déformation à appliquer au modèle afin d'obtenir l'animation.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
ITTO98A0000842 | 1998-10-07 | ||
IT1998TO000842A IT1314671B1 (it) | 1998-10-07 | 1998-10-07 | Procedimento e apparecchiatura per l'animazione di un modellosintetizzato di volto umano pilotata da un segnale audio. |
Publications (2)
Publication Number | Publication Date |
---|---|
CA2285158A1 CA2285158A1 (fr) | 2000-04-07 |
CA2285158C true CA2285158C (fr) | 2006-04-11 |
Family
ID=11417087
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CA002285158A Expired - Fee Related CA2285158C (fr) | 1998-10-07 | 1999-10-06 | Methode et appareillage pour animer un modele synthetise d'un visage humain au moyen d'un signal audio |
Country Status (6)
Country | Link |
---|---|
US (1) | US6665643B1 (fr) |
EP (1) | EP0993197B1 (fr) |
JP (1) | JP3215823B2 (fr) |
CA (1) | CA2285158C (fr) |
DE (1) | DE69941942D1 (fr) |
IT (1) | IT1314671B1 (fr) |
Families Citing this family (31)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6826540B1 (en) | 1999-12-29 | 2004-11-30 | Virtual Personalities, Inc. | Virtual human interface for conducting surveys |
US7080473B2 (en) * | 2000-05-24 | 2006-07-25 | Virtual Video Uk Ltd. | Novelty animated device with synchronized audio output, and method for achieving synchronized audio output therein |
US6661418B1 (en) | 2001-01-22 | 2003-12-09 | Digital Animations Limited | Character animation system |
US20020120643A1 (en) * | 2001-02-28 | 2002-08-29 | Ibm Corporation | Audio-visual data collection system |
US20020140718A1 (en) * | 2001-03-29 | 2002-10-03 | Philips Electronics North America Corporation | Method of providing sign language animation to a monitor and process therefor |
US7343082B2 (en) * | 2001-09-12 | 2008-03-11 | Ryshco Media Inc. | Universal guide track |
US20030058932A1 (en) * | 2001-09-24 | 2003-03-27 | Koninklijke Philips Electronics N.V. | Viseme based video coding |
US7076430B1 (en) * | 2002-05-16 | 2006-07-11 | At&T Corp. | System and method of providing conversational visual prosody for talking heads |
ITTO20020724A1 (it) * | 2002-08-14 | 2004-02-15 | Telecom Italia Lab Spa | Procedimento e sistema per la trasmissione di messaggi su |
US20050049005A1 (en) * | 2003-08-29 | 2005-03-03 | Ken Young | Mobile telephone with enhanced display visualization |
WO2005031701A2 (fr) * | 2003-09-29 | 2005-04-07 | Siemens Aktiengesellschaft | Production automatique de representations graphiques en plusieurs dimensions d'elements de langage des signes |
US8965771B2 (en) * | 2003-12-08 | 2015-02-24 | Kurzweil Ainetworks, Inc. | Use of avatar with event processing |
US20080228497A1 (en) * | 2005-07-11 | 2008-09-18 | Koninklijke Philips Electronics, N.V. | Method For Communication and Communication Device |
US7567251B2 (en) * | 2006-01-10 | 2009-07-28 | Sony Corporation | Techniques for creating facial animation using a face mesh |
US8224652B2 (en) * | 2008-09-26 | 2012-07-17 | Microsoft Corporation | Speech and text driven HMM-based body animation synthesis |
KR101541907B1 (ko) * | 2008-10-14 | 2015-08-03 | 삼성전자 주식회사 | 음성 기반 얼굴 캐릭터 형성 장치 및 방법 |
CN101436312B (zh) * | 2008-12-03 | 2011-04-06 | 腾讯科技(深圳)有限公司 | 一种生成视频动画的方法及装置 |
JP5178607B2 (ja) * | 2009-03-31 | 2013-04-10 | 株式会社バンダイナムコゲームス | プログラム、情報記憶媒体、口形状制御方法及び口形状制御装置 |
BRPI0904540B1 (pt) * | 2009-11-27 | 2021-01-26 | Samsung Eletrônica Da Amazônia Ltda | método para animar rostos/cabeças/personagens virtuais via processamento de voz |
US8594993B2 (en) | 2011-04-04 | 2013-11-26 | Microsoft Corporation | Frame mapping approach for cross-lingual voice transformation |
US20120276504A1 (en) * | 2011-04-29 | 2012-11-01 | Microsoft Corporation | Talking Teacher Visualization for Language Learning |
TW201301148A (zh) * | 2011-06-21 | 2013-01-01 | Hon Hai Prec Ind Co Ltd | 網頁瀏覽控制系統及方法 |
US8655152B2 (en) | 2012-01-31 | 2014-02-18 | Golden Monkey Entertainment | Method and system of presenting foreign films in a native language |
CN102609969B (zh) * | 2012-02-17 | 2013-08-07 | 上海交通大学 | 基于汉语文本驱动的人脸语音同步动画的处理方法 |
US20150279364A1 (en) * | 2014-03-29 | 2015-10-01 | Ajay Krishnan | Mouth-Phoneme Model for Computerized Lip Reading |
US10839825B2 (en) * | 2017-03-03 | 2020-11-17 | The Governing Council Of The University Of Toronto | System and method for animated lip synchronization |
US10910001B2 (en) * | 2017-12-25 | 2021-02-02 | Casio Computer Co., Ltd. | Voice recognition device, robot, voice recognition method, and storage medium |
GB201804807D0 (en) * | 2018-03-26 | 2018-05-09 | Orbital Media And Advertising Ltd | Interaactive systems and methods |
US10699705B2 (en) * | 2018-06-22 | 2020-06-30 | Adobe Inc. | Using machine-learning models to determine movements of a mouth corresponding to live speech |
CN111970540B (zh) * | 2020-08-19 | 2021-05-04 | 王磊 | 基于远程互动和云计算的媒体数据处理方法及大数据平台 |
CN117877509B (zh) * | 2024-03-13 | 2024-06-04 | 亚信科技(中国)有限公司 | 一种数字人实时交互方法及装置、电子设备、存储介质 |
Family Cites Families (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB8528143D0 (en) * | 1985-11-14 | 1985-12-18 | British Telecomm | Image encoding & synthesis |
US6122616A (en) * | 1993-01-21 | 2000-09-19 | Apple Computer, Inc. | Method and apparatus for diphone aliasing |
US5608839A (en) * | 1994-03-18 | 1997-03-04 | Lucent Technologies Inc. | Sound-synchronized video system |
US6330023B1 (en) * | 1994-03-18 | 2001-12-11 | American Telephone And Telegraph Corporation | Video signal processing systems and methods utilizing automated speech analysis |
US5657426A (en) * | 1994-06-10 | 1997-08-12 | Digital Equipment Corporation | Method and apparatus for producing audio-visual synthetic speech |
MX9504648A (es) * | 1994-11-07 | 1997-02-28 | At & T Corp | Metodo y aparato para el procesamiento de imagenes, asistido por acustica. |
AU2167097A (en) * | 1996-03-26 | 1997-10-17 | British Telecommunications Public Limited Company | Image synthesis |
US5818463A (en) * | 1997-02-13 | 1998-10-06 | Rockwell Science Center, Inc. | Data compression for animated three dimensional objects |
US6208356B1 (en) * | 1997-03-24 | 2001-03-27 | British Telecommunications Public Limited Company | Image synthesis |
US6154222A (en) * | 1997-03-27 | 2000-11-28 | At&T Corp | Method for defining animation parameters for an animation definition interface |
US5995119A (en) * | 1997-06-06 | 1999-11-30 | At&T Corp. | Method for generating photo-realistic animated characters |
US6177928B1 (en) * | 1997-08-22 | 2001-01-23 | At&T Corp. | Flexible synchronization framework for multimedia streams having inserted time stamp |
US6112177A (en) * | 1997-11-07 | 2000-08-29 | At&T Corp. | Coarticulation method for audio-visual text-to-speech synthesis |
US6250928B1 (en) * | 1998-06-22 | 2001-06-26 | Massachusetts Institute Of Technology | Talking facial display method and apparatus |
-
1998
- 1998-10-07 IT IT1998TO000842A patent/IT1314671B1/it active
-
1999
- 1999-09-28 US US09/407,027 patent/US6665643B1/en not_active Expired - Lifetime
- 1999-10-06 JP JP28550899A patent/JP3215823B2/ja not_active Expired - Fee Related
- 1999-10-06 CA CA002285158A patent/CA2285158C/fr not_active Expired - Fee Related
- 1999-10-07 DE DE69941942T patent/DE69941942D1/de not_active Expired - Lifetime
- 1999-10-07 EP EP99119180A patent/EP0993197B1/fr not_active Expired - Lifetime
Also Published As
Publication number | Publication date |
---|---|
CA2285158A1 (fr) | 2000-04-07 |
ITTO980842A1 (it) | 2000-04-07 |
US6665643B1 (en) | 2003-12-16 |
IT1314671B1 (it) | 2002-12-31 |
DE69941942D1 (de) | 2010-03-11 |
EP0993197B1 (fr) | 2010-01-20 |
JP2000113216A (ja) | 2000-04-21 |
EP0993197A2 (fr) | 2000-04-12 |
JP3215823B2 (ja) | 2001-10-09 |
EP0993197A3 (fr) | 2002-03-27 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CA2285158C (fr) | Methode et appareillage pour animer un modele synthetise d'un visage humain au moyen d'un signal audio | |
CN113192161B (zh) | 一种虚拟人形象视频生成方法、***、装置及存储介质 | |
CN113194348B (zh) | 一种虚拟人讲课视频生成方法、***、装置及存储介质 | |
US7145606B2 (en) | Post-synchronizing an information stream including lip objects replacement | |
Cosatto et al. | Lifelike talking faces for interactive services | |
US20100082345A1 (en) | Speech and text driven hmm-based body animation synthesis | |
KR102098734B1 (ko) | 대화 상대의 외형을 반영한 수어 영상 제공 방법, 장치 및 단말 | |
CN110880198A (zh) | 动画生成方法和装置 | |
US6839672B1 (en) | Integration of talking heads and text-to-speech synthesizers for visual TTS | |
Albrecht et al. | " May I talk to you?:-)"-facial animation from text | |
CN114793300A (zh) | 一种基于生成对抗网络的虚拟视频客服机器人合成方法和*** | |
KR20090040014A (ko) | 텍스트 분석 기반의 입 모양 동기화 장치 및 방법 | |
CN114155321B (zh) | 一种基于自监督和混合密度网络的人脸动画生成方法 | |
CN115529500A (zh) | 动态影像的生成方法和装置 | |
EP0970467B1 (fr) | Procede de synthese vocale | |
CN115170702A (zh) | 数字人面部生成方法和装置、计算机装置和存储介质 | |
EP0982684A1 (fr) | Dispositif de generation d'images en mouvement et dispositif d'apprentissage via reseau de controle d'images | |
Beautemps et al. | Telma: Telephony for the hearing-impaired people. from models to user tests | |
CN113763924B (zh) | 声学深度学习模型训练方法、语音生成方法及设备 | |
Faruquie et al. | Translingual visual speech synthesis | |
Sato et al. | HMM-based photo-realistic talking face synthesis using facial expression parameter mapping with deep neural networks | |
Fagel | Merging methods of speech visualization | |
CN117750060A (zh) | 一种基于多模态ai手语生成***、方法 | |
Anitha et al. | NextGen Dynamic Video Generator using AI | |
Kawano et al. | Facial and head movements of a sign interpreter and their application to Japanese sign animation |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
EEER | Examination request | ||
MKLA | Lapsed |
Effective date: 20171006 |
|
MKLA | Lapsed |
Effective date: 20171006 |