CN106600667B - 一种基于卷积神经网络的视频驱动人脸动画方法 - Google Patents
一种基于卷积神经网络的视频驱动人脸动画方法 Download PDFInfo
- Publication number
- CN106600667B CN106600667B CN201611138441.3A CN201611138441A CN106600667B CN 106600667 B CN106600667 B CN 106600667B CN 201611138441 A CN201611138441 A CN 201611138441A CN 106600667 B CN106600667 B CN 106600667B
- Authority
- CN
- China
- Prior art keywords
- expression
- neural network
- dimensional
- training
- convolutional neural
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 38
- 230000014509 gene expression Effects 0.000 claims abstract description 162
- 238000012549 training Methods 0.000 claims abstract description 51
- 238000013528 artificial neural network Methods 0.000 claims abstract description 22
- 230000001815 facial effect Effects 0.000 claims abstract description 20
- 230000036544 posture Effects 0.000 claims abstract description 13
- 238000005286 illumination Methods 0.000 claims abstract description 11
- 238000004364 calculation method Methods 0.000 claims abstract description 6
- 239000011159 matrix material Substances 0.000 claims description 17
- 230000008569 process Effects 0.000 claims description 10
- 239000013598 vector Substances 0.000 claims description 6
- 230000008921 facial expression Effects 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 3
- 238000009826 distribution Methods 0.000 claims description 2
- 230000001902 propagating effect Effects 0.000 claims description 2
- 230000003287 optical effect Effects 0.000 claims 1
- 238000009877 rendering Methods 0.000 claims 1
- 238000001514 detection method Methods 0.000 abstract description 7
- 238000005516 engineering process Methods 0.000 description 18
- 238000013135 deep learning Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 238000011176 pooling Methods 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000007935 neutral effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T13/00—Animation
- G06T13/20—3D [Three Dimensional] animation
- G06T13/40—3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Processing Or Creating Images (AREA)
- Image Analysis (AREA)
Abstract
Description
Claims (4)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201611138441.3A CN106600667B (zh) | 2016-12-12 | 2016-12-12 | 一种基于卷积神经网络的视频驱动人脸动画方法 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201611138441.3A CN106600667B (zh) | 2016-12-12 | 2016-12-12 | 一种基于卷积神经网络的视频驱动人脸动画方法 |
Publications (2)
Publication Number | Publication Date |
---|---|
CN106600667A CN106600667A (zh) | 2017-04-26 |
CN106600667B true CN106600667B (zh) | 2020-04-21 |
Family
ID=58599184
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201611138441.3A Expired - Fee Related CN106600667B (zh) | 2016-12-12 | 2016-12-12 | 一种基于卷积神经网络的视频驱动人脸动画方法 |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN106600667B (zh) |
Families Citing this family (30)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10572773B2 (en) * | 2017-05-05 | 2020-02-25 | Intel Corporation | On the fly deep learning in machine learning for autonomous machines |
CN109903360A (zh) * | 2017-12-08 | 2019-06-18 | 浙江舜宇智能光学技术有限公司 | 三维人脸动画控制***及其控制方法 |
CN107886568B (zh) * | 2017-12-09 | 2020-03-03 | 东方梦幻文化产业投资有限公司 | 一种利用3D Avatar重建人脸表情的方法及*** |
CN110096925B (zh) * | 2018-01-30 | 2021-05-14 | 普天信息技术有限公司 | 人脸表情图像的增强方法、获取方法和装置 |
CN108960288B (zh) * | 2018-06-07 | 2021-05-04 | 山东师范大学 | 基于卷积神经网络的三维模型分类方法及*** |
CN111971713A (zh) * | 2018-06-14 | 2020-11-20 | 英特尔公司 | 使用图像和时间跟踪神经网络进行的3d面部捕获和修改 |
CN109087380B (zh) * | 2018-08-02 | 2023-10-20 | 咪咕文化科技有限公司 | 一种漫画动图生成方法、装置及存储介质 |
CN109165578A (zh) * | 2018-08-08 | 2019-01-08 | 盎锐(上海)信息科技有限公司 | 基于拍摄装置的表情检测装置及数据处理方法 |
CN109308731B (zh) * | 2018-08-24 | 2023-04-25 | 浙江大学 | 级联卷积lstm的语音驱动唇形同步人脸视频合成算法 |
CN109239914B (zh) * | 2018-09-12 | 2020-05-01 | 南京大学 | 一种实现高空间带宽积的成像方法 |
CN109255831B (zh) * | 2018-09-21 | 2020-06-12 | 南京大学 | 基于多任务学习的单视图人脸三维重建及纹理生成的方法 |
CN109493403A (zh) * | 2018-11-13 | 2019-03-19 | 北京中科嘉宁科技有限公司 | 一种基于运动单元表情映射实现人脸动画的方法 |
RU2703327C1 (ru) * | 2018-12-10 | 2019-10-16 | Самсунг Электроникс Ко., Лтд. | Способ обработки двухмерного изображения и реализующее его вычислительное устройство пользователя |
WO2020122513A1 (ko) | 2018-12-10 | 2020-06-18 | 삼성전자주식회사 | 2차원 이미지 처리 방법 및 이 방법을 실행하는 디바이스 |
CN109886250A (zh) * | 2019-03-11 | 2019-06-14 | 重庆科技学院 | 一种基于kf-pls的加重毯体验评价方法及*** |
US10953334B2 (en) * | 2019-03-27 | 2021-03-23 | Electronic Arts Inc. | Virtual character generation from image or video data |
CN110503201A (zh) * | 2019-08-29 | 2019-11-26 | 苏州浪潮智能科技有限公司 | 一种神经网络分布式并行训练方法与装置 |
CN111091624B (zh) * | 2019-12-19 | 2021-09-28 | 南京大学 | 一种从单张图片生成高精度可驱动人脸三维模型的方法 |
CN111222459B (zh) * | 2020-01-06 | 2023-05-12 | 上海交通大学 | 一种视角无关的视频三维人体姿态识别方法 |
CN111311712B (zh) * | 2020-02-24 | 2023-06-16 | 北京百度网讯科技有限公司 | 视频帧处理方法和装置 |
CN112734887B (zh) * | 2021-01-20 | 2022-09-20 | 清华大学 | 基于深度学习的人脸混合-变形生成方法和装置 |
CN112581520A (zh) * | 2021-01-29 | 2021-03-30 | 秒影工场(北京)科技有限公司 | 一种基于帧连续四维扫描的人脸形状表情模型构建方法 |
CN113255457A (zh) * | 2021-04-28 | 2021-08-13 | 上海交通大学 | 基于人脸表情识别的动画角色面部表情生成方法及*** |
CN113191282A (zh) * | 2021-05-07 | 2021-07-30 | 深圳市灼华网络科技有限公司 | 人脸表情动画的生成方法、装置、设备及存储介质 |
WO2023282913A1 (en) * | 2021-07-09 | 2023-01-12 | Hewlett-Packard Development Company, L.P. | Blendshape weights prediction for facial expression of hmd wearer using machine learning model trained on rendered avatar training images |
CN114092649B (zh) * | 2021-11-25 | 2022-10-18 | 马上消费金融股份有限公司 | 基于神经网络的图片生成方法及装置 |
CN114049678B (zh) * | 2022-01-11 | 2022-04-12 | 之江实验室 | 一种基于深度学习的面部动作捕捉方法及*** |
CN114463817B (zh) * | 2022-02-10 | 2022-09-30 | 北京中科深智科技有限公司 | 一种轻量级的基于2d视频的人脸表情驱动方法和*** |
CN114898020A (zh) * | 2022-05-26 | 2022-08-12 | 唯物(杭州)科技有限公司 | 一种3d角色实时面部驱动方法、装置、电子设备及存储介质 |
CN115346262A (zh) * | 2022-08-23 | 2022-11-15 | 北京字跳网络技术有限公司 | 一种表情驱动参数的确定方法、装置、设备及存储介质 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103093490A (zh) * | 2013-02-02 | 2013-05-08 | 浙江大学 | 基于单个视频摄像机的实时人脸动画方法 |
CN103927531A (zh) * | 2014-05-13 | 2014-07-16 | 江苏科技大学 | 一种基于局部二值和粒子群优化bp神经网络的人脸识别方法 |
CN103942822A (zh) * | 2014-04-11 | 2014-07-23 | 浙江大学 | 一种基于单视频摄像机的面部特征点跟踪和人脸动画方法 |
CN104809719A (zh) * | 2015-04-01 | 2015-07-29 | 华南理工大学 | 基于单应性矩阵分割的虚拟视点合成的方法 |
-
2016
- 2016-12-12 CN CN201611138441.3A patent/CN106600667B/zh not_active Expired - Fee Related
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103093490A (zh) * | 2013-02-02 | 2013-05-08 | 浙江大学 | 基于单个视频摄像机的实时人脸动画方法 |
CN103942822A (zh) * | 2014-04-11 | 2014-07-23 | 浙江大学 | 一种基于单视频摄像机的面部特征点跟踪和人脸动画方法 |
CN103927531A (zh) * | 2014-05-13 | 2014-07-16 | 江苏科技大学 | 一种基于局部二值和粒子群优化bp神经网络的人脸识别方法 |
CN104809719A (zh) * | 2015-04-01 | 2015-07-29 | 华南理工大学 | 基于单应性矩阵分割的虚拟视点合成的方法 |
Non-Patent Citations (1)
Title |
---|
基于Kinect的主动外观模型及在表情动画上的应用;王庆祥;《博士学位论文》;20140615;说明书第58页 * |
Also Published As
Publication number | Publication date |
---|---|
CN106600667A (zh) | 2017-04-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN106600667B (zh) | 一种基于卷积神经网络的视频驱动人脸动画方法 | |
US20210241521A1 (en) | Face image generation method and apparatus, device, and storage medium | |
CN109255831B (zh) | 基于多任务学习的单视图人脸三维重建及纹理生成的方法 | |
CN108154550B (zh) | 基于rgbd相机的人脸实时三维重建方法 | |
WO2021093453A1 (zh) | 三维表情基的生成方法、语音互动方法、装置及介质 | |
US10846903B2 (en) | Single shot capture to animated VR avatar | |
CN106710003B (zh) | 一种基于OpenGL ES的三维拍照方法和*** | |
CN108573527A (zh) | 一种表情图片生成方法及其设备、存储介质 | |
US20130127827A1 (en) | Multiview Face Content Creation | |
CN112784621B (zh) | 图像显示方法及设备 | |
CN101916454A (zh) | 基于网格变形和连续优化的高分辨率人脸重建方法 | |
CN102306386B (zh) | 从单张树木图像快速构建真实感树木模型的方法 | |
CN101968891A (zh) | 用于游戏的照片自动生成三维图形*** | |
US20230419600A1 (en) | Volumetric performance capture with neural rendering | |
Zhu et al. | Facescape: 3d facial dataset and benchmark for single-view 3d face reconstruction | |
CN110070605A (zh) | 一种实时人体无标记动作捕获方法 | |
Ye et al. | High-fidelity 3D real-time facial animation using infrared structured light sensing system | |
CN114998514A (zh) | 一种虚拟角色的生成方法及设备 | |
CN109903360A (zh) | 三维人脸动画控制***及其控制方法 | |
Yao et al. | Neural Radiance Field-based Visual Rendering: A Comprehensive Review | |
Zhang et al. | Face animation making method based on facial motion capture | |
Gao et al. | EvaSurf: Efficient View-Aware Implicit Textured Surface Reconstruction on Mobile Devices | |
US20240119671A1 (en) | Systems and methods for face asset creation and models from one or more images | |
Ma-Thi et al. | Image-Based Approach for Preparing Artifacts for Virtual Puppetry Museum | |
CN117671110A (zh) | 基于人工智能的实时渲染***及方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
CB03 | Change of inventor or designer information |
Inventor after: Cao Xun Inventor after: Zhao Rui Inventor after: Li Ke Inventor after: Yue Tao Inventor after: Zhu Hao Inventor after: Ji Shengyu Inventor after: Jin Wei Inventor before: Cao Xun Inventor before: Li Ke Inventor before: Yue Tao Inventor before: Zhu Hao Inventor before: Ji Shengyu Inventor before: Jin Wei |
|
CB03 | Change of inventor or designer information | ||
TA01 | Transfer of patent application right |
Effective date of registration: 20170825 Address after: 210093 Nanjing, Gulou District, Jiangsu, No. 22 Hankou Road Applicant after: NANJING University Applicant after: Original Force, Ltd. Address before: 210093 Nanjing, Gulou District, Jiangsu, No. 22 Hankou Road Applicant before: Nanjing University |
|
TA01 | Transfer of patent application right | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
CP01 | Change in the name or title of a patent holder |
Address after: 210093 Nanjing, Gulou District, Jiangsu, No. 22 Hankou Road Co-patentee after: Jiangsu Yuanli Digital Technology Co.,Ltd. Patentee after: NANJING University Address before: 210093 Nanjing, Gulou District, Jiangsu, No. 22 Hankou Road Co-patentee before: Jiangsu force animation production Limited by Share Ltd. Patentee before: NANJING University |
|
CP01 | Change in the name or title of a patent holder | ||
CP03 | Change of name, title or address |
Address after: 210093 Nanjing, Gulou District, Jiangsu, No. 22 Hankou Road Co-patentee after: Jiangsu force animation production Limited by Share Ltd. Patentee after: NANJING University Address before: 22 Hankou Road, Gulou District, Nanjing Co-patentee before: Original Force, Ltd. Patentee before: NANJING University |
|
CP03 | Change of name, title or address | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20200421 |
|
CF01 | Termination of patent right due to non-payment of annual fee |