SG11202108498RA - Method and device for generating video, electronic equipment, and computer storage medium - Google Patents

Method and device for generating video, electronic equipment, and computer storage medium

Info

Publication number
SG11202108498RA
SG11202108498RA SG11202108498RA SG11202108498RA SG11202108498RA SG 11202108498R A SG11202108498R A SG 11202108498RA SG 11202108498R A SG11202108498R A SG 11202108498RA SG 11202108498R A SG11202108498R A SG 11202108498RA SG 11202108498R A SG11202108498R A SG 11202108498RA
Authority
SG
Singapore
Prior art keywords
storage medium
electronic equipment
computer storage
generating video
video
Prior art date
Application number
SG11202108498RA
Inventor
Linsen Song
Wenyan Wu
Chen Qian
Ran He
Original Assignee
Beijing Sensetime Technology Development Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Sensetime Technology Development Co Ltd filed Critical Beijing Sensetime Technology Development Co Ltd
Publication of SG11202108498RA publication Critical patent/SG11202108498RA/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—3D [Three Dimensional] animation
    • G06T13/40—3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/7834—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using audio features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G06T5/75—Unsharp masking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/77—Retouching; Inpainting; Scratch removal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G06V40/176—Dynamic expression
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/27—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique
    • G10L25/30—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique using neural networks
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G10L25/57—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for processing of video signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/265—Mixing

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Signal Processing (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Computational Linguistics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Acoustics & Sound (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Library & Information Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Geometry (AREA)
  • Databases & Information Systems (AREA)
  • Probability & Statistics with Applications (AREA)
  • Computer Graphics (AREA)
  • Image Analysis (AREA)
  • Processing Or Creating Images (AREA)
  • Studio Devices (AREA)
SG11202108498RA 2019-09-18 2020-09-08 Method and device for generating video, electronic equipment, and computer storage medium SG11202108498RA (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910883605.2A CN110677598B (en) 2019-09-18 2019-09-18 Video generation method and device, electronic equipment and computer storage medium
PCT/CN2020/114103 WO2021052224A1 (en) 2019-09-18 2020-09-08 Video generation method and apparatus, electronic device, and computer storage medium

Publications (1)

Publication Number Publication Date
SG11202108498RA true SG11202108498RA (en) 2021-09-29

Family

ID=69078255

Family Applications (1)

Application Number Title Priority Date Filing Date
SG11202108498RA SG11202108498RA (en) 2019-09-18 2020-09-08 Method and device for generating video, electronic equipment, and computer storage medium

Country Status (6)

Country Link
US (1) US20210357625A1 (en)
JP (1) JP2022526148A (en)
KR (1) KR20210140762A (en)
CN (1) CN110677598B (en)
SG (1) SG11202108498RA (en)
WO (1) WO2021052224A1 (en)

Families Citing this family (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210390937A1 (en) * 2018-10-29 2021-12-16 Artrendex, Inc. System And Method Generating Synchronized Reactive Video Stream From Auditory Input
CN110677598B (en) * 2019-09-18 2022-04-12 北京市商汤科技开发有限公司 Video generation method and device, electronic equipment and computer storage medium
CN111294665B (en) * 2020-02-12 2021-07-20 百度在线网络技术（北京）有限公司 Video generation method and device, electronic equipment and readable storage medium
CN111368137A (en) * 2020-02-12 2020-07-03 百度在线网络技术（北京）有限公司 Video generation method and device, electronic equipment and readable storage medium
SG10202001693VA (en) * 2020-02-26 2021-09-29 Pensees Pte Ltd Methods and Apparatus for AI (Artificial Intelligence) Movie Producer System
CN111429885B (en) * 2020-03-02 2022-05-13 北京理工大学 Method for mapping audio clip to human face-mouth type key point
CN113689527B (en) * 2020-05-15 2024-02-20 武汉Tcl集团工业研究院有限公司 Training method of face conversion model and face image conversion method
CN113689538B (en) * 2020-05-18 2024-05-21 北京达佳互联信息技术有限公司 Video generation method and device, electronic equipment and storage medium
US11538140B2 (en) * 2020-11-13 2022-12-27 Adobe Inc. Image inpainting based on multiple image transformations
CN112669441B (en) * 2020-12-09 2023-10-17 北京达佳互联信息技术有限公司 Object reconstruction method and device, electronic equipment and storage medium
CN112489036A (en) * 2020-12-14 2021-03-12 Oppo（重庆）智能科技有限公司 Image evaluation method, image evaluation device, storage medium, and electronic apparatus
CN112699263B (en) * 2021-01-08 2023-05-23 郑州科技学院 AI-based two-dimensional art image dynamic display method and device
CN112927712B (en) * 2021-01-25 2024-06-04 网易（杭州）网络有限公司 Video generation method and device and electronic equipment
CN113132815A (en) * 2021-04-22 2021-07-16 北京房江湖科技有限公司 Video generation method and device, computer-readable storage medium and electronic equipment
CN113077537B (en) * 2021-04-29 2023-04-25 广州虎牙科技有限公司 Video generation method, storage medium and device
CN113299312B (en) * 2021-05-21 2023-04-28 北京市商汤科技开发有限公司 Image generation method, device, equipment and storage medium
CN113378697B (en) * 2021-06-08 2022-12-09 安徽大学 Method and device for generating speaking face video based on convolutional neural network
US20230035306A1 (en) * 2021-07-21 2023-02-02 Nvidia Corporation Synthesizing video from audio using one or more neural networks
CN114466179A (en) * 2021-09-09 2022-05-10 马上消费金融股份有限公司 Method and device for measuring synchronism of voice and image
CN113868472A (en) * 2021-10-18 2021-12-31 深圳追一科技有限公司 Method for generating digital human video and related equipment
CN114093384A (en) * 2021-11-22 2022-02-25 上海商汤科技开发有限公司 Speaking video generation method, device, equipment and storage medium
WO2023097633A1 (en) * 2021-12-03 2023-06-08 Citrix Systems, Inc. Telephone call information collection and retrieval
CN114373033A (en) * 2022-01-10 2022-04-19 腾讯科技（深圳）有限公司 Image processing method, image processing apparatus, image processing device, storage medium, and computer program
CN116152122B (en) * 2023-04-21 2023-08-25 荣耀终端有限公司 Image processing method and electronic device
CN117593442B (en) * 2023-11-28 2024-05-03 拓元(广州)智慧科技有限公司 Portrait generation method based on multi-stage fine grain rendering
CN117474807B (en) * 2023-12-27 2024-05-31 科大讯飞股份有限公司 Image restoration method, device, equipment and storage medium
CN117556084B (en) * 2023-12-27 2024-03-26 环球数科集团有限公司 Video emotion analysis system based on multiple modes
CN117523051B (en) * 2024-01-08 2024-05-07 南京硅基智能科技有限公司 Method, device, equipment and storage medium for generating dynamic image based on audio

Family Cites Families (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2795084B2 (en) * 1992-07-27 1998-09-10 国際電信電話株式会社 Mouth shape image synthesis method and apparatus
JPH1166272A (en) * 1997-08-13 1999-03-09 Sony Corp Processor and processing method for image or voice and record medium
JPH11149285A (en) * 1997-11-17 1999-06-02 Matsushita Electric Ind Co Ltd Image acoustic system
KR100411760B1 (en) * 2000-05-08 2003-12-18 주식회사 모리아테크놀로지 Apparatus and method for an animation image synthesis
CN100476877C (en) * 2006-11-10 2009-04-08 中国科学院计算技术研究所 Generating method of cartoon face driven by voice and text together
JP5109038B2 (en) * 2007-09-10 2012-12-26 株式会社国際電気通信基礎技術研究所 Lip sync animation creation device and computer program
JP2010086178A (en) * 2008-09-30 2010-04-15 Fujifilm Corp Image synthesis device and control method thereof
FR2958487A1 (en) * 2010-04-06 2011-10-07 Alcatel Lucent A METHOD OF REAL TIME DISTORTION OF A REAL ENTITY RECORDED IN A VIDEO SEQUENCE
CN101944238B (en) * 2010-09-27 2011-11-23 浙江大学 Data driving face expression synthesis method based on Laplace transformation
CN103093490B (en) * 2013-02-02 2015-08-26 浙江大学 Based on the real-time face animation method of single video camera
CN103279970B (en) * 2013-05-10 2016-12-28 中国科学技术大学 A kind of method of real-time voice-driven human face animation
US10283162B2 (en) * 2014-02-05 2019-05-07 Avatar Merger Sub II, LLC Method for triggering events in a video
US9779775B2 (en) * 2014-02-24 2017-10-03 Lyve Minds, Inc. Automatic generation of compilation videos from an original video based on metadata associated with the original video
CN105551071B (en) * 2015-12-02 2018-08-10 中国科学院计算技术研究所 A kind of the human face animation generation method and system of text voice driving
CN105957129B (en) * 2016-04-27 2019-08-30 上海河马动画设计股份有限公司 A kind of video display animation method based on voice driven and image recognition
CN107818785A (en) * 2017-09-26 2018-03-20 平安普惠企业管理有限公司 A kind of method and terminal device that information is extracted from multimedia file
CN107832746A (en) * 2017-12-01 2018-03-23 北京小米移动软件有限公司 Expression recognition method and device
CN108197604A (en) * 2018-01-31 2018-06-22 上海敏识网络科技有限公司 Fast face positioning and tracing method based on embedded device
JP2019201360A (en) * 2018-05-17 2019-11-21 住友電気工業株式会社 Image processing apparatus, computer program, video call system, and image processing method
CN109101919B (en) * 2018-08-03 2022-05-10 北京字节跳动网络技术有限公司 Method and apparatus for generating information
CN108985257A (en) * 2018-08-03 2018-12-11 北京字节跳动网络技术有限公司 Method and apparatus for generating information
CN109522818B (en) * 2018-10-29 2021-03-30 中国科学院深圳先进技术研究院 Expression recognition method and device, terminal equipment and storage medium
CN109409296B (en) * 2018-10-30 2020-12-01 河北工业大学 Video emotion recognition method integrating facial expression recognition and voice emotion recognition
CN109801349B (en) * 2018-12-19 2023-01-24 武汉西山艺创文化有限公司 Sound-driven three-dimensional animation character real-time expression generation method and system
CN109829431B (en) * 2019-01-31 2021-02-12 北京字节跳动网络技术有限公司 Method and apparatus for generating information
CN110147737B (en) * 2019-04-25 2021-06-18 北京百度网讯科技有限公司 Method, apparatus, device and storage medium for generating video
CN110516696B (en) * 2019-07-12 2023-07-25 东南大学 Self-adaptive weight bimodal fusion emotion recognition method based on voice and expression
CN110381266A (en) * 2019-07-31 2019-10-25 百度在线网络技术（北京）有限公司 A kind of video generation method, device and terminal
CN110677598B (en) * 2019-09-18 2022-04-12 北京市商汤科技开发有限公司 Video generation method and device, electronic equipment and computer storage medium

Also Published As

Publication number Publication date
US20210357625A1 (en) 2021-11-18
CN110677598B (en) 2022-04-12
JP2022526148A (en) 2022-05-23
WO2021052224A1 (en) 2021-03-25
KR20210140762A (en) 2021-11-23
CN110677598A (en) 2020-01-10

Similar Documents

Publication Publication Date Title
SG11202108498RA (en) Method and device for generating video, electronic equipment, and computer storage medium
GB2593005B (en) Video generating method, apparatus, electronic device and computer storage medium
SG11202010145WA (en) Video repair method and apparatus, electronic device, and storage medium
GB202017755D0 (en) Video photographing method and apparatus, electronic device and computer readable storage medium
GB2600309B (en) Video processing method and apparatus, and electronic device and storage medium
EP4044616A4 (en) Method and apparatus for generating video, electronic device, and computer readable medium
SG11202103527XA (en) Interactive plot implementation method, device, computer apparatus, and storage medium
SG11202008134YA (en) Method and device for video processing, electronic device, and storage medium
SG11202010572RA (en) Image generating method and apparatus, electronic device, and storage medium
SG11202012469TA (en) Image generation method, device, electronic apparatus, and storage medium
EP3893513A4 (en) Video stitching method and apparatus, electronic device, and computer storage medium
SG11202102262YA (en) Method and device for previewing video, electronic equipment, and computer-readable storage medium
SG11201913916QA (en) Question data generation method and apparatus, computer device, and storage medium
SG11202106271XA (en) Image processing method and device, electronic equipment and storage medium
SG11202003999QA (en) Video summary generation method and apparatus, electronic device, and computer storage medium
EP3780637A4 (en) Webpage video playback method and apparatus, electronic device and storage medium
EP3886436A4 (en) Video encoding method and apparatus, electronic device, and computer readable storage medium
EP4262221A4 (en) Video generation method and apparatus, electronic device, and storage medium
SG11202103326QA (en) Video cutting method and apparatus, computer device and storage medium
SG11202011781UA (en) Video processing method, apparatus, electronic device and storage medium
SG11202004541WA (en) Chatbot configuration method and apparatus, computer device, and storage medium
EP4184926A4 (en) Video interaction methods and apparatus, electronic device, and storage medium
SG11202106680UA (en) Method and device for image processing, processor, electronic equipment and storage medium
SG11202106335SA (en) Video processing method and apparatus, electronic device, and storage medium
GB202101362D0 (en) Video playback method, electronic device, and storage medium