US20120249761A1 - Motion Picture Personalization by Face and Voice Image Replacement - Google Patents

Motion Picture Personalization by Face and Voice Image Replacement Download PDF

Info

Publication number
US20120249761A1
US20120249761A1 US13/078,955 US201113078955A US2012249761A1 US 20120249761 A1 US20120249761 A1 US 20120249761A1 US 201113078955 A US201113078955 A US 201113078955A US 2012249761 A1 US2012249761 A1 US 2012249761A1
Authority
US
United States
Prior art keywords
voice
person
face
motion picture
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US13/078,955
Inventor
Joonbum Byun
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to US13/078,955 priority Critical patent/US20120249761A1/en
Publication of US20120249761A1 publication Critical patent/US20120249761A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • the disclosed art enables the customization of existing movie, drama, documentary film or concert video by replacing main character or player's face and voice image with your face and voice. From now on, for the shake of argument, we will call a person “client” who wishes his or her face and voice image be embedded in a motion picture.
  • Client's face and voice image must be first captured. If an existing movie is in traditional film, it is digitally scanned into computer video and audio image.
  • a computer software an embodiment of current invention, examine the video frame by frame to recognize each character's face from the video image and arrange them in such as way that it can be easily replaced with client's face image. Once it is done, only the face image in the video is replaced with client's face image frame by frame.
  • the software captures actor's subtle facial expression and uses it to render client's facial image that corresponds to actor's facial expression.
  • the voice also goes through similar process. Words, language, accent in actor's or singer's voice remain unchanged. But only the characteristic of the voice is altered to that of client's so that it sounds like client is talking in the video.
  • Current invention is related with motion picture, computer graphic and personalized entertainment.
  • FIG. 1 shows overall flow of motion picture images from original movie to personalized one. If original video is in film format, it is converted into digital format by digital scanner and stored in digital computer. If original motion picture is in digital format, it is used as is. Client's face and voice image are captured. Face image includes as many expression as possible such as when client is happy, angry, sad, laughing, talking, sleeping.
  • FIG. 2 shows another embodiment of current invention, which inspects each frame by frame to recognize human face image and organize them into a data structure such that they can be later easily replaced by client's face image. It also alters actor's voice in the video similar to client's one while leaving the words, accent and language without change.
  • the software replaces the actor's face image with client's image frame by frame. It senses actor's expression from the facial image and transplant them into client's face image frame by frame. Running through whole video from start to beginning, it completes a movie personalized to a specific client.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Processing Or Creating Images (AREA)

Abstract

Disclosed invention enables ordinary person who are not non-professional movie star or singer to take a role in motion picture video. It replaces facial and voice image in existing video with someone else's facial and voice. The words, emotion, expression and feelings shown in original video are fused into that of non-professionals so that the normal person is playing a role in a movie just like the actual movie star or singer does.

Description

    BACKGROUND OF THE INVENTION
  • There is no doubt that everybody want to be a movie star or famous singer. In reality, not all the wishes do not come true. However, thanks to advancement in computer graphics technology, the wish can be realized at least in motion picture. In a historic hit movie such as “Gone with the Wind” or “Star Wars”, wouldn't it be great if you were a main character? Or each of your family members could take an appropriate role in such a movie.
  • Current invention enables the dream come true at least in motion picture.
    • BRIEF SUMMARY OF THE INVENTION
  • The disclosed art enables the customization of existing movie, drama, documentary film or concert video by replacing main character or player's face and voice image with your face and voice. From now on, for the shake of argument, we will call a person “client” who wishes his or her face and voice image be embedded in a motion picture.
  • Client's face and voice image must be first captured. If an existing movie is in traditional film, it is digitally scanned into computer video and audio image. A computer software, an embodiment of current invention, examine the video frame by frame to recognize each character's face from the video image and arrange them in such as way that it can be easily replaced with client's face image. Once it is done, only the face image in the video is replaced with client's face image frame by frame. The software captures actor's subtle facial expression and uses it to render client's facial image that corresponds to actor's facial expression.
  • The voice also goes through similar process. Words, language, accent in actor's or singer's voice remain unchanged. But only the characteristic of the voice is altered to that of client's so that it sounds like client is talking in the video.
    • FIELD OF INVENTION
  • Current invention is related with motion picture, computer graphic and personalized entertainment.
    • DESCRIPTION OF RELATED ART Detailed Description of Invention Brief Description of Drawing
  • FIG. 1 shows overall flow of motion picture images from original movie to personalized one. If original video is in film format, it is converted into digital format by digital scanner and stored in digital computer. If original motion picture is in digital format, it is used as is. Client's face and voice image are captured. Face image includes as many expression as possible such as when client is happy, angry, sad, laughing, talking, sleeping.
  • FIG. 2 shows another embodiment of current invention, which inspects each frame by frame to recognize human face image and organize them into a data structure such that they can be later easily replaced by client's face image. It also alters actor's voice in the video similar to client's one while leaving the words, accent and language without change.
    •
  • The software replaces the actor's face image with client's image frame by frame. It senses actor's expression from the facial image and transplant them into client's face image frame by frame. Running through whole video from start to beginning, it completes a movie personalized to a specific client.

Claims (1)

1. An algorithm that replace person A's face and voice image in a video or motion picture with person B's face and voice image while having the person A's facial expression, emotion and speaking expression transplanted to person B's facial image and talking that;
converts still image of person B's face into person B's live face with person A's facial expression and emotion.
transforms person A's voice in motion picture into person B's voice by applying person B's voice characteristics into person A's voice.
US13/078,955 2011-04-02 2011-04-02 Motion Picture Personalization by Face and Voice Image Replacement Abandoned US20120249761A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US13/078,955 US20120249761A1 (en) 2011-04-02 2011-04-02 Motion Picture Personalization by Face and Voice Image Replacement

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US13/078,955 US20120249761A1 (en) 2011-04-02 2011-04-02 Motion Picture Personalization by Face and Voice Image Replacement

Publications (1)

Publication Number Publication Date
US20120249761A1 true US20120249761A1 (en) 2012-10-04

Family

ID=46926721

Family Applications (1)

Application Number Title Priority Date Filing Date
US13/078,955 Abandoned US20120249761A1 (en) 2011-04-02 2011-04-02 Motion Picture Personalization by Face and Voice Image Replacement

Country Status (1)

Country Link
US (1) US20120249761A1 (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104008296A (en) * 2014-06-08 2014-08-27 蒋小辉 Method for converting video into game, video game and achieving method thereof
CN106295568A (en) * 2016-08-11 2017-01-04 上海电力学院 The mankind's naturalness emotion identification method combined based on expression and behavior bimodal
CN109308466A (en) * 2018-09-18 2019-02-05 宁波众鑫网络科技股份有限公司 The method that a kind of pair of interactive language carries out Emotion identification
CN109618223A (en) * 2019-01-28 2019-04-12 北京易捷胜科技有限公司 A kind of sound replacement method
CN112423081A (en) * 2020-11-09 2021-02-26 腾讯科技（深圳）有限公司 Video data processing method, device and equipment and readable storage medium
CN113160799A (en) * 2021-04-22 2021-07-23 北京房江湖科技有限公司 Video generation method and device, computer-readable storage medium and electronic equipment
CN113411655A (en) * 2021-05-18 2021-09-17 北京达佳互联信息技术有限公司 Method and device for generating video on demand, electronic equipment and storage medium

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090135177A1 (en) * 2007-11-20 2009-05-28 Big Stage Entertainment, Inc. Systems and methods for voice personalization of video content

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090135177A1 (en) * 2007-11-20 2009-05-28 Big Stage Entertainment, Inc. Systems and methods for voice personalization of video content

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104008296A (en) * 2014-06-08 2014-08-27 蒋小辉 Method for converting video into game, video game and achieving method thereof
CN106295568A (en) * 2016-08-11 2017-01-04 上海电力学院 The mankind's naturalness emotion identification method combined based on expression and behavior bimodal
CN109308466A (en) * 2018-09-18 2019-02-05 宁波众鑫网络科技股份有限公司 The method that a kind of pair of interactive language carries out Emotion identification
CN109618223A (en) * 2019-01-28 2019-04-12 北京易捷胜科技有限公司 A kind of sound replacement method
CN112423081A (en) * 2020-11-09 2021-02-26 腾讯科技（深圳）有限公司 Video data processing method, device and equipment and readable storage medium
CN113160799A (en) * 2021-04-22 2021-07-23 北京房江湖科技有限公司 Video generation method and device, computer-readable storage medium and electronic equipment
CN113411655A (en) * 2021-05-18 2021-09-17 北京达佳互联信息技术有限公司 Method and device for generating video on demand, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
US20120249761A1 (en) Motion Picture Personalization by Face and Voice Image Replacement
CN101356800B (en) Content imaging apparatus
US10541000B1 (en) User input-based video summarization
KR101492359B1 (en) Input support device, input support method, and recording medium
CN111480156A (en) System and method for selectively storing audiovisual content using deep learning
US20140192140A1 (en) Visual Content Modification for Distributed Story Reading
US20140195222A1 (en) Speech Modification for Distributed Story Reading
EP3050604A1 (en) Location based augmentation for story reading
WO2011099299A1 (en) Video extraction device, image capturing apparatus, program, and recording medium
US20170242833A1 (en) Systems and Methods to Generate Comic Books or Graphic Novels from Videos
JP2016118991A (en) Image generation device, image generation method, and program
CN110677707A (en) Interactive video generation method, generation device, equipment and readable medium
US20150281586A1 (en) Method and apparatus for forming a video sequence
JP2011523821A (en) Apparatus and method for adjusting audiovisual system to viewer's attention level
US20230156245A1 (en) Systems and methods for processing and presenting media data to allow virtual engagement in events
US11183219B2 (en) Movies with user defined alternate endings
JP2007026090A (en) Video preparation device
JP5310682B2 (en) Karaoke equipment
KR102398848B1 (en) Content Transforming Apparatus Responsive to Audience Reaction
US20170287521A1 (en) Methods, circuits, devices, systems and associated computer executable code for composing composite content
KR20190111642A (en) Image processing system and method using talking head animation based on the pixel of real picture
US11989231B2 (en) Audio recommendation based on text information and video content
CN110708595A (en) Interactive video generation method, device, equipment and medium based on artificial intelligence
Metz Will the real rhapsodist please stand up?
JP2006039705A (en) Content creation device, content creation method, and content creation system

Legal Events

Date Code Title Description
STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION