DE112018004717T5 - System für audiovisuelle Effekte zur Erweiterung einer aufgenommenen Darbietung basierend auf deren Inhalt - Google Patents

System für audiovisuelle Effekte zur Erweiterung einer aufgenommenen Darbietung basierend auf deren Inhalt Download PDF

Info

Publication number: DE112018004717T5
Authority: DE; Germany
Prior art keywords: audiovisual; performance; visual effects; procedure according; vocal
Prior art date: 2017-08-21
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.): Pending

Application number

DE112018004717.2T

Other languages

German (de)

English (en)

Inventor

David Steinwedel

Perry R. Cook

Paul T. Chi

Wei Zhou

Jon Moldover

Anton Holmberg

Jingxi Li

Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)

Smule Inc

Original Assignee

Smule Inc

Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)

2017-08-21

Filing date

2018-08-21

Publication date

2020-06-10

2018-08-21 Application filed by Smule Inc filed Critical Smule Inc

2020-06-10 Publication of DE112018004717T5 publication Critical patent/DE112018004717T5/de

Status Pending legal-status Critical Current

Links

230000000694 effects Effects 0.000 title claims description 44
230000001755 vocal effect Effects 0.000 claims abstract description 158
230000000007 visual effect Effects 0.000 claims abstract description 151
238000000034 method Methods 0.000 claims abstract description 77
230000011218 segmentation Effects 0.000 claims abstract description 29
230000007704 transition Effects 0.000 claims description 25
230000036651 mood Effects 0.000 claims description 24
238000004891 communication Methods 0.000 claims description 22
239000002245 particle Substances 0.000 claims description 9
230000001360 synchronised effect Effects 0.000 claims description 8
230000033001 locomotion Effects 0.000 claims description 6
238000009877 rendering Methods 0.000 claims description 5
238000002156 mixing Methods 0.000 claims description 4
230000005236 sound signal Effects 0.000 claims description 4
238000004458 analytical method Methods 0.000 claims description 3
238000004590 computer program Methods 0.000 claims description 3
230000008859 change Effects 0.000 claims description 2
230000008451 emotion Effects 0.000 claims description 2
230000007246 mechanism Effects 0.000 claims description 2
230000004044 response Effects 0.000 claims description 2
238000010801 machine learning Methods 0.000 abstract description 2
239000011295 pitch Substances 0.000 description 57
238000012937 correction Methods 0.000 description 22
239000000203 mixture Substances 0.000 description 18
238000012545 processing Methods 0.000 description 14
230000033458 reproduction Effects 0.000 description 13
230000001413 cellular effect Effects 0.000 description 10
230000008901 benefit Effects 0.000 description 9
238000010586 diagram Methods 0.000 description 6
241001342895 Chorus Species 0.000 description 5
230000005540 biological transmission Effects 0.000 description 5
238000004422 calculation algorithm Methods 0.000 description 5
HAORKNGNJCEJBX-UHFFFAOYSA-N cyprodinil Chemical compound N=1C(C)=CC(C2CC2)=NC=1NC1=CC=CC=C1 HAORKNGNJCEJBX-UHFFFAOYSA-N 0.000 description 5
238000000605 extraction Methods 0.000 description 5
239000002131 composite material Substances 0.000 description 4
230000003993 interaction Effects 0.000 description 4
238000012384 transportation and delivery Methods 0.000 description 4
230000015572 biosynthetic process Effects 0.000 description 3
230000006870 function Effects 0.000 description 3
230000002452 interceptive effect Effects 0.000 description 3
238000012546 transfer Methods 0.000 description 3
230000006978 adaptation Effects 0.000 description 2
238000007792 addition Methods 0.000 description 2
239000003086 colorant Substances 0.000 description 2
230000006835 compression Effects 0.000 description 2
238000007906 compression Methods 0.000 description 2
239000000470 constituent Substances 0.000 description 2
238000011161 development Methods 0.000 description 2
230000018109 developmental process Effects 0.000 description 2
238000003384 imaging method Methods 0.000 description 2
230000006872 improvement Effects 0.000 description 2
230000001404 mediated effect Effects 0.000 description 2
238000010295 mobile communication Methods 0.000 description 2
238000012986 modification Methods 0.000 description 2
230000004048 modification Effects 0.000 description 2
238000013439 planning Methods 0.000 description 2
238000012805 post-processing Methods 0.000 description 2
230000011664 signaling Effects 0.000 description 2
230000002123 temporal effect Effects 0.000 description 2
206010041243 Social avoidant behaviour Diseases 0.000 description 1
METKIMKYRPQLGS-UHFFFAOYSA-N atenolol Chemical compound CC(C)NCC(O)COC1=CC=C(CC(N)=O)C=C1 METKIMKYRPQLGS-UHFFFAOYSA-N 0.000 description 1
230000004888 barrier function Effects 0.000 description 1
230000002457 bidirectional effect Effects 0.000 description 1
238000005352 clarification Methods 0.000 description 1
238000012790 confirmation Methods 0.000 description 1
230000003111 delayed effect Effects 0.000 description 1
238000001514 detection method Methods 0.000 description 1
238000005516 engineering process Methods 0.000 description 1
230000003203 everyday effect Effects 0.000 description 1
230000001771 impaired effect Effects 0.000 description 1
238000007726 management method Methods 0.000 description 1
238000013507 mapping Methods 0.000 description 1
239000003550 marker Substances 0.000 description 1
239000000463 material Substances 0.000 description 1
239000011317 mixed pitch Substances 0.000 description 1
238000003032 molecular docking Methods 0.000 description 1
230000006855 networking Effects 0.000 description 1
230000003287 optical effect Effects 0.000 description 1
230000001151 other effect Effects 0.000 description 1
230000008569 process Effects 0.000 description 1
230000003252 repetitive effect Effects 0.000 description 1
238000011160 research Methods 0.000 description 1
238000000926 separation method Methods 0.000 description 1
238000012163 sequencing technique Methods 0.000 description 1
238000003786 synthesis reaction Methods 0.000 description 1

Images

Classifications

- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440236—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
- H04N7/147—Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
- H04N7/152—Multipoint control units therefor
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
- H04N7/142—Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
- H04N2007/145—Handheld terminals

Landscapes

Engineering & Computer Science (AREA)
Multimedia (AREA)
Signal Processing (AREA)
Auxiliary Devices For Music (AREA)
Reverberation, Karaoke And Other Acoustics (AREA)

DE112018004717.2T 2017-08-21 2018-08-21 System für audiovisuelle Effekte zur Erweiterung einer aufgenommenen Darbietung basierend auf deren Inhalt Pending DE112018004717T5 (de)

Applications Claiming Priority (3)

Application Number	Priority Date	Filing Date	Title
US201762548122P	2017-08-21	2017-08-21
US62/548,122		2017-08-21
PCT/US2018/047325 WO2019040492A1 (en)	2017-08-21	2018-08-21	AUDIOVISUAL EFFECT SYSTEM FOR INCREASING CAPTURED INTERPRETATION BASED ON ITS CONTENT

Publications (1)

Publication Number	Publication Date
DE112018004717T5 true DE112018004717T5 (de)	2020-06-10

Family

ID=65439230

Family Applications (1)

Application Number	Title	Priority Date	Filing Date
DE112018004717.2T Pending DE112018004717T5 (de)	2017-08-21	2018-08-21	System für audiovisuelle Effekte zur Erweiterung einer aufgenommenen Darbietung basierend auf deren Inhalt

Country Status (3)

Country	Link
CN (1)	CN111345044B (zh)
DE (1)	DE112018004717T5 (zh)
WO (1)	WO2019040492A1 (zh)

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
JP2006047754A (ja) *	2004-08-05	2006-02-16	Namco Ltd	カラオケ情報配信システム、プログラム、情報記憶媒体およびカラオケ情報配信方法
US8370747B2 (en) *	2006-07-31	2013-02-05	Sony Mobile Communications Ab	Method and system for adapting a visual user interface of a mobile radio terminal in coordination with music
JP2010060627A (ja) *	2008-09-01	2010-03-18	Bmb Corp	カラオケシステム
US20110126103A1 (en) *	2009-11-24	2011-05-26	Tunewiki Ltd.	Method and system for a "karaoke collage"
US9147385B2 (en) *	2009-12-15	2015-09-29	Smule, Inc.	Continuous score-coded pitch correction
US9866731B2 (en) *	2011-04-12	2018-01-09	Smule, Inc.	Coordinating and mixing audiovisual content captured from geographically distributed performers
KR101377208B1 (ko) *	2012-08-17	2014-03-21	주식회사 디자인피버	합성 영상을 이용한 노래방 시스템
KR20150033757A (ko) *	2013-09-23	2015-04-02	조경환	휴대폰어플이용 노래방 텔레비젼 시스템
CN104580838A (zh) *	2015-01-27	2015-04-29	苏州乐聚一堂电子科技有限公司	演唱视觉特效***及演唱视觉特效处理方法

2018
- 2018-08-21 CN CN201880054029.4A patent/CN111345044B/zh active Active
- 2018-08-21 DE DE112018004717.2T patent/DE112018004717T5/de active Pending
- 2018-08-21 WO PCT/US2018/047325 patent/WO2019040492A1/en active Application Filing

Also Published As

Publication number	Publication date
CN111345044B (zh)	2023-03-21
WO2019040492A1 (en)	2019-02-28
CN111345044A (zh)	2020-06-26

Publication	Publication Date	Title
US11862198B2 (en)	2024-01-02	Synthesizing a presentation from multiple media clips
US11488569B2 (en)	2022-11-01	Audio-visual effects system for augmentation of captured performance based on content thereof
DE112018001871T5 (de)	2020-02-27	Audiovisuelles Kollaborationsverfahren mit Latenzverwaltung für großflächige Übertragung
US11310538B2 (en)	2022-04-19	Audiovisual collaboration system and method with latency management for wide-area broadcast and social media-type user interface mechanics
KR20180027423A (ko)	2018-03-14	지리적으로 분산된 연주자로부터 캡처된 콘텐츠에 기반하여 조율된 오디오비주얼 작품을 자동 생성하는 기법
US20220051448A1 (en)	2022-02-17	Augmented reality filters for captured audiovisual performances
US11900967B2 (en)	2024-02-13	Template-based excerpting and rendering of multimedia performance
US8782176B2 (en)	2014-07-15	Synchronized video system
US20220122573A1 (en)	2022-04-21	Augmented Reality Filters for Captured Audiovisual Performances
US11848032B2 (en)	2023-12-19	User-generated templates for segmented multimedia performance
US20230095431A1 (en)	2023-03-30	Template-based excerpting and rendering of multimedia performance
DE112018004717T5 (de)	2020-06-10	System für audiovisuelle Effekte zur Erweiterung einer aufgenommenen Darbietung basierend auf deren Inhalt