WO2009087641A3 - System and method for real-time super-resolution image reconstruction - Google Patents

System and method for real-time super-resolution image reconstruction Download PDF

Info

Publication number
WO2009087641A3
WO2009087641A3 PCT/IL2009/000045 IL2009000045W WO2009087641A3 WO 2009087641 A3 WO2009087641 A3 WO 2009087641A3 IL 2009000045 W IL2009000045 W IL 2009000045W WO 2009087641 A3 WO2009087641 A3 WO 2009087641A3
Authority
WO
WIPO (PCT)
Prior art keywords
representations
reference representation
resolution image
image reconstruction
sampled
Prior art date
Application number
PCT/IL2009/000045
Other languages
French (fr)
Other versions
WO2009087641A2 (en
Inventor
Barak Fishbain
Lanir Ideses
Leonid Yaroslavsky
Original Assignee
Ramot At Tel-Aviv University Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ramot At Tel-Aviv University Ltd. filed Critical Ramot At Tel-Aviv University Ltd.
Publication of WO2009087641A2 publication Critical patent/WO2009087641A2/en
Publication of WO2009087641A3 publication Critical patent/WO2009087641A3/en
Priority to US12/833,973 priority Critical patent/US20100272184A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G06T3/4069—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution by subpixel displacements

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

A method and system are presented for real time Super-Resolution image reconstruction. Video data compressed by motion compensated compression technique is processed, and representations of video objects (VOs) appearing in frames of said video are obtained. At least one of these representations is utilized as reference and motion vectors, associating said representations with said reference, are obtained from said video data. The representations and the motion vectors are processed, and pixel displacement maps are generated, each associating at least some pixels of one of the representations with locations on said at least one reference representation. The reference representation is re-sampled according to the sub-pixel accuracy of the displacement maps, and a re-sampled reference representation is obtained. Pixels of said representations are registered against the re-sampled reference representation according to the displacement maps, thereby providing super-resolved image of the reference representation of said one or more VOs.
PCT/IL2009/000045 2008-01-10 2009-01-11 System and method for real-time super-resolution WO2009087641A2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US12/833,973 US20100272184A1 (en) 2008-01-10 2010-07-10 System and Method for Real-Time Super-Resolution

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US2030508P 2008-01-10 2008-01-10
US61/020,305 2008-01-10

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US12/833,973 Continuation US20100272184A1 (en) 2008-01-10 2010-07-10 System and Method for Real-Time Super-Resolution

Publications (2)

Publication Number Publication Date
WO2009087641A2 WO2009087641A2 (en) 2009-07-16
WO2009087641A3 true WO2009087641A3 (en) 2010-03-11

Family

ID=40853537

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IL2009/000045 WO2009087641A2 (en) 2008-01-10 2009-01-11 System and method for real-time super-resolution

Country Status (2)

Country Link
US (1) US20100272184A1 (en)
WO (1) WO2009087641A2 (en)

Families Citing this family (48)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4525692B2 (en) * 2007-03-27 2010-08-18 株式会社日立製作所 Image processing apparatus, image processing method, and image display apparatus
JP4356777B2 (en) * 2007-06-18 2009-11-04 ソニー株式会社 Image processing apparatus, image processing method, program, and recording medium
US8290212B2 (en) * 2007-07-17 2012-10-16 Lockheed Martin Corporation Super-resolving moving vehicles in an unregistered set of video frames
ATE547775T1 (en) * 2009-08-21 2012-03-15 Ericsson Telefon Ab L M METHOD AND DEVICE FOR ESTIMATING INTERFRAME MOTION FIELDS
SE0901125A1 (en) 2009-08-27 2010-11-16 Flir Systems Ab Method for compensating turbulence when recording movie sequences
KR101791919B1 (en) 2010-01-22 2017-11-02 톰슨 라이센싱 Data pruning for video compression using example-based super-resolution
CN102823242B (en) 2010-01-22 2016-08-10 汤姆森特许公司 Based on sampling super-resolution Video coding and the method and apparatus of decoding
US8665367B2 (en) * 2010-01-29 2014-03-04 Sharp Laboratories Of America, Inc. Video resolution enhancement technique
EP2410746A1 (en) * 2010-07-20 2012-01-25 Siemens Aktiengesellschaft Video coding with reference frames of high resolution
JP6042813B2 (en) * 2010-09-10 2016-12-14 トムソン ライセンシングＴｈｏｍｓｏｎ Ｌｉｃｅｎｓｉｎｇ Method and apparatus for encoding video signals using motion compensated case-based super-resolution for video compression
WO2012033970A1 (en) 2010-09-10 2012-03-15 Thomson Licensing Encoding of a picture in a video sequence by example - based data pruning using intra- frame patch similarity
US9544598B2 (en) 2010-09-10 2017-01-10 Thomson Licensing Methods and apparatus for pruning decision optimization in example-based data pruning compression
US9554142B2 (en) 2011-01-28 2017-01-24 Eye IO, LLC Encoding of video stream based on scene type
CN103493481A (en) * 2011-01-28 2014-01-01 艾艾欧有限公司 Adaptive bit rate control based on scenes
CN103167218B (en) * 2011-12-14 2016-04-06 北京大学 A kind of super resolution ratio reconstruction method based on non-locality and equipment
US8976254B2 (en) * 2012-06-08 2015-03-10 Apple Inc. Temporal aliasing reduction and coding of upsampled video
CN103533212A (en) * 2012-07-04 2014-01-22 腾讯科技（深圳）有限公司 Image synthesizing method and apparatus
US9185437B2 (en) 2012-11-01 2015-11-10 Microsoft Technology Licensing, Llc Video data
DE102013114996A1 (en) * 2013-01-07 2014-07-10 GM Global Technology Operations LLC (n. d. Gesetzen des Staates Delaware) Method for applying super-resolution to images detected by camera device of vehicle e.g. motor car, involves applying spatial super-resolution to area-of-interest within frame to increase the image sharpness within area-of-interest
US9336574B2 (en) * 2013-01-07 2016-05-10 GM Global Technology Operations LLC Image super-resolution for dynamic rearview mirror
JP5738904B2 (en) * 2013-01-28 2015-06-24 オリンパス株式会社 Image processing apparatus, imaging apparatus, image processing method, and program
US9668019B2 (en) 2013-03-15 2017-05-30 Samsung Electronics Co., Ltd. Electronic system with adaptive enhancement mechanism and method of operation thereof
US9445124B2 (en) 2013-03-15 2016-09-13 Samsung Electronics Co., Ltd. Electronic system with frequency mechanism and method of operation thereof
CN104103052B (en) * 2013-04-11 2017-08-04 北京大学 A kind of image super-resolution rebuilding method based on rarefaction representation
US9196021B2 (en) * 2013-05-29 2015-11-24 Adobe Systems Incorporated Video enhancement using related content
US9418398B2 (en) * 2014-06-12 2016-08-16 Samsung Electronics Co., Ltd. Low power subpixel rendering on RGBW display
CN105447824B (en) * 2014-08-26 2018-05-15 北京大学 Image processing method and device
CN105488776B (en) * 2014-10-10 2018-05-08 北京大学 Super-resolution image reconstruction method and device
CN105590304B (en) * 2014-10-24 2019-01-08 北京大学 Super-resolution image reconstruction method and device
WO2016145406A1 (en) 2015-03-11 2016-09-15 Massachusetts Institute Of Technology Methods and apparatus for modeling deformations of an object
WO2017204886A1 (en) * 2016-05-23 2017-11-30 Massachusetts Institute Of Technology System and method for providing real-time super-resolution for compressed videos
US10380745B2 (en) * 2016-09-01 2019-08-13 Massachusetts Institute Of Technology Methods and devices for measuring object motion using camera images
US11032494B2 (en) * 2016-09-28 2021-06-08 Versitech Limited Recovery of pixel resolution in scanning imaging
US10924755B2 (en) * 2017-10-19 2021-02-16 Arizona Board Of Regents On Behalf Of Arizona State University Real time end-to-end learning system for a high frame rate video compressive sensing network
JP7223117B2 (en) * 2018-08-09 2023-02-15 グーグル エルエルシー Super-resolution using natural hand-held motion applied to the user device
US11468542B2 (en) 2019-01-18 2022-10-11 Arizona Board Of Regents On Behalf Of Arizona State University LAPRAN: a scalable Laplacian pyramid reconstructive adversarial network for flexible compressive sensing reconstruction
JP7304418B2 (en) 2019-02-06 2023-07-06 ウィリアム イー バトラー， Spatio-temporal reconstruction of moving vascular pulse waves from multiple low-dimensional angiographic projections
GB2596702B (en) 2019-04-04 2023-03-01 E Butler William Intrinsic contrast optical cross-correlated wavelet angiography
CN110415172B (en) * 2019-07-10 2023-03-17 武汉大学苏州研究院 Super-resolution reconstruction method for face area in mixed resolution code stream
CN111083359B (en) * 2019-12-06 2021-06-25 Oppo广东移动通信有限公司 Image processing method and apparatus, electronic device, and computer-readable storage medium
CN111246250B (en) * 2020-03-13 2022-07-01 广州市百果园信息技术有限公司 Video processing method, device, equipment and storage medium
US11777520B2 (en) 2020-03-31 2023-10-03 Arizona Board Of Regents On Behalf Of Arizona State University Generic compression ratio adapter for end-to-end data-driven compressive sensing reconstruction frameworks
US11763165B2 (en) 2020-05-11 2023-09-19 Arizona Board Of Regents On Behalf Of Arizona State University Selective sensing: a data-driven nonuniform subsampling approach for computation-free on-sensor data dimensionality reduction
WO2021248349A1 (en) * 2020-06-10 2021-12-16 Plantronics, Inc. Combining high-quality foreground with enhanced low-quality background
CN113132647B (en) * 2021-04-25 2023-01-20 京东方科技集团股份有限公司 Data processing method, data processing device, storage medium and electronic equipment
US20220361834A1 (en) * 2021-05-12 2022-11-17 Angiowave Imaging, Llc Motion-compensated wavelet angiography
CN113066014B (en) * 2021-05-19 2022-09-02 云南电网有限责任公司电力科学研究院 Image super-resolution method and device
CN113822801B (en) * 2021-06-28 2023-08-18 浙江工商大学 Compressed video super-resolution reconstruction method based on multi-branch convolutional neural network

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6728317B1 (en) * 1996-01-30 2004-04-27 Dolby Laboratories Licensing Corporation Moving image compression quality enhancement using displacement filters with negative lobes
US6735253B1 (en) * 1997-05-16 2004-05-11 The Trustees Of Columbia University In The City Of New York Methods and architecture for indexing and editing compressed video over the world wide web
US20060157574A1 (en) * 2004-12-21 2006-07-20 Canon Kabushiki Kaisha Printed data storage and retrieval

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6782132B1 (en) * 1998-08-12 2004-08-24 Pixonics, Inc. Video coding and reconstruction apparatus and methods
US8279341B1 (en) * 2007-02-26 2012-10-02 MotionDSP, Inc. Enhancing the resolution and quality of sequential digital images

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6728317B1 (en) * 1996-01-30 2004-04-27 Dolby Laboratories Licensing Corporation Moving image compression quality enhancement using displacement filters with negative lobes
US6735253B1 (en) * 1997-05-16 2004-05-11 The Trustees Of Columbia University In The City Of New York Methods and architecture for indexing and editing compressed video over the world wide web
US20060157574A1 (en) * 2004-12-21 2006-07-20 Canon Kabushiki Kaisha Printed data storage and retrieval

Also Published As

Publication number Publication date
US20100272184A1 (en) 2010-10-28
WO2009087641A2 (en) 2009-07-16

Similar Documents

Publication Publication Date Title
WO2009087641A3 (en) System and method for real-time super-resolution image reconstruction
RU2009130114A (en) DEVICE AND METHOD FOR PREPARING IMAGES WITH MORE HIGH RESOLUTION IN THE BUILT-IN DEVICE
EP1995972A3 (en) Sub-pixel resolution motion refinement for video encoding
WO2010101789A3 (en) Multi-modal tone-mapping of images
MX343351B (en) Generating subpixel values for different color sampling formats.
EP4401404A3 (en) Method, apparatus and device for obtaining motion information of video images and template construction method
EP2571249A3 (en) Image synthesizing device, image synthesizing method and computer readable medium
EP1708138A3 (en) Image processing method, three-dimensional position measuring method and image processing apparatus
WO2007015746A3 (en) Method of rendering pixel images from abstract datasets
EP2299703A3 (en) System and method for producing high definition video from low definition video
WO2012019417A8 (en) Device, system and method for online video condensation
EP2445189A3 (en) Information processing apparatus and method thereof
EP2570989A3 (en) Resolution and contrast enhancement with fusion in low-resolution IR images
WO2007087640A3 (en) Adaptive filtering to enhance video encoder performance
WO2009102480A3 (en) System and method for multi-resolution storage of images
EP1978746A3 (en) Method, medium and apparatus effectively compressing and restoring edge position images
EP2306399A3 (en) Image processing method, apparatus and system
EP1796044A3 (en) Image processing apparatus and method, recording medium, and program
WO2007124360A3 (en) Image stabilization method
EP2012532A3 (en) Video displaying apparatus, video signal processing apparatus and video signal processing method
JP2013537380A5 (en)
US20170206633A1 (en) Method and apparatus for up-scaling an image
EP2869551A3 (en) Super-resolution in processing images such as from multi-layer sensors
MY201632A (en) Projection image construction method and device
TW200943973A (en) Dynamic image processing device, dynamic image processing method, and dynamic image processing program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09700867

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 09700867

Country of ref document: EP

Kind code of ref document: A2