US20170085787A1 - Systems, Methods, and Media for Recording an Image Using an Optical Diffuser - Google Patents

Systems, Methods, and Media for Recording an Image Using an Optical Diffuser Download PDF

Info

Publication number
US20170085787A1
US20170085787A1 US15/225,552 US201615225552A US2017085787A1 US 20170085787 A1 US20170085787 A1 US 20170085787A1 US 201615225552 A US201615225552 A US 201615225552A US 2017085787 A1 US2017085787 A1 US 2017085787A1
Authority
US
United States
Prior art keywords
diffuser
image
scene
light representing
function
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US15/225,552
Inventor
Oliver Cossairt
Changyin Zhou
Shree K. Nayar
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Columbia University in the City of New York
Original Assignee
Columbia University in the City of New York
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Columbia University in the City of New York filed Critical Columbia University in the City of New York
Priority to US15/225,552 priority Critical patent/US20170085787A1/en
Publication of US20170085787A1 publication Critical patent/US20170085787A1/en
Assigned to NAVY, SECRETARY OF THE UNITED STATES OF AMERICA reassignment NAVY, SECRETARY OF THE UNITED STATES OF AMERICA CONFIRMATORY LICENSE (SEE DOCUMENT FOR DETAILS). Assignors: COLUMBIA UNIVERSITY
Abandoned legal-status Critical Current

Links

Images

Classifications

    • H04N5/23229
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80Camera processing pipelines; Components thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/73Deblurring; Sharpening
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/0075Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for altering, e.g. increasing, the depth of field or depth of focus
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B5/00Optical elements other than lenses
    • G02B5/02Diffusing elements; Afocal elements
    • G02B5/0273Diffusing elements; Afocal elements characterized by the use
    • G06T5/003
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/67Focus control based on electronic image sensor signals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70Circuitry for compensating brightness variation in the scene
    • H04N23/75Circuitry for compensating brightness variation in the scene by influencing optical camera components
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/95Computational photography systems, e.g. light-field imaging systems
    • H04N23/958Computational photography systems, e.g. light-field imaging systems for extended depth of field imaging
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/61Noise processing, e.g. detecting, correcting, reducing or removing noise the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
    • H04N25/615Noise processing, e.g. detecting, correcting, reducing or removing noise the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4" involving a transfer function modelling the optical system, e.g. optical transfer function [OTF], phase transfer function [PhTF] or modulation transfer function [MTF]
    • H04N5/2254
    • H04N5/23293
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/72Modifying the appearance of television pictures by optical filters or diffusing screens
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10004Still image; Photographic image
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/63Control of cameras or camera modules by using electronic viewfinders

Definitions

  • the disclosed subject matter relates to systems, methods, and media for recording an image using an optical diffuser.
  • DOF depth of field
  • noise there is a fundamental trade-off between depth of field (DOF) and noise.
  • DOF depth of field
  • cameras have a single focal plane, and objects that deviate from this plane are blurred due to defocus.
  • the amount of defocus blur depends on the aperture size and the distance from the focal plane.
  • the aperture size must be decreased, reducing the signal strength of the recorded image as well.
  • stopping down the lens aperture is not always an option, especially in low light conditions, because it can increase noise, which in turn can materially impact the recorded image.
  • systems, methods, and media for recording an image of a scene comprising: a diffuser that diffuses light representing the scene and that has a scattering function that is independent of aperture coordinates; a sensor that receives diffused light representing the scene and generates data representing an image; and a hardware processor that uses a point spread function to deblur the image.
  • methods for recording an image of a scene comprising: diffusing light representing the scene using a diffuser that has a scattering function that is independent of aperture coordinates; receiving diffused light representing the scene and generating data representing an image; and using a point spread function to deblur the image
  • FIG. 1 is a diagram of a mechanism for recording an image in accordance with some embodiments.
  • FIG. 2 is a combination of two images, one not using a diffuser (a) and one using a diffuser (b), in accordance with some embodiments.
  • FIG. 3 is a diagram of a lens and a sensor in accordance with some embodiments.
  • FIG. 4 is a diagram of a lens, a diffuser, and a sensor in accordance with some embodiments.
  • FIG. 5 is a diagram illustrating a light field on a sensor in accordance with some embodiments.
  • FIG. 6 is a diagram of a ray and the scattering of the ray in accordance with some embodiments.
  • FIG. 7 is a combination of point-spread-function and modulation-transfer-function graph pairs in accordance with some embodiments.
  • FIG. 8 is a diagram an optical system including a wedge (a) and a random varying surface in accordance with some embodiments.
  • FIG. 9 is combination of diagrams of a diffuser profile (a), a diffuser height map (b), a diffuser scatter PDF (c), and a diffuser (d) in accordance with some embodiments.
  • FIG. 1 an illustration of an image recording mechanism 102 (e.g., a camera, video camera, mobile phone incorporating a camera, and/or any other suitable image recording mechanism) that is being used to capture an image including three objects, A 104 , B 106 , and C 108 , is shown. As can be seen, these objects are at different depths with respect to mechanism 102 . Because of limitations in the depth of field of mechanism 102 , objects A 104 and C 108 may be out of focus when mechanism 102 is focused on object B 106 . For example, these objects may be the toys shown in FIG. 2 . As illustrated in FIG. 2( a ) , when a camera is focused on the center object (which may correspond to object B 106 of FIG.
  • the center object which may correspond to object B 106 of FIG.
  • the other objects may be out of focus.
  • an image can be recorded of such objects so that they appear to be in focus as illustrated in FIG. 2( b ) .
  • This can be referred to as mechanism 102 having an extended depth of field.
  • extended depth of field can be achieved by incorporating a diffuser 110 or 112 into an image recording mechanism 102 .
  • Recording an image using a diffuser in the pupil plane of an image recording mechanism can be referred to as diffusion coding.
  • a diffuser can be located at any suitable point in the image recording mechanism.
  • a diffuser 110 can be positioned between a light source (e.g., objects 104 , 106 , and 108 ) and a lens 114 (e.g., as a lens attachment), a diffuser 112 can be positioned between a lens 114 and a sensor 116 (e.g., as part of a lens or a camera body), etc.
  • the diffusion coding image can then be detected by sensor 116 and then provided to a hardware processor 118 (incorporated into mechanism 102 ) and/or a hardware processor 120 (external to mechanism 102 ) for subsequent processing.
  • This processing can include deblurring the sensed image using a PSF that is matched to the PSF of the diffuser. Any other suitable processing can additionally or alternatively be used.
  • an extended depth of field image can be presented on a display 124 (internal to mechanism 102 ) and/or a display 122 (external to mechanism 102 ).
  • a light field L( ⁇ , x ) can be used to represent a four-dimensional set of rays propagating from an ideal lens with effective focal length (EFL) f to a sensor.
  • the irradiance E( x ) observed on the sensor can be defined as the light field integrated over all ray angles:
  • the captured image E( x ) can be modeled as a convolution between a depth-dependent PSF kernel P( x ) and an all-in-focus image I( x ).
  • a camera PSF can be shaped so that an image I( x ) can be recovered from the captured image E( x ) by deblurring with a single PSF P( x ).
  • the depth-dependence of the camera PSF can be analyzed by considering the image produced by a unit energy point source. For example, as shown in FIG. 3 , consider a point source whose image comes to focus at a distance d 0 from the aperture of the lens. Assuming a rectangular aperture of width A, the light field produced by this point can be represented as:
  • ⁇ ( x _ w ) ⁇ 1 if ⁇ ⁇ ⁇ x _ i ⁇ ⁇ 1 2 , ⁇ i 0 otherwise . ( 3 )
  • the image of this point is the camera PSF at the depth d 0 , which is a box shaped PSF with defocus blur width s 0 A:
  • the effect of a general kernel D applied to a light field L can next be analyzed.
  • the kernel can produce a new filtered light field ⁇ circumflex over (L) ⁇ , from which the modified PSF ⁇ circumflex over (P) ⁇ can be derived as:
  • Equation 7 For this diffuser kernel, substituting Equation 7 into Equations 5 and 6 gives:
  • Equation 7 ⁇ circle around ( ⁇ ) ⁇ denotes convolution.
  • the modified PSF can be the camera PSF blurred with a box function. Therefore, the effect of the diffuser is to blur the image that would be captured were it not present.
  • the diffuser given by the kernel in Equation 7 may not be useful for extending depth of field because it does not increase depth independence or preserve high frequencies in the camera PSF.
  • the kernel for any diffuser that is placed in the aperture can be represented as:
  • the diffuser has no effect in the ⁇ domain, but has the effect of a convolution in the x domain.
  • the scatter function can be represented as a two-dimensional box function:
  • a slice of L ⁇ ( x , y ) represents a single ray
  • a slice L( ⁇ ,r) represents a 2D set of rays.
  • a slice of the light field represents a conic surface connecting a circle with radius ⁇ in the aperture plane to a circle of radius r on the sensor (see FIG. 5 ).
  • Equation 7 A radially symmetric diffuser produces a drastically different effect than the diffuser given by Equation 7.
  • Equation 7 When a radially symmetric diffuser is introduced, neither the diffuser nor the lens deflects rays tangentially, and therefore the diffuser kernel and modified light field can be represented using the reduced coordinates ( ⁇ ,r). Equations 5 and 6 then become:
  • Equation 13 The same box-shaped scattering function as was used for the diffuser kernel in Equation 7 can be used for Equation 13:
  • this diffuser is different than the previous diffuser.
  • each ray in the light field is scattered so that it spreads across a square on the sensor.
  • the effect of the scattering function in Equation 14, however, is as illustrated in FIG. 6 .
  • light from an annulus of width d ⁇ and radius ⁇ in the aperture plane projects to an annulus of width dr and radius r on the sensor.
  • the effect of the scatter function in Equation 14 is to spread the light incident on the sensor so that it produces an annulus of width w instead.
  • a ray in polar coordinates, can be a small annular section that travels from the aperture plane to the sensor plane.
  • the effect of the diffuser, which is to scatter a ray along a radial line of width w, can be as illustrated by volume 604 .
  • a box-shaped scatter function can be used here for notational convenience, but a Gaussian scattering function (e.g., as illustrated in FIG. 9( c ) ) can be superior for extended DOF imaging.
  • the light field of a point source filtered by this diffuser kernel and PSF can be shown to be:
  • the analytic solution for this PSF is a piecewise function due to the contribution from the term in brackets, which is a convolution between the two rect functions (one weighted by
  • FIG. 7 shows several PSF 702 and Modulation Transfer Function (MTF) 704 graph pairs for a camera with ( 714 , 716 , 718 , 720 , 722 , 724 , 726 , and 728 ) and without ( 715 , 717 , 719 , 721 , 723 , 725 , 727 , and 729 ) the diffuser given by Equation 16.
  • the defocus blur diameter s 0 A changes between 0 pixels 706 , 25 pixels 708 , 50 pixels 710 , and 100 pixels 712 .
  • the diffuser when the diffuser is present, there is little variation with depth for either the PSF or MTF. Introducing the diffuser also eliminates the zero crossings in the MTF. For smaller defocus values, the diffuser suppresses high frequencies in the MTF. However, because the diffuser MTF does not vary significantly with depth, high frequencies can be recovered via deconvolution.
  • diffusers of the “kinoform” type (as described in Caufield, H. J., “Kinoform Diffusers,” In SPIE Conference Series, vol. 25, p. 111, 1971, which is hereby incorporated by reference herein in its entirety) where the scattering effect is caused entirely by roughness variations across a surface
  • Such a diffuser can be considered to be a random phase screen, and according to statistical optics, for a camera with effective focal length f, and center wavelength X, the effect of placing this screen in the aperture of the camera can result in the following:
  • Equation 18 a diffuser can be implemented by creating an optical element with thickness t(u,v), where the gradient of this surface ⁇ t(u,v) is sampled from a probability distribution which is also a desired PSF.
  • this equation can be understood as follows: p ⁇ u , ⁇ v denotes the fraction of the surface t(u,v) with slope ( ⁇ u , ⁇ v ).
  • the quantity p ⁇ u , ⁇ v also reflects the portion of light that will be deflected by the slope ( ⁇ x , ⁇ y ).
  • a Kinoform diffuser has a randomly varying surface with a general probability distribution of slopes as illustrated in FIG. 8( b ) .
  • Kinoform diffusers can be thought of as generalized phase plates.
  • the result of placing this phase plate in the pupil plane of a camera is to shift the PSF, which can be thought of as convolving p( ⁇ u ) with the PSF.
  • the diffuser surface can be implemented as a sequence of quadratic elements whose diameter and sag is drawn from a random distribution as described in Sales, T. R. M., “Structured microlens arrays for beam shaping,” Optical Engineering 42, 11, pp. 3084-3085, 2003, which is hereby incorporated by reference herein in its entirety.
  • a one-dimensional random profile can be created and then a polar transformation applied to create a two-dimensional surface (see, e.g., FIGS. 9( a ) and 9( b ) ).
  • a diffuser can be made using laser etching.
  • the maximum height of the diffuser surface can be 3 ⁇ m, and the diffuser can be fabricated using a laser machining technology which has a minimum spot size of about 10 ⁇ m.
  • the minimum diameter of a single element can be chosen to be 200 ⁇ m, resulting in a diffuser with 42 different annular sections.
  • any suitable hardware can be used to implement a mechanism 102 in accordance with some embodiments.
  • a Canon EOS 450D sensor from Canon U.S.A., Inc. can be used as sensor 116
  • a 22 mm diameter diffuser e.g., as illustrated in FIG. 9( d )
  • a 50 mm f/1.8 lens from Canon U.S.A., Inc. can be used as lens 114
  • lens 114 can have any focal length and consist of refractive optics, reflective optics, or both.
  • a 3048 mm focal length Meade LX200 telescope available from can be used in some embodiments.
  • any suitable processing can be performed to deblur the image hitting a camera sensor after passing through a lens and diffuser (in either order).
  • the Wiener deconvolution with the PSF at the center depth can be used to deblur the sensed images.
  • Any suitable additional or alternative processing on the images can be used.
  • additional deblurring of diffusion coded images can be performed using the BM3D deblurring algorithm as described in Dabov, K., Foi, A., Katkovnik, V., and Egiazarian, K., “Image restoration by sparse 3D transform-domain collaborative filtering,” In SPIE Conference Series, vol. 6812, 681207, 2008, which is hereby incorporated by reference herein in its entirety.
  • the BM3D deblurring algorithm enforces a piecewise smoothness prior that suppresses the noise amplified by the deblurring process.
  • Any suitable hardware processor such as a microprocessor, digital signal processor, special purpose computer (which can include a microprocessor, digital signal processor, a controller, etc., memory, communication interfaces, display controllers, input devices, etc.), general purpose computer suitably programmed (which can include a microprocessor, digital signal processor, a controller, etc., memory, communication interfaces, display controllers, input devices, etc.), server, programmable gate array, etc. can be used to deblur the image captured by the sensor. Any suitable hardware can by used to transfer the image from the sensor to the processor. Any suitable display, storage device, or printer can then be used to display, store, or print the deblurred image.
  • a microprocessor, digital signal processor, special purpose computer which can include a microprocessor, digital signal processor, a controller, etc., memory, communication interfaces, display controllers, input devices, etc.
  • general purpose computer suitably programmed (which can include a microprocessor, digital signal processor, a controller, etc., memory, communication interfaces,
  • any suitable computer readable media can be used for storing instructions for performing the processes described herein.
  • computer readable media can be transitory or non-transitory.
  • non-transitory computer readable media can include media such as magnetic media (such as hard disks, floppy disks, etc.), optical media (such as compact discs, digital video discs, Blu-ray discs, etc.), semiconductor media (such as flash memory, electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), etc.), any suitable media that is not fleeting or devoid of any semblance of permanence during transmission, and/or any suitable tangible media.
  • transitory computer readable media can include signals on networks, in wires, conductors, optical fibers, circuits, any suitable media that is fleeting and devoid of any semblance of permanence during transmission, and/or any suitable intangible media.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Optics & Photonics (AREA)
  • Computing Systems (AREA)
  • Image Processing (AREA)
  • Studio Devices (AREA)
  • Facsimile Image Signal Circuits (AREA)
  • Holo Graphy (AREA)
  • Blocking Light For Cameras (AREA)

Abstract

Systems, methods, and media for recording an image of a scene are provided. In accordance with some embodiments, systems for recording an image of a scene are provided, comprising: a diffuser that diffuses light representing the scene and that has a scattering function that is independent of aperture coordinates; a sensor that receives diffused light representing the scene and generates data representing an image; and a hardware processor that uses a point spread function to deblur the image.

Description

    CROSS REFERENCE TO RELATED APPLICATIONS
  • This application is a continuation of U.S. patent application Ser. No. 13/522,943, filed Mar. 22, 2013, which is a national stage application under 35 U.S.C. §371 of International Application No. PCT/US2011/022248, filed Jan. 24, 2011, which claims the benefit of U.S. Provisional Patent Application No. 61/297,667, filed Jan. 22, 2010, each of which is hereby incorporated by reference herein in its entirety.
  • This invention was made with government support under Grant Numbers N00014-08-1-0329 and N00014-08-1-0638 awarded by the Navy/Office of Naval Research. The government has certain rights to the invention.
  • TECHNICAL FIELD
  • The disclosed subject matter relates to systems, methods, and media for recording an image using an optical diffuser.
  • BACKGROUND
  • For conventional cameras, there is a fundamental trade-off between depth of field (DOF) and noise. Generally, cameras have a single focal plane, and objects that deviate from this plane are blurred due to defocus. The amount of defocus blur depends on the aperture size and the distance from the focal plane. To decrease defocus blur and increase DOF, the aperture size must be decreased, reducing the signal strength of the recorded image as well. In many cases, it is desirable to have a DOF that is as large as possible so that all details in the scene are preserved. This is the case, for instance, in machine vision applications such as object detection and recognition, where it is desirable that all objects of interest be in focus. However, stopping down the lens aperture is not always an option, especially in low light conditions, because it can increase noise, which in turn can materially impact the recorded image.
  • SUMMARY
  • Systems, methods, and media for recording an image of a scene are provided. In accordance with some embodiments, systems for recording an image of a scene are provided, comprising: a diffuser that diffuses light representing the scene and that has a scattering function that is independent of aperture coordinates; a sensor that receives diffused light representing the scene and generates data representing an image; and a hardware processor that uses a point spread function to deblur the image.
  • In accordance with some embodiments, methods for recording an image of a scene are provided, the methods comprising: diffusing light representing the scene using a diffuser that has a scattering function that is independent of aperture coordinates; receiving diffused light representing the scene and generating data representing an image; and using a point spread function to deblur the image
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 is a diagram of a mechanism for recording an image in accordance with some embodiments.
  • FIG. 2 is a combination of two images, one not using a diffuser (a) and one using a diffuser (b), in accordance with some embodiments.
  • FIG. 3 is a diagram of a lens and a sensor in accordance with some embodiments.
  • FIG. 4 is a diagram of a lens, a diffuser, and a sensor in accordance with some embodiments.
  • FIG. 5 is a diagram illustrating a light field on a sensor in accordance with some embodiments.
  • FIG. 6 is a diagram of a ray and the scattering of the ray in accordance with some embodiments.
  • FIG. 7 is a combination of point-spread-function and modulation-transfer-function graph pairs in accordance with some embodiments.
  • FIG. 8 is a diagram an optical system including a wedge (a) and a random varying surface in accordance with some embodiments.
  • FIG. 9 is combination of diagrams of a diffuser profile (a), a diffuser height map (b), a diffuser scatter PDF (c), and a diffuser (d) in accordance with some embodiments.
  • DETAILED DESCRIPTION
  • Systems, methods, and media for recording an image using an optical diffuser are provided.
  • Turning to FIG. 1, an illustration of an image recording mechanism 102 (e.g., a camera, video camera, mobile phone incorporating a camera, and/or any other suitable image recording mechanism) that is being used to capture an image including three objects, A 104, B 106, and C 108, is shown. As can be seen, these objects are at different depths with respect to mechanism 102. Because of limitations in the depth of field of mechanism 102, objects A 104 and C 108 may be out of focus when mechanism 102 is focused on object B 106. For example, these objects may be the toys shown in FIG. 2. As illustrated in FIG. 2(a), when a camera is focused on the center object (which may correspond to object B 106 of FIG. 1), the other objects may be out of focus. By using the mechanisms as described herein, however, an image can be recorded of such objects so that they appear to be in focus as illustrated in FIG. 2(b). This can be referred to as mechanism 102 having an extended depth of field.
  • In accordance with some embodiments, extended depth of field can be achieved by incorporating a diffuser 110 or 112 into an image recording mechanism 102. Recording an image using a diffuser in the pupil plane of an image recording mechanism can be referred to as diffusion coding. Such a diffuser can be located at any suitable point in the image recording mechanism. For example, a diffuser 110 can be positioned between a light source (e.g., objects 104, 106, and 108) and a lens 114 (e.g., as a lens attachment), a diffuser 112 can be positioned between a lens 114 and a sensor 116 (e.g., as part of a lens or a camera body), etc.
  • The diffusion coding image can then be detected by sensor 116 and then provided to a hardware processor 118 (incorporated into mechanism 102) and/or a hardware processor 120 (external to mechanism 102) for subsequent processing. This processing can include deblurring the sensed image using a PSF that is matched to the PSF of the diffuser. Any other suitable processing can additionally or alternatively be used. After such processing, an extended depth of field image can be presented on a display 124 (internal to mechanism 102) and/or a display 122 (external to mechanism 102).
  • In order to illustrate how such an image can be recorded using a diffuser, the optics of some embodiments are now described.
  • As shown in FIG. 3, a light field L(ū,x) can be used to represent a four-dimensional set of rays propagating from an ideal lens with effective focal length (EFL) f to a sensor. A vector ū=(u,v) can be used to denote the coordinates on the u-v plane, which is coincident with the exit pupil of the lens, and a vector x=(x,y) can be used to denote the coordinates on the x-y plane that is coincident with the sensor. The irradiance E(x) observed on the sensor can be defined as the light field integrated over all ray angles:
  • E ( x _ ) = Ω u _ L ( u _ , x _ ) u _ , ( 1 )
  • where, Ωū is the domain of ū. For a scene with smooth depth variation, locally, the captured image E(x) can be modeled as a convolution between a depth-dependent PSF kernel P(x) and an all-in-focus image I(x).
  • As described further below, in accordance with some embodiments, a camera PSF can be shaped so that an image I(x) can be recovered from the captured image E(x) by deblurring with a single PSF P(x). The depth-dependence of the camera PSF can be analyzed by considering the image produced by a unit energy point source. For example, as shown in FIG. 3, consider a point source whose image comes to focus at a distance d0 from the aperture of the lens. Assuming a rectangular aperture of width A, the light field produced by this point can be represented as:
  • L δ ( u _ , x _ ) = 1 A 2 ( u _ A ) δ ( x _ - s 0 u _ ) , ( 2 )
  • where s0=(d0−f)/d is the defocus slope in light field space, and
    Figure US20170085787A1-20170323-P00001
    is the box function:
  • ( x _ w ) = { 1 if x _ i < 1 2 , i 0 otherwise . ( 3 )
  • The image of this point is the camera PSF at the depth d0, which is a box shaped PSF with defocus blur width s0A:
  • P ( x ) = 1 s 0 2 A 2 ( x _ s 0 A ) . ( 4 )
  • The effect of a general kernel D applied to a light field L, which represents the effect of a diffuser placed in the aperture of a camera lens, can next be analyzed. The kernel can produce a new filtered light field {circumflex over (L)}, from which the modified PSF {circumflex over (P)} can be derived as:
  • L ^ ( u _ , x _ ) = Ω u _ Ω x _ D ( u _ , u _ , x _ , x _ ) L ( u _ , x _ ) u _ x _ , ( 5 ) P ^ ( x _ ) = Ω u L ^ ( u _ , x _ ) u _ , ( 6 )
  • where Ω{circumflex over (x)} is the domain of x. This approach allows a large class of operations applied to a light field to be expressed. For instance, consider a kernel of the form
  • D ( u _ , u _ , x _ , x _ ) = 1 w 2 δ ( u _ - u _ ) ( x _ - x _ w ) . ( 7 )
  • Note that here D takes the form of a separable convolution kernel with finite support in the x domain. The geometric meaning of this kernel can be illustrated as shown in FIG. 4. As shown, each ray in the light field is blurred so that, instead of piercing the sensor at a single location, it contributes to a square of width w. In order to understand the effect of the diffuser, an image E captured without the diffuser can be compared to an image Ê captured with it. For this diffuser kernel, substituting Equation 7 into Equations 5 and 6 gives:
  • P ^ ( x _ ) = 1 w 2 ( x _ w ) P ( x _ ) , ( 8 )
  • where {circle around (×)} denotes convolution. The modified PSF can be the camera PSF blurred with a box function. Therefore, the effect of the diffuser is to blur the image that would be captured were it not present. However, the diffuser given by the kernel in Equation 7 may not be useful for extending depth of field because it does not increase depth independence or preserve high frequencies in the camera PSF.
  • In general, the kernel for any diffuser that is placed in the aperture can be represented as:

  • D(ū,ū′,x,x ′)=δ(ū−ū′)k(ū,xx ′),  (9)
  • where k is called the scatter function. As can be seen, the diffuser has no effect in the ū domain, but has the effect of a convolution in the x domain. For the diffuser given by Equation 7, the scatter function can be represented as a two-dimensional box function:
  • k ( u _ , x _ ) = 1 w 2 ( x _ w ) .
  • By changing from rectangular coordinates (u,v,x,y) to polar coordinates (ρ,φ,r,θ) using the relations u=ρ cos φ, v=ρ sin φ, x=r cos θ, and y=r sin θ, a polar system where ρ,rε(—∞,∞) and θ,φε(0,π) and a circular aperture with diameter A can be considered. In this system, the light field representing a unit-energy point source located at distance d0 can be written as:
  • L δ ( ρ , r ) = 4 π A 2 ( ρ A ) δ ( r - s 0 ρ ) π r , ( 10 )
  • which is independent of both θ and φ because the source is isotropic. Note that verifying unit-energy can be carried out trivially by integrating Lδ(ρ,r) in polar coordinates. Comparing the parameterizations for the light field of a point source in Equations 2 and 10, it can be seen than a slice of Lδ(x,y) represents a single ray, while a slice L(ρ,r) represents a 2D set of rays. In the radially symmetric parameterization, a slice of the light field represents a conic surface connecting a circle with radius ρ in the aperture plane to a circle of radius r on the sensor (see FIG. 5).
  • A radially symmetric diffuser produces a drastically different effect than the diffuser given by Equation 7. When a radially symmetric diffuser is introduced, neither the diffuser nor the lens deflects rays tangentially, and therefore the diffuser kernel and modified light field can be represented using the reduced coordinates (ρ,r). Equations 5 and 6 then become:
  • L ^ ( ρ , r ) = π 2 Ω ρ Ω r D ( ρ , ρ , r , r ) L ( ρ , r ) ρ ρ r r , ( 11 ) E ( r ) = π Ω ρ L ^ ( ρ , r ) ρ ρ , ( 12 )
  • and the general form of the diffuser kernel becomes:
  • D ( ρ , ρ , r , r ) = δ ( ρ - ρ ) π ρ k ( r - r , ρ ) π r ( 13 )
  • The same box-shaped scattering function as was used for the diffuser kernel in Equation 7 can be used for Equation 13:
  • k ( r , ρ ) = 1 w ( r w ) . ( 14 )
  • However, the physical interpretation of this diffuser is different than the previous diffuser. For the previous diffuser, each ray in the light field is scattered so that it spreads across a square on the sensor. The effect of the scattering function in Equation 14, however, is as illustrated in FIG. 6. As shown, in the absence of the diffuser, light from an annulus of width dρ and radius ρ in the aperture plane projects to an annulus of width dr and radius r on the sensor. The effect of the scatter function in Equation 14 is to spread the light incident on the sensor so that it produces an annulus of width w instead.
  • As illustrated by volume 602 in FIG. 6, in polar coordinates, a ray can be a small annular section that travels from the aperture plane to the sensor plane. The effect of the diffuser, which is to scatter a ray along a radial line of width w, can be as illustrated by volume 604.
  • A box-shaped scatter function can be used here for notational convenience, but a Gaussian scattering function (e.g., as illustrated in FIG. 9(c)) can be superior for extended DOF imaging. The light field of a point source filtered by this diffuser kernel and PSF can be shown to be:
  • L ^ ( ρ , r ) = 4 π A 2 ( ρ A ) ( r - s 0 ρ w ) π w r , ( 15 ) P ^ ( r ) = 4 π s 0 2 A 2 1 w r [ ( r w ) ( ( r s 0 A ) · r ) ] . ( 16 )
  • The analytic solution for this PSF is a piecewise function due to the contribution from the term in brackets, which is a convolution between the two rect functions (one weighted by |r|). Note that as the scattering width w is reduced to zero, the first rect (combined with 1/w) approaches a delta function and the result is a pillbox-shaped defocus PSF. Also note that if a different diffuser with different scattering function is used, the first rect is simply replaced with the new scattering function. However, the convolution term is far less significant than the 1/|r| term, whose effect dominates, resulting in a PSF which can be strongly depth independent while still maintaining a strong peak and preserving high frequencies.
  • As illustrated in FIG. 6, light incident on a small annular region of width δr and radius r emanates from an annulus in the aperture, and its energy can be proportional to ρ or equivalently to r/s0. This explains the presence of the |r| multiplier within the term in brackets of Equation 16. This term in brackets states that the energy in a pillbox defocus PSF annulus is spread uniformly along radial lines of width w by the diffuser, as shown on the right hand side of FIG. 6. The 1/|r| term in Equation 16 can be attributed to the fact that the energy density becomes larger for light that is scattered closer to the center of the PSF.
  • FIG. 7 shows several PSF 702 and Modulation Transfer Function (MTF) 704 graph pairs for a camera with (714, 716, 718, 720, 722, 724, 726, and 728) and without (715, 717, 719, 721, 723, 725, 727, and 729) the diffuser given by Equation 16. The defocus blur diameter s0A changes between 0 pixels 706, 25 pixels 708, 50 pixels 710, and 100 pixels 712. The scatter function of Equation 14 is a Gaussian instead of a box function, and the diffuser parameter w (the variance of the Gaussian) is chosen so that w=100 pixels. Note that when the diffuser is present, there is little variation with depth for either the PSF or MTF. Introducing the diffuser also eliminates the zero crossings in the MTF. For smaller defocus values, the diffuser suppresses high frequencies in the MTF. However, because the diffuser MTF does not vary significantly with depth, high frequencies can be recovered via deconvolution.
  • In accordance with some embodiments, diffusers of the “kinoform” type (as described in Caufield, H. J., “Kinoform Diffusers,” In SPIE Conference Series, vol. 25, p. 111, 1971, which is hereby incorporated by reference herein in its entirety) where the scattering effect is caused entirely by roughness variations across a surface can be used. Such a diffuser can be considered to be a random phase screen, and according to statistical optics, for a camera with effective focal length f, and center wavelength X, the effect of placing this screen in the aperture of the camera can result in the following:
  • P ^ ( x , y ) p φ u , φ v ( x λ _ k , y λ _ k ) , ( 18 )
  • where φu and φv are the u and v derivatives of the phase shift induced by the surface, and pφ x y is the joint probability of these derivatives. The result of Equation 18 is that a diffuser can be implemented by creating an optical element with thickness t(u,v), where the gradient of this surface ∇t(u,v) is sampled from a probability distribution which is also a desired PSF. Intuitively, this equation can be understood as follows: pφ u v denotes the fraction of the surface t(u,v) with slope (φuv). For small angles, all incoming rays incident on this fraction of the surface will be deflected at the same angle, since the slope is constant over this region. Thus, the quantity pφ u v also reflects the portion of light that will be deflected by the slope (φxy).
  • A Kinoform diffuser has a randomly varying surface with a general probability distribution of slopes as illustrated in FIG. 8(b). Kinoform diffusers can be thought of as generalized phase plates. For example, a regular deterministic phase plate with thickness t(u)=aλu, as shown in FIG. 8(a), can also be thought of as having a slope drawn from a probability function p(φu) which is a delta function. The result of placing this phase plate in the pupil plane of a camera is to shift the PSF, which can be thought of as convolving p(φu) with the PSF.
  • To implement the diffuser defined in Equation 14, the diffuser surface can be implemented as a sequence of quadratic elements whose diameter and sag is drawn from a random distribution as described in Sales, T. R. M., “Structured microlens arrays for beam shaping,” Optical Engineering 42, 11, pp. 3084-3085, 2003, which is hereby incorporated by reference herein in its entirety. The scatter function of the diffuser can be designed to be roughly Gaussian with 0.5 mm variance (corresponding to w=1 mm in Equation 16) as shown in FIG. 9(c). To create a radially symmetric diffuser, a one-dimensional random profile can be created and then a polar transformation applied to create a two-dimensional surface (see, e.g., FIGS. 9(a) and 9(b)).
  • In some embodiments, a diffuser can be made using laser etching.
  • In some embodiments, the maximum height of the diffuser surface can be 3 μm, and the diffuser can be fabricated using a laser machining technology which has a minimum spot size of about 10 μm. To ensure that each quadratic element in the diffuser is fabricated with high accuracy, the minimum diameter of a single element can be chosen to be 200 μm, resulting in a diffuser with 42 different annular sections.
  • Any suitable hardware can be used to implement a mechanism 102 in accordance with some embodiments. For example, a Canon EOS 450D sensor from Canon U.S.A., Inc. can be used as sensor 116, a 22 mm diameter diffuser (e.g., as illustrated in FIG. 9(d)) that is laser etched in a piece of suitable optical glass by RPC Photonics of Rochester, N.Y. can be used as diffuser 110 or 112, and a 50 mm f/1.8 lens from Canon U.S.A., Inc. can be used as lens 114. As another example, lens 114 can have any focal length and consist of refractive optics, reflective optics, or both. For instance, a 3048 mm focal length Meade LX200 telescope (available from) can be used in some embodiments.
  • In accordance with some embodiments, any suitable processing can be performed to deblur the image hitting a camera sensor after passing through a lens and diffuser (in either order). For example, the Wiener deconvolution with the PSF at the center depth can be used to deblur the sensed images. Any suitable additional or alternative processing on the images can be used. For example, additional deblurring of diffusion coded images can performed using the BM3D deblurring algorithm as described in Dabov, K., Foi, A., Katkovnik, V., and Egiazarian, K., “Image restoration by sparse 3D transform-domain collaborative filtering,” In SPIE Conference Series, vol. 6812, 681207, 2008, which is hereby incorporated by reference herein in its entirety. In some embodiments, the BM3D deblurring algorithm enforces a piecewise smoothness prior that suppresses the noise amplified by the deblurring process.
  • Any suitable hardware processor, such as a microprocessor, digital signal processor, special purpose computer (which can include a microprocessor, digital signal processor, a controller, etc., memory, communication interfaces, display controllers, input devices, etc.), general purpose computer suitably programmed (which can include a microprocessor, digital signal processor, a controller, etc., memory, communication interfaces, display controllers, input devices, etc.), server, programmable gate array, etc. can be used to deblur the image captured by the sensor. Any suitable hardware can by used to transfer the image from the sensor to the processor. Any suitable display, storage device, or printer can then be used to display, store, or print the deblurred image.
  • In some embodiments, any suitable computer readable media can be used for storing instructions for performing the processes described herein. For example, in some embodiments, computer readable media can be transitory or non-transitory. For example, non-transitory computer readable media can include media such as magnetic media (such as hard disks, floppy disks, etc.), optical media (such as compact discs, digital video discs, Blu-ray discs, etc.), semiconductor media (such as flash memory, electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), etc.), any suitable media that is not fleeting or devoid of any semblance of permanence during transmission, and/or any suitable tangible media. As another example, transitory computer readable media can include signals on networks, in wires, conductors, optical fibers, circuits, any suitable media that is fleeting and devoid of any semblance of permanence during transmission, and/or any suitable intangible media.
  • Although the invention has been described and illustrated in the foregoing illustrative embodiments, it is understood that the present disclosure has been made only by way of example, and that numerous changes in the details of implementation of the invention can be made without departing from the spirit and scope of the invention, which is only limited by the claims which follow. Features of the disclosed embodiments can be combined and rearranged in various ways.

Claims (18)

What is claimed is:
1. A system for recording an image of a scene, comprising:
a diffuser that behaves in accordance with a scattering function so that light representing the scene entering a first side of the diffuser is output by a second side of the diffuser as a blurred form of the light representing the scene;
an image sensor that receives the blurred form of the light representing the scene and generates image data representing a first image based on the blurred form of the light representing the scene; and
a hardware processor that uses a point spread function corresponding to the scattering function to generate a second image based on the image data representing the first image and that corresponds to a deblurred version of the first image.
2. The system of claim 1, wherein the diffuser is radially symmetric.
3. The system of claim 1, wherein a one-dimensional thickness profile of the radially symmetric diffuser along a radial direction is random.
4. The system of claim 3, wherein the diffuser is a kinoform type diffuser.
5. The system of claim 1, wherein the scattering function is a symmetric distribution function.
6. The system of claim 1, wherein the system comprises a camera having an aperture, and wherein the scattering function does not vary as a function of aperture coordinates of describing the aperture.
7. The system of claim 1, further comprising a lens through which the light representing the scene passes before it is incident on the first side of the diffuser.
8. The system of claim 1, further comprising a lens through which the blurred form of the light representing the scene passes before it is incident on the image sensor.
9. The system of claim 1, further comprising a display that displays the deblurred image.
10. A method for recording an image of a scene, comprising:
diffusing light representing the scene using a diffuser that behaves in accordance with a scattering function so that the light representing the scene entering a first side of the diffuser is output by a second side of the diffuser as a blurred form of the light representing the scene;
receiving the blurred form of the light representing the scene using an image sensor;
generating image data representing a first image based on the blurred form of the light representing the scene received by the image sensor; and
generating a second image that corresponds to a deblurred version of the first image based on the image data representing the first image using a point spread function corresponding to the scattering function.
11. The method of claim 10, wherein the diffuser is radially symmetric.
12. The method of claim 10, wherein a one-dimensional thickness profile of the radially symmetric diffuser along a radial direction is random.
13. The method of claim 12, wherein the diffuser is a kinoform type diffuser.
14. The method of claim 10, wherein the scattering function is a symmetric distribution function.
15. The method of claim 10, further comprising passing the light representing the scene through a camera aperture before it is incident on the first side of the diffuser, wherein the scattering function does not vary as a function of aperture coordinates of describing the aperture.
16. The method of claim 10, further comprising passing the light representing the scene through a lens before it is incident on the first side of the diffuser.
17. The method of claim 10, further comprising passing the blurred form of the light representing the scene through a lens before it is incident on the image sensor.
18. The method of claim 10, further comprising displaying the deblurred image using a display.
US15/225,552 2010-01-22 2016-08-01 Systems, Methods, and Media for Recording an Image Using an Optical Diffuser Abandoned US20170085787A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US15/225,552 US20170085787A1 (en) 2010-01-22 2016-08-01 Systems, Methods, and Media for Recording an Image Using an Optical Diffuser

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US29766710P 2010-01-22 2010-01-22
PCT/US2011/022248 WO2011091358A1 (en) 2010-01-22 2011-01-24 Systems, methods, and media for recording an image using an optical diffuser
US201313522943A 2013-03-22 2013-03-22
US15/225,552 US20170085787A1 (en) 2010-01-22 2016-08-01 Systems, Methods, and Media for Recording an Image Using an Optical Diffuser

Related Parent Applications (2)

Application Number Title Priority Date Filing Date
PCT/US2011/022248 Continuation WO2011091358A1 (en) 2010-01-22 2011-01-24 Systems, methods, and media for recording an image using an optical diffuser
US13/522,943 Continuation US9407833B2 (en) 2010-01-22 2011-01-24 Systems, methods, and media for recording an image using an optical diffuser

Publications (1)

Publication Number Publication Date
US20170085787A1 true US20170085787A1 (en) 2017-03-23

Family

ID=44307258

Family Applications (2)

Application Number Title Priority Date Filing Date
US13/522,943 Active US9407833B2 (en) 2010-01-22 2011-01-24 Systems, methods, and media for recording an image using an optical diffuser
US15/225,552 Abandoned US20170085787A1 (en) 2010-01-22 2016-08-01 Systems, Methods, and Media for Recording an Image Using an Optical Diffuser

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US13/522,943 Active US9407833B2 (en) 2010-01-22 2011-01-24 Systems, methods, and media for recording an image using an optical diffuser

Country Status (5)

Country Link
US (2) US9407833B2 (en)
JP (2) JP5567692B2 (en)
KR (1) KR20120116003A (en)
CN (1) CN102770873B (en)
WO (1) WO2011091358A1 (en)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2503364A1 (en) * 2011-03-22 2012-09-26 Koninklijke Philips Electronics N.V. Camera system comprising a camera, camera, method of operating a camera and method for deconvoluting a recorded image
US9752869B2 (en) 2011-08-15 2017-09-05 The Trustees Of Columbia University In The City Of New York Systems and methods for performing machine vision using diffuse structured light
US9325971B2 (en) * 2013-01-10 2016-04-26 The Regents Of The University Of Colorado, A Body Corporate Engineered point spread function for simultaneous extended depth of field and 3D ranging
US9131159B2 (en) * 2013-06-28 2015-09-08 Nokia Technologies Oy Optical field communication
CN104506771B (en) * 2014-12-18 2018-04-27 北京智谷睿拓技术服务有限公司 Image processing method and device
KR101538395B1 (en) * 2015-02-16 2015-07-23 주식회사 나무가 A light diffuser combination module capable of adjusting an aiming angle, 3D camera and method for optimizing an aiming angle
US10666848B2 (en) * 2015-05-05 2020-05-26 Microsoft Technology Licensing, Llc Remote depth sensing via relayed depth from diffusion
EP3360012A4 (en) * 2015-10-09 2019-05-29 Visby Camera Corporation Holographic light field imaging device and method of using the same
JP6834134B2 (en) * 2016-01-21 2021-02-24 オムロン株式会社 Imaging device and information code reader
TWI578787B (en) * 2016-03-18 2017-04-11 圓展科技股份有限公司 Image capture device with automatic white balance function and adjustment method for automatic white balance
CN106296811A (en) * 2016-08-17 2017-01-04 李思嘉 A kind of object three-dimensional reconstruction method based on single light-field camera
US10691926B2 (en) * 2018-05-03 2020-06-23 Analog Devices, Inc. Single-pixel sensor
US11119253B2 (en) 2018-09-28 2021-09-14 Avalon Holographics Inc. Direct projection light field display
WO2020061714A1 (en) * 2018-09-28 2020-04-02 Peckham Jordan Direct projection light field display
US11671698B1 (en) * 2020-12-28 2023-06-06 Zachary Kapela Universal lens-taking optical viewfinder
JP7103448B2 (en) * 2021-02-01 2022-07-20 オムロン株式会社 Imaging device and information code reader

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050117035A1 (en) * 2002-04-16 2005-06-02 Bernd Hofflinger Method and arrangement for FPN correction in an image signal from an image sensor
US20070268585A1 (en) * 2000-07-14 2007-11-22 Scott Santoro Light control devices implemented with diffusers having controllable diffusion characteristics
US20090244355A1 (en) * 2008-03-27 2009-10-01 Olympus Corporation Filter switching device, photographing lens, camera, and image pickup system
US20090273843A1 (en) * 2008-05-01 2009-11-05 Ramesh Raskar Apparatus and Method for Reducing Glare in Images
US7701489B1 (en) * 2003-05-27 2010-04-20 Apple Inc. Method and apparatus for color correction

Family Cites Families (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0829678A (en) * 1994-07-18 1996-02-02 Olympus Optical Co Ltd Rigid endoscope observing optical system
JP3275010B2 (en) 1995-02-03 2002-04-15 ザ・リジェンツ・オブ・ザ・ユニバーシティ・オブ・コロラド Optical system with extended depth of field
JP3872852B2 (en) * 1996-02-26 2007-01-24 オリンパス株式会社 Endoscope TV observation system, light source unit used for endoscope TV observation system, and small light source unit for endoscope
JP2002513951A (en) * 1998-05-01 2002-05-14 ユニバーシティ テクノロジー コーポレイション Expanded depth of field optical system
US7176960B1 (en) * 1999-09-20 2007-02-13 The Trustees Of Columbia University In The City Of New York System and methods for generating spherical mosaic images
JP3791777B2 (en) * 2001-12-28 2006-06-28 オリンパス株式会社 Electronic endoscope
US7260251B2 (en) * 2003-03-31 2007-08-21 Cdm Optics, Inc. Systems and methods for minimizing aberrating effects in imaging systems
US7139067B2 (en) * 2003-09-12 2006-11-21 Textron Systems Corporation Three-dimensional imaging with multiframe blind deconvolution
US20060055807A1 (en) * 2004-09-14 2006-03-16 Pemer Frederick A Imaging methods, image sensors, imaging systems, and articles of manufacture
JP2006317605A (en) 2005-05-11 2006-11-24 Canon Inc Photographic optical system and imaging apparatus having the same
JP4500875B2 (en) * 2005-07-12 2010-07-14 エヌエックスピー ビー ヴィ Method and apparatus for removing motion blur effect
US7720368B2 (en) * 2005-07-15 2010-05-18 Redrock Microsystems, Llc System, method and apparatus for enhancing a projected image
US7304731B2 (en) * 2005-09-02 2007-12-04 Kla-Tencor Technologies Corp. Systems and methods for providing illumination of a specimen for inspection
US7620309B2 (en) 2006-04-04 2009-11-17 Adobe Systems, Incorporated Plenoptic camera
JP2008245157A (en) * 2007-03-28 2008-10-09 Kyocera Corp Imaging device and method therefor
JP2009134023A (en) * 2007-11-29 2009-06-18 Kyocera Corp Imaging device and information code reading device
JPWO2009119838A1 (en) 2008-03-27 2011-07-28 京セラ株式会社 Optical system, imaging device, and information code reader
JP4658162B2 (en) * 2008-06-27 2011-03-23 京セラ株式会社 Imaging apparatus and electronic apparatus
FR2933501B1 (en) * 2008-07-02 2010-09-10 St Microelectronics Grenoble OPTICAL PASTILLE WITH VARYING REFRACTIVE INDEX, NEGATIVE DISTORTION CORRECTION, AND METHOD FOR MANUFACTURING SUCH A PASTILLE
JP2010021645A (en) * 2008-07-08 2010-01-28 Nikon Corp Image processing apparatus, image capturing apparatus, and image processing program
JP5150937B2 (en) * 2008-12-08 2013-02-27 富士フイルム株式会社 Optical system and imaging apparatus

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070268585A1 (en) * 2000-07-14 2007-11-22 Scott Santoro Light control devices implemented with diffusers having controllable diffusion characteristics
US20050117035A1 (en) * 2002-04-16 2005-06-02 Bernd Hofflinger Method and arrangement for FPN correction in an image signal from an image sensor
US7701489B1 (en) * 2003-05-27 2010-04-20 Apple Inc. Method and apparatus for color correction
US20090244355A1 (en) * 2008-03-27 2009-10-01 Olympus Corporation Filter switching device, photographing lens, camera, and image pickup system
US20090273843A1 (en) * 2008-05-01 2009-11-05 Ramesh Raskar Apparatus and Method for Reducing Glare in Images

Also Published As

Publication number Publication date
JP2013518474A (en) 2013-05-20
CN102770873B (en) 2017-09-19
CN102770873A (en) 2012-11-07
KR20120116003A (en) 2012-10-19
JP5824113B2 (en) 2015-11-25
JP5567692B2 (en) 2014-08-06
WO2011091358A1 (en) 2011-07-28
US9407833B2 (en) 2016-08-02
JP2014222893A (en) 2014-11-27
US20130194458A1 (en) 2013-08-01

Similar Documents

Publication Publication Date Title
US20170085787A1 (en) Systems, Methods, and Media for Recording an Image Using an Optical Diffuser
Cossairt et al. Diffusion coded photography for extended depth of field
US10547786B2 (en) Image processing for turbulence compensation
Hahne et al. Refocusing distance of a standard plenoptic camera
Quirin et al. Depth estimation and image recovery using broadband, incoherent illumination with engineered point spread functions
Berlich et al. Single shot three-dimensional imaging using an engineered point spread function
Hardie et al. Fusion of interpolated frames superresolution in the presence of atmospheric optical turbulence
EP2909671B1 (en) Method for designing a single-path imager able to estimate the depth of field
Javidnia et al. Application of preconditioned alternating direction method of multipliers in depth from focal stack
Nakamura et al. Computational phase modulation in light field imaging
JP6149717B2 (en) Imaging apparatus and imaging method
US8436892B2 (en) Image data obtaining method and apparatus therefor
CN109257524B (en) Full-focus scanning imaging method based on Wigner distribution function
Opatovski et al. Monocular kilometer-scale passive ranging by point-spread function engineering
Laurenzis et al. Comparison of super-resolution and noise reduction for passive single-photon imaging
Leach Jr et al. Analysis of the confluence of three patterns using the Centering and Pointing System (CAPS) images for the Advanced Radiographic Capability (ARC) at the National Ignition Facility
EP0280588A1 (en) Method and device for taking a picture with a great depth of view
Estrada et al. Multi-frame GAN-based machine learning image restoration for degraded visual environments
Bos Imaging theory and mitigation in extreme anisoplanatism
Jeong et al. Optic design and image processing considering angle of incidence via end-to-end optimization method
EP4294030A1 (en) Method and apparatus with image quality improvement
Zhou Point Spread Function Engineering for Scene Recovery
Ikeoka et al. Depth estimation with tilted optics by multi-aperture using color filter
Li et al. Hybrid imaging with extended depth of field based on binocular ranging
Scrymgeour et al. Advanced Imaging Optics Utilizing Wavefront Coding.

Legal Events

Date Code Title Description
AS Assignment

Owner name: NAVY, SECRETARY OF THE UNITED STATES OF AMERICA, V

Free format text: CONFIRMATORY LICENSE;ASSIGNOR:COLUMBIA UNIVERSITY;REEL/FRAME:042569/0479

Effective date: 20161103

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION