IMAGE INPUT DEVICE HAVING OPTICAL DEFLECTION ELEMENTS FOR CAPTURING MULTIPLE SUB-IMAGES Background of the Invention
The present invention relates to an imaging system for selectively providing either a low resolution real-time image or a high resolution scanned image. More particularly, the invention relates to a camera-like imaging system in which high resolution digital capture of an image is provided by combining a series of sequentially obtained sub-images.
It is known to increase the resolution of a video camera either by increasing the number of pixels in the image sensor or by increasing the number of image sensors. Either approach typically increases the cost of the camera.
An alternative method of increasing resolution uses just one image sensor, but uses that image sensor to capture only a portion (sub-image) of the desired image. A series of sub-images called "tiles" are sequentially captured and stored in the memory of an image processor. The image processor combines ("stitches") the tiles together to form a composite ("mosaic") image. Tiling systems, however, present a number of problems involving the capture of the sub-images. The sub-images should fit together tightly, avoiding both overlap of tiles or gaps between tiles. A high degree of repeatability in the tile
capture process is desirable, because repeatable (characterizable) errors can be corrected. Because of the complexity involved in acquiring the tiles, known tiling systems have been limited to highly sophisticated and costly applications, such as satellite imaging of the earth. The present invention uses tiling to provide a low cost scanner for commercial applications. A low cost tiling system, incorporated as part of a scanning camera, could find application, for example, as an aid to the visually impaired, as the imaging component of a face-up copying system for books, as an x-ray film scanner, as a variable resolution desktop scanner, as a microfilm copying system or as a video teleconferencing input device.
It would be desirable to provide a scanning camera with both a real-time mode and a high resolution mode.
It would also be desirable to provide a scanning camera with a highly repeatable, low cost tiling system.
It would further be desirable to provide a tiling mechanism with minimal settling time.
It would still further be desirable to provide a scanning camera with zoom and prescan capabilities.
Summary of the Invention
It is an object of the present invention to provide a scanning camera with both a real-time mode and a high resolution mode.
It is also an object of the present invention to provide a highly repeatable, low cost tiling system.
It is further an object of the present invention to provide a tiling mechanism with minimal settling time.
It is still further an object of the present invention to provide a scanning camera with zoom and prescan capabilities.
This invention provides an image input device for providing a composite image of an object composed of a plurality of sub-images of the object. The system comprises an image sensor for receiving a sub-image, a lens for focusing the sub-image on the image sensor, an optical deflecting means positioned between the lens and the object, the deflecting means having a plurality of optical elements, each such element for deflecting one of a plurality of sub-images to the lens, and control means for causing the optical deflection means to sequence through the sub-images comprising the composite image to form a composite image.
Brief Description Of The Drawings The above and other objects and advantages of this invention will be apparent on consideration of the following detailed description, taken in conjunction with the accompanying drawings, in which like reference characters refer to like parts throughout, and in which:
FIG. 1 is a schematic illustration of a first embodiment of the invention;
FIG. 2A is a bottom view of the disk of holographic optical elements of the first embodiment of the invention;
FIG. 2B is a bottom view of an alternate embodiment of the disk of FIG. 2A;
FIG. 3A is an illustration of a tiling pattern used in the first embodiment of the invention;
FIG. 3B is an illustration of an embodiment of the invention including a focal correction lens;
FIG. 3C is an illustration of an embodiment of the invention including a motor-driven focus adjustment;
FIG. 4 is a schematic illustration of a second embodiment of the invention;
FIG. 5 is a schematic illustration of a third embodiment of the invention; FIG. 6 is a schematic illustration of a fourth embodiment of the invention;
FIG. 7 is a schematic illustration of a fifth embodiment of the invention; and
FIG. 8 is a schematic embodiment of a linear slide of optical deflectors.
Detailed Description of the Invention
The imaging system of the present invention provides both a real-time (television-frame rate) camera and a high resolution scanning camera. The camera uses an area array charge coupled device ("CCD") sensor to provide the real-time imaging capability. The high resolution scanning mode of the camera is implemented by stepping the point of gaze of the CCD sensor over an object image, such as an 8.5 by 12.5 inch document, so that the document image is acquired as a series of tiles. In a first embodiment, a 32-element disk functions as an optical deflector. Step-wise rotation of the optical deflection disk steps the point of gaze of the CCD array over an object plane so that 32 image "tiles" are acquired. The tiles are "stitched" together in the scanner's memory to form a complete composite image. In a preferred embodiment discussed below, a sampling resolution of 300 dots per inch can be achieved over an 8.5 by 12.5 inch area,
with a time to scan from 2 to 4 seconds, based on photometric calculations for the optics and CCD, and including the predicted settling time for disk motion. A first preferred embodiment of the imaging system of the present invention, as shown in FIG. 1, comprises an image sensor 10, a lens 20, an optical deflection means 30, a positioning means 50 having a shaft 52, and an image processor 70. Object plane 25, which is not a part of the invention, indicates the location of the image to be scanned.
Image sensor 10 receives an optical image and provides an electrical representation of the image as an output. Image sensors are well known in the art. An illuminator or monochromatic light source 15 may be used to improve the signal-to-noise ratio of image sensor 10. Image sensor 10 is preferably an area array CCD, such as the TC245 available from Texas Instruments Incorporated, Dallas, Texas. The TC245 has 365,420 pixels arranged as 484 lines of 755 pixels. The TC245 generates an NTSC interlaced video format; therefore, two fields, one comprising odd-numbered scan lines and one comprising even-numbered scan lines, comprise a frame. Each of the two fields is transferred at a 60 Hz rate; therefore, the frame transfer rate is 30 Hz. The output frames from image sensor 10 are coupled to sensor interface 72, which is shown in FIG. 1 as part of image processor 70. Sensor interface 72 is preferably a TCK 244/245 CCD image sensor evaluation kit available from Texas Instruments, or an integrated circuit chip set, also available from Texas Instruments, for driving the TC245. Sensor interface 72 receives frame information from image sensor 10 and passes it to frame grabber 74. Frame grabber 74 is preferably model number SVM-BW-1MB-20-14.3 available from EPIX, Inc., Northbrook, Illinois.
Frame grabber 74 stores and formats the frame information from sensor interface 72, and provides the frame information to personal computer 76. Although it is shown outside personal computer 76, frame grabber 74 is available as a circuit card which plugs into personal computer 76. Personal computer 76, which is an IBM PC in the first preferred embodiment, executes known software algorithms to combine the frames into a composite image stored in memory 90, which is shown as part of personal computer 76 in the first preferred embodiment. One skilled in the art will appreciate that a wide variety of commercially available products may be substituted for image sensor 10, sensor interface 72, frame grabber 74 and personal computer 76 without departing from the scope of the present invention.
Deflection means 30 comprises a moveable array of fixed optical deflection elements 35 for dividing the object image into a plurality of image tiles. One embodiment of deflection means 30 comprises a disk 32, rotatable about its central axis, having 32 holographic optical elements 201-232 disposed about its circumference as shown in FIG. 2A. For clarity, only one of the 32 holographic optical elements is shown in FIG. 1; however, it is to be understood that element 35 represents one of a plurality of elements. It is further to be understood that the deflection means of the present invention is not limited to 32 elements. More or fewer elements may be selected depending on the application. Disk 32 serves as an optical deflector, with each element 201-232 along its circumference corresponding to a tile position. Positioning means 50 causes disk 32 to rotate, thereby bringing each of the holographic optical elements 201-232 sequentially into the optical path to deflect its corresponding sub-image
to image sensor 10. Holographic optical elements 201- 232 are manufactured, according to known methods, to incorporate microscopic ridges to serve as a diffraction grating on a portion of disk 32, which is preferably a molded plastic material.
Imaging lens 20 is provided in the optical path between deflection element 35 and image sensor 10 to project the object onto image sensor 10. Deflection element 35 is positioned in front of lens 20 (between lens 20 and the object to be imaged on object plane 25) to avoid off-axis image distortion. The angle subtended by the image from deflection element 35 is much smaller than the angle subtended by the image on object plane 25. Positioning lens 20 in front of deflection element 35 is undesirable, because lens 20 would then have to subtend a much larger angle. A larger angle would require lens 20 to be a much more costly wide-angle lens, and such a lens would be more prone to aberrations. In the first embodiment, disk 32 is mounted on shaft 52 of a stepping motor, which serves as positioning means 50. After moving to a new position, a stepping mechanical scanning mechanism requires a settling time in which to stop vibrating, in order to capture a sub-image. The settling time of the tiling scanning mechanism determines, in part, the rate at which images may be captured. If the image capture rate does not allow the tiling scanning mechanism to settle, the captured images will be distorted; allowing too great a time between capturing images unduly slows the image acquisition process. Accordingly, it is preferable to provide a scanning mechanism with minimal settling time. Disk 32 should preferably be constructed of a thin, lightweight material such as plastic in order to minimize its settling time. For
each tile to be acquired, the disk is positioned and allowed to settle. Alignment of disk 32 is not critical on any axis. Moreover, disk 32 has a low mass. From these considerations it follows that positioning will be possible within one frame time of image sensor 10 per tile, and that disk 32 may be supported by shaft 52 alone.
Positioning means 50 is electrically coupled to control means 80. In response to signals from control means 80, positioning means 50 positions disk 32 in one of the 32 angular positions occupied by holographic optical elements 201-232 shown in FIG. 2. Each position corresponds to one setting of deflection means 30. Because alignment of holographic element 35 is not critical, positioning means 50 may comprise an open-loop positioning mechanism. An open-loop system is inherently faster than a closed-loop system, because the closed-loop system operates by making a series of adjustments to achieve precise positioning, whereas the open-loop system seeks its desired position in just one step.
Disk 37, shown in FIG. 2B, comprises a disk encircled by a band of holographic elements 251-282 disposed along its circumference. Because elements
251-282 provide a continuum of deflectors, positioning means 50 can comprise a continually rotating motor instead of a stepping motor. A further benefit of the deflection continuum provided by elements 251-282 is that minor defects in the holographic material average out. An image is acquired when the boundaries of an element, such as element 251, are in alignment with a desired tile. Each element 251-282 corresponds to a frame transfer from image sensor 10. Accordingly, disk 37 should rotate at a speed that allows image sensor 10
to transfer both fields comprising a frame at a single deflection setting.
Control means 80 may be a conventional stand¬ alone computer or microprocessor, or may be part of image processor 70 as shown in FIG. 1. Control means 80 causes positioning means 50 to stop deflection means 30 at known positions or settings. Control means 80 further directs memory 90 to store the images received from image sensor 10 at appropriate times, allowing for the settling time of deflection means 30.
In an exemplary embodiment of the system of the present invention lens 20 would be 15 inches (26.67 cm.) from an object on object plane 25, and would have a minimum depth of focus of 10 mm. Based on the pixel dimensions of the TC245 image sensor and the desired
300 dot per inch resolution, a demagnification of 8.41X is required from object plane 25 to sensor 10. Using these parameters and basic lens equations, expected working values for the optical path are that lens 20 have a 40 mm focal length and f#10, 4.5 mm aperture, working distance of 380 mm, and depth of field of 14 mm. Assuming 800 lux room level illumination and 50% page reflectivity yields sensor illumination of 1.0 lux. This corresponds to 4 times the dark current of the TC245 sensor when operated at television frame rates (16.6 ms integration time per frame). An illuminator 15 can improve the signal-to-noise ratio for sensor 10. A near-IR filter may be included in the optical path to compensate for the chromaticity of holographic elements 201-232. Scanning time is expected to be limited by the sum of the tile stepping time (assumed to be 33.3 ms per tile) and frame acquisition time; therefore, the minimum time to acquire all 32 tiles is expected to be 2.13 seconds.
FIG. 3A illustrates the tiling scheme adopted in the first preferred embodiment of the invention. Tiles 301-332 correspond to the 32 tiling elements 201- 232 shown in FIG. 2A. Tiles 301-332 comprise the object, for example a page of printed text, located on object plane 25. A 4X8 array of tiles is shown, based on the layout of the TC245 sensor, with the horizontal scanning axis of the sensor aligned with the short dimension of the page. This alignment is preferred for a page of printed text, because page images can be properly displayed without image rotation. FIG. 3A shows the optical path of tile 312 from object plane 25, through holographic optical element 35 and lens 20, to image sensor 10. Line 350 indicates the uncorrected focal surface and its relation to object plane 25. It is to be understood that line 350 is a cross-section of the focal surface, which is spherical.
As shown in FIG. 3A, uncorrected focal surface 350 curves through object plan 25. If optimum focus is obtained at a point falling on tile 310, then tile 301 will be out of focus. Lens 22, shown in FIG. 3B, corrects the disparity between object plane 25 and uncorrected focal surface 350. Lens 22 is mounted on the upper surface of disk 32, above representative holographic optical element 35. It is to be understood that a lens corresponding to lens 22 would be provided, as needed, for each optical element 201-232, and each such lens 22 would correct for the particular focal length disparity of each corresponding optical element 201-232. Some tiles intersect the focal surface and therefore, require no correction. Lens 22 will slightly alter the size of its corresponding tile as seen by image sensor 10. Correction for this scaling, as well as distortion imposed by the angle of view, may take place in software.
Because holographic optical elements 201-232 are diffractive optics, the diffraction rulings can be arranged to include both a lens and a prism by appropriate micromachining or molding operations. Therefore, holographic optical elements 201-232 can incorporate lens 22, instead of adding a plano-convex lens to the upper surface of disk 32.
The 32 tiles acquired must adjoin each other perfectly in order to assemble a single composite image from the 32 acquired sub-images. Although disk 32 may not yield perfect alignment accuracy, it will provide highly repeatable alignment for any disk, and from disk to disk. Errors in the master disk may be characterized and modeled in software. The magnification changes created by focal length correction, the angle of view distortion and errors in deflection angle produced by individual deflection elements also can be modeled. It thus follows that known post-scan electronic correction may be applied using known image warping methods to correct for the aggregate error. (See, Wolberg, Digital Image Warping) . It is only necessary to insure that HOE deflection errors create an overlap of tiles, rather than gaps between tile regions. Element 233, shown in FIG. 2A, provides for a real-time zoom function. Element 233 is an optically neutral element (such as a hole) that allows light to pass through undeflected. One skilled in the art will recognize that additional known optical elements can be inserted into the optical path between disk 32 and lens 20 to implement fully a zoom function and also to provide an extra motor-driven focus adjustment 24, shown in FIG. 3C. Focus adjustment 24 brings into optimum focus objects on object plane 25 that stand higher than the depth of field or which are not flat,
such as books or household objects. Both focus adjustment 24 and optical zoom may be implemented using standard optical techniques.
A second preferred embodiment of the system of the present invention is shown in FIG. 4. In the second embodiment, mirror 36 serves as the optical deflection element. Wedge support 34 is used to affix mirror 36 to disk 32. Only one mirror 36 is depicted in FIG. 4; however, it is to be understood that a plurality of mirrors are disposed around the circumference of disk 32, similar to the holographic optical elements encircling disk 32 in FIG. 2. Each mirror 36 is affixed to disk 32 at a different angle, so that a two dimensional tiling pattern is obtained as in the first embodiment.
Mirrors, in contrast to transmissive deflection elements such as prisms and holographic optical elements, tend to provide blurred images if the settling time of disk 32 is inadequate. Moreover, even with adequate settling time, mirrors require absolute positional accuracy. Accordingly, positioning means 50 preferably uses a closed-loop system to avoid the image distortion problem with mirrors. The closed-loop system is slower, but more precise, than an open-loop system, because the closed-loop positioning means hunts for the proper position until it is successful.
A third preferred embodiment of the system of the present invention is shown in FIG. 5. In the third embodiment, a plurality of offset plano-convex lens pairs 44, 45 are disposed around the circumference of transparent disk 42. Lens 20 is not required in this embodiment, because the offset lens pair 44, 45 can adequately focus the sub-image onto image sensor 10. A fourth preferred embodiment is shown in FIG. 6, wherein wedge prisms 47 act as optical
deflection means, disposed around the circumference of transparent disk 42. As in the previous figures, prism 47 represents one of a plurality of optical deflectors. A correction lens, such as lens 22 shown in FIG. 3B, may also be used in conjunction with prism 47.
In a fifth embodiment of the invention, shown in FIG. 7, deflection means 30 comprises a pair of mirror galvanometers 53, 55. Mirror galvanometers are available from General Scanning, Inc., Watertown, Massachusetts. Mirror galvanometer 55 comprises mirror 38, shaft 56 and motor 58. Mirror 38 rotates on shaft 56 of motor 58, in response to signals from control 80. Mirror 37 rotates on axis 54, which is the shaft of the motor (not shown) of mirror galvanometer 53. Mirrors 38 and 37 have mutually perpendicular axes of rotation, an X-axis and a Y-axis, thereby allowing scanning of object plane 25 in two directions. Each tile setting for mirror galvanometers 53, 55, comprises an X- coordinate, corresponding to rotation about axis 56 and a Y-coordinate corresponding to rotation about axis 54. One skilled in the art will appreciate that circular, cylindrical or spherical coordinates could be substituted for the X-Y coordinate system, and that a single gimbel-mounted mirror could provide a scanning means as well. Other mirror arrangements can also be employed, provided that a suitable drive motor is used that can position the mirrors with a high degree of repeatabi1ity.
In contrast to previous embodiments comprising a plurality of fixed optical deflectors, mirror galvanometers 53, 55 provide a variable number of scanning settings. Therefore, mirror galvanometers 53, 55 provide a variable scanning means, capable of scanning either a discrete series or a continuum of sub-images.
Prisms are a preferred deflection element, because they are inexpensive, relatively position insensitive and easily manufactured in arrays, with each array in a manufacturing run exhibiting the same characteristics as the other arrays in the batch.
Thus, if several deflector arrays are manufactured from the same mold, element 231 in each array will exhibit the same characteristics. Therefore, any minor optical deviation in element 231 can be corrected either in software or by additional optical elements.
Holographic optical elements also provide highly consistent characteristics within a manufacturing batch; however, holographic optical elements are more highly chromatic than prisms. Prisms have far less chromaticity than holographic optical elements for a given amount of deflection. Accordingly, prisms are preferred deflection elements in applications where chromaticity is a problem. As an alternative, a monochromatic light source 15, shown in FIG. 1A, can be used with holographic optical elements.
Other embodiments of the invention will be apparent to those skilled in the art. For example, disk 32 could easily be replaced by an array of image deflectors disposed on linear slide 33 as shown in FIG. 8, or on a loop or a cylinder. Similarly, image deflectors other than mirrors, prisms, offset lens pairs, mirror galvanometers and holographic optical elements can be used without departing from the spirit of the invention. The scanning camera of the present invention can obtain a pre-scan image in the time required by one extra step of the deflection means, by adding an additional optical focusing element that, in conjunction with motor-driven focus adjustment 24, focuses the entire object image onto image sensor 10.
The low resolution grey-scale image so obtained can be analyzed in software by an area-histogram to produce a thresholding map. This map may then be used dynamically to vary the binarizing threshold applied to incoming pixels during high resolution scanning. This permits the acquisition of accurate text images over tinted backgrounds, even on the multicolor pages commonly found in periodicals.
Thus, it may be seen that the real-time imaging capacity inherent in the scanning camera of the present invention permits the simplified provision of essential features in image scanning applications which do not need real-time display capability.
It will be understood that the particular embodiments described above are only illustrative of the principles of the present invention, and that various modifications could be made by those skilled in the art without departing from the scope and spirit of the present invention, which is limited only by the claims that follow.