WO2020039759A1 - Imaging device and imaging system - Google Patents

Imaging device and imaging system Download PDF

Info

Publication number
WO2020039759A1
WO2020039759A1 PCT/JP2019/026285 JP2019026285W WO2020039759A1 WO 2020039759 A1 WO2020039759 A1 WO 2020039759A1 JP 2019026285 W JP2019026285 W JP 2019026285W WO 2020039759 A1 WO2020039759 A1 WO 2020039759A1
Authority
WO
WIPO (PCT)
Prior art keywords
imaging
optical system
image
imaging device
present
Prior art date
Application number
PCT/JP2019/026285
Other languages
French (fr)
Japanese (ja)
Inventor
善夫 松村
Original Assignee
パナソニックＩｐマネジメント株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by パナソニックＩｐマネジメント株式会社 filed Critical パナソニックＩｐマネジメント株式会社
Publication of WO2020039759A1 publication Critical patent/WO2020039759A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B13/00—Optical objectives specially designed for the purposes specified below
    • G02B13/04—Reversed telephoto objectives
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B13/00—Optical objectives specially designed for the purposes specified below
    • G02B13/06—Panoramic objectives; So-called "sky lenses" including panoramic objectives having reflecting surfaces
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B13/00—Optical objectives specially designed for the purposes specified below
    • G02B13/18—Optical objectives specially designed for the purposes specified below with lenses having one or more non-spherical faces, e.g. for reducing geometrical aberration
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B15/00—Special procedures for taking photographs; Apparatus therefor
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B37/00—Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules

Definitions

  • the present disclosure relates to an imaging device including an optical system and an imaging system.
  • Patent Document 1 discloses a method of capturing a digital panoramic image using a rectangular image sensor.
  • a deformed panoramic image is projected onto an image sector so as to occupy a larger number of pixels than that occupied by a disk-shaped image, by using an annular lens as a fish-eye objective lens.
  • an annular lens as a fish-eye objective lens.
  • the present disclosure provides an imaging apparatus and an imaging system capable of easily capturing a panoramic image.
  • the imaging device includes an imaging element and an optical system.
  • the imaging device forms an imaging surface having first and second directions that intersect each other.
  • the optical system is arranged so as to form an image on the imaging surface.
  • the optical system includes a free-form surface lens that is asymmetric between the first and second directions according to a projection relationship with respect to a cylindrical surface that is a virtual curved surface curved so as to cover the side opposite to the imaging surface.
  • a straight line direction intersecting the circular arc direction on the cylindrical surface and a second direction of the imaging surface correspond to each other while the arc direction in which the cylindrical surface is curved corresponds to the first direction of the imaging surface.
  • the imaging system includes the imaging device described above and an image processing unit.
  • the image processing unit generates a panoramic image based on a captured image captured by the imaging device.
  • the imaging device and the imaging system according to the present disclosure it is possible to easily capture a panoramic image based on the projection relationship to the cylindrical surface.
  • FIG. 1 is a diagram for describing an overview of an imaging system according to a first embodiment of the present disclosure.
  • FIG. 1 is a diagram illustrating a configuration of an imaging system according to a first embodiment.
  • Diagram for explaining an imaging device in an imaging system FIG. 2 illustrates a configuration of an optical system in an imaging system.
  • Diagram for explaining a cylindrical projection method in an imaging system The figure which illustrates the correspondence between the horizontal plane and the vertical plane in the cylindrical projection method Diagram showing another example of the correspondence between the horizontal plane and the vertical plane in the cylindrical projection method
  • FIG. 3 is a diagram for explaining the optical characteristics of the optical system according to the first embodiment. 3 is a scatter diagram illustrating a relationship between an angle of view and an image point in the optical system according to the first embodiment.
  • Scatter plot illustrating the relationship between the angle of view and the image point in the central projection method 9 is a scatter diagram illustrating a relationship between an angle of view and an image point in the optical system according to the second embodiment. Scatter plot showing the relationship between the angle of view and the image point in the optical system according to the third embodiment.
  • 10 is a flowchart illustrating an example of a process of synthesizing a panoramic image in the imaging system.
  • FIG. 4 is a diagram illustrating an operation of the imaging device in the imaging system. Diagram for explaining a panoramic image synthesizing process in an imaging system Diagram for explaining a modification of the imaging system
  • Embodiment 1 of the present disclosure will be described with reference to the drawings.
  • an imaging apparatus and an imaging system that employ a novel cylindrical projection method for an optical system will be described.
  • FIG. 1 is a diagram for describing an overview of an imaging system according to the present embodiment.
  • FIG. 1A shows an example of the cylindrical surface 20 for the imaging device 11 in the imaging system of the present embodiment.
  • FIG. 1B illustrates a captured image captured by the imaging device 11 in the example of FIG.
  • the imaging system employs a cylindrical projection method for optical imaging when imaging is performed by the imaging device 11.
  • the cylindrical projection method is a projection method based on a cylindrical surface 20, which is a virtual curved surface having a cylindrical shape, as illustrated in FIG.
  • the cylindrical surface 20 is set so as to cover the light incident side of the imaging device 11 by, for example, an arc direction d1 curved along a horizontal plane with respect to the optical axis of the imaging device 11 and a linear direction d2 perpendicular to the horizontal plane. You. On the cylindrical surface 20, for example, an image of a scene around the imaging device 11 is developed in a panoramic manner.
  • the imaging device 11 captures a scene in a range defined by a horizontal angle of view ⁇ 1 and a vertical angle of view ⁇ 2 on a cylindrical surface 20, for example, as shown in FIG. FIG. 1B shows a captured image generated per such image capturing.
  • the horizontal angle of view ⁇ 1 indicates a range of angles that can be imaged in a horizontal direction orthogonal to the optical axis of the imaging device 11 on a horizontal plane.
  • the vertical angle of view ⁇ 2 indicates, for example, an angle range in which imaging can be performed in a vertical direction orthogonal to a horizontal plane (that is, a linear direction d2).
  • an image of a scene developed on the cylindrical surface 20 is formed on the imaging surface inside the imaging device 11 so as not to be distorted, and a captured image is generated. You. Thereby, for example, even in one imaging, a panoramic captured image can be obtained with uniform image quality in the direction of the horizontal angle of view ⁇ 1 corresponding to the arc direction d1 of the cylindrical surface 20.
  • the present imaging system is applicable to various applications such as robot vision, and is applicable to, for example, a mobile robot, a Visual-SLAM (simultaneous positioning map creation), a 360 ° camera, a conference system, a vehicle-mounted sensor, and the like.
  • FIG. 2 is a diagram illustrating a configuration of the imaging system.
  • the imaging system includes an imaging device 11 and an image processing unit 13, for example, as illustrated in FIG.
  • the imaging device 11 includes an optical system IL and an imaging device 12.
  • the imaging device 11 is a device that captures images of various objects as subjects, and constitutes, for example, various cameras.
  • the image processing unit 13 may be configured by an information processing device such as a PC, or may be incorporated in a camera or the like.
  • the direction of the optical axis d0 of the optical system IL in the imaging device 11 is defined as the Z direction
  • the horizontal direction orthogonal to the Z direction is defined as the X direction
  • X directions is defined as the Y direction.
  • the + Z side of the optical system IL may be referred to as an image plane side
  • the -Z side may be referred to as an object side.
  • the optical system IL captures light incident from the outside of the imaging device 11, and forms an image such as an image circle by the captured light.
  • the optical system IL forms, for example, a single focus lens or a zoom lens.
  • the cylindrical surface 20 illustrated in FIG. 1 is rotationally asymmetric about the optical axis d0 of the imaging device 11.
  • the imaging device 11 of the present embodiment implements a cylindrical projection method by using an optical system IL that guides light rotationally asymmetrically. Details of the optical system IL and the cylindrical projection method will be described later.
  • the image sensor 12 is, for example, a CCD or CMOS image sensor.
  • the imaging element 12 has an imaging surface in which a plurality of pixels are two-dimensionally arranged at equal intervals.
  • the imaging element 12 is arranged so that the imaging surface of the imaging device 11 is located on the image plane of the optical system IL.
  • the imaging device 12 captures an image formed on the imaging surface via the optical system IL, and generates an image signal indicating a captured image.
  • the image processing unit 13 performs predetermined image processing on an image captured by the imaging device 11 based on an image signal from the image sensor 12, and generates, for example, a panoramic image based on a plurality of captured images.
  • Image processing by the image processing unit 13 includes stitching for connecting a plurality of images.
  • the image processing may include, for example, gamma correction and distortion correction.
  • the image processing unit 13 includes, for example, a CPU or an MPU that implements various functions by executing a program stored in an internal memory.
  • the image processing unit 13 may include a dedicated hardware circuit designed to realize a desired function.
  • the image processing unit 13 may include a CPU, an MPU, a GPU, a DSP, an FPGA, an ASIC, or the like.
  • the imaging surface of the imaging element 12 is formed in, for example, a rectangular shape.
  • the imaging surface of the imaging element 12 will be described with reference to FIG.
  • FIG. 3 illustrates a case where the imaging surface 12a of the imaging element 12 is rectangular.
  • the imaging element 12 has a long side Dx and a short side Dy that define the imaging surface 12a.
  • the long side Dx is orthogonal to the short side Dy and is larger than the short side Dy.
  • the long side Dx is parallel to the X direction
  • the short side Dy is parallel to the Y direction.
  • FIG. 3 illustrates the positional relationship between the image circle Is by the optical system IL and the imaging surface 12a of the imaging device 12.
  • the image circle Is of the present embodiment has a shape distorted from a circular shape to an ellipse or the like, and has a major axis Ix and a minor axis Iy.
  • the major axis Ix is orthogonal to the minor axis Iy and larger than the minor axis Iy.
  • the optical system IL is arranged so that the major axis Ix of the image circle Is is parallel to the X direction and the minor axis Iy is parallel to the Y direction, corresponding to the long side Dx and the short side Dy of the image sensor 12.
  • the X direction is an example of a first direction
  • the Y direction is an example of a second direction.
  • the image circle Is of the optical system IL has, for example, a portion that is not included in the range of the imaging surface 12a of the imaging element 12.
  • the major axis Ix of the image circle Is is larger than the major side Dx of the image sensor 12.
  • the minor diameter Iy of the image circle Is is larger than the minor side Dy of the image sensor 12.
  • the imaging device 12 captures an image by the image circle Is within the range of the imaging surface 12a.
  • FIG. 4 is a diagram illustrating a configuration of the optical system IL in the imaging system.
  • FIG. 4A shows a lens arrangement diagram in the YZ section of the optical system IL.
  • FIG. 4B shows a lens arrangement diagram in an XZ section of the optical system IL. In FIG. 4B, various symbols are omitted.
  • the YZ section in FIG. 4A is a virtual section along the Y direction and the optical axis d0 of the optical system IL.
  • the YZ section is also referred to as a vertical plane.
  • the XZ cross section in FIG. 4B is a virtual cross section along the horizontal X direction and the optical axis d0, and corresponds to the horizontal plane of the imaging device 11.
  • the optical system IL of the present embodiment includes a plurality of lens elements Ln and an aperture A, as shown in FIG.
  • the plurality of lens elements Ln are arranged in order from the object side to the image plane side along the optical axis d0 in the optical system IL.
  • the stop A is an aperture stop and is arranged between the plurality of lens elements Ln.
  • aperture A has an F-number less than 16. This makes it easy to avoid a situation where a good image cannot be obtained from the viewpoint of the influence of the diffraction limit.
  • the F value of the aperture A may be set to less than 8 from the above viewpoint.
  • the plurality of lens elements Ln in the optical system IL include a free-form surface lens Lf.
  • the free-form surface lens Lf is a lens element having a free-form surface on at least one of the object side and the image plane side.
  • the free-form surface is a curved surface that is rotationally asymmetric with respect to the optical axis d0.
  • the free-form surface lens Lf is arranged on the object side and the image plane side of the stop A, respectively.
  • the lens element Ln in the optical system IL may include a rotationally symmetric lens Lr such as a spherical lens and an aspherical lens.
  • the optical system IL may include a cover glass of the image sensor 12.
  • the free-form surface lens Lf has a free-form surface that is asymmetric between the X direction and the Y direction, for example, as shown in FIGS.
  • the optical system IL of the present embodiment controls the incident light beam to be rotationally asymmetric in accordance with the cylindrical projection method.
  • the free-form surface in the free-form surface lens Lf is, for example, an XY polynomial surface.
  • various parameters can be set for the lens element Ln including the free-form surface lens Lf so that the cylindrical projection method is implemented.
  • the various parameters include various coefficients that define the corresponding surface shape, such as various coefficients of an XY polynomial that defines an XY polynomial surface.
  • the various parameters may include a radius of curvature, a surface interval, a refractive index, an Abbe number, and the like.
  • the optical system IL of the present embodiment has, for example, a focal length that is larger than 10 times the focal length of the optical system IL and smaller than 1000 times.
  • a focal length that is larger than 10 times the focal length of the optical system IL and smaller than 1000 times.
  • the half angle of view ⁇ 1 / 2 of the horizontal angle of view ⁇ 1 is, for example, larger than 20 ° and smaller than 85 °. Below the lower limit, the corresponding cylindrical surface 20 becomes too narrow. If the upper limit is exceeded, it becomes difficult to control astigmatism in the optical system IL. From the above viewpoint, 40 ° ⁇ 1 / 2 ⁇ 70 ° may be satisfied.
  • Cylindrical Projection Method A cylindrical projection method by the imaging system according to the present embodiment will be described with reference to FIGS.
  • FIG. 5 is a diagram for explaining the cylindrical projection method in the imaging system.
  • FIG. 6 is a diagram exemplifying a correspondence between a horizontal plane and a vertical plane in the cylindrical projection method.
  • FIG. 7 is a diagram illustrating another example of the correspondence between the horizontal plane and the vertical plane.
  • FIG. 6A illustrates a projection relationship on a horizontal plane (that is, an XZ section) by the cylindrical projection optical system IL.
  • FIG. 6B illustrates a projection relationship on a vertical plane (that is, a YZ cross section) by the cylindrical projection optical system IL.
  • x indicates an image height in the X direction, and is defined for each image point with reference to the optical axis d0.
  • ⁇ x is an angle indicating a half angle of view of the image point in the X direction (see FIG. 5).
  • fx is a paraxial focal length of the optical system IL in the X direction.
  • y indicates the image height of each image point in the Y direction
  • ⁇ y is an angle indicating the half angle of view of the image point in the Y direction
  • fy is the paraxial focus in the Y direction. Distance. Note that the above equations (E1) and (E2) can be appropriately defined even in a region where various variables x, ⁇ x, y, and ⁇ y are negative.
  • the cylindrical projection optical system IL controls (ie, guides) incoming light rays according to equidistant projection on the horizontal plane.
  • An image is formed at each image point P1a arranged in the X direction.
  • the angular width ⁇ at regular intervals along the arc direction d1 of the cylindrical surface 20 is equal to the image points at regular intervals in the X direction on the imaging surface 12a. It corresponds to the distance ⁇ x between P1a.
  • images of a plurality of points P2b arranged on the cylindrical surface 20 for each width ⁇ d in the linear direction d2 are formed on the imaging surface 12a at the image points P1b arranged in the Y direction on the imaging surface 12a via the optical system IL. .
  • the equally-spaced width ⁇ d on the cylindrical surface 20 is equal to the distance ⁇ y between equally-spaced image points P1b in the Y direction on the imaging surface 12a.
  • FIG. 7A shows the same projection relationship as FIG. 6A.
  • FIG. 7B illustrates the projection relationship of the optical system IL on the YZ section shifted from the optical axis d0 in FIG. 7A.
  • the center projection light guide is performed between and the image point P1c in the same manner as in FIG. 6B.
  • the optical system IL of the present embodiment has optical characteristics unique to the cylindrical projection method as described above.
  • the optical system IL has asymmetric distortion between the X direction and the Y direction. The distortion of the optical system IL will be described with reference to FIG.
  • FIG. 8 is a diagram for explaining the optical characteristics of the optical system IL according to the present embodiment.
  • FIG. 8A illustrates a characteristic of distortion of the optical system IL.
  • FIG. 8B illustrates a subject serving as a reference for distortion.
  • FIG. 8A corresponds to a captured image in which distortion has occurred when the imaging apparatus 11 of the present embodiment captures an image of a square lattice-shaped subject as shown in FIG. 8B.
  • the origin O in FIG. 8A corresponds to the position of the optical axis d0.
  • the characteristic of the distortion in the X direction is similar to the case of the equidistant projection method.
  • the optical system IL has a larger negative distortion in the X direction as the distance from the optical axis d0 increases.
  • an image can be formed by associating the arc direction d1 of the cylindrical surface 20 with the X direction of the imaging surface 12a.
  • the characteristic of the distortion in the Y direction fluctuates negatively according to the position in the X direction, for example, as shown in FIG.
  • an image can be formed by associating the linear direction d2 of the cylindrical surface 20 with the Y direction of the imaging surface 12a.
  • Optical characteristics specific to the cylindrical projection system in the optical system IL can be set by, for example, the various parameters described above. Examples 1 to 3 of the optical system IL according to the present embodiment will be described with reference to FIGS.
  • FIG. 9 is a scatter diagram showing the relationship between the angle of view and the image point P11 in the optical system IL of Example 1 according to the present embodiment.
  • the optical system IL of the present embodiment satisfies the theoretical expressions (E1) and (E2) of the cylindrical projection system.
  • the optical system IL according to the present embodiment has a maximum horizontal image height of 3 mm and a vertical image height of 1.85 mm.
  • the half angle of view of the horizontal angle of view ⁇ 1 in the optical system IL was 65.0 °, and the half angle of view of the vertical angle of view ⁇ 2 was 35.0 °.
  • FIG. 9 an image point P11 at which incident light forms an image on the image plane is plotted for each predetermined angle width over the entire angle of view of the optical system IL.
  • the angle width was set to 5 ° in each of the horizontal direction and the vertical direction.
  • FIG. 9 illustrates the image point P11 in the first quadrant on the XY plane of the image plane whose origin is the position of the optical axis d0. Since the optical system IL of the present embodiment is line-symmetric with respect to the X axis and the Y axis, the same applies to the second to fourth quadrants as in FIG.
  • FIG. 10 is a scatter diagram illustrating the relationship between the angle of view and the image point P0 in the central projection method.
  • an image point P0 plotted in the same manner as in FIG. 9 is shown for an optical system that is symmetrical between the X and Y directions and simply follows the central projection method.
  • the interval between the image points P0 for each predetermined angular width increases.
  • the image quality is not uniform in the horizontal direction.
  • the intervals between the image points P11 are equal in the horizontal direction, and uniform image quality is obtained.
  • an image can be efficiently formed on the entire image plane by the optical system IL of the cylindrical projection system.
  • the optical system IL in the imaging system of the present embodiment does not need to strictly satisfy the theoretical expressions (E1) and (E2), but implements the cylindrical projection method by satisfying the following expressions (1) and (2). Is also good. 0.5 ⁇ x / (fx * ⁇ x) ⁇ 2.0 (1) 0.5 ⁇ y / (fy * cos ⁇ x * tan ⁇ y) ⁇ 2.0 (2) A panoramic image can be formed with sufficient accuracy for practical use also by the projection relationship of the optical system IL that satisfies the above equations (1) and (2).
  • FIG. 11 is a scatter diagram illustrating the relationship between the angle of view and the image point P12 in the optical system IL according to the second embodiment.
  • the value of the middle side of the above equation (1) is in the range of 0.97 to 1.00, and the value of the middle side of the above equation (2) is 0.74 to 1. 00, which satisfies the above equations (1) and (2). Even with the optical system IL of the present embodiment, as shown in FIG. 11, uniform image quality can be obtained in the horizontal direction.
  • FIG. 12 is a scatter diagram illustrating the relationship between the angle of view and the image point P13 in the optical system IL according to the third embodiment.
  • the value of the middle side of the above equation (1) is in the range of 1.00 to 1.37, and the value of the middle side of the above equation (2) is 0.79 to 1.37. 00, which satisfies the above equations (1) and (2). Also with the optical system IL of the present embodiment, as shown in FIG. 12, uniform image quality can be obtained in the horizontal direction.
  • the imaging system performs, for example, a plurality of imagings in the imaging device 11 and combines a plurality of the captured images in the image processing unit 13, so that a panorama image in which scenes of the imaging results for the multiple times are continuously reflected.
  • the optical system IL forms an image of a scene developed on the cylindrical surface 20 on the imaging surface 12a of the imaging device 12 according to the above-described cylindrical projection method. Accordingly, the image processing unit 13 can easily perform stitching image processing for connecting a plurality of captured images, and can improve the image quality of a panoramic image.
  • the image processing unit 13 synthesizes a panoramic image in the imaging system according to the present embodiment, with reference to FIGS.
  • FIG. 13 is a flowchart illustrating a panoramic image synthesizing process in the imaging system. Each process in the flowchart of FIG. 13 is executed by the image processing unit 13 of the imaging system.
  • the image processing unit 13 of the imaging system acquires a plurality of captured images from, for example, one imaging device 11 (S1).
  • the imaging device 11 captures the surrounding scene a plurality of times, for example, in a time-division manner, and outputs a plurality of captured images to the image processing unit 13.
  • FIGS. 14A and 14B show an example of the state of imaging by the imaging device 11 in step S1.
  • FIG. 14A illustrates an example in which the imaging device 11 performs the first imaging.
  • FIG. 14B illustrates a state in which the imaging device 11 performs the second imaging, following FIG. 14A.
  • a plurality of subjects 31 and 32 are arranged around the imaging device 11.
  • the number of times of imaging by the imaging device 11 is not particularly limited to two, but may be three or more.
  • step S1 the imaging device 11 sequentially performs imaging while changing the direction of the optical axis d0 with the center of the circular arc direction d1 of the cylindrical surface 20 as a rotation axis, as shown in FIGS. 14A and 14B, for example.
  • Such an operation of the imaging device 11 may be operated by a user or may be automatically controlled.
  • the imaging system may include a driving device that drives the imaging device 11.
  • step S1 the captured image obtained from the imaging device 11 follows the cylindrical projection method within an allowable error range.
  • the image processing unit 13 may perform image processing for correcting an error with respect to the ideal cylindrical projection method on the acquired captured image.
  • the image processing unit 13 performs stitching image processing on the acquired plurality of captured images (S2) to synthesize a panoramic image. Stitching image processing by the image processing unit 13 will be described with reference to FIG.
  • FIG. 15A illustrates a captured image 41 obtained at the time of imaging in FIG.
  • FIG. 15B illustrates a captured image 42 obtained at the time of the imaging in FIG. 14B.
  • FIG. 15C illustrates the panoramic image 4 after the stitching for the captured images 41 and 42 of FIGS. 15A and 15B.
  • step S2 the image processing unit 13 connects the captured images 41 and 42 shown in FIGS. 15A and 15B, for example, and synthesizes the panoramic image 4 as illustrated in FIG. 15C.
  • the image processing unit 13 may determine an overlapping area 40 between the captured images 41 and 42 by, for example, image analysis, and perform averaging or replacement processing on the determined area 40.
  • the image processing unit 13 indicates the direction of each of the acquired captured images 41 and 42, for example, the direction of the optical axis d0 in each example of FIGS. 14 (a) and 14 (b).
  • Information may be used.
  • the information may be input by a user operation, for example, or may be acquired from a gyro sensor or the like incorporated in the imaging device 11.
  • the image processing unit 13 outputs the synthesized panoramic image 4 to various display devices or storage devices (S3), and ends the processing according to this flowchart.
  • the captured image (S1) acquired from the imaging device 11 reflects the surrounding scene with uniform image quality along the arc direction d1 of the cylindrical surface 20 according to the cylindrical projection method.
  • the same subject 32 appears at different positions between the captured images 41 and 42 in FIGS.
  • the cylindrical projection method although the direction of the optical axis d0 with respect to the subject 32 is rotated (FIGS. 14A and 14B), the shape of the subject 32 reflected in the captured images 41 and 42 is not changed. Has not changed.
  • the panoramic image 4 can be synthesized without particularly performing a significant coordinate transformation that deforms the subject 32 shown in the captured images 41 and 42. Processing load can be reduced. Further, since such a large coordinate transformation as described above can be avoided, stitching of the captured image can be performed without deteriorating the image quality from the time of imaging, and the image quality of the panoramic image 4 can be improved.
  • the imaging device 11 includes the imaging element 12 and the optical system IL.
  • the imaging element 12 forms an imaging surface 12a having X and Y directions as an example of first and second directions intersecting each other.
  • the optical system IL is arranged so as to form an image on the imaging surface 12a.
  • the optical system IL includes a free-form surface lens Lf that is asymmetric between the X and Y directions according to a projection relationship with respect to the cylindrical surface 20 that is a virtual curved surface that is curved so as to cover the side opposite to the imaging surface 12a.
  • the projection relationship of the optical system IL is such that a linear direction d2 intersecting the circular arc direction d1 on the cylindrical surface 20 and a Y direction of the image capturing surface 12a correspond to the circular arc direction d1 at which the cylindrical surface 20 is curved and the X direction of the image capturing surface 12a. Make it correspond.
  • the imaging device 11 described above by projecting the optical system IL onto the cylindrical surface 20 according to the cylindrical projection method, it is possible to reduce the distortion of the scene on the cylindrical surface 20 and image the panoramic image. it can.
  • the optical system IL has asymmetric distortion between the X and Y directions so that the linear direction d2 and the Y direction correspond while the arc direction d1 and the X direction correspond (see FIG. 8). .
  • the optical system IL having the distortion characteristic peculiar to the cylindrical projection method, it is possible to easily capture a panoramic image along the cylindrical surface 20.
  • the projection relationship of the optical system IL is based on equidistant projection in the X direction and based on center projection in the Y direction (see FIGS. 6 and 7). Accordingly, it is possible to easily capture a panoramic image along each of the arc direction d1 and the linear direction d2.
  • the image height x in the X direction and the image height y in the Y direction on the imaging surface 12a satisfy the following Expressions (1) and (2).
  • 0.5 ⁇ x / (fx * ⁇ x) ⁇ 2 (1)
  • ⁇ x half angle of view of image height x in X direction
  • ⁇ y half angle of view of image height y in Y direction
  • fx paraxial focal length in optical system in X direction
  • fy paraxial focal length in optical system in Y direction .
  • the half angle of view ⁇ x of the image height x in the X direction is larger than 20 ° and smaller than 85 °. Therefore, the range of the cylindrical surface 20 to be imaged can be sufficiently secured while controlling astigmatism.
  • the paraxial focal length fx in the X direction and the paraxial focal length fy in the Y direction may have the same size. In this case, it is easy to obtain a smooth distortion in both the X and Y directions.
  • the optical system IL sets a focusing distance which is larger than at least one of the paraxial focal lengths fx and fy in the X and Y directions and smaller than 1000 times, for example. Have. As described above, the blurring of the object near the imaging device 11 can be easily reduced while controlling the image plane distortion by the long focusing distance.
  • the optical system IL has an F value smaller than 16, for example. This makes it easy to avoid a situation where it is difficult to obtain a good image from the viewpoint of the diffraction limit.
  • the imaging system includes an imaging device 11 and an image processing unit 13.
  • the image processing unit 13 generates a panoramic image 4 based on the captured images 41 and 42 captured by the imaging device 11. According to the imaging system of the present embodiment, the panorama image 4 is easily generated by the optical system IL of the imaging device 11.
  • the imaging system of the present embodiment may include a plurality of imaging devices 11. This modification will be described with reference to FIG.
  • FIG. 16 illustrates an example in which two imaging devices 11 included in the imaging system perform imaging.
  • Each imaging device 11 is connected to, for example, a common image processing unit 13 to configure an imaging system of the present modification.
  • the imaging system of the present embodiment may include three or more imaging devices 11.
  • the respective imaging devices 11 are arranged so as to image different regions on the common cylindrical surface 20.
  • the image processing unit 13 acquires a captured image from each imaging device 11, and performs stitching image processing (S2). Also in this case, since a uniform captured image is obtained along the arc direction d1 of the cylindrical surface 20 in each of the imaging devices 11, the image processing unit 13 can easily perform stitching, and can perform high-quality panoramic images. Can be generated.
  • the cylindrical projection method based on the cylindrical surface 20 has been described (see theoretical formulas (E1) and (E2)).
  • the cylindrical surface 20 may be appropriately set to a curved surface that is distorted within an allowable error range.
  • the circular arc direction d1 of the cylindrical surface 20 may not be a circular arc in a perfect circle but may be a direction by a convex curve as appropriate.
  • the linear direction d2 may not be a direction based on a strict straight line, and may be a direction that is more linear than the arc direction d1.
  • the arc direction d1 and the linear direction d2 may intersect each other from a right angle within an allowable angle range as appropriate.
  • the projection relation of the optical system IL according to the cylindrical projection method is based on the equidistant projection in the X direction and the center projection in the Y direction.
  • the projection relationship of the optical system IL according to the present embodiment may be based on each stereoscopic projection, orthographic projection or stereoscopic projection in the X direction, or may be based on weak center projection or pseudo center projection in the Y direction.
  • the rectangular imaging surface 12a is illustrated in FIG. 3, but the imaging surface 12a of the imaging element 12 is not limited to this.
  • the imaging surface 12a of the imaging element 12 may have various rectangular shapes other than a rectangular shape, or may be partially masked. Further, the imaging surface 12a of the imaging element 12 may be curved.
  • the long side Dx and the short side Dy of the image sensor 12 of the present embodiment need not be orthogonal to each other, and may intersect at various angles.
  • the imaging element 12 may have two sides having the same length instead of the long side Dx and the short side Dy.
  • the first and second directions defined by the major axis Ix and the minor axis Iy of the image circle Is not necessarily orthogonal to each other, and may intersect at various angles.
  • the diameters of the image circle Is in the first and second directions may be the same.
  • the image circle Is does not necessarily have to be distorted from a circle.
  • the image circle Is has a portion that is not included in the imaging surface 12a of the imaging element 12, but the image circle Is may include the imaging surface 12a.
  • the XY polynomial surface has been exemplified as an example of the free-form surface.
  • the free-form surface is not limited to the above, and may include, for example, an anamorphic aspheric surface or a toric surface.
  • the free-form surface lens may have a free-form surface that is not anamorphic. Free-form surfaces that are not anamorphic include XY polynomial surfaces but do not include anamorphic aspheric surfaces. A free-form surface that is not anamorphic may not have, for example, a plane of symmetry.
  • the present disclosure is applicable to various applications that perform imaging, for example, a mobile robot, a Visual-SLAM, a 360 ° camera, a conference system, and a vehicle-mounted sensor.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Stereoscopic And Panoramic Photography (AREA)
  • Studio Devices (AREA)
  • Lenses (AREA)

Abstract

An imaging device (11) is provided with an imaging element (12) and an optical system (IL). The imaging element is constituted from an imaging surface (12a) having mutually intersecting first and second directions (X, Y). The optical system is disposed so as to form an image on the imaging surface. The optical system is provided with a free-form lens (Lf) which is asymmetrical between the first and second directions and corresponding to a projection relationship to a cylindrical surface (20) which is a virtual curved surface curved so as to cover the opposite side from the imaging surface. The projection relationship of the optical system causes the straight-line direction (d2) intersecting with the arc direction in the cylindrical surface to correspond to the second direction of the imaging surface while causing the arc direction (d1) in which the cylindrical surface curves to correspond to the first direction of the imaging surface.

Description

撮像装置、及び撮像システムImaging device and imaging system
　本開示は、光学系を備えた撮像装置、及び撮像システムに関する。 The present disclosure relates to an imaging device including an optical system and an imaging system.
　特許文献１は、長方形の画像センサによりデジタルパノラマ画像を撮像する方法を開示している。特許文献１は、魚眼対物レンズに円環レンズを用いて、変形されたパノラマ画像を、円板形画像が占めるピクセル数よりも多いピクセル数を占めるように、画像セクタに投影している。これにより、使用する画像のゾーンの精細度を向上し、デジタルズームの品質を高めている。 Patent Document 1 discloses a method of capturing a digital panoramic image using a rectangular image sensor. In Patent Literature 1, a deformed panoramic image is projected onto an image sector so as to occupy a larger number of pixels than that occupied by a disk-shaped image, by using an annular lens as a fish-eye objective lens. Thereby, the definition of the zone of the image to be used is improved, and the quality of the digital zoom is improved.
国際公開第２００３／０１０５９９号International Publication No. WO 2003/010599
　本開示は、パノラマ状の画像を撮像し易くすることができる撮像装置、及び撮像システムを提供する。 The present disclosure provides an imaging apparatus and an imaging system capable of easily capturing a panoramic image.
　本開示に係る撮像装置は、撮像素子と、光学系とを備える。撮像素子は、互いに交差する第１及び第２方向を有する撮像面を構成する。光学系は、撮像面に結像するように配置される。光学系は、撮像面とは反対側を覆うように湾曲した仮想的な曲面である円筒面に対する射影関係に応じて第１及び第２方向の間で非対称な自由曲面レンズを備える。光学系の射影関係は、円筒面が湾曲する円弧方向と撮像面の第１方向を対応させながら、円筒面において円弧方向に交差する直線方向と撮像面の第２方向を対応させる。 撮 像 The imaging device according to the present disclosure includes an imaging element and an optical system. The imaging device forms an imaging surface having first and second directions that intersect each other. The optical system is arranged so as to form an image on the imaging surface. The optical system includes a free-form surface lens that is asymmetric between the first and second directions according to a projection relationship with respect to a cylindrical surface that is a virtual curved surface curved so as to cover the side opposite to the imaging surface. In the projection relationship of the optical system, a straight line direction intersecting the circular arc direction on the cylindrical surface and a second direction of the imaging surface correspond to each other while the arc direction in which the cylindrical surface is curved corresponds to the first direction of the imaging surface.
　本開示に係る撮像システムは、上記の撮像装置と、画像処理部とを備える。画像処理部は、撮像装置によって撮像された撮像画像に基づいて、パノラマ画像を生成する。 撮 像 The imaging system according to the present disclosure includes the imaging device described above and an image processing unit. The image processing unit generates a panoramic image based on a captured image captured by the imaging device.
　本開示に係る撮像装置及び撮像システムによると、円筒面に対する射影関係に基づいて、パノラマ状の画像を撮像し易くすることができる。 According to the imaging device and the imaging system according to the present disclosure, it is possible to easily capture a panoramic image based on the projection relationship to the cylindrical surface.
本開示の実施形態１に係る撮像システムの概要を説明するための図FIG. 1 is a diagram for describing an overview of an imaging system according to a first embodiment of the present disclosure. 実施形態１に係る撮像システムの構成を示す図FIG. 1 is a diagram illustrating a configuration of an imaging system according to a first embodiment. 撮像システムにおける撮像素子を説明するための図Diagram for explaining an imaging device in an imaging system 撮像システムにおける光学系の構成を例示する図FIG. 2 illustrates a configuration of an optical system in an imaging system. 撮像システムにおける円筒射影方式を説明するための図Diagram for explaining a cylindrical projection method in an imaging system 円筒射影方式における水平面と垂直面との対応関係を例示する図The figure which illustrates the correspondence between the horizontal plane and the vertical plane in the cylindrical projection method 円筒射影方式における水平面と垂直面との対応関係の別例を示す図Diagram showing another example of the correspondence between the horizontal plane and the vertical plane in the cylindrical projection method 実施形態１における光学系の光学特性を説明するための図FIG. 3 is a diagram for explaining the optical characteristics of the optical system according to the first embodiment. 実施形態１に係る実施例１の光学系における画角と像点との関係を示す散布図3 is a scatter diagram illustrating a relationship between an angle of view and an image point in the optical system according to the first embodiment. 中心射影方式における画角と像点との関係を例示する散布図Scatter plot illustrating the relationship between the angle of view and the image point in the central projection method 実施例２の光学系における画角と像点との関係を示す散布図9 is a scatter diagram illustrating a relationship between an angle of view and an image point in the optical system according to the second embodiment. 実施例３の光学系における画角と像点との関係を示す散布図Scatter plot showing the relationship between the angle of view and the image point in the optical system according to the third embodiment. 撮像システムにおけるパノラマ画像の合成処理を例示するフローチャート10 is a flowchart illustrating an example of a process of synthesizing a panoramic image in the imaging system. 撮像システムにおける撮像装置の動作を例示する図FIG. 4 is a diagram illustrating an operation of the imaging device in the imaging system. 撮像システムにおけるパノラマ画像の合成処理を説明するための図Diagram for explaining a panoramic image synthesizing process in an imaging system 撮像システムの変形例を説明するための図Diagram for explaining a modification of the imaging system
　以下、適宜図面を参照しながら、実施の形態を詳細に説明する。但し、必要以上に詳細な説明は省略する場合がある。例えば、既によく知られた事項の詳細説明、或いは実質的に同一の構成に対する重複説明を省略する場合がある。これは、以下の説明が不必要に冗長になるのを避け、当業者の理解を容易にするためである。 Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, an unnecessary detailed description may be omitted. For example, detailed descriptions of well-known matters or redundant description of substantially the same configuration may be omitted. This is to prevent the following description from being unnecessarily redundant and to facilitate understanding by those skilled in the art.
　なお、出願人は、当業者が本開示を十分に理解するために添付図面および以下の説明を提供するのであって、これらによって特許請求の範囲に記載の主題を限定することを意図するものではない。 The applicant provides the accompanying drawings and the following description so that those skilled in the art can fully understand the present disclosure, and it is not intended that the invention be limited to the claimed subject matter. Absent.
（実施形態１）
　以下、本開示の実施形態１を、図面を用いて説明する。実施形態１では、光学系に新規の円筒射影方式を採用した撮像装置及び撮像システムについて説明する。 (Embodiment 1)
Hereinafter, Embodiment 1 of the present disclosure will be described with reference to the drawings. In a first embodiment, an imaging apparatus and an imaging system that employ a novel cylindrical projection method for an optical system will be described.
１．構成
　実施形態１に係る撮像システムについて、図１を用いて説明する。図１は、本実施形態に係る撮像システムの概要を説明するための図である。 1. Configuration The imaging system according to the first embodiment will be described with reference to FIG. FIG. 1 is a diagram for describing an overview of an imaging system according to the present embodiment.
　図１（ａ）は、本実施形態の撮像システムにおける撮像装置１１に対する円筒面２０の一例を示す。図１（ｂ）は、図１（ａ）の例において撮像装置１１が撮像した撮像画像を示す。 FIG. 1A shows an example of the cylindrical surface 20 for the imaging device 11 in the imaging system of the present embodiment. FIG. 1B illustrates a captured image captured by the imaging device 11 in the example of FIG.
　本実施形態の撮像システムは、撮像装置１１において撮像を行う際の光学的な結像に、円筒射影方式を採用する。円筒射影方式は、図１（ａ）に例示するように、円筒形状を有する仮想的な曲面である円筒面２０に基づく射影方式である。円筒面２０は、例えば撮像装置１１の光軸を基準とする水平面上に沿って湾曲する円弧方向ｄ１および水平面に直交する直線方向ｄ２によって、撮像装置１１の光の入射側を覆うように設定される。円筒面２０上には、例えば撮像装置１１周囲のシーンの像がパノラマ状に展開することとなる。 The imaging system according to the present embodiment employs a cylindrical projection method for optical imaging when imaging is performed by the imaging device 11. The cylindrical projection method is a projection method based on a cylindrical surface 20, which is a virtual curved surface having a cylindrical shape, as illustrated in FIG. The cylindrical surface 20 is set so as to cover the light incident side of the imaging device 11 by, for example, an arc direction d1 curved along a horizontal plane with respect to the optical axis of the imaging device 11 and a linear direction d2 perpendicular to the horizontal plane. You. On the cylindrical surface 20, for example, an image of a scene around the imaging device 11 is developed in a panoramic manner.
　本実施形態の撮像装置１１は、例えば図１（ａ）に示すように、円筒面２０上で水平画角θ１および垂直画角θ２によって規定される範囲のシーンを撮像する。図１（ｂ）は、このような撮像１回あたりで生成された撮像画像を示している。水平画角θ１は、水平面上で撮像装置１１の光軸に直交する水平方向において撮像可能な角度の範囲を示す。垂直画角θ２は、例えば水平面に直交する垂直方向（即ち直線方向ｄ２）において撮像可能な角度範囲を示す。 The imaging device 11 according to the present embodiment captures a scene in a range defined by a horizontal angle of view θ1 and a vertical angle of view θ2 on a cylindrical surface 20, for example, as shown in FIG. FIG. 1B shows a captured image generated per such image capturing. The horizontal angle of view θ1 indicates a range of angles that can be imaged in a horizontal direction orthogonal to the optical axis of the imaging device 11 on a horizontal plane. The vertical angle of view θ2 indicates, for example, an angle range in which imaging can be performed in a vertical direction orthogonal to a horizontal plane (that is, a linear direction d2).
　本実施形態の撮像システムによると、円筒射影方式に基づき、円筒面２０上に展開するシーンの像を、撮像装置１１内部の撮像面上で歪めないように結像して、撮像画像が生成される。これにより、例えば１回の撮像においても、パノラマ状の撮像画像が、円筒面２０の円弧方向ｄ１に対応する水平画角θ１の方向にわたって均一な画質で得られる。本撮像システムは、ロボットビジョン等の各種アプリケーションに適用可能であり、例えば移動ロボット、Ｖｉｓｕａｌ－ＳＬＡＭ（同時位置決め地図作成）、３６０°カメラ、会議システム及び車載センサなどに適用可能である。 According to the imaging system of the present embodiment, based on the cylindrical projection method, an image of a scene developed on the cylindrical surface 20 is formed on the imaging surface inside the imaging device 11 so as not to be distorted, and a captured image is generated. You. Thereby, for example, even in one imaging, a panoramic captured image can be obtained with uniform image quality in the direction of the horizontal angle of view θ1 corresponding to the arc direction d1 of the cylindrical surface 20. The present imaging system is applicable to various applications such as robot vision, and is applicable to, for example, a mobile robot, a Visual-SLAM (simultaneous positioning map creation), a 360 ° camera, a conference system, a vehicle-mounted sensor, and the like.
１－１．システム構成
　本実施形態に係る撮像システムの構成を、図２を用いて説明する。図２は、撮像システムの構成を示す図である。 1-1. System Configuration The configuration of the imaging system according to the present embodiment will be described with reference to FIG. FIG. 2 is a diagram illustrating a configuration of the imaging system.
　本実施形態に係る撮像システムは、例えば図２に示すように、撮像装置１１と、画像処理部１３とを備える。撮像装置１１は、光学系ＩＬと、撮像素子１２とを備える。撮像装置１１は、種々の物体を被写体とする画像を撮像する装置であり、例えば各種のカメラを構成する。画像処理部１３は、ＰＣなどの情報処理装置で構成されてもよいし、カメラ等に組み込まれてもよい。 The imaging system according to the present embodiment includes an imaging device 11 and an image processing unit 13, for example, as illustrated in FIG. The imaging device 11 includes an optical system IL and an imaging device 12. The imaging device 11 is a device that captures images of various objects as subjects, and constitutes, for example, various cameras. The image processing unit 13 may be configured by an information processing device such as a PC, or may be incorporated in a camera or the like.
　以下、撮像装置１１における光学系ＩＬの光軸ｄ０の方向をＺ方向とし、Ｚ方向に直交する水平方向をＸ方向とし、Ｚ，Ｘ方向に直交する垂直方向をＹ方向とする。また、光学系ＩＬにおける＋Ｚ側を像面側といい、－Ｚ側を物体側という場合がある。 Hereinafter, the direction of the optical axis d0 of the optical system IL in the imaging device 11 is defined as the Z direction, the horizontal direction orthogonal to the Z direction is defined as the X direction, and the vertical direction orthogonal to the Z, X directions is defined as the Y direction. Further, the + Z side of the optical system IL may be referred to as an image plane side, and the -Z side may be referred to as an object side.
　光学系ＩＬは、撮像装置１１の外部から入射する光を取り込んで、取り込んだ光によるイメージサークル等の像を結像する。光学系ＩＬは、例えば単焦点レンズ又はズームレンズを構成する。図１に例示した円筒面２０は、撮像装置１１の光軸ｄ０の周りには回転非対称になる。本実施形態の撮像装置１１は、回転非対称に導光する光学系ＩＬを用いて、円筒射影方式を実装する。光学系ＩＬ及び円筒射影方式の詳細については後述する。 The optical system IL captures light incident from the outside of the imaging device 11, and forms an image such as an image circle by the captured light. The optical system IL forms, for example, a single focus lens or a zoom lens. The cylindrical surface 20 illustrated in FIG. 1 is rotationally asymmetric about the optical axis d0 of the imaging device 11. The imaging device 11 of the present embodiment implements a cylindrical projection method by using an optical system IL that guides light rotationally asymmetrically. Details of the optical system IL and the cylindrical projection method will be described later.
　撮像素子１２は、例えばＣＣＤ又はＣＭＯＳイメージセンサである。撮像素子１２は、複数の画素が等間隔で二次元的に配置された撮像面を有する。撮像素子１２は、撮像装置１１において撮像面が光学系ＩＬの像面に位置するように配置される。撮像素子１２は、光学系ＩＬを介して撮像面に結像した像を撮像し、撮像画像を示す画像信号を生成する。 The image sensor 12 is, for example, a CCD or CMOS image sensor. The imaging element 12 has an imaging surface in which a plurality of pixels are two-dimensionally arranged at equal intervals. The imaging element 12 is arranged so that the imaging surface of the imaging device 11 is located on the image plane of the optical system IL. The imaging device 12 captures an image formed on the imaging surface via the optical system IL, and generates an image signal indicating a captured image.
　画像処理部１３は、撮像素子１２からの画像信号に基づいて、撮像装置１１による撮像画像に所定の画像処理を行って、例えば複数の撮像画像に基づくパノラマ画像を生成する。画像処理部１３による画像処理は、複数の画像を連結するスティッチングを含む。画像処理は、例えばガンマ補正および歪曲補正等を含んでもよい。 The image processing unit 13 performs predetermined image processing on an image captured by the imaging device 11 based on an image signal from the image sensor 12, and generates, for example, a panoramic image based on a plurality of captured images. Image processing by the image processing unit 13 includes stitching for connecting a plurality of images. The image processing may include, for example, gamma correction and distortion correction.
　画像処理部１３は、例えば内部メモリに格納されたプログラムを実行することで種々の機能を実現するＣＰＵ又はＭＰＵ等を含む。画像処理部１３は、所望の機能を実現するように設計された専用のハードウェア回路を含んでもよい。画像処理部１３は、ＣＰＵ、ＭＰＵ、ＧＰＵ、ＤＳＰ、ＦＰＧＡ又はＡＳＩＣ等を含んでもよい。 The image processing unit 13 includes, for example, a CPU or an MPU that implements various functions by executing a program stored in an internal memory. The image processing unit 13 may include a dedicated hardware circuit designed to realize a desired function. The image processing unit 13 may include a CPU, an MPU, a GPU, a DSP, an FPGA, an ASIC, or the like.
　本実施形態の撮像システムにおいて、撮像素子１２の撮像面は、例えば矩形状に構成される。撮像素子１２の撮像面について、図３を用いて説明する。 に お い て In the imaging system according to the present embodiment, the imaging surface of the imaging element 12 is formed in, for example, a rectangular shape. The imaging surface of the imaging element 12 will be described with reference to FIG.
　図３は、撮像素子１２の撮像面１２ａが長方形である場合を例示している。撮像素子１２は、撮像面１２ａを規定する長辺Ｄｘおよび短辺Ｄｙを有する。図３の例において、長辺Ｄｘは、短辺Ｄｙに直交し、且つ短辺Ｄｙよりも大きい。本例の撮像素子１２において、長辺ＤｘはＸ方向に平行であり、短辺ＤｙはＹ方向に平行である。 FIG. 3 illustrates a case where the imaging surface 12a of the imaging element 12 is rectangular. The imaging element 12 has a long side Dx and a short side Dy that define the imaging surface 12a. In the example of FIG. 3, the long side Dx is orthogonal to the short side Dy and is larger than the short side Dy. In the image sensor 12 of this example, the long side Dx is parallel to the X direction, and the short side Dy is parallel to the Y direction.
　図３では、光学系ＩＬによるイメージサークルＩｓと撮像素子１２の撮像面１２ａとの位置関係を例示している。本実施形態のイメージサークルＩｓは、円形状から楕円などに歪んだ形状であり、長径Ｉｘ及び短径Ｉｙを有する。図３の例において、長径Ｉｘは、短径Ｉｙに直交し、且つ短径Ｉｙよりも大きい。光学系ＩＬは、撮像素子１２の長辺Ｄｘ及び短辺Ｄｙに対応して、イメージサークルＩｓの長径ＩｘがＸ方向に平行であり、短径ＩｙがＹ方向に平行であるように配置される。本実施形態の撮像素子１２において、Ｘ方向は第１方向の一例であり、Ｙ方向は第２方向の一例である。 FIG. 3 illustrates the positional relationship between the image circle Is by the optical system IL and the imaging surface 12a of the imaging device 12. The image circle Is of the present embodiment has a shape distorted from a circular shape to an ellipse or the like, and has a major axis Ix and a minor axis Iy. In the example of FIG. 3, the major axis Ix is orthogonal to the minor axis Iy and larger than the minor axis Iy. The optical system IL is arranged so that the major axis Ix of the image circle Is is parallel to the X direction and the minor axis Iy is parallel to the Y direction, corresponding to the long side Dx and the short side Dy of the image sensor 12. . In the image sensor 12 of the present embodiment, the X direction is an example of a first direction, and the Y direction is an example of a second direction.
　光学系ＩＬのイメージサークルＩｓは、例えば、撮像素子１２の撮像面１２ａの範囲に包含されない部分を有する。図３の例において、イメージサークルＩｓの長径Ｉｘは、撮像素子１２の長辺Ｄｘよりも大きい。また、イメージサークルＩｓの短径Ｉｙは、撮像素子１２の短辺Ｄｙよりも大きい。撮像素子１２は、撮像面１２ａの範囲内のイメージサークルＩｓによる画像を撮像する。 The image circle Is of the optical system IL has, for example, a portion that is not included in the range of the imaging surface 12a of the imaging element 12. In the example of FIG. 3, the major axis Ix of the image circle Is is larger than the major side Dx of the image sensor 12. The minor diameter Iy of the image circle Is is larger than the minor side Dy of the image sensor 12. The imaging device 12 captures an image by the image circle Is within the range of the imaging surface 12a.
１－２．光学系について
　本実施形態における光学系ＩＬの詳細について、図４を用いて説明する。図４は、撮像システムにおける光学系ＩＬの構成を例示する図である。 1-2. Optical System Details of the optical system IL according to the present embodiment will be described with reference to FIG. FIG. 4 is a diagram illustrating a configuration of the optical system IL in the imaging system.
　図４（ａ）は、光学系ＩＬのＹＺ断面におけるレンズ配置図を示す。図４（ｂ）は、光学系ＩＬのＸＺ断面におけるレンズ配置図を示す。なお、図４（ｂ）では、各種符号を省略している。 FIG. 4A shows a lens arrangement diagram in the YZ section of the optical system IL. FIG. 4B shows a lens arrangement diagram in an XZ section of the optical system IL. In FIG. 4B, various symbols are omitted.
　図４（ａ），（ｂ）の各レンズ配置図は、光学系ＩＬにおける各種レンズ等の配置を示す。図４（ａ）のＹＺ断面は、Ｙ方向と光学系ＩＬの光軸ｄ０に沿った仮想的な断面である。以下ではＹＺ断面を垂直面ともいう。図４（ｂ）のＸＺ断面は、水平方向であるＸ方向と光軸ｄ０に沿った仮想的な断面であり、撮像装置１１の水平面に対応する。 4 (a) and 4 (b) show the arrangement of various lenses and the like in the optical system IL. The YZ section in FIG. 4A is a virtual section along the Y direction and the optical axis d0 of the optical system IL. Hereinafter, the YZ section is also referred to as a vertical plane. The XZ cross section in FIG. 4B is a virtual cross section along the horizontal X direction and the optical axis d0, and corresponds to the horizontal plane of the imaging device 11.
　本実施形態の光学系ＩＬは、図４（ａ）に示すように、複数のレンズ素子Ｌｎと、絞りＡとを備える。複数のレンズ素子Ｌｎは、光学系ＩＬにおける光軸ｄ０に沿って、物体側から像面側へ順番に並んでいる。 光学 The optical system IL of the present embodiment includes a plurality of lens elements Ln and an aperture A, as shown in FIG. The plurality of lens elements Ln are arranged in order from the object side to the image plane side along the optical axis d0 in the optical system IL.
　絞りＡは、開口絞りであり、複数のレンズ素子Ｌｎの間に配置される。例えば、絞りＡは１６未満のＦ値を有する。これにより、回折限界の影響の観点から良好な像が得られない事態を回避し易い。絞りＡのＦ値は、上記の観点から８未満に設定されてもよい。 The stop A is an aperture stop and is arranged between the plurality of lens elements Ln. For example, aperture A has an F-number less than 16. This makes it easy to avoid a situation where a good image cannot be obtained from the viewpoint of the influence of the diffraction limit. The F value of the aperture A may be set to less than 8 from the above viewpoint.
　光学系ＩＬにおける複数のレンズ素子Ｌｎは、自由曲面レンズＬｆを含む。自由曲面レンズＬｆは、物体側と像面側の少なくとも一方に自由曲面を有するレンズ素子である。自由曲面は、光軸ｄ０に対して回転非対称な曲面である。本例において、自由曲面レンズＬｆは、絞りＡよりも物体側と像面側とにそれぞれ配置されている。また、光学系ＩＬにおけるレンズ素子Ｌｎは、球面レンズおよび非球面レンズなどの回転対称レンズＬｒを含んでもよい。光学系ＩＬは、撮像素子１２のカバーガラスを含んでもよい。 複数 The plurality of lens elements Ln in the optical system IL include a free-form surface lens Lf. The free-form surface lens Lf is a lens element having a free-form surface on at least one of the object side and the image plane side. The free-form surface is a curved surface that is rotationally asymmetric with respect to the optical axis d0. In this example, the free-form surface lens Lf is arranged on the object side and the image plane side of the stop A, respectively. Further, the lens element Ln in the optical system IL may include a rotationally symmetric lens Lr such as a spherical lens and an aspherical lens. The optical system IL may include a cover glass of the image sensor 12.
　本実施形態の光学系ＩＬにおいては、自由曲面レンズＬｆが、例えば図４（ａ），（ｂ）に示すように、Ｘ方向とＹ方向との間で非対称な自由曲面を有する。このような自由曲面レンズＬｆによって、本実施形態の光学系ＩＬは、円筒射影方式に応じて、入射する光線を回転非対称に制御する。自由曲面レンズＬｆにおける自由曲面は、例えばＸＹ多項式面である。 自由 In the optical system IL of the present embodiment, the free-form surface lens Lf has a free-form surface that is asymmetric between the X direction and the Y direction, for example, as shown in FIGS. With such a free-form surface lens Lf, the optical system IL of the present embodiment controls the incident light beam to be rotationally asymmetric in accordance with the cylindrical projection method. The free-form surface in the free-form surface lens Lf is, for example, an XY polynomial surface.
　本実施形態の光学系ＩＬにおいては、円筒射影方式が実装されるように、自由曲面レンズＬｆを含むレンズ素子Ｌｎについての各種パラメータが設定可能である。各種パラメータは、例えばＸＹ多項式面を規定するＸＹ多項式の各種係数のように、対応する面形状を規定する種々の係数を含む。各種パラメータは、曲率半径、面間隔、屈折率、及びアッベ数等を含んでもよい。 In the optical system IL of the present embodiment, various parameters can be set for the lens element Ln including the free-form surface lens Lf so that the cylindrical projection method is implemented. The various parameters include various coefficients that define the corresponding surface shape, such as various coefficients of an XY polynomial that defines an XY polynomial surface. The various parameters may include a radius of curvature, a surface interval, a refractive index, an Abbe number, and the like.
　本実施形態の光学系ＩＬは、例えば光学系ＩＬの焦点距離の１０倍よりも大きくて且つ１０００倍よりも小さい合焦距離を有する。上記の下限値を下回ると、像面歪曲の制御が困難になり得る。また、上記の上限値を上回ると、被写界深度の関係から近距離物体の像にボケが生じ得る。 光学 The optical system IL of the present embodiment has, for example, a focal length that is larger than 10 times the focal length of the optical system IL and smaller than 1000 times. When the value is below the above lower limit, it may be difficult to control the field curvature. If the upper limit is exceeded, the image of a close-range object may be blurred due to the depth of field.
　また、本実施形態の光学系ＩＬにおいて、水平画角θ１の半画角θ１／２は、例えば２０°よりも大きくて且つ８５°よりも小さい。上記の下限値を下回ると、対応する円筒面２０が狭くなり過ぎる。また、上記の上限値を上回ると、光学系ＩＬにおける非点収差の制御が難しくなる。以上のような観点から、４０°＜θ１／２＜７０°であってもよい。 Also, in the optical system IL of the present embodiment, the half angle of view θ1 / 2 of the horizontal angle of view θ1 is, for example, larger than 20 ° and smaller than 85 °. Below the lower limit, the corresponding cylindrical surface 20 becomes too narrow. If the upper limit is exceeded, it becomes difficult to control astigmatism in the optical system IL. From the above viewpoint, 40 ° <θ1 / 2 <70 ° may be satisfied.
１－３．円筒射影方式について
　本実施形態の撮像システムによる円筒射影方式について、図５～１２を用いて説明する。 1-3. Cylindrical Projection Method A cylindrical projection method by the imaging system according to the present embodiment will be described with reference to FIGS.
　図５は、撮像システムにおける円筒射影方式を説明するための図である。図６は、円筒射影方式における水平面と垂直面との対応関係を例示する図である。図７は、水平面と垂直面との対応関係の別例を示す図である。 FIG. 5 is a diagram for explaining the cylindrical projection method in the imaging system. FIG. 6 is a diagram exemplifying a correspondence between a horizontal plane and a vertical plane in the cylindrical projection method. FIG. 7 is a diagram illustrating another example of the correspondence between the horizontal plane and the vertical plane.
　図６（ａ）は、円筒射影方式の光学系ＩＬによる水平面（即ちＸＺ断面）上の射影関係を例示する。図６（ｂ）は、円筒射影方式の光学系ＩＬによる垂直面（即ちＹＺ断面）上の射影関係を例示する。 FIG. 6A illustrates a projection relationship on a horizontal plane (that is, an XZ section) by the cylindrical projection optical system IL. FIG. 6B illustrates a projection relationship on a vertical plane (that is, a YZ cross section) by the cylindrical projection optical system IL.
　円筒射影方式の理論式は、理想的には次式（Ｅ１），（Ｅ２）のように表される。
ｘ＝ｆｘ＊θｘ　　　　　　　　　　…（Ｅ１）
ｙ＝ｆｙ＊ｃｏｓθｘ＊ｔａｎθｙ　…（Ｅ２） The theoretical expression of the cylindrical projection method is ideally expressed by the following expressions (E1) and (E2).
x = fx * θx (E1)
y = fy * cos θx * tan θy (E2)
　上式（Ｅ１）において、ｘはＸ方向における像高を示し、光軸ｄ０を基準として像点毎に規定される。θｘは、Ｘ方向における当該像点の半画角を示す角度である（図５参照）。ｆｘは、光学系ＩＬについてのＸ方向の近軸焦点距離である。同様に、上式（Ｅ２）において、ｙはＹ方向における各種像点の像高を示し、θｙはＹ方向における当該像点の半画角を示す角度であり、ｆｙはＹ方向の近軸焦点距離である。なお、上式（Ｅ１），（Ｅ２）は各種変数ｘ，θｘ，ｙ，θｙが負の領域においても適宜、定義可能である。 Ｘ In the above equation (E1), x indicates an image height in the X direction, and is defined for each image point with reference to the optical axis d0. θx is an angle indicating a half angle of view of the image point in the X direction (see FIG. 5). fx is a paraxial focal length of the optical system IL in the X direction. Similarly, in the above equation (E2), y indicates the image height of each image point in the Y direction, θy is an angle indicating the half angle of view of the image point in the Y direction, and fy is the paraxial focus in the Y direction. Distance. Note that the above equations (E1) and (E2) can be appropriately defined even in a region where various variables x, θx, y, and θy are negative.
　上式（Ｅ１）のように、円筒射影方式の光学系ＩＬは、水平面においては等距離射影に従って、入射する光線の制御（即ち導光）を行う。これにより、例えば図６（ａ）に示すように、水平画角における角度幅Δθ毎に円筒面２０上に並んだ複数の点Ｐ２ａの像は、光学系ＩＬを介して、撮像面１２ａ上でＸ方向に並ぶ各々の像点Ｐ１ａに結像する。光学系ＩＬの射影関係が理想的な場合（式（Ｅ１）参照）、円筒面２０の円弧方向ｄ１に沿って等間隔の角度幅Δθは、撮像面１２ａ上ではＸ方向において等間隔な像点Ｐ１ａ間の距離Δｘに対応する。 円 筒 As in the above equation (E1), the cylindrical projection optical system IL controls (ie, guides) incoming light rays according to equidistant projection on the horizontal plane. Thus, for example, as shown in FIG. 6A, the images of the plurality of points P2a arranged on the cylindrical surface 20 for each angular width Δθ in the horizontal angle of view on the imaging surface 12a via the optical system IL. An image is formed at each image point P1a arranged in the X direction. When the projection relationship of the optical system IL is ideal (see equation (E1)), the angular width Δθ at regular intervals along the arc direction d1 of the cylindrical surface 20 is equal to the image points at regular intervals in the X direction on the imaging surface 12a. It corresponds to the distance Δx between P1a.
　以上のような水平面における等距離射影と共に、円筒射影方式の光学系ＩＬは、垂直面（即ちＹＺ断面）において上式（Ｅ２）のように、中心射影に従った導光を行う。図６（ｂ）では、光軸ｄ０を通ったＸＺ断面（即ちθｘ＝０）における光学系ＩＬの射影関係を示している。例えば、円筒面２０上で直線方向ｄ２の幅Δｄ毎に並んだ複数の点Ｐ２ｂの像は、光学系ＩＬを介して、撮像面１２ａ上でＹ方向に並ぶ各々の像点Ｐ１ｂに結像する。光学系ＩＬの射影関係が理想的な場合（式（Ｅ２）参照）、円筒面２０上で等間隔の幅Δｄは、撮像面１２ａ上でＹ方向において等間隔な像点Ｐ１ｂ間の距離Δｙに対応する。 Along with the equidistant projection on the horizontal plane as described above, the cylindrical projection optical system IL guides the light on the vertical plane (that is, the YZ section) according to the central projection as in the above equation (E2). FIG. 6B shows the projection relationship of the optical system IL on the XZ section (that is, θx = 0) passing through the optical axis d0. For example, images of a plurality of points P2b arranged on the cylindrical surface 20 for each width Δd in the linear direction d2 are formed on the imaging surface 12a at the image points P1b arranged in the Y direction on the imaging surface 12a via the optical system IL. . When the projection relationship of the optical system IL is ideal (see equation (E2)), the equally-spaced width Δd on the cylindrical surface 20 is equal to the distance Δy between equally-spaced image points P1b in the Y direction on the imaging surface 12a. Corresponding.
　また、光学系ＩＬによる中心射影の導光は、上式（Ｅ２）に示すように、水平画角における角度θｘ毎に変化する。図７（ａ）は、図６（ａ）と同様の射影関係を示す。図７（ｂ）は、図７（ａ）の光軸ｄ０からずれたＹＺ断面における光学系ＩＬの射影関係を例示している。理想的な光学系ＩＬの射影関係によると、θｘ≠０の各種位置において上式（Ｅ２）におけるｃｏｓθｘ＜１の分、図７（ｂ）に示すように近付いた円筒面２０上の各点Ｐ２ｃと像点Ｐ１ｃとの間で、図６（ｂ）と同様に中心射影の導光が行われる。 {Circle around (5)} The center projection light guide by the optical system IL changes for each angle θx in the horizontal angle of view as shown in the above equation (E2). FIG. 7A shows the same projection relationship as FIG. 6A. FIG. 7B illustrates the projection relationship of the optical system IL on the YZ section shifted from the optical axis d0 in FIG. 7A. According to the projection relationship of the ideal optical system IL, at various positions where θx ≠ 0, each point P2c on the cylindrical surface 20 approached as shown in FIG. 7B by cosθx <1 in the above equation (E2). The center projection light guide is performed between and the image point P1c in the same manner as in FIG. 6B.
　本実施形態の光学系ＩＬは、以上のような円筒射影方式に特有の光学特性を有する。例えば、光学系ＩＬは、Ｘ方向とＹ方向との間で非対称な歪曲収差を有する。図８を用いて、光学系ＩＬの歪曲収差について説明する。 光学 The optical system IL of the present embodiment has optical characteristics unique to the cylindrical projection method as described above. For example, the optical system IL has asymmetric distortion between the X direction and the Y direction. The distortion of the optical system IL will be described with reference to FIG.
　図８は、本実施形態における光学系ＩＬの光学特性を説明するための図である。図８（ａ）は、光学系ＩＬの歪曲収差の特性を例示する。図８（ｂ）は、歪曲収差の基準となる被写体を例示する。 FIG. 8 is a diagram for explaining the optical characteristics of the optical system IL according to the present embodiment. FIG. 8A illustrates a characteristic of distortion of the optical system IL. FIG. 8B illustrates a subject serving as a reference for distortion.
　図８（ａ）は、本実施形態の撮像装置１１が、図８（ｂ）のような正方格子状の被写体を撮像した場合に、歪曲収差が生じた撮像画像に対応する。図８（ａ）の原点Ｏは、光軸ｄ０の位置に対応する。なお、図８（ａ）では、光学系ＩＬがｆｘ＝ｆｙの場合の歪曲特性を例示している。 FIG. 8A corresponds to a captured image in which distortion has occurred when the imaging apparatus 11 of the present embodiment captures an image of a square lattice-shaped subject as shown in FIG. 8B. The origin O in FIG. 8A corresponds to the position of the optical axis d0. FIG. 8A illustrates the distortion characteristics when the optical system IL is fx = fy.
　本実施形態の光学系ＩＬにおいて、Ｘ方向における歪曲収差の特性は、等距離射影方式の場合と類似している。例えば図８（ａ）に示すように、光学系ＩＬは、Ｘ方向においては光軸ｄ０から離れる程、負に大きい歪曲収差を有する。これにより、円筒面２０の円弧方向ｄ１と撮像面１２ａのＸ方向とを対応させて結像を行える。また、本実施形態の光学系ＩＬにおいて、Ｙ方向における歪曲収差の特性は、例えば図８（ａ）に示すように、Ｘ方向における位置に応じて負に変動する。これにより、Ｘ方向の角度θｘ毎に、円筒面２０の直線方向ｄ２と撮像面１２ａのＹ方向とを対応させて結像を行える。 に お い て In the optical system IL of the present embodiment, the characteristic of the distortion in the X direction is similar to the case of the equidistant projection method. For example, as shown in FIG. 8A, the optical system IL has a larger negative distortion in the X direction as the distance from the optical axis d0 increases. Thereby, an image can be formed by associating the arc direction d1 of the cylindrical surface 20 with the X direction of the imaging surface 12a. Further, in the optical system IL of the present embodiment, the characteristic of the distortion in the Y direction fluctuates negatively according to the position in the X direction, for example, as shown in FIG. Thus, for each angle θx in the X direction, an image can be formed by associating the linear direction d2 of the cylindrical surface 20 with the Y direction of the imaging surface 12a.
　上述した例においては、ｆｘ＝ｆｙにより、Ｘ，Ｙ方向の双方において滑らかな歪曲特性を得やすい。光学系ＩＬにおける円筒射影方式に特有の光学特性は、例えば上述した各種パラメータにより設定できる。本実施形態に係る光学系ＩＬの実施例１～３について、図９～図１２を用いて説明する。 に お い て In the above example, fx = fy makes it easy to obtain smooth distortion characteristics in both the X and Y directions. Optical characteristics specific to the cylindrical projection system in the optical system IL can be set by, for example, the various parameters described above. Examples 1 to 3 of the optical system IL according to the present embodiment will be described with reference to FIGS.
　図９は、本実施形態に係る実施例１の光学系ＩＬにおける画角と像点Ｐ１１との関係を示す散布図である。本実施例の光学系ＩＬは、円筒射影方式の理論式（Ｅ１），（Ｅ２）を満たす。本実施例の光学系ＩＬは、最大値としての水平像高３ｍｍおよび垂直像高１．８５ｍｍを有する。また、光学系ＩＬの焦点距離は、焦点距離ｆｘ＝ｆｙ＝２．６４ｍｍであった。また、光学系ＩＬにおける水平画角θ１の半画角は、６５．０°であり、垂直画角θ２の半画角は３５．０°であった。 FIG. 9 is a scatter diagram showing the relationship between the angle of view and the image point P11 in the optical system IL of Example 1 according to the present embodiment. The optical system IL of the present embodiment satisfies the theoretical expressions (E1) and (E2) of the cylindrical projection system. The optical system IL according to the present embodiment has a maximum horizontal image height of 3 mm and a vertical image height of 1.85 mm. The focal length of the optical system IL was fx = fy = 2.64 mm. The half angle of view of the horizontal angle of view θ1 in the optical system IL was 65.0 °, and the half angle of view of the vertical angle of view θ2 was 35.0 °.
　図９においては、光学系ＩＬの画角全体における所定の角度幅毎に、入射する光が像面上で結像する像点Ｐ１１をプロットしている。当該角度幅は、水平方向及び垂直方向の各々について５°に設定した。図９では、光軸ｄ０の位置を原点とする像面のＸＹ平面において、第１象限の像点Ｐ１１を例示している。本実施例の光学系ＩＬは、Ｘ軸及びＹ軸に対してそれぞれ線対称であることから、第２～第４象限についても図９と同様である。 In FIG. 9, an image point P11 at which incident light forms an image on the image plane is plotted for each predetermined angle width over the entire angle of view of the optical system IL. The angle width was set to 5 ° in each of the horizontal direction and the vertical direction. FIG. 9 illustrates the image point P11 in the first quadrant on the XY plane of the image plane whose origin is the position of the optical axis d0. Since the optical system IL of the present embodiment is line-symmetric with respect to the X axis and the Y axis, the same applies to the second to fourth quadrants as in FIG.
　図１０は、中心射影方式における画角と像点Ｐ０との関係を例示する散布図である。図１０の例では、Ｘ，Ｙ方向間で対称な、単なる中心射影方式に従う光学系について、図９と同様にプロットされた像点Ｐ０を示している。図１０によると、水平像高が大きくなるほど、所定の角度幅毎の像点Ｐ０間の間隔が拡がっている。このように、単なる中心射影方式の場合、水平方向において画質が不均一になっている。 FIG. 10 is a scatter diagram illustrating the relationship between the angle of view and the image point P0 in the central projection method. In the example of FIG. 10, an image point P0 plotted in the same manner as in FIG. 9 is shown for an optical system that is symmetrical between the X and Y directions and simply follows the central projection method. According to FIG. 10, as the horizontal image height increases, the interval between the image points P0 for each predetermined angular width increases. Thus, in the case of the simple center projection method, the image quality is not uniform in the horizontal direction.
　これに対して、円筒射影方式に基づく本実施例の光学系ＩＬによると、図９に示すように、水平方向において像点Ｐ１１の間隔が等間隔であり、均一な画質が得られている。このように、本実施形態の撮像システムによると、円筒射影方式の光学系ＩＬにより像面全体で効率良く結像を行うことができる。 On the other hand, according to the optical system IL of this embodiment based on the cylindrical projection method, as shown in FIG. 9, the intervals between the image points P11 are equal in the horizontal direction, and uniform image quality is obtained. As described above, according to the imaging system of the present embodiment, an image can be efficiently formed on the entire image plane by the optical system IL of the cylindrical projection system.
　本実施形態の撮像システムにおける光学系ＩＬは、理論式（Ｅ１），（Ｅ２）を厳密に満たす必要はなく、次式（１），（２）を満たすことによって、円筒射影方式を実装してもよい。
０．５＜ｘ／（ｆｘ＊θｘ）＜２．０　　　　　　　　　　…（１）
０．５＜ｙ／（ｆｙ＊ｃｏｓθｘ＊ｔａｎθｙ）＜２．０　…（２）
　上式（１），（２）を満たす光学系ＩＬの射影関係によっても、実用上充分に精度良くパノラマ状の画像を結像することができる。 The optical system IL in the imaging system of the present embodiment does not need to strictly satisfy the theoretical expressions (E1) and (E2), but implements the cylindrical projection method by satisfying the following expressions (1) and (2). Is also good.
0.5 <x / (fx * θx) <2.0 (1)
0.5 <y / (fy * cos θx * tan θy) <2.0 (2)
A panoramic image can be formed with sufficient accuracy for practical use also by the projection relationship of the optical system IL that satisfies the above equations (1) and (2).
　図１１は、実施例２の光学系ＩＬにおける画角と像点Ｐ１２との関係を示す散布図である。本実施例の光学系ＩＬは、水平像高３ｍｍ、垂直像高２．１４ｍｍ、Ｘ方向の近軸焦点距離ｆｘ＝２．６４ｍｍ、Ｙ方向の近軸焦点距離ｆｙ＝３．５０ｍｍ、水平半画角θ１／２＝６５°、及び垂直半画角θ２／２＝３５°を有する。 FIG. 11 is a scatter diagram illustrating the relationship between the angle of view and the image point P12 in the optical system IL according to the second embodiment. The optical system IL according to this embodiment has a horizontal image height of 3 mm, a vertical image height of 2.14 mm, a paraxial focal length fx in the X direction fx = 2.64 mm, a paraxial focal length fy in the Y direction fy = 3.50 mm, and a horizontal half image. It has an angle θ1 / 2 = 65 ° and a vertical half angle of view θ2 / 2 = 35 °.
　本実施例の光学系ＩＬでは、上式（１）の中辺の値が０．９７～１．００の範囲内であり、上式（２）の中辺の値が０．７４～１．００の範囲内であり、上式（１），（２）を満たしている。本実施例の光学系ＩＬによっても、図１１に示すように、水平方向において均一な画質を得ることができる。 In the optical system IL of the present embodiment, the value of the middle side of the above equation (1) is in the range of 0.97 to 1.00, and the value of the middle side of the above equation (2) is 0.74 to 1. 00, which satisfies the above equations (1) and (2). Even with the optical system IL of the present embodiment, as shown in FIG. 11, uniform image quality can be obtained in the horizontal direction.
　図１２は、実施例３の光学系ＩＬにおける画角と像点Ｐ１３との関係を示す散布図である。本実施例の光学系ＩＬは、水平像高３．１５ｍｍ、垂直像高２．１４ｍｍ、Ｘ方向の近軸焦点距離ｆｘ＝２．６４ｍｍ、Ｙ方向の近軸焦点距離ｆｙ＝３．５０ｍｍ、水平半画角θ１／２＝５０°、及び垂直半画角θ２／２＝３５°を有する。 FIG. 12 is a scatter diagram illustrating the relationship between the angle of view and the image point P13 in the optical system IL according to the third embodiment. The optical system IL of the present embodiment has a horizontal image height of 3.15 mm, a vertical image height of 2.14 mm, a paraxial focal length fx in the X direction fx = 2.64 mm, a paraxial focal length fy in the Y direction fy = 3.50 mm, and a horizontal image height of 3.50 mm. It has a half angle of view θ1 / 2 = 50 ° and a vertical half angle of view θ2 / 2 = 35 °.
　本実施例の光学系ＩＬでは、上式（１）の中辺の値が１．００～１．３７の範囲内であり、上式（２）の中辺の値が０．７９～１．００の範囲内であり、上式（１），（２）を満たしている。本実施例の光学系ＩＬによっても、図１２に示すように、水平方向において均一な画質を得ることができる。 In the optical system IL of this embodiment, the value of the middle side of the above equation (1) is in the range of 1.00 to 1.37, and the value of the middle side of the above equation (2) is 0.79 to 1.37. 00, which satisfies the above equations (1) and (2). Also with the optical system IL of the present embodiment, as shown in FIG. 12, uniform image quality can be obtained in the horizontal direction.
２．動作
　以上のように構成される撮像システム及び撮像装置１１の動作について、以下説明する。 2. Operation The operation of the imaging system and the imaging device 11 configured as described above will be described below.
　本実施形態に係る撮像システムは、例えば撮像装置１１において複数回の撮像を行い、画像処理部１３において複数の撮像画像を合成することによって、複数回分の撮像結果のシーンが連続的に映るパノラマ画像を生成する。撮像装置１１が各撮像画像を生成する際に、光学系ＩＬは、上述した円筒射影方式に従って、撮像素子１２の撮像面１２ａに、円筒面２０に展開するシーンの像を結像する。これにより、画像処理部１３において複数の撮像画像を連結するスティッチングの画像処理を行い易くし、パノラマ画像の画質を良くすることができる。 The imaging system according to the present embodiment performs, for example, a plurality of imagings in the imaging device 11 and combines a plurality of the captured images in the image processing unit 13, so that a panorama image in which scenes of the imaging results for the multiple times are continuously reflected. Generate When the imaging device 11 generates each captured image, the optical system IL forms an image of a scene developed on the cylindrical surface 20 on the imaging surface 12a of the imaging device 12 according to the above-described cylindrical projection method. Accordingly, the image processing unit 13 can easily perform stitching image processing for connecting a plurality of captured images, and can improve the image quality of a panoramic image.
２－１．パノラマ画像の合成処理
　本実施形態の撮像システムにおいて、画像処理部１３がパノラマ画像を合成する処理について、図１３～１５を用いて説明する。 2-1. Panoramic Image Synthesizing Process The image processing unit 13 synthesizes a panoramic image in the imaging system according to the present embodiment, with reference to FIGS.
　図１３は、撮像システムにおけるパノラマ画像の合成処理を例示するフローチャートである。図１３のフローチャートによる各処理は、撮像システムの画像処理部１３によって実行される。 FIG. 13 is a flowchart illustrating a panoramic image synthesizing process in the imaging system. Each process in the flowchart of FIG. 13 is executed by the image processing unit 13 of the imaging system.
　まず、撮像システムの画像処理部１３は、例えば１台の撮像装置１１から複数の撮像画像を取得する（Ｓ１）。撮像装置１１は、例えば時分割で複数回、周囲のシーンを撮像して、複数の撮像画像を画像処理部１３に出力する。ステップＳ１における撮像装置１１の撮像の様子の一例を、図１４（ａ），（ｂ）に示す。 First, the image processing unit 13 of the imaging system acquires a plurality of captured images from, for example, one imaging device 11 (S1). The imaging device 11 captures the surrounding scene a plurality of times, for example, in a time-division manner, and outputs a plurality of captured images to the image processing unit 13. FIGS. 14A and 14B show an example of the state of imaging by the imaging device 11 in step S1.
　図１４（ａ）は、撮像装置１１が１回目の撮像を行う様子を例示する。図１４（ｂ）は、図１４（ａ）に続いて撮像装置１１が２回目の撮像を行う様子を例示する。図１４（ａ），（ｂ）の例では、撮像装置１１の周囲に、複数の被写体３１，３２が配置されている。撮像装置１１による撮像の回数は特に２回に限らず、３回以上であってもよい。 FIG. 14A illustrates an example in which the imaging device 11 performs the first imaging. FIG. 14B illustrates a state in which the imaging device 11 performs the second imaging, following FIG. 14A. In the example of FIGS. 14A and 14B, a plurality of subjects 31 and 32 are arranged around the imaging device 11. The number of times of imaging by the imaging device 11 is not particularly limited to two, but may be three or more.
　ステップＳ１において撮像装置１１は、例えば図１４（ａ），（ｂ）に示すように、円筒面２０の円弧方向ｄ１の中心を回転軸として、光軸ｄ０の向きを変えながら順次、撮像を行う。このような撮像装置１１の動作は、ユーザによって操作されてもよいし、自動制御されてもよい。撮像システムは、撮像装置１１を駆動する駆動装置を含んでもよい。 In step S1, the imaging device 11 sequentially performs imaging while changing the direction of the optical axis d0 with the center of the circular arc direction d1 of the cylindrical surface 20 as a rotation axis, as shown in FIGS. 14A and 14B, for example. . Such an operation of the imaging device 11 may be operated by a user or may be automatically controlled. The imaging system may include a driving device that drives the imaging device 11.
　ステップＳ１において、撮像装置１１から取得される撮像画像は、適宜許容誤差の範囲内で、円筒射影方式に従っている。画像処理部１３は、取得した撮像画像に対して、理想的な円筒射影方式に対する誤差を補正するための画像処理などを行ってもよい。 In step S1, the captured image obtained from the imaging device 11 follows the cylindrical projection method within an allowable error range. The image processing unit 13 may perform image processing for correcting an error with respect to the ideal cylindrical projection method on the acquired captured image.
　次に、画像処理部１３は、取得した複数の撮像画像にスティッチングの画像処理を行って（Ｓ２）、パノラマ画像を合成する。画像処理部１３によるスティッチングの画像処理について、図１５を用いて説明する。 Next, the image processing unit 13 performs stitching image processing on the acquired plurality of captured images (S2) to synthesize a panoramic image. Stitching image processing by the image processing unit 13 will be described with reference to FIG.
　図１５（ａ）は、図１４（ａ）の撮像時に得られた撮像画像４１を例示する。図１５（ｂ）は、図１４（ｂ）の撮像時に得られた撮像画像４２を例示する。図１５（ｃ）は、図１５（ａ），（ｂ）の撮像画像４１，４２に対するスティッチング後のパノラマ画像４を例示する。 FIG. 15A illustrates a captured image 41 obtained at the time of imaging in FIG. FIG. 15B illustrates a captured image 42 obtained at the time of the imaging in FIG. 14B. FIG. 15C illustrates the panoramic image 4 after the stitching for the captured images 41 and 42 of FIGS. 15A and 15B.
　ステップＳ２において、画像処理部１３は、例えば図１５（ａ），（ｂ）に示す撮像画像４１，４２を互いに連結して、図１５（ｃ）に例示するようにパノラマ画像４を合成する。この際、画像処理部１３は、例えば画像解析によって撮像画像４１，４２間の重なりの領域４０を判定し、判定した領域４０について平均化或いは置換等の処理を行ってもよい。 In step S2, the image processing unit 13 connects the captured images 41 and 42 shown in FIGS. 15A and 15B, for example, and synthesizes the panoramic image 4 as illustrated in FIG. 15C. At this time, the image processing unit 13 may determine an overlapping area 40 between the captured images 41 and 42 by, for example, image analysis, and perform averaging or replacement processing on the determined area 40.
　ステップＳ２の画像処理時に、画像処理部１３は、例えば図１４（ａ），（ｂ）の各例における光軸ｄ０の向きのように、取得した各々の撮像画像４１，４２についての向きを示す情報を用いてもよい。当該情報は、例えばユーザの操作で入力されてもよいし、撮像装置１１に組み込まれたジャイロセンサ等から取得されてもよい。 At the time of the image processing in step S2, the image processing unit 13 indicates the direction of each of the acquired captured images 41 and 42, for example, the direction of the optical axis d0 in each example of FIGS. 14 (a) and 14 (b). Information may be used. The information may be input by a user operation, for example, or may be acquired from a gyro sensor or the like incorporated in the imaging device 11.
　図１３に戻り、画像処理部１３は、合成したパノラマ画像４を、各種の表示デバイス又は記憶デバイス等に出力して（Ｓ３）、本フローチャートによる処理を終了する。 Returning to FIG. 13, the image processing unit 13 outputs the synthesized panoramic image 4 to various display devices or storage devices (S3), and ends the processing according to this flowchart.
　以上の処理によると、撮像装置１１から取得される撮像画像（Ｓ１）が円筒射影方式に従って、円筒面２０の円弧方向ｄ１に沿って均一な画質で周囲のシーンを映している。例えば、図１５（ａ），（ｂ）の撮像画像４１，４２間では、同じ被写体３２が、違う位置に映っている。この際、円筒射影方式によると、被写体３２に対する光軸ｄ０の向きが回転しているにも拘わらず（図１４（ａ），（ｂ））、被写体３２が撮像画像４１，４２に映る形状は変化していない。 According to the above processing, the captured image (S1) acquired from the imaging device 11 reflects the surrounding scene with uniform image quality along the arc direction d1 of the cylindrical surface 20 according to the cylindrical projection method. For example, the same subject 32 appears at different positions between the captured images 41 and 42 in FIGS. At this time, according to the cylindrical projection method, although the direction of the optical axis d0 with respect to the subject 32 is rotated (FIGS. 14A and 14B), the shape of the subject 32 reflected in the captured images 41 and 42 is not changed. Has not changed.
　よって、スティッチングの画像処理時（Ｓ２）に、撮像画像４１，４２に映る被写体３２を変形するような大幅な座標変換を特に行うことなく、パノラマ画像４を合成でき、スティッチングの画像処理における処理負荷を低減できる。また、上記のような大幅な座標変換を回避できることから、撮像時から画質を劣化させることなく撮像画像のスティッチングを行え、パノラマ画像４の画質を良くすることができる。 Therefore, at the time of the stitching image processing (S2), the panoramic image 4 can be synthesized without particularly performing a significant coordinate transformation that deforms the subject 32 shown in the captured images 41 and 42. Processing load can be reduced. Further, since such a large coordinate transformation as described above can be avoided, stitching of the captured image can be performed without deteriorating the image quality from the time of imaging, and the image quality of the panoramic image 4 can be improved.
３．まとめ
　以上のように、本実施形態における撮像装置１１は、撮像素子１２と、光学系ＩＬとを備える。撮像素子１２は、互いに交差する第１及び第２方向の一例としてＸ，Ｙ方向を有する撮像面１２ａを構成する。光学系ＩＬは、撮像面１２ａに結像するように配置される。光学系ＩＬは、撮像面１２ａとは反対側を覆うように湾曲した仮想的な曲面である円筒面２０に対する射影関係に応じてＸ，Ｙ方向の間で非対称な自由曲面レンズＬｆを備える。光学系ＩＬの射影関係は、円筒面２０が湾曲する円弧方向ｄ１と撮像面１２ａのＸ方向を対応させながら、円筒面２０において円弧方向ｄ１に交差する直線方向ｄ２と撮像面１２ａのＹ方向を対応させる。 3. Conclusion As described above, the imaging device 11 according to the present embodiment includes the imaging element 12 and the optical system IL. The imaging element 12 forms an imaging surface 12a having X and Y directions as an example of first and second directions intersecting each other. The optical system IL is arranged so as to form an image on the imaging surface 12a. The optical system IL includes a free-form surface lens Lf that is asymmetric between the X and Y directions according to a projection relationship with respect to the cylindrical surface 20 that is a virtual curved surface that is curved so as to cover the side opposite to the imaging surface 12a. The projection relationship of the optical system IL is such that a linear direction d2 intersecting the circular arc direction d1 on the cylindrical surface 20 and a Y direction of the image capturing surface 12a correspond to the circular arc direction d1 at which the cylindrical surface 20 is curved and the X direction of the image capturing surface 12a. Make it correspond.
　以上の撮像装置１１によると、円筒射影方式に従う光学系ＩＬの円筒面２０に対する射影関係によって、円筒面２０上のシーンの歪みを低減して撮像でき、パノラマ状の画像を撮像し易くすることができる。 According to the imaging device 11 described above, by projecting the optical system IL onto the cylindrical surface 20 according to the cylindrical projection method, it is possible to reduce the distortion of the scene on the cylindrical surface 20 and image the panoramic image. it can.
　本実施形態において、光学系ＩＬは、円弧方向ｄ１とＸ方向を対応させながら直線方向ｄ２とＹ方向を対応させるように、Ｘ，Ｙ方向の間で非対称な歪曲収差を有する（図８参照）。円筒射影方式に特有の歪曲特性を有する光学系ＩＬによって、円筒面２０に沿ったパノラマ状の画像を撮像し易くすることができる。 In this embodiment, the optical system IL has asymmetric distortion between the X and Y directions so that the linear direction d2 and the Y direction correspond while the arc direction d1 and the X direction correspond (see FIG. 8). . By the optical system IL having the distortion characteristic peculiar to the cylindrical projection method, it is possible to easily capture a panoramic image along the cylindrical surface 20.
　また、本実施形態において、光学系ＩＬの射影関係は、Ｘ方向においては等距離射影に基づき、Ｙ方向においては中心射影に基づく（図６，７参照）。これにより、円弧方向ｄ１と直線方向ｄ２との各々に沿ったパノラマ状の画像を撮像し易くすることができる。 Also, in the present embodiment, the projection relationship of the optical system IL is based on equidistant projection in the X direction and based on center projection in the Y direction (see FIGS. 6 and 7). Accordingly, it is possible to easily capture a panoramic image along each of the arc direction d1 and the linear direction d2.
　また、本実施形態の撮像装置１１において、撮像面１２ａ上で、Ｘ方向における像高ｘ及びＹ方向における像高ｙが、以下の式（１）及び式（２）を満たす。
０．５＜ｘ／（ｆｘ＊θｘ）＜２　　　　　　　　　　…（１）
０．５＜ｙ／（ｆｙ＊ｃｏｓθｘ＊ｔａｎθｙ）＜２　…（２）
　ここで、
θｘ：Ｘ方向における像高ｘの半画角
θｙ：Ｙ方向における像高ｙの半画角
ｆｘ：光学系におけるＸ方向の近軸焦点距離
ｆｙ：光学系におけるＹ方向の近軸焦点距離
である。これにより、式（１），（２）のように円筒射影方式を実装することにより、パノラマ状の画像を実用上、充分に撮像し易くすることができる。 In the imaging device 11 of the present embodiment, the image height x in the X direction and the image height y in the Y direction on the imaging surface 12a satisfy the following Expressions (1) and (2).
0.5 <x / (fx * θx) <2 (1)
0.5 <y / (fy * cos θx * tan θy) <2 (2)
here,
θx: half angle of view of image height x in X direction θy: half angle of view of image height y in Y direction fx: paraxial focal length in optical system in X direction fy: paraxial focal length in optical system in Y direction . Thus, by implementing the cylindrical projection method as in Expressions (1) and (2), it is possible to easily capture a panoramic image practically and sufficiently.
　また、本実施形態の撮像装置１１において、Ｘ方向における像高ｘの半画角θｘは、２０°よりも大きくて且つ８５°よりも小さい。これにより、非点収差を制御しながら、撮像される円筒面２０の範囲を充分に確保することができる。 In addition, in the imaging device 11 of the present embodiment, the half angle of view θx of the image height x in the X direction is larger than 20 ° and smaller than 85 °. Thereby, the range of the cylindrical surface 20 to be imaged can be sufficiently secured while controlling astigmatism.
　また、本実施形態の撮像装置１１において、Ｘ方向の近軸焦点距離ｆｘとＹ方向の近軸焦点距離ｆｙとは、同じ大きさを有してもよい。この場合、Ｘ，Ｙ方向の双方において滑らかな歪曲収差を得やすい。 In the imaging device 11 of the present embodiment, the paraxial focal length fx in the X direction and the paraxial focal length fy in the Y direction may have the same size. In this case, it is easy to obtain a smooth distortion in both the X and Y directions.
　また、本実施形態の撮像装置１１において、光学系ＩＬは、例えばＸ，Ｙ方向の近軸焦点距離ｆｘ，ｆｙの少なくとも一方の１０倍よりも大きくて、且つ１０００倍よりも小さい合焦距離を有する。このように遠い合焦距離により、像面歪曲を制御しながら、撮像装置１１付近の物体のボケの低減を実現し易い。 Further, in the imaging apparatus 11 of the present embodiment, the optical system IL sets a focusing distance which is larger than at least one of the paraxial focal lengths fx and fy in the X and Y directions and smaller than 1000 times, for example. Have. As described above, the blurring of the object near the imaging device 11 can be easily reduced while controlling the image plane distortion by the long focusing distance.
　また、本実施形態において、光学系ＩＬは、例えば１６よりも小さいＦ値を有する。これにより、回折限界の観点から良好な像が得難くなる事態を回避しやすい。 Also, in the present embodiment, the optical system IL has an F value smaller than 16, for example. This makes it easy to avoid a situation where it is difficult to obtain a good image from the viewpoint of the diffraction limit.
　また、本実施形態における撮像システムは、撮像装置１１と、画像処理部１３とを備える。画像処理部１３は、撮像装置１１によって撮像された撮像画像４１，４２に基づいて、パノラマ画像４を生成する。本実施形態の撮像システムによると、撮像装置１１の光学系ＩＬにより、パノラマ画像４の生成を行い易い。 The imaging system according to the present embodiment includes an imaging device 11 and an image processing unit 13. The image processing unit 13 generates a panoramic image 4 based on the captured images 41 and 42 captured by the imaging device 11. According to the imaging system of the present embodiment, the panorama image 4 is easily generated by the optical system IL of the imaging device 11.
（他の実施形態）
　以上のように、本出願において開示する技術の例示として、実施形態１を説明した。しかしながら、本開示における技術は、これに限定されず、適宜、変更、置換、付加、省略などを行った実施の形態にも適用可能である。また、上記各実施形態で説明した各構成要素を組み合わせて、新たな実施の形態とすることも可能である。そこで、以下、他の実施形態を例示する。 (Other embodiments)
As described above, the first embodiment has been described as an example of the technology disclosed in the present application. However, the technology in the present disclosure is not limited to this, and can be applied to embodiments in which changes, substitutions, additions, omissions, and the like are made as appropriate. Further, it is also possible to form a new embodiment by combining the components described in the above embodiments. Therefore, other embodiments will be exemplified below.
　上記の実施形態１では、１台の撮像装置１１を備える撮像システムの例を説明した。本実施形態の撮像システムは、複数台の撮像装置１１を備えてもよい。本変形例について、図１６を用いて説明する。 In the first embodiment, the example of the imaging system including one imaging device 11 has been described. The imaging system of the present embodiment may include a plurality of imaging devices 11. This modification will be described with reference to FIG.
　図１６では、撮像システムに含まれる２台の撮像装置１１が撮像を行う様子を例示する。各撮像装置１１は、例えば共通の画像処理部１３に接続して、本変形例の撮像システムを構成する。なお、本実施形態の撮像システムは、３台以上の撮像装置１１を含んでもよい。 FIG. 16 illustrates an example in which two imaging devices 11 included in the imaging system perform imaging. Each imaging device 11 is connected to, for example, a common image processing unit 13 to configure an imaging system of the present modification. Note that the imaging system of the present embodiment may include three or more imaging devices 11.
　図１６の例において、各々の撮像装置１１は、共通の円筒面２０上で異なる領域を撮像するように配置されている。画像処理部１３は、図１３のステップＳ１において、各撮像装置１１から撮像画像を取得し、スティッチングの画像処理を行う（Ｓ２）。この場合においても、各々の撮像装置１１において円筒面２０の円弧方向ｄ１に沿って均質な撮像画像が得られることから、画像処理部１３は簡単にスティッチングを実行でき、高画質のパノラマ画像を生成できる。 In the example of FIG. 16, the respective imaging devices 11 are arranged so as to image different regions on the common cylindrical surface 20. In step S1 of FIG. 13, the image processing unit 13 acquires a captured image from each imaging device 11, and performs stitching image processing (S2). Also in this case, since a uniform captured image is obtained along the arc direction d1 of the cylindrical surface 20 in each of the imaging devices 11, the image processing unit 13 can easily perform stitching, and can perform high-quality panoramic images. Can be generated.
　また、上記の各実施形態では、円筒面２０に基づく円筒射影方式について説明した（理論式（Ｅ１），（Ｅ２）参照）。本実施形態の撮像システムにおいて、円筒面２０は適宜、許容誤差の範囲内で歪んだ曲面に設定されてもよい。例えば、円筒面２０の円弧方向ｄ１は、真円における円弧でなくてもよく、適宜凸状の曲線による方向であってもよい。また、直線方向ｄ２は、厳密な直線による方向でなくてもよく、円弧方向ｄ１よりも直線的な方向であってもよい。また、円弧方向ｄ１と直線方向ｄ２とは、直角から適宜、許容誤差の角度範囲内で互いに交差してもよい。 In each of the above embodiments, the cylindrical projection method based on the cylindrical surface 20 has been described (see theoretical formulas (E1) and (E2)). In the imaging system of the present embodiment, the cylindrical surface 20 may be appropriately set to a curved surface that is distorted within an allowable error range. For example, the circular arc direction d1 of the cylindrical surface 20 may not be a circular arc in a perfect circle but may be a direction by a convex curve as appropriate. Further, the linear direction d2 may not be a direction based on a strict straight line, and may be a direction that is more linear than the arc direction d1. Further, the arc direction d1 and the linear direction d2 may intersect each other from a right angle within an allowable angle range as appropriate.
　また、上記の各実施形態では、円筒射影方式に従う光学系ＩＬの射影関係として、Ｘ方向においては等距離射影に基づき、Ｙ方向においては中心射影に基づく場合を例示した。本実施形態の光学系ＩＬの射影関係は、Ｘ方向において等立体各射影、正射影あるいは立体射影に基づいてもよいし、Ｙ方向において弱中心射影あるいは疑似中心射影に基づいてもよい。 In each of the above embodiments, the projection relation of the optical system IL according to the cylindrical projection method is based on the equidistant projection in the X direction and the center projection in the Y direction. The projection relationship of the optical system IL according to the present embodiment may be based on each stereoscopic projection, orthographic projection or stereoscopic projection in the X direction, or may be based on weak center projection or pseudo center projection in the Y direction.
　また、上記の各実施形態では、図３において長方形の撮像面１２ａを例示したが、撮像素子１２の撮像面１２ａはこれに限らない。本実施形態において、撮像素子１２の撮像面１２ａは、長方形でない各種の矩形状であってもよく、部分的にマスクされていてもよい。また、撮像素子１２の撮像面１２ａは、湾曲していてもよい。 In each of the above embodiments, the rectangular imaging surface 12a is illustrated in FIG. 3, but the imaging surface 12a of the imaging element 12 is not limited to this. In the present embodiment, the imaging surface 12a of the imaging element 12 may have various rectangular shapes other than a rectangular shape, or may be partially masked. Further, the imaging surface 12a of the imaging element 12 may be curved.
　例えば、本実施形態の撮像素子１２の長辺Ｄｘと短辺Ｄｙとは直交していなくてもよく、各種の角度で交差してもよい。また、撮像素子１２は、長辺Ｄｘ及び短辺Ｄｙの代わりに同じ長さの二辺を有してもよい。本実施形態の光学系ＩＬでは、イメージサークルＩｓの長径Ｉｘと短径Ｉｙとによる第１及び第２方向も、互いに直交していなくてもよく、各種の角度で交差してもよい。また、イメージサークルＩｓにおける第１及び第２方向の径の長さが同じであってもよい。イメージサークルＩｓは、必ずしも円形から歪んでいなくてもよい。また、本実施形態ではイメージサークルＩｓは撮像素子１２の撮像面１２ａに包含されない部分を有していたが、イメージサークルＩｓは撮像面１２ａを包含していてもよい。 For example, the long side Dx and the short side Dy of the image sensor 12 of the present embodiment need not be orthogonal to each other, and may intersect at various angles. Further, the imaging element 12 may have two sides having the same length instead of the long side Dx and the short side Dy. In the optical system IL of the present embodiment, the first and second directions defined by the major axis Ix and the minor axis Iy of the image circle Is not necessarily orthogonal to each other, and may intersect at various angles. Further, the diameters of the image circle Is in the first and second directions may be the same. The image circle Is does not necessarily have to be distorted from a circle. In the present embodiment, the image circle Is has a portion that is not included in the imaging surface 12a of the imaging element 12, but the image circle Is may include the imaging surface 12a.
　上記の各実施形態では、自由曲面の一例として、ＸＹ多項式面を例示した。本実施形態において、自由曲面は上記に限らず、例えばアナモルフィック非球面、或いはトーリック面を含んでもよい。また、本実施形態の光学系ＩＬにおいて、自由曲面レンズは、アナモルフィックではない自由曲面を有してもよい。アナモルフィックではない自由曲面は、ＸＹ多項式面は含む一方でアナモルフィック非球面を含まない。アナモルフィックではない自由曲面は、例えば対称面を有さなくてもよい。 In each of the above embodiments, the XY polynomial surface has been exemplified as an example of the free-form surface. In the present embodiment, the free-form surface is not limited to the above, and may include, for example, an anamorphic aspheric surface or a toric surface. Further, in the optical system IL of the present embodiment, the free-form surface lens may have a free-form surface that is not anamorphic. Free-form surfaces that are not anamorphic include XY polynomial surfaces but do not include anamorphic aspheric surfaces. A free-form surface that is not anamorphic may not have, for example, a plane of symmetry.
　以上のように、本開示における技術の例示として、実施の形態を説明した。そのために、添付図面および詳細な説明を提供した。 As described above, the embodiments have been described as examples of the technology in the present disclosure. For that purpose, the accompanying drawings and the detailed description have been provided.
　したがって、添付図面および詳細な説明に記載された構成要素の中には、課題解決のために必須な構成要素だけでなく、上記技術を例示するために、課題解決のためには必須でない構成要素も含まれ得る。そのため、それらの必須ではない構成要素が添付図面や詳細な説明に記載されていることをもって、直ちに、それらの必須ではない構成要素が必須であるとの認定をするべきではない。 Accordingly, among the components described in the accompanying drawings and the detailed description, not only those components that are essential for solving the problem, but also those that are not essential for solving the problem in order to exemplify the technology. May also be included. Therefore, it should not be immediately recognized that these non-essential components are essential based on the fact that the non-essential components are described in the accompanying drawings and the detailed description.
　また、上述の実施の形態は、本開示における技術を例示するためのものであるから、特許請求の範囲またはその均等の範囲において、種々の変更、置換、付加、省略などを行うことができる。 In addition, since the above-described embodiments are intended to exemplify the technology of the present disclosure, various changes, substitutions, additions, omissions, and the like can be made within the scope of the claims or the equivalents thereof.
　本開示は、撮像を行う各種アプリケーションに適用可能であり、例えば移動ロボット、Ｖｉｓｕａｌ－ＳＬＡＭ、３６０°カメラ、会議システム及び車載センサなどに適用可能である。 The present disclosure is applicable to various applications that perform imaging, for example, a mobile robot, a Visual-SLAM, a 360 ° camera, a conference system, and a vehicle-mounted sensor.

Claims (9)

  1. 　互いに交差する第１及び第２方向を有する撮像面を構成する撮像素子と、
    　前記撮像面に結像するように配置された光学系と
    を備え、
    　前記光学系は、前記撮像面とは反対側を覆うように湾曲した仮想的な曲面である円筒面に対する射影関係に応じて前記第１及び第２方向の間で非対称な自由曲面レンズを備え、
    　前記光学系の射影関係は、前記円筒面が湾曲する円弧方向と前記撮像面の第１方向を対応させながら、前記円筒面において前記円弧方向に交差する直線方向と前記撮像面の第２方向を対応させる
    撮像装置。     An image sensor that forms an imaging surface having first and second directions that intersect each other;
    An optical system arranged to form an image on the imaging surface,
    The optical system includes a free-form surface lens that is asymmetric between the first and second directions according to a projection relationship with respect to a cylindrical surface that is a virtual curved surface that is curved so as to cover an opposite side to the imaging surface,
    The projection relationship of the optical system is such that, while making the arc direction in which the cylindrical surface is curved correspond to the first direction of the imaging surface, the linear direction intersecting the arc direction in the cylindrical surface and the second direction of the imaging surface are set. The corresponding imaging device.
  2. 　前記光学系は、前記円弧方向と前記第１方向を対応させながら前記直線方向と前記第２方向を対応させるように、前記第１及び第２方向の間で非対称な歪曲収差を有する
    請求項１に記載の撮像装置。     The optical system has an asymmetric distortion between the first and second directions so that the linear direction corresponds to the second direction while the arc direction corresponds to the first direction. An imaging device according to claim 1.
  3. 　前記光学系の射影関係は、前記第１方向においては等距離射影に基づき、前記第２方向においては中心射影に基づく
    請求項１又は２に記載の撮像装置。     The imaging apparatus according to claim 1, wherein the projection relationship of the optical system is based on equidistant projection in the first direction and based on center projection in the second direction.
  4. 　前記撮像面上で、前記第１方向における像高ｘ及び前記第２方向における像高ｙが、以下の式（１）及び式（２）を満たし、
    ０．５＜ｘ／（ｆｘ＊θｘ）＜２　　　　　　　　　　…（１）
    ０．５＜ｙ／（ｆｙ＊ｃｏｓθｘ＊ｔａｎθｙ）＜２　…（２）
    　ここで、
    θｘ：前記第１方向における像高ｘの半画角
    θｙ：前記第２方向における像高ｙの半画角
    ｆｘ：前記光学系における前記第１方向の近軸焦点距離
    ｆｙ：前記光学系における前記第２方向の近軸焦点距離
    である、請求項１～３のいずれか１項に記載の撮像装置。     An image height x in the first direction and an image height y in the second direction satisfy the following Expressions (1) and (2) on the imaging surface;
    0.5 <x / (fx * θx) <2 (1)
    0.5 <y / (fy * cos θx * tan θy) <2 (2)
    here,
    θx: half angle of view of the image height x in the first direction θy: half angle of view fx of the image height y in the second direction fx: paraxial focal length fy of the optical system in the first direction: the halfway angle of the optical system The imaging device according to any one of claims 1 to 3, wherein the imaging device has a paraxial focal length in a second direction.
  5. 　前記第１方向における像高ｘの半画角θｘは、２０°よりも大きくて且つ８５°よりも小さい
    請求項４に記載の撮像装置。     The imaging device according to claim 4, wherein a half angle of view θx of the image height x in the first direction is larger than 20 ° and smaller than 85 °.
  6. 　前記第１方向の近軸焦点距離ｆｘと前記第２方向の近軸焦点距離ｆｙとが、同じ大きさを有する
    請求項４又は５に記載の撮像装置。     The imaging device according to claim 4, wherein the paraxial focal length fx in the first direction and the paraxial focal length fy in the second direction have the same size.
  7. 　前記光学系は、前記第１及び第２方向の近軸焦点距離ｆｘ，ｆｙの少なくとも一方の１０倍よりも大きくて、且つ１０００倍よりも小さい合焦距離を有する
    請求項４～６のいずれか１項に記載の撮像装置。     The optical system according to any one of claims 4 to 6, wherein the optical system has a focusing distance that is greater than 10 times and less than 1000 times at least one of the paraxial focal lengths fx and fy in the first and second directions. 2. The imaging device according to claim 1.
  8. 　前記光学系は、１６よりも小さいＦ値を有する
    請求項１～７のいずれか１項に記載の撮像装置。     The imaging device according to claim 1, wherein the optical system has an F-number smaller than 16.
  9. 　請求項１～８のいずれか１項に記載の撮像装置と、
    　前記撮像装置によって撮像された撮像画像に基づいて、パノラマ画像を生成する画像処理部と
    を備える撮像システム。     An imaging device according to any one of claims 1 to 8,
    An imaging system comprising: an image processing unit configured to generate a panoramic image based on a captured image captured by the imaging device.
PCT/JP2019/026285 2018-08-22 2019-07-02 Imaging device and imaging system WO2020039759A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2018155493A JP2021182019A (en) 2018-08-22 2018-08-22 Imaging apparatus and imaging system
JP2018-155493 2018-08-22

Publications (1)

Publication Number Publication Date
WO2020039759A1 true WO2020039759A1 (en) 2020-02-27

Family

ID=69591869

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/026285 WO2020039759A1 (en) 2018-08-22 2019-07-02 Imaging device and imaging system

Country Status (2)

Country Link
JP (1) JP2021182019A (en)
WO (1) WO2020039759A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114019659A (en) * 2021-11-30 2022-02-08 江西晶超光学有限公司 Optical system, image capturing module and electronic equipment

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10206985A (en) * 1997-01-21 1998-08-07 Tec Corp Image pickup device
JP2011130327A (en) * 2009-12-21 2011-06-30 Sony Corp Image processing apparatus, method and program
JP2017134394A (en) * 2016-01-21 2017-08-03 パナソニックＩｐマネジメント株式会社 Lens system, and camera system having the same

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10206985A (en) * 1997-01-21 1998-08-07 Tec Corp Image pickup device
JP2011130327A (en) * 2009-12-21 2011-06-30 Sony Corp Image processing apparatus, method and program
JP2017134394A (en) * 2016-01-21 2017-08-03 パナソニックＩｐマネジメント株式会社 Lens system, and camera system having the same

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114019659A (en) * 2021-11-30 2022-02-08 江西晶超光学有限公司 Optical system, image capturing module and electronic equipment

Also Published As

Publication number Publication date
JP2021182019A (en) 2021-11-25

Similar Documents

Publication Publication Date Title
TW469348B (en) Super wide-angle panoramic imaging apparatus
JP5169499B2 (en) Imaging device and imaging apparatus
JP2018201214A (en) Multi-camera system using folded optics
KR20110068994A (en) Three-mirror panoramic camera
JP7108935B2 (en) Lens system, camera system and imaging system
US10462363B2 (en) Optical apparatus
CN108227159A (en) Optical system, Optical devices and image pick-up device
WO2020017200A1 (en) Imaging optical system, imaging device, and imaging system
US20220221694A1 (en) Lens system, imaging device, and imaging system
US20220214528A1 (en) Lens system, imaging device, and imaging system
JP7382568B2 (en) Lens system, imaging device and imaging system
US20140340472A1 (en) Panoramic bifocal objective lens
JP4981359B2 (en) Element image group imaging device and stereoscopic image display device
WO2020039759A1 (en) Imaging device and imaging system
JPWO2019187221A1 (en) Lens system, imaging device and imaging system
KR20090089234A (en) Image capturing lens system
JP2015191186A (en) Stereo adapter and stereo image-capturing device
JP2008256796A (en) Imaging pick-up device
JP2011227306A (en) Complex eye camera
JP2006126322A (en) Optical system
US8503109B2 (en) Optical system and imaging device
JP2008197541A (en) Optical image forming system, optical device and imaging apparatus
US11019260B2 (en) Image capturing apparatus
JP2018098738A (en) Image processing device, imaging device, image processing method and image processing program
WO2014168110A1 (en) Image-capturing device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19852967

Country of ref document: EP

Kind code of ref document: A1

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19852967

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP